Evaluation of knowledge-augmented DeepSeek models on nephrology examination questions

Clicks: 2
ID: 338117
2026
Free PDF
Article Quality & Performance Metrics
Overall Quality
Not rated
Combines reader engagement with the AI quality analysis. This article has not been analysed, so there is no overall score — reader engagement is measured and shown alongside.
AI Quality Assessment
Not analyzed
Readership in this journal
Emerging

Ranked #227 of 228 articles by views in clinical kidney journal

Most read Least read

Bar heights use a square-root scale. Only the 120 most-read articles are drawn; the journal has 228 in total.

Mint this article as an NFT
Not yet minted

Create a permanent, verifiable on-chain record of this article on the Scimatic Network. The NFT is held in your Journament account, and you can withdraw it to your own wallet at any time.

5 SUSD one-off · no wallet required
Sign in to mint
Abstract
Abstract Background Large language models (LLMs) demonstrate significant potential in medical applications. DeepSeek, as an open-source and customizable LLM, offers potential for nephrology applications through flexible deployment, potential cost reduction, and enhanced data privacy. However, its specific performance and applicability in the nephrology specialty remain to be systematically evaluated. Methods We evaluated six large language models (DeepSeek-R1, DeepSeek-V3, GPT-4, GPT-4o, Qwen-2.5, and Gemini-2.5) using 1 294 nephrology multiple-choice questions derived from the ASN NephSAP and KSAP educational programs. The DeepSeek-R1, DeepSeek-V3, GPT-4o, and Gemini-2.5 models were further enhanced through a retrieval-augmented generation (RAG) framework incorporating a nephrology comorbidity knowledge base (NCKB). The NCKB, reviewed by board-certified nephrologists, integrated authoritative sources including standard textbooks, clinical guidelines, and peer-reviewed literature. The NCKB included 50 347 structured knowledge units. Model accuracy, response time, and error types were analyzed using chi-square tests and visualized with bar and radar charts. Results A total of 1 294 nephrology exam questions were used for evaluation. The knowledge-enhanced DeepSeek models demonstrated significantly improved performance compared with their base versions (both P < 0.001), with DSR1 + NCKB and DSV3 + NCKB achieving accuracies of 81.22% and 79.13%, respectively. The performance difference between DSR1 + NCKB and DSV3 + NCKB did not reach statistical significance (P = 0.171). Knowledge gaps were the main cause of DSR1 + NCKB’s errors (52.26%), followed by reasoning and comprehension errors (20.16%). Conclusion Integrating an expert-curated NCKB significantly improves performance on standardized nephrology questions compared with baseline models. The magnitude of improvement varies across models, with DSR1 + NCKB achieving performance comparable to or approaching that of advanced proprietary models evaluated in this study.
Reference Key
openalex_W7225087313 Use this key to autocite in the manuscript while using SciMatic Manuscript Manager or Thesis Manager
Authors Youlu Zhao, Hongyu Yang, Xizi Zheng, Damin Xu, Lingyi Xu, Qingqing Zhou, Mengrui Li, Yuhui Zhang, Liu Lijun, Jicheng Lv, Li Yang
Are you one of these authors? Claim your profile
Journal clinical kidney journal
Year 2026
DOI
10.1093/ckj/sfag363
URL
https://doi.org/10.1093/ckj/sfag363
Keywords Keywords not found

Citations

No citations found. To add a citation, contact the admin at info@scimatic.org

Comments

No comments yet. Be the first to comment on this article.