Systematic Reviews of Medical Machine Learning: Limitations of Pooling AUCs
Clicks: 1
ID: 320341
2026
Article Quality & Performance Metrics
Overall Quality
Not rated
Combines reader engagement with the AI quality analysis. This
article has not been analysed, so there is no overall score —
reader engagement is measured and shown alongside.
Reader Engagement
0.0
/100
1 views
0 readers
AI Quality Assessment
Not analyzed
Readership in this journal
Ranked #32 of 55 articles by views in European Heart Journal - Digital Health
Most read
Least read
Bar heights use a square-root scale.
Mint this article as an NFT
Not yet mintedCreate a permanent, verifiable on-chain record of this article on the Scimatic Network. The NFT is held in your Journament account, and you can withdraw it to your own wallet at any time.
5
SUSD
one-off · no wallet required
Abstract
Abstract The medical literature has seen rapid growth in studies that develop and evaluate machine learning (ML) classifiers for diagnostics, accompanied by an increase in systematic reviews for synthesizing this evidence. Many such reviews use meta-analyses that pool performance metrics such as area under the receiver-operating characteristic curve (AUC), sensitivity, and specificity across ML studies. In our view, quantitative pooling of such metrics is statistically and conceptually inappropriate in most settings, except when the same model is being tested across different samples. ML classifiers differ fundamentally in training data, model specification, and validation strategies. As a result, there is no underlying “true AUC” that pooling aims to recover. In addition, AUC is a sampling-dependent, non-linear, and bounded measure whose value depends on relative ranking within a dataset, which makes it unsuitable for aggregation. Pooling sensitivity and specificity presents similar problems due to threshold dependence, inconsistent reporting practices, and inappropriate weighting by test-set size. Outside limited scenarios in which aggregation may be reasonable, undue emphasis on pooled performance estimates obfuscates critical descriptors of model validity and generalizability. We propose that systematic reviews of medical ML models prioritize structured, descriptive synthesis of key study characteristics, including dataset composition, validation strategy, input modalities, outcome definitions, data augmentation, independent validation, and deployability. When summary performance measures are presented, they are better presented as stratified summaries in subgroups of comparable studies. A concerted discourse on standardizing reporting standards is essential to guide systematic reviews that meaningfully inform clinical utility of ML models.
| Reference Key |
openalex_W7167852722
Use this key to autocite in the manuscript while using
SciMatic Manuscript Manager or Thesis Manager
|
|---|---|
| Authors | Amulya Gupta, Rahel Sileshi, Jeffrey Thompson, Amit Noheria |
| Journal | European Heart Journal - Digital Health |
| Year | 2026 |
| DOI |
10.1093/ehjdh/ztag107
|
| URL | |
| Keywords | Keywords not found |
Citations
No citations found. To add a citation, contact the admin at info@scimatic.org
Comments
No comments yet. Be the first to comment on this article.