Accuracy and Quality Assessment of Artificial Intelligence Platforms and a Hybrid Evidence-Assisted Model in Responding to Drug Information Questions

Clicks: 3
ID: 338923
2026
Free PDF
Article Quality & Performance Metrics
Overall Quality
Not rated
Combines reader engagement with the AI quality analysis. This article has not been analysed, so there is no overall score — reader engagement is measured and shown alongside.
AI Quality Assessment
Not analyzed
Readership in this journal
Emerging

Ranked #251 of 255 articles by views in American journal of health-system pharmacy : AJHP : official journal of the American Society of Health-System Pharmacists

Most read Least read

Bar heights use a square-root scale. Only the 120 most-read articles are drawn; the journal has 255 in total.

Mint this article as an NFT
Not yet minted

Create a permanent, verifiable on-chain record of this article on the Scimatic Network. The NFT is held in your Journament account, and you can withdraw it to your own wallet at any time.

5 SUSD one-off · no wallet required
Sign in to mint
Abstract
PURPOSE: This descriptive report evaluated the accuracy and response quality of ChatGPT 5.1, OpenEvidence, Clair, and a hybrid evidence-assisted ChatGPT 5.1 model in responding to drug information questions, using drug information specialist-developed reference responses as the comparator. SUMMARY: Twenty drug information questions were selected from the UNC Health Drug Information Center archive from July 2021 through July 2025, stratified across 10 categories, and classified as simple or complex. Responses were generated in November 2025 using ChatGPT 5.1, OpenEvidence, Clair, and a hybrid evidence-assisted model in which drug information specialist-curated evidence packets were provided to ChatGPT 5.1. Drug information specialist-developed reference responses included a definitive answer, evidence synthesis, a recommendation when applicable, and verifiable references. Deidentified and randomized responses were evaluated through a blinded two-phase review process assessing accuracy and response quality. ChatGPT 5.1 had the numerically highest accuracy at 70%, followed by the hybrid evidence-assisted model at 60%, OpenEvidence at 40%, and Clair at 15%. Drug information specialist-developed reference responses and OpenEvidence had the highest mean overall quality performance at 82.5% each. CONCLUSION: AI platforms demonstrated variable accuracy and quality when responding to drug information questions. In this limited sample, ChatGPT 5.1 had the numerically highest accuracy, while OpenEvidence had the highest mean overall quality performance among AI-generated responses. Despite strengths in organization and readability, limitations in accuracy, reference identifiability, and evidence synthesis support the continued need for drug information specialist oversight.
Reference Key
openalex_W7225294548 Use this key to autocite in the manuscript while using SciMatic Manuscript Manager or Thesis Manager
Authors Nasr Issa, Evan W Colmenares, Andrew Douglas, Gregory A. Heindel, L. Lee, Erin Maxwell, Erin Mazur, Zach Pedretti, John Killian Rodgers, Kirsten Smith, Ina Liu
Are you one of these authors? Claim your profile
Journal American journal of health-system pharmacy : AJHP : official journal of the American Society of Health-System Pharmacists
Year 2026
DOI
10.1093/ajhp/zxag379
URL
https://doi.org/10.1093/ajhp/zxag379
Keywords Keywords not found

Citations

No citations found. To add a citation, contact the admin at info@scimatic.org

Comments

No comments yet. Be the first to comment on this article.