risk assessment of student performance in the international foundations of medicine clinical science examination by the use of statistical modeling

Michael C David,1 Diann S Eley,2 Jennifer Schafer,2 Leo Davies,3 1School of Public Health, 2School of Medicine, The University of Queensland, Herston, QLD, 3Sydney Medical School, The University of Sydney, NSW, Australia Purpose: The primary aim of this study was to assess the predictive validity of cumulative grade point average (GPA) for performance in the International Foundations of Medicine (IFOM) Clinical Science Examination (CSE). A secondary aim was to develop a strategy for identifying students at risk of performing poorly in the IFOM CSE as determined by the National Board of Medical Examiners’ International Standard of Competence. Methods: Final year medical students from an Australian university medical school took the IFOM CSE as a formative assessment. Measures included overall IFOM CSE score as the dependent variable, cumulative GPA as the predictor, and the factors age, gender, year of enrollment, international or domestic status of student, and language spoken at home as covariates. Multivariable linear regression was used to measure predictor and covariate effects. Optimal thresholds of risk assessment were based on receiver-operating characteristic (ROC) curves. Results: Cumulative GPA (nonstandardized regression coefficient [B]: 81.83; 95% confidence interval [CI]: 68.13 to 95.53) and international status (B: –37.40; 95% CI: –57.85 to –16.96) from 427 students were found to be statistically associated with increased IFOM CSE ­performance. Cumulative GPAs of 5.30 (area under ROC [AROC]: 0.77; 95% CI: 0.72 to 0.82) and 4.90 (AROC: 0.72; 95% CI: 0.66 to 0.78) were identified as being thresholds of significant risk for domestic and international students, respectively. Conclusion: Using cumulative GPA as a predictor of IFOM CSE performance and accommodating for differences in international status, it is possible to identify students who are at risk of failing to satisfy the National Board of Medical Examiners’ International Standard of Competence. Keywords: academic performance, benchmarking, medical education, predictive validity, risk assessment, students at risk