Should Physicians Take the Rap? Normative Analysis of Clinician Perspectives on Responsible Use of 'Black Box' AI Tools.

Increasing interest in deploying artificial intelligence tools in clinical contexts has raised several ethical questions of both normative and empirical interest. One such question in the literature is whether "responsibility gaps" (r-gaps) are created when clinicians utilize or rely on such tools for providing care, and if so, what to do about them. These gaps are particularly likely to arise when using opaque, "black box" AI tools. Compared to normative and legal analysis of AI-generated responsibility gaps in health care, little is known, empirically, about health care providers views on this issue. The present study examines clinician perspectives on this issue in the context of black box AI decisional support systems (BBAI-DSS) in advanced heart failure. Semi-structured interviews were conducted with 20 clinicians (14 cardiologists and 6 LVAD nurse coordinators). Interviews were transcribed, coded, and thematically analyzed for salient themes. All study procedures were approved by local IRB. We found that clinicians voiced that, if someone were responsible for the use and outcomes of black box AI, it would be physicians. We compare clinician perspectives on the existence of r-gaps and their impact on responsibility for errors or adverse outcomes when BBAI-DSS tools are used against a taxonomy from the literature, finding some clinicians acknowledging an r-gap and others denying it or its relevance in medical decision-making. Clinicians varied in their view about the existence of r-gaps but were united in their ascriptions of physician responsibility for the use of BBAI-DSS in clinical care. It was unclear at times whether these were descriptive or normative judgments (i.e., is it merely inevitable physicians will be responsible, or is it morally appropriate that they be held responsible?) or both. We discuss the likely normative inadequacy of such a conception of physician responsibility for BBAI tool use.