A Scoping Study of Evaluation Practices for Responsible AI Tools: Steps Towards Effectiveness Evaluations
Clicks: 22
ID: 283233
2024
Article Quality & Performance Metrics
Overall Quality
Not rated
Combines reader engagement with the AI quality analysis. This
article has not been analysed, so there is no overall score —
reader engagement is measured and shown alongside.
Reader Engagement
Emerging Content
6.3
/100
22 views
10 readers
AI Quality Assessment
Not analyzed
Readership in this journal
EmergingRanked #773 of 803 articles by views in arXiv
Most read
Least read
Bar heights use a square-root scale. Only the 120 most-read articles are drawn; the journal has 803 in total.
Mint this article as an NFT
Not yet mintedCreate a permanent, verifiable on-chain record of this article on the Scimatic Network. The NFT is held in your Journament account, and you can withdraw it to your own wallet at any time.
5
SUSD
one-off · no wallet required
Abstract
Responsible design of AI systems is a shared goal across HCI and AI
communities. Responsible AI (RAI) tools have been developed to support
practitioners to identify, assess, and mitigate ethical issues during AI
development. These tools take many forms (e.g., design playbooks, software
toolkits, documentation protocols). However, research suggests that use of RAI
tools is shaped by organizational contexts, raising questions about how
effective such tools are in practice. To better understand how RAI tools are --
and might be -- evaluated, we conducted a qualitative analysis of 37
publications that discuss evaluations of RAI tools. We find that most
evaluations focus on usability, while questions of tools' effectiveness in
changing AI development are sidelined. While usability evaluations are an
important approach to evaluate RAI tools, we draw on evaluation approaches from
other fields to highlight developer- and community-level steps to support
evaluations of RAI tools' effectiveness in shaping AI development practices and
outcomes.
| Reference Key |
madaio2024a
Use this key to autocite in the manuscript while using
SciMatic Manuscript Manager or Thesis Manager
|
|---|---|
| Authors | Glen Berman; Nitesh Goyal; Michael Madaio |
| Journal | arXiv |
| Year | 2024 |
| DOI |
DOI not found
|
| URL | |
| Keywords |
Citations
No citations found. To add a citation, contact the admin at info@scimatic.org
Comments
No comments yet. Be the first to comment on this article.