Towards AI Accountability Infrastructure: Gaps and Opportunities in AI Audit Tooling
Clicks: 45
ID: 283229
2024
Article Quality & Performance Metrics
Overall Quality
Not rated
Combines reader engagement with the AI quality analysis. This
article has not been analysed, so there is no overall score —
reader engagement is measured and shown alongside.
Reader Engagement
Emerging Content
13.2
/100
45 views
28 readers
AI Quality Assessment
Not analyzed
Readership in this journal
EmergingRanked #591 of 803 articles by views in arXiv
Most read
Least read
Bar heights use a square-root scale. Only the 120 most-read articles are drawn; the journal has 803 in total.
Mint this article as an NFT
Not yet mintedCreate a permanent, verifiable on-chain record of this article on the Scimatic Network. The NFT is held in your Journament account, and you can withdraw it to your own wallet at any time.
5
SUSD
one-off · no wallet required
Abstract
Audits are critical mechanisms for identifying the risks and limitations of
deployed artificial intelligence (AI) systems. However, the effective execution
of AI audits remains incredibly difficult, and practitioners often need to make
use of various tools to support their efforts. Drawing on interviews with 35 AI
audit practitioners and a landscape analysis of 435 tools, we compare the
current ecosystem of AI audit tooling to practitioner needs. While many tools
are designed to help set standards and evaluate AI systems, they often fall
short in supporting accountability. We outline challenges practitioners faced
in their efforts to use AI audit tools and highlight areas for future tool
development beyond evaluation -- from harms discovery to advocacy. We conclude
that the available resources do not currently support the full scope of AI
audit practitioners' needs and recommend that the field move beyond tools for
just evaluation and towards more comprehensive infrastructure for AI
accountability.
| Reference Key |
raji2024towards
Use this key to autocite in the manuscript while using
SciMatic Manuscript Manager or Thesis Manager
|
|---|---|
| Authors | Victor Ojewale; Ryan Steed; Briana Vecchione; Abeba Birhane; Inioluwa Deborah Raji |
| Journal | arXiv |
| Year | 2024 |
| DOI |
DOI not found
|
| URL | |
| Keywords |
Citations
No citations found. To add a citation, contact the admin at info@scimatic.org
Comments
No comments yet. Be the first to comment on this article.