Risk
Existing protections tend to address only high-impact, high-risk decisions; however, persons with disabilities face many small, low-impact AI decisions every day (i.e., resume filters, course recommendations, scheduling tools, etc.). Over time, both small and large harms compound and accumulate, causing multi-step harm that becomes particularly significant when accommodations are not made, considered, nor inputted into the AI tracking the data.
Mitigation
Governance should be extended to cover repeated low-impact decisions, with cumulative effect over time treated as the unit of analysis as opposed to the single decision. Longitudinal harm tracking should be required, as opposed to relying on one-off audits.
Illustrative Examples
Education
Small barriers stacking across the learning journey
Students with disabilities may interact with many AI systems over time, including admissions tools, course recommendation systems, exam monitoring, and grading tools. Small issues in each system may build up over time, leading to repeated disadvantages that affect long-term outcomes.
Employment
Layered disadvantages across the employment lifecycle
Workers with disabilities may encounter multiple AI systems across their employment journey, including job ads, screening tools, interview scoring, and performance reviews. Each system may introduce small disadvantages, and over time these effects may build up. This can shape hiring, promotion, and retention outcomes in ways that are hard to detect.
Healthcare
Compounding gaps across the care pathway
AI systems are often used at many points in care, such as triage, scheduling, diagnosis, and treatment planning. When each system has small gaps or biases, these effects can build up over time. A patient may be repeatedly under-served at each step, which can worsen health outcomes.
Services
Barriers accumulating across public systems
People may interact with many AI systems across public services, such as benefits, taxes, transportation, and identification systems. Small issues or biases in each interaction may build up over time. This can affect access to services and create ongoing challenges.