Risk
AI text and image classifiers are often trained on datasets which perpetuate social biases against persons with disabilities; as a result, disability-related terms tend to be flagged as toxic, negative, or harmful. This produces stigmatizing, patronizing, or pity-driven representations of disability, as well as content moderation systems that suppress disability-related content, including speech and images.
Mitigation
Toxicity classifiers should be audited before deployment to remove disability-language scoring, and disability sentiment benchmarks should be incorporated into model evaluation. Partnering with disabled creators and disabled persons’ organizations (DPOs) is essential to ensure that representation testing reflects lived experiences.
Illustrative Examples
Education
Course content blocked by filters
Captioning systems may incorrectly censor or filter certain words, including terms related to bodies, health, or relationships, when they are used in an educational context. This may limit access to accurate information and remove important meaning from lesson content, especially in subjects like health education.
Employment
Disability conversations filtered out at work
Content moderation systems used on professional platforms may incorrectly flag disability-related language as sensitive or negative. This can lead to disability advocacy posts or discussions about accommodations being hidden, removed, or shown to fewer people. Over time, this may reduce visibility of disability-related knowledge and professional contributions.
Healthcare
Patronizing clinical language
Patient-facing chatbots may produce responses about disability that sound overly cautious, patronizing, pitying, or indirect. This tone can make the information seem less trustworthy or less respectful, which may reduce patient confidence in the healthcare they receive.
Services
Public messaging that reinforces disability stigma
Generative tools used in public communications may produce language about disability that is overly negative, simplified, or stigmatizing. This can affect how programs are described and understood, and may reduce trust or clarity for people who rely on these services.