Illustrative Examples

Adaptive learning systems are often designed around typical patterns of student behavior. As a result, they may be less effective for students with disabilities whose ways of learning or engaging fall outside those patterns. This may lead to lower-quality feedback, inappropriate difficulty levels, or missed learning needs.

Captioning systems may incorrectly censor or filter certain words, including terms related to bodies, health, or relationships, when they are used in an educational context. This may limit access to accurate information and remove important meaning from lesson content, especially in subjects like health education.

When educators use image generation tools to create teaching materials, the outputs may rarely include assistive devices or disability-relevant situations, even when prompted. This can lead to classroom content that does not reflect a diverse range of learners.

Online exam proctoring systems may flag certain behaviors or tools as suspicious, even when they are related to disability. For example, assistive devices, different eye movements, or physical differences may be misinterpreted as signs of cheating. This can lead to unfair scrutiny or penalties.

Admissions systems that rely on patterns such as continuous enrollment may disadvantage students whose records reflect disability-related disruptions. These systems may categorize applicants who have taken medical leave as less academically strong even when the gap in studies is not related to student performance.

Students with disabilities may interact with many AI systems over time, including admissions tools, course recommendation systems, exam monitoring, and grading tools. Small issues in each system may build up over time, leading to repeated disadvantages that affect long-term outcomes.

Tools used to create and manage AI in education may not be accessible to instructional designers with disabilities. This can limit the instructor’s ability to design inclusive course materials or identify accessibility issues, reducing the quality of learning environments.

Students who rely on AI-generated descriptions of images, diagrams, or charts may receive inaccurate or incomplete descriptions. If the student cannot independently verify the content, this may affect understanding of key material, especially in subjects like science or mathematics.

Systems that track student engagement may collect detailed data about behavior and accommodations. Because some disability-related patterns are unique, this data may make students easier to identify, even when it is meant to be anonymous. This can create privacy risks.

Students who rely heavily on AI tools for writing may begin to use more standard or neutral language generated by the system. Over time, this may reduce the visibility of their own voice, including perspectives shaped by their lived experience with disability.

Speech recognition tools may be introduced to support students who prefer speaking over typing. However, they may not work reliably for students with different speech patterns. This can make it harder for those students to complete assignments, even though the tool is meant to improve access.

Accessibility features in educational tools are often only available in higher-cost versions or in widely used languages. This may limit access for students in lower-resource settings and can create dependency on tools that may change or become unavailable over time.

A student with a disability may leave the education system early due to lack of accessible supports, barriers in learning environments, or unmet accommodation needs. Without formal credentials or stable pathways into employment, they may be more likely to seek flexible online work to earn income. AI-related gig work such as data labelling or content moderation can appear accessible because it is remote and does not require formal qualifications. As a result, individuals may become more vulnerable to exploitative conditions that can worsen their health or limit opportunities to move into more secure employment.

When students report issues with tools (e.g., speech recognition or captioning errors), these may be viewed as individual problems rather than part of a broader pattern of errors. This can reduce the likelihood of system-wide fixes. Over time, similar issues may be reported by multiple students but treated separately, which can prevent patterns from being identified. As a result, the tool may continue to perform poorly for certain users without triggering meaningful updates or improvements.

A workplace performance evaluation system may compare employees against common productivity patterns, such as consistent work hours, typing speed, or communication style. An employee with a disability who works in shorter bursts, uses assistive tools, or communicates in different ways may fall outside these patterns. As a result, the system may rate their performance as lower or less consistent, even when their work quality is strong. These differences may be seen as normal variation in the data rather than a sign of bias, which can prevent managers from recognizing and addressing unfair evaluation practices.

Content moderation systems used on professional platforms may incorrectly flag disability-related language as sensitive or negative. This can lead to disability advocacy posts or discussions about accommodations being hidden, removed, or shown to fewer people. Over time, this may reduce visibility of disability-related knowledge and professional contributions.

Generative tools used to create résumés or portfolios may suggest removing or rewording gaps, part-time work, or non-traditional roles. As a result, disability-related experiences such as medical leave or flexible work may be omitted, creating a more “standard” profile that does not reflect the person’s full background.

Workplace monitoring systems may flag certain behaviors as unusual or suspicious, even when they are related to disability. For example, taking breaks for health reasons or using assistive technology may be marked as low productivity or potential misuse. This can lead to unfair scrutiny or incorrect performance assessments.

Résumé-screening systems may use patterns in work history to filter candidates, which can disadvantage applicants with gaps due to medical leave or disability-related needs. Even when disability-based discrimination is not allowed, these systems may still pick up indirect signals and exclude candidates. As a result, qualified applicants may be overlooked.

Workers with disabilities may encounter multiple AI systems across their employment journey, including job ads, screening tools, interview scoring, and performance reviews. Each system may introduce small disadvantages, and over time these effects may build up. This can shape hiring, promotion, and retention outcomes in ways that are hard to detect.

Hiring processes for technical roles often use coding tests and development platforms that may not be fully accessible. This can make it harder for developers with disabilities to demonstrate their skills or complete assessments. As a result, fewer individuals with disabilities may be included in AI-related roles.

Human resources chatbots may provide incorrect information about policies,  workplace rights or accommodation processes while appearing confident. Employees who rely on these tools may not know the information is wrong. This can lead to missed supports or misunderstandings about available options.

Workplace systems that track employee activity may collect detailed data about accommodations or working patterns. Because some of these patterns are unique, they may make individuals easier to identify, even when data is meant to be anonymous. This information may then influence management decisions in ways that are not always clear or fair.

When AI tools are used to draft emails, reports, or other workplace communication, they may suggest standard language and tone. Employees who rely on these tools may gradually adopt this style, which can reduce the visibility of their own voice and perspective. This may affect how their contributions are understood.

A company may introduce automated captions in meetings and events and assume that captioning needs have been fully addressed. However, automated captions may contain frequent errors, missed words, or incorrect terminology, especially in fast-paced or technical discussions. As a result, Deaf and hard of hearing employees may still not be able to fully follow the conversation. Despite this, the company may reduce or stop providing human captioning services (such as Communication Access Realtime Translation, or CART), creating a situation where access appears to be improved but is less effective for the people who rely on it most.

AI tools that support workplace accommodations are often only available in certain regions or languages, or at higher cost. This may limit access for workers in lower-resource settings. Over time, individuals may become dependent on specific tools that may change, become more expensive, or be discontinued.

People with disabilities may be more likely to take on flexible work such as data labeling or content review to earn income. While this work may provide short-term income, it is often low paid and may lack work-place protections or supports. Because AI gig work is often not classified as formal employment, workers may not have access to workplace accommodations, ergonomic supports, or health benefits. This may lead to untreated injuries or conditions, increasing long-term healthcare needs. In some cases, the tasks require review of graphic or sensitive material and may worsen existing physical or mental health conditions.

Patterns such as more frequent rejection of job candidates with non-standard work histories may be explained away as normal variation in the data. This can prevent employers from recognizing and correcting barriers linked to disability. Over time, each decision may be treated as an isolated case rather than part of a broader issue. As a result, qualified candidates may continue to be overlooked without triggering review or change in the system.

Diagnostic artificial intelligence (AI) systems are often trained on common symptom patterns. As a result, these systems may be more likely to miss or under-detect conditions in people with atypical symptoms or physiology that do not match the data the system was trained on. This can lead to delayed diagnosis or incomplete care when the system does not recognize less common presentations.

Patient-facing chatbots may produce responses about disability that sound overly cautious, patronizing, pitying, or indirect. This tone can make the information seem less trustworthy or less respectful, which may reduce patient confidence in the healthcare they receive.

Patient-information chatbots in many settings may default to content in dominant languages. This can leave people who use other languages, including ASL speakers, without clear or accessible health information. As a result, they may miss important guidance or receive less useful support.

Insurance fraud detection systems may be more likely to flag certain patterns linked to disabilities as suspicious. Because people with disabilities often have ongoing care needs, multiple conditions, or complex treatment histories, their claims may be treated as unusual or higher risk. This can lead to claims being denied or flagged for review more often, creating barriers to care, delays in treatment, added administrative burden, and increased stress for patients who may already be managing significant health needs; in serious cases, these delays may disrupt essential care and contribute to worsening health or life-threatening outcomes.

Care-prioritization systems may use past data to decide how to allocate resources. Because people with disabilities have sometimes received less care in the past, these systems may underestimate their needs. This may result in fewer services or lower priority, even when support is needed.

AI systems are often used at many points in care, such as triage, scheduling, diagnosis, and treatment planning. When each system has small gaps or biases, these effects can build up over time. A patient may be repeatedly under-served at each step, which can worsen health outcomes.

Tools used to operate or review clinical AI systems, such as dashboards and monitoring platforms, may not be fully accessible. This can make it harder for clinicians with disabilities to use, oversee, or question these systems. As a result, fewer perspectives may shape how the AI is improved or corrected.

Symptom-checker chatbots may sometimes give incorrect answers while appearing very confident. They may miss or misinterpret disability-related symptoms, or suggest conditions that sound believable but are not accurate. This can lead to confusion or delay in seeking the right care.

Wearable devices and monitoring tools can collect ongoing health data. Even when this data is labeled as anonymous, it may still be possible to re-identify people with rare conditions because their patterns are unique. This may create privacy risks that are not always clear to patients.

AI tools that summarize patient input may simplify,reshape, or change what a patient has said. Important details about disability-related experiences may be removed or softened in the process. Over time, this may give a reduced or oversimplified view of the patient’s situation to their care team, not capturing the full context of their experience.

Some health monitoring tools may not measure vital signs as accurately for people with different body types, movement patterns, or assistive devices. This can lead to less reliable data for people with disabilities. If clinicians rely on this data, decisions may not reflect the patient’s actual condition.

Many AI health tools are developed for well-funded systems and may be costly to use or maintain. Clinics with fewer resources may not be able to continue using them if prices change or vendors shift direction. This can interrupt care, especially for tools that support people with disabilities over time.

Healthcare systems may not recognize AI gig work as a source of occupational strain. Healthcare providers may not always ask about or record AI gig work when discussing a patient’s daily activities or employment. As a result, the demands of this work, such as long hours of screen use, repetitive clicking or typing, irregular schedules, or exposure to distressing content, may not be recognized as part of the person’s health context. This can delay recovery, worsen existing conditions, impact eligibility for care, or increase reliance on healthcare services without reducing the source of strain.

An AI diagnostic system can make more mistakes for people with less common conditions, but these errors may be considered too rare to address. The impact may be serious or life-threatening, even a small number of missed or incorrect diagnoses may delay treatment, lead to inappropriate care, or worsen existing conditions. Over time, treating these errors as rare may allow the same risks to continue for others with similar needs without being corrected.

Eligibility systems are often built on common spending and usage patterns. As a result, they may be less accurate for people whose expenses reflect disability-related needs. This can lead to misclassification or incorrect decisions about benefit eligibility.

Generative tools used in public communications may produce language about disability that is overly negative, simplified, or stigmatizing. This can affect how programs are described and understood, and may reduce trust or clarity for people who rely on these services.

A government agency may introduce a chatbot to help people understand eligibility for a service. Because the system’s training data may include limited information about disability-related situations, it may provide incomplete or incorrect answers when users ask about complex needs, such as how fluctuating conditions affect eligibility or what documentation is required for specific accommodations. As a result, people with disabilities may be given guidance that does not reflect their situation, leading to incorrect applications, delays in receiving support, or missed access to services they are eligible for.

Systems used in areas such as border security or fraud detection may flag certain behaviors or tools as suspicious. People with disabilities, including those using assistive devices or with different movement patterns, may be more likely to be flagged. This can lead to delays, additional checks, or stress.

Eligibility systems may use patterns from past applications to make decisions. If people with disabilities were previously denied at higher rates, these systems may continue to reflect those patterns. This can result in applicants being excluded based on indirect signals rather than their actual eligibility.

People may interact with many AI systems across public services, such as benefits, taxes, transportation, and identification systems. Small issues or biases in each interaction may build up over time. This can affect access to services and create ongoing challenges.

Tools used to design, purchase, and review public-sector AI systems may not be fully accessible. This can limit the ability of people with disabilities to take part in oversight and decision-making. As a result, fewer accessibility concerns may be identified and addressed.

Chatbots used for public services may provide incorrect information about eligibility or processes while appearing confident. People who rely on these tools may not know the information is wrong. This can lead to missed benefits or incorrect applications.

Systems that track service use may collect detailed information about behavior and accommodations. Because some patterns are unique, this data may make it easier to identify individuals, even when intended to be anonymous. This can raise privacy concerns and affect how people are treated.

A government service may use AI tools to summarize or relay user feedback from people with disabilities, for example, comments about barriers in public transportation, healthcare access, or community services. The AI tool may reshape feedback into more general or neutral language. As a result, specific details about lived experience, such as how a barrier affects daily routines or safety, may be simplified or left out. Over time, the feedback received by the service may reflect more uniform patterns rather than the full range of needs, making it harder for decision-makers to understand and respond to real issues.

A government service may introduce a chatbot to help people apply for benefits or get information, and assume this improves access. However, while the chatbot can be helpful for some people with disabilities, it may not perform as well in more complex cases. As a result, people with disabilities who are more extreme outliers may receive incomplete or unclear answers. If access to human support is reduced because the organization believes they are addressing access needs through the chatbot, this can make it harder to get accurate guidance, even though the service appears to be more accessible.

Services. Accessible AI tools for public services may only be available in widely used languages or higher-cost systems. This can limit access in lower-resource settings and create dependence on specific vendors. If these tools change or are removed, users may lose access to important services.

Services. Public service programs may not account for newer forms of digital gig work when setting eligibility rules. AI gig work is often irregular, with income changing from week to week. Benefits systems may interpret short-term increases in earnings as a sign that the person is no longer eligible for funded supports. This can lead to sudden loss or reduction of services, even when the income is not reliable enough to replace those supports. This can create gaps where people earning income through AI-related tasks do not clearly fit into existing categories. As a result, they may face uncertainty about eligibility or fall through the cracks entirely.

When a benefits system incorrectly denies or delays applications from people with disabilities at slightly higher rates, it may not trigger action if the difference is seen as too small within the data sets. Being treated as anecdotal isolated cases can allow the same issue to continue over time, creating continual barriers to access without leading to system-wide fixes.