What is AI Bias Testing?

Definition

Bias testing is the technical work that measures whether outcomes differ across demographic or other sensitive slices. It can support a formal audit or stand alone where statutes ask deployers to test and document discrimination risk.

See also our AI compliance glossary for short definitions of common terms.

Which regulations require this

• Colorado AI Act (SB 24-205)

  Requires developers and deployers of high-risk AI systems to use reasonable care to protect consumers from algorithmi...

  COAI-specificHighUpcoming
• Illinois Human Rights Act - AI Amendment (HB 3773)

  Amends Illinois Human Rights Act to prohibit employer use of AI that results in discrimination against protected clas...

  ILAI-specificHighIn effect
• EEOC Guidance on AI in Employment Selection

  EEOC guidance clarifying that Title VII and ADA apply to employer use of algorithmic decision-making and AI in hiring...

  FEDERALFederalMediumIn effect
• FTC Enforcement Policy on AI and Algorithmic Fairness

  FTC has signaled it will use existing Section 5 authority to pursue deceptive or unfair AI practices, including biase...

  FEDERALFederalMediumIn effect
• NIST AI Risk Management Framework (AI RMF 1.0)

  NIST AI RMF is a voluntary framework used as a practical benchmark by regulators and lawmakers. Following it can supp...

  FEDERALFederalLowIn effect

Which states reference this obligation

What you should do next

• Define the outcomes and populations you must compare under each statute.
• Gather statistically sound samples with counsel on data use.
• Run disparity tests with agreed thresholds for escalation.
• Document mitigations when results exceed your internal or regulatory triggers.
• Record who ran the test, when, and what changed afterward.