ZeonTools
What is Algorithmic Bias?
As Artificial Intelligence and Machine Learning models increasingly make automated decisions regarding hiring, lending, housing, and healthcare, the risk of Algorithmic Bias has become a massive legal liability. If a training dataset contains historical human prejudice, the AI will mathematically encode and scale that discrimination.
A Compliance Auditor mathematically measures the outcomes of an AI model across different demographic groups to ensure that no specific protected class (e.g., race, gender, age) is suffering from disparate impact compared to the baseline privileged group. Checking for intersectional bias (e.g. Hispanic Women) rather than just single dimensions is critical for thorough compliance.
Disparate Impact Ratio (DIR) Formula
The Disparate Impact Ratio (DIR) is the industry-standard mathematical metric used by data scientists to quantify fairness. It compares the probability of a favorable outcome (like getting a loan) for an unprivileged group against the privileged group.
÷
( Favorable Outcomes Privileged / Total Privileged )
A DIR of 1.0 represents perfect demographic parity. A DIR of 0.50 means the unprivileged group is receiving favorable outcomes at only half the rate of the privileged group, which is highly illegal in most jurisdictions.
Statistical Parity Difference (SPD)
While DIR calculates a ratio, Statistical Parity Difference (SPD) calculates the absolute difference in outcome probabilities.
For example, if the privileged group has an 80% success rate, and the unprivileged group has a 60% success rate, the SPD is -0.20. An SPD of 0.0 indicates perfect fairness. Values below -0.10 are generally considered significant red flags requiring dataset re-weighing.
The EEOC Four-Fifths (4/5ths) Rule
In the United States, the Equal Employment Opportunity Commission (EEOC) uses the Four-Fifths Rule as an administrative rule of thumb to determine adverse impact in hiring.
Under this rule, a selection rate for any race, sex, or ethnic group which is less than four-fifths (4/5 or 80%) of the rate for the group with the highest rate will generally be regarded as evidence of adverse impact. Mathematically, this means your DIR must be > 0.80. If your AI resume screener has a DIR of 0.75 for female applicants compared to male applicants, you are liable for a discrimination lawsuit.
EU AI Act & NYC Local Law 144
Modern AI legislation is aggressively tightening the definitions of algorithmic fairness:
- New York City Local Law 144: Requires employers using Automated Employment Decision Tools (AEDTs) to undergo an independent bias audit annually. The industry standard for passing this audit is maintaining a DIR > 0.90.
- EU AI Act: Classifies AI used in employment, education, and credit scoring as "High-Risk." These systems require strict fundamental rights impact assessments, pushing the compliance threshold closer to Strict Parity (DIR > 0.95).
How to Mitigate Dataset Bias (Re-weighing)
If this calculator flags your algorithm as non-compliant, you can use the built-in Mitigation Simulator to generate a mathematical fix. Data scientists generally deploy the following mitigation techniques:
- Pre-processing (Re-weighing): The Mitigation Simulator outputs a Re-weighing Multiplier. By assigning this multiplier to training examples of the unprivileged group, you can artificially balance the dataset before the model even learns. This is easily done using libraries like IBM's AI Fairness 360 (AIF360).
- In-processing (Adversarial Debiasing): Training two neural networks—one to predict the outcome, and an adversary to predict the protected attribute. The model is forced to learn representations that fool the adversary.
- Post-processing (Threshold Optimization): Adjusting the decision thresholds (e.g., probability cutoffs) differently for different groups to achieve statistical parity on the final output. The Mitigation Simulator calculates exactly how many more positive classifications are needed to shift the threshold correctly.