The Danger of Synthetic Data Bias
Generative AI models (GANs, VAEs, Diffusion Models) are increasingly used to generate synthetic datasets for privacy compliance and data augmentation. However, these models suffer from mode collapse and amplification bias.
If an original dataset has a slight imbalance (e.g., 60% Group A, 40% Group B), the generative model will often exaggerate this (e.g., outputting 80% Group A, 20% Group B). This is called data drift. Training downstream AI models on this biased synthetic data leads to catastrophic fairness failures.
Kullback-Leibler (KL) Divergence
KL Divergence (relative entropy) is a measure of how one probability distribution (the synthetic data, Q) differs from a second, reference probability distribution (the original data, P).
A KL divergence of 0 indicates the distributions are perfectly identical. Note: KL Divergence is asymmetric.
Jensen-Shannon (JS) Divergence
JS Divergence is a smoothed, symmetric version of KL divergence. It measures the KL divergence of both P and Q against their mathematical average (M = (P+Q)/2). JS Divergence is elegantly bounded between 0 (identical) and 1 (completely disjoint).
Population Stability Index (PSI)
In financial services, the Population Stability Index (PSI) is the gold standard for measuring data drift. It is a symmetric metric calculated as:
- PSI < 0.1: Green Zone. No significant drift. Safe to use.
- 0.1 ≤ PSI < 0.25: Yellow Zone. Moderate drift.
- PSI ≥ 0.25: Red Zone. Significant drift. DO NOT USE without mitigation.
Chi-Square & Statistical Significance
While PSI looks purely at distribution percentages, the Chi-Square Goodness-of-Fit Test factors in your sample size (N). A 1% drift in a dataset of 100 rows might be random noise, but a 1% drift in 1,000,000 rows is highly statistically significant (p < 0.05). This tool calculates the chi-square statistic against the theoretical expected counts to alert you if the drift is mathematically proven rather than anecdotal.
Mitigating Data Drift (Rejection Sampling)
If your synthetic data falls into the PSI Red Zone, you must mitigate the generative bias before proceeding. Use the Auto-Mitigate Bias button in this tool to simulate Rejection Sampling.
Rejection sampling works by finding the most under-represented demographic bucket in the synthetic data. It sets that bucket as the baseline, and mathematically computes the exact percentage of rows that must be randomly deleted (rejected) from every other over-represented bucket to perfectly realign the PSI back to 0.