Synthetic Data Distribution & Bias Mapper

Cross-reference generative data outputs against baseline curves to isolate synthetic drift and data decay.

Dataset Distribution Builder

Demographic / Bucket Original % Synthetic %

Statistical Bias Dashboard

KL Divergence
0.000
Bits / Nats
JS Divergence
0.000
Bounded (0 to 1)
PSI (Drift)
0.000
No Shift
Chi-Square (χ²)
0.00
p-value

Mitigation & Analysis Table

Category PSI Contrib. Mitigation Action

The Danger of Synthetic Data Bias

Generative AI models (GANs, VAEs, Diffusion Models) are increasingly used to generate synthetic datasets for privacy compliance and data augmentation. However, these models suffer from mode collapse and amplification bias.

If an original dataset has a slight imbalance (e.g., 60% Group A, 40% Group B), the generative model will often exaggerate this (e.g., outputting 80% Group A, 20% Group B). This is called data drift. Training downstream AI models on this biased synthetic data leads to catastrophic fairness failures.

Kullback-Leibler (KL) Divergence

KL Divergence (relative entropy) is a measure of how one probability distribution (the synthetic data, Q) differs from a second, reference probability distribution (the original data, P).

D_KL(P || Q) = Σ P(x) * log( P(x) / Q(x) )

A KL divergence of 0 indicates the distributions are perfectly identical. Note: KL Divergence is asymmetric.

Jensen-Shannon (JS) Divergence

JS Divergence is a smoothed, symmetric version of KL divergence. It measures the KL divergence of both P and Q against their mathematical average (M = (P+Q)/2). JS Divergence is elegantly bounded between 0 (identical) and 1 (completely disjoint).

Population Stability Index (PSI)

In financial services, the Population Stability Index (PSI) is the gold standard for measuring data drift. It is a symmetric metric calculated as:

PSI = Σ (Original% - Synthetic%) * ln( Original% / Synthetic% )
  • PSI < 0.1: Green Zone. No significant drift. Safe to use.
  • 0.1 ≤ PSI < 0.25: Yellow Zone. Moderate drift.
  • PSI ≥ 0.25: Red Zone. Significant drift. DO NOT USE without mitigation.

Chi-Square & Statistical Significance

While PSI looks purely at distribution percentages, the Chi-Square Goodness-of-Fit Test factors in your sample size (N). A 1% drift in a dataset of 100 rows might be random noise, but a 1% drift in 1,000,000 rows is highly statistically significant (p < 0.05). This tool calculates the chi-square statistic against the theoretical expected counts to alert you if the drift is mathematically proven rather than anecdotal.

Mitigating Data Drift (Rejection Sampling)

If your synthetic data falls into the PSI Red Zone, you must mitigate the generative bias before proceeding. Use the Auto-Mitigate Bias button in this tool to simulate Rejection Sampling.

Rejection sampling works by finding the most under-represented demographic bucket in the synthetic data. It sets that bucket as the baseline, and mathematically computes the exact percentage of rows that must be randomly deleted (rejected) from every other over-represented bucket to perfectly realign the PSI back to 0.

Frequently Asked Questions

Algorithmic Basics

How does the Synthetic Data Distribution & Bias Mapper work?
The mapper analyzes generated synthetic datasets against baseline real-world data to identify distributional shifts, over-representation, or under-representation of specific demographics or features.

Model Fairness

Why is mapping bias in synthetic data crucial?
Synthetic data often amplifies the subtle biases present in the training models. Mapping these biases ensures that the generated data remains fair, representative, and safe for downstream machine learning tasks.

Data Visualization

What types of visualizations does the mapper provide?
It provides comprehensive statistical charts, including heatmaps, feature distribution overlays, and multidimensional scaling plots to easily spot discrepancies between real and synthetic data.

Rate Synthetic Data Distribution & Bias Mapper

Help us improve by rating this tool.

4.9/5
1,029 reviews