What is differential privacy?
Differential privacy is a mathematical standard for publishing statistics or training models on a dataset while limiting what anyone can learn about a single person in it. It adds calibrated random noise, so results look almost the same whether or not any one person’s record is included. The strength of that promise is set by a parameter called epsilon.
For example, a hospital network wants to publish how many patients in each ZIP code were treated for diabetes last year. One small ZIP code has only two patients, so an exact count, combined with a local news story, could expose them. With differential privacy, the network adds a small amount of random noise to every count before release. Totals across a region stay useful, but no single published number confirms whether a specific person is in the data.
How differential privacy works
- Sensitivity. The analyst works out how much one person’s record could change a result. A count changes by at most one.
- Noise. The system adds random noise scaled to that sensitivity, often drawn from a Laplace or Gaussian distribution.
- Privacy budget. Epsilon sets the noise level. A smaller epsilon means more noise and stronger privacy. Every query spends part of the budget, so repeated questions about the same data add up.
- Central or local. In central differential privacy, a trusted party holds the raw data and adds noise to outputs. In local differential privacy, each device adds noise before data leaves it.
The US Census Bureau used differential privacy to protect the 2020 Census redistricting data, one of the largest public uses of the method so far.
Differential privacy vs masking and k-anonymity
| Method | What it protects | Best fit |
|---|---|---|
| Differential privacy | Aggregate outputs, such as counts or model weights | Published statistics, analytics, model training on population trends |
| K-anonymity | A released table, by making each record match at least k others on quasi-identifiers | Sharing structured datasets for research |
| Masking and tokenization | Individual values in each record | Apps and AI systems that work on specific people’s records |
Why differential privacy matters for AI systems
Large models can memorize rare strings from training data, such as a phone number that appears once. Training with differentially private methods, such as DP-SGD, limits how much any one record shapes the model, at some cost to accuracy. Differential privacy does not help at inference time, though. When a support rep asks an AI assistant about one customer’s claim, or an AI agent pulls a record through a tool call, the task needs that specific record. Protection there has to work at the record level.
How Protecto helps
Protecto works at the record level, so it complements differential privacy. Protecto detects personal data in training sets, prompts and retrieved documents and replaces it with consistent tokens, so models learn patterns and complete tasks without seeing real identities. See AI training data de-identification.
Frequently asked questions
What is epsilon in differential privacy?
Epsilon is the privacy loss parameter. A smaller epsilon adds more noise and gives stronger privacy, while a larger epsilon gives more accurate results with weaker privacy.
Is differentially private data anonymous?
Differential privacy describes how results are produced, not a label for a dataset. Whether a release counts as anonymous depends on the epsilon, the release and the law that applies, so teams should still assess re-identification risk.
Does differential privacy reduce accuracy?
Yes, some. The added noise lowers precision, especially for small groups and rare events. Large datasets and aggregate questions lose the least accuracy.
Related terms
K-anonymity · Synthetic data · Data anonymization · Quasi-identifier · Data masking