Quick answer: Masking and tokenization are transformations. Classification is the step that decides which transformation is correct. Masking replaces a value with a substitute that keeps enough shape for downstream systems to work. Tokenization replaces it with a reference that maps back under controlled conditions. Neither can be applied correctly until the pipeline knows the data type.
Most comparisons of tokenization vs masking treat the two as a choice you make once, write into a design doc, and implement. That framing skips the step that determines the answer. Classification sits upstream of both. A transformation is something you apply to a value, and you cannot select it correctly until you know what the value is.
We ran an internal comparison of OpenAI PrivacyFilter and Protecto in July 2026 that made this concrete. Across 344 expected annotations, PrivacyFilter detected roughly 68% under some label, but only 37% arrived with a data type precise enough for a policy engine to act on. Protecto correctly classified 90% of the same 344 annotations under the same documented scoring rules. Detection was not the gap. Classification was. Scope limits and our own errors are below.
What tokenization vs masking actually compares
The three terms get used interchangeably in vendor material. They differ in reversibility, in whether they preserve referential integrity across records, and in what a downstream system can still do with the output.
Classification assigns a specific data type to a span: US Social Security number, India PAN, driving license number, medical procedure. Nothing about the data changes. The output is a label, and that label is what a policy engine selects on. Retention windows, redaction strictness, regional handling, and access rules are written against data types.
Masking replaces a sensitive value with a substitute. Format-preserving masking keeps the substitute in the same shape as the original, so downstream validation does not break. It may be reversible or irreversible depending on implementation. In our run, replacements preserved shape on structured identifiers and were weaker on dates and IP addresses, where separators were retained but character classes and field widths were not.
Tokenization replaces a value with a reference token that carries no information about the original. The mapping lives in a separate, secured store, and reversing it requires a controlled lookup. That fits values a downstream system genuinely needs back, such as a card number a billing service will later charge.
| Classification | Masking | Tokenization | |
|---|---|---|---|
| What it produces | A specific data type label | A substitute value | A reference token plus a governed mapping |
| Reversible | Not applicable, data unchanged | Depends on implementation | Yes, under controlled lookup |
| Referential integrity across records | Not applicable | Only when substitution is deterministic | Yes, same input maps to same token |
| What downstream systems can still do | Select the correct policy rule | Read a value with the original shape | Re-identify under policy, otherwise treat as opaque |
| Common failure mode | Label too broad to act on | Format destroyed, or value needed reversing | Token store is not governed |
Read that table left to right. The first column is the input to the other two. If the label is wrong or too broad, the rest of the row is decided on bad information.
Why classification sits upstream of the tokenization vs masking decision
The correct treatment depends on the type. A payment card usually needs tokenization, because a payment system may need the real value under strict controls. A customer name in a support ticket may only need format-preserving masking, because nothing downstream reverses it. Clinical language needs something else again. There is no discrete identifier to tokenize, only a sentence carrying protected meaning, and recognizing it requires semantic classification rather than format matching.
This is where AI data protection builds go wrong. A team implements a transformation layer without a classification layer granular enough to route different types to different treatments. The output looks protected. Every sensitive-looking value has been replaced with something. But a card masked instead of tokenized, or a license number and a bank account routed through one generic rule, means the protection does not match the data underneath it.
What breaks when classification is wrong: a payroll example
In this run, PrivacyFilter returned 111 raw ACCOUNT_NUMBER labels. Thirteen were actual bank account numbers. Ninety-two were identifiers Protecto classified across approximately 22 other specific types, including 17 Social Security numbers, 9 India PAN values, 8 credit cards, and 7 passport numbers. Six had no corresponding Protecto value. Not all 111 of those labels are incorrect, and the 111 figure is a raw label count that is not comparable with the 344-annotation scorecard. Never combine the two.
Here is why the generic label matters, using a synthetic prompt from the report:
New employee Sarah Johnson’s PAN is AAAPS4321Q and her bank account for salary credit is 100234567890.
| Tax identifier | Salary account | |
|---|---|---|
| Protecto | PAN | BANK_ACC_NO |
| PrivacyFilter | ACCOUNT_NUMBER | ACCOUNT_NUMBER |
Both values were detected. Both were labeled. A payroll system consuming that output still cannot determine which value is the destination account. The tokenization vs masking question is now unanswerable for those two spans, because the correct answer differs between them and the pipeline cannot tell them apart. Tokenize both and a tax identifier lands in a payments token store. Mask both under one rule and the payroll run loses the account it needs.
The weakness is not uniform. Within the 72 government and national ID annotations, PrivacyFilter correctly classified 0%, generalized 75%, misclassified 7%, and did not detect 18%. Within the 40 healthcare and PHI annotations, it correctly classified 15% and did not detect 72%. Two different problems in two different families.
A specific wrong label can be harder to manage than a broad one. Fourteen expected annotations were detected and given a type that contradicts the value: three IP addresses returned as PRIVATE_URL, three US Social Security numbers as PRIVATE_PHONE, two card CVVs as PRIVATE_ADDRESS. A broad label signals uncertainty and can be routed for review. A confident wrong label gets acted on.

The transformation is bounded by the span you were given
Classification decides which transformation is correct. Detection decides what you are allowed to transform at all.
My driving license number is DL-0420110149646, renewed last year, and my current address is 45 Baker Street, London, NW1 6XE.
| Protecto | PrivacyFilter |
|---|---|
| DL-0420110149646 to DRIVING_LICENSE_NUMBER | DL-0420110149646 to ACCOUNT_NUMBER |
| 45 Baker Street to ADDRESS | 45 Baker Street, London, NW1 6XE to PRIVATE_ADDRESS |
| London to CITY | |
| NW1 6XE to PINCODE |
A merged span is not unsafe in itself. It reduces the policy granularity available to you and can remove context the application still needs. With separate spans you can mask the street line and the license number while keeping the city and postcode for a residency check, regional entitlement, or fraud analysis. With one merged span, masking and tokenization are both all-or-nothing across the whole address.
The four-layer framework
| Layer | What it produces | What depends on it | Evaluated in our comparison |
|---|---|---|---|
| 1. Detection | A span of sensitive text | Whether the value is visible at all | Evaluated |
| 2. Classification | A specific data type | Which policy rule a system can select | Evaluated |
| 3. Transformation | Protected text the application can consume | Whether the pipeline runs without a custom substitution layer | Evaluated |
| 4. Enterprise control | Governed re-identification, audit, tenancy | Whether it can be deployed in a regulated environment | Not evaluated |
Layer 4 is a hard boundary here. API output tells you what a product detected, classified, and returned. It says nothing about architecture, tenancy, access controls, deployment model, or vault capability, for either product. Evaluate those against documentation and in a POC of your own.
Each layer inherits the weakness of the one before it. A strong transformation layer on a weak classification layer is still a weak pipeline, whatever the masking or tokenization algorithm does on its own.
Where Protecto fell short in the same run
Protecto did not score perfectly. One US Social Security number was classified as a Canadian SIN. One phone number written in Arabic-Indic numerals was not detected. One email address using Cyrillic homoglyphs was not detected. Protecto correctly classified 79% of the 43 edge and adversarial annotations and 67% of the 6 character-spaced values. These are specific observed cases rather than a complete breakdown of our results.
How we ran this comparison
This comparison was designed, run, and scored by Protecto. It was not independently validated. Both products were called through their public APIs under default configuration, on the same input set, in a single run on 29 July 2026. The 150-prompt set intentionally includes difficult and adversarial cases, so these rates describe this test set rather than universal production performance. False positives were outside the scope, so the results are not precision or F-score measurements. The full prompt set, expected annotations, scoring rules, and raw API responses are available on request from sales@protecto.ai.
FAQ
Is tokenization better than masking?
Neither is better in the abstract. Tokenization is correct when a downstream system must recover the original value under control. Masking is correct when nothing needs the original back and the record still has to be readable. The data type decides, which is why classification comes first.
Can we skip classification and mask everything that looks sensitive?
You can, and the output will look protected. What you lose is policy selection. Retention, regional handling, and access rules are written against data types. A generic label gives the pipeline nothing to select on, as the payroll example shows.
Does masking break AI output quality?
It can, when the replacement removes context the model needed. If a service prompt says the April invoice is wrong and the protection layer replaces “April” with a generic person tag, nothing leaked and the answer is still wrong. Context-preserving replacement exists to avoid that.
Read next
The full four-layer breakdown, the scorecard by data family, and every scope limit are in the pillar post, OpenAI PrivacyFilter vs Protecto: what 150 prompts and 344 annotations told us about classification. If you are evaluating tools rather than designing a pipeline, the sibling post PII redaction software: a buyer’s checklist for AI pipelines turns these findings into tests you can run in a POC.
To see how Protecto classifies and transforms your own prompt patterns, schedule a demo. For the underlying data, write to sales@protecto.ai. More at www.protecto.ai.
