Tokenization vs masking vs classification: why the order matters in AI pipelines

Tokenization vs masking is the wrong first question. Classification decides which transformation is correct, and each layer inherits the one before it.
TL;DR: key takeaways
  • Masking and tokenization are transformations. Classification decides which one is correct.
  • Tokenization suits values a downstream system must recover. Masking suits values nothing needs back.
  • A generic ACCOUNT_NUMBER label left a payroll system unable to tell a tax identifier from a salary account.
  • Separate spans let you mask a street line while keeping the city and postcode. A merged span is all-or-nothing.
  • Each layer inherits the weakness of the one before it.

Quick answer: Masking and tokenization are transformations. Classification is the step that decides which transformation is correct. Masking replaces a value with a substitute that keeps enough shape for downstream systems to work. Tokenization replaces it with a reference that maps back under controlled conditions. Neither can be applied correctly until the pipeline knows the data type.

Most comparisons of tokenization vs masking treat the two as a choice you make once, write into a design doc, and implement. That framing skips the step that determines the answer. Classification sits upstream of both. A transformation is something you apply to a value, and you cannot select it correctly until you know what the value is.

We ran an internal comparison of OpenAI PrivacyFilter and Protecto in July 2026 that made this concrete. Across 344 expected annotations, PrivacyFilter detected roughly 68% under some label, but only 37% arrived with a data type precise enough for a policy engine to act on. Protecto correctly classified 90% of the same 344 annotations under the same documented scoring rules. Detection was not the gap. Classification was. Scope limits and our own errors are below.

What tokenization vs masking actually compares

The three terms get used interchangeably in vendor material. They differ in reversibility, in whether they preserve referential integrity across records, and in what a downstream system can still do with the output.

Classification assigns a specific data type to a span: US Social Security number, India PAN, driving license number, medical procedure. Nothing about the data changes. The output is a label, and that label is what a policy engine selects on. Retention windows, redaction strictness, regional handling, and access rules are written against data types.

Masking replaces a sensitive value with a substitute. Format-preserving masking keeps the substitute in the same shape as the original, so downstream validation does not break. It may be reversible or irreversible depending on implementation. In our run, replacements preserved shape on structured identifiers and were weaker on dates and IP addresses, where separators were retained but character classes and field widths were not.

Tokenization replaces a value with a reference token that carries no information about the original. The mapping lives in a separate, secured store, and reversing it requires a controlled lookup. That fits values a downstream system genuinely needs back, such as a card number a billing service will later charge.

Classification Masking Tokenization
What it produces A specific data type label A substitute value A reference token plus a governed mapping
Reversible Not applicable, data unchanged Depends on implementation Yes, under controlled lookup
Referential integrity across records Not applicable Only when substitution is deterministic Yes, same input maps to same token
What downstream systems can still do Select the correct policy rule Read a value with the original shape Re-identify under policy, otherwise treat as opaque
Common failure mode Label too broad to act on Format destroyed, or value needed reversing Token store is not governed

Read that table left to right. The first column is the input to the other two. If the label is wrong or too broad, the rest of the row is decided on bad information.

Why classification sits upstream of the tokenization vs masking decision

The correct treatment depends on the type. A payment card usually needs tokenization, because a payment system may need the real value under strict controls. A customer name in a support ticket may only need format-preserving masking, because nothing downstream reverses it. Clinical language needs something else again. There is no discrete identifier to tokenize, only a sentence carrying protected meaning, and recognizing it requires semantic classification rather than format matching.

This is where AI data protection builds go wrong. A team implements a transformation layer without a classification layer granular enough to route different types to different treatments. The output looks protected. Every sensitive-looking value has been replaced with something. But a card masked instead of tokenized, or a license number and a bank account routed through one generic rule, means the protection does not match the data underneath it.

What breaks when classification is wrong: a payroll example

In this run, PrivacyFilter returned 111 raw ACCOUNT_NUMBER labels. Thirteen were actual bank account numbers. Ninety-two were identifiers Protecto classified across approximately 22 other specific types, including 17 Social Security numbers, 9 India PAN values, 8 credit cards, and 7 passport numbers. Six had no corresponding Protecto value. Not all 111 of those labels are incorrect, and the 111 figure is a raw label count that is not comparable with the 344-annotation scorecard. Never combine the two.

Here is why the generic label matters, using a synthetic prompt from the report:

New employee Sarah Johnson’s PAN is AAAPS4321Q and her bank account for salary credit is 100234567890.

Tax identifier Salary account
Protecto PAN BANK_ACC_NO
PrivacyFilter ACCOUNT_NUMBER ACCOUNT_NUMBER

Both values were detected. Both were labeled. A payroll system consuming that output still cannot determine which value is the destination account. The tokenization vs masking question is now unanswerable for those two spans, because the correct answer differs between them and the pipeline cannot tell them apart. Tokenize both and a tax identifier lands in a payments token store. Mask both under one rule and the payroll run loses the account it needs.

The weakness is not uniform. Within the 72 government and national ID annotations, PrivacyFilter correctly classified 0%, generalized 75%, misclassified 7%, and did not detect 18%. Within the 40 healthcare and PHI annotations, it correctly classified 15% and did not detect 72%. Two different problems in two different families.

A specific wrong label can be harder to manage than a broad one. Fourteen expected annotations were detected and given a type that contradicts the value: three IP addresses returned as PRIVATE_URL, three US Social Security numbers as PRIVATE_PHONE, two card CVVs as PRIVATE_ADDRESS. A broad label signals uncertainty and can be routed for review. A confident wrong label gets acted on.

Tokenization Vs Masking Vs Classification

The transformation is bounded by the span you were given

Classification decides which transformation is correct. Detection decides what you are allowed to transform at all.

My driving license number is DL-0420110149646, renewed last year, and my current address is 45 Baker Street, London, NW1 6XE.

Protecto PrivacyFilter
DL-0420110149646 to DRIVING_LICENSE_NUMBER DL-0420110149646 to ACCOUNT_NUMBER
45 Baker Street to ADDRESS 45 Baker Street, London, NW1 6XE to PRIVATE_ADDRESS
London to CITY
NW1 6XE to PINCODE

A merged span is not unsafe in itself. It reduces the policy granularity available to you and can remove context the application still needs. With separate spans you can mask the street line and the license number while keeping the city and postcode for a residency check, regional entitlement, or fraud analysis. With one merged span, masking and tokenization are both all-or-nothing across the whole address.

The four-layer framework

Layer What it produces What depends on it Evaluated in our comparison
1. Detection A span of sensitive text Whether the value is visible at all Evaluated
2. Classification A specific data type Which policy rule a system can select Evaluated
3. Transformation Protected text the application can consume Whether the pipeline runs without a custom substitution layer Evaluated
4. Enterprise control Governed re-identification, audit, tenancy Whether it can be deployed in a regulated environment Not evaluated

Layer 4 is a hard boundary here. API output tells you what a product detected, classified, and returned. It says nothing about architecture, tenancy, access controls, deployment model, or vault capability, for either product. Evaluate those against documentation and in a POC of your own.

Each layer inherits the weakness of the one before it. A strong transformation layer on a weak classification layer is still a weak pipeline, whatever the masking or tokenization algorithm does on its own.

Where Protecto fell short in the same run

Protecto did not score perfectly. One US Social Security number was classified as a Canadian SIN. One phone number written in Arabic-Indic numerals was not detected. One email address using Cyrillic homoglyphs was not detected. Protecto correctly classified 79% of the 43 edge and adversarial annotations and 67% of the 6 character-spaced values. These are specific observed cases rather than a complete breakdown of our results.

How we ran this comparison

This comparison was designed, run, and scored by Protecto. It was not independently validated. Both products were called through their public APIs under default configuration, on the same input set, in a single run on 29 July 2026. The 150-prompt set intentionally includes difficult and adversarial cases, so these rates describe this test set rather than universal production performance. False positives were outside the scope, so the results are not precision or F-score measurements. The full prompt set, expected annotations, scoring rules, and raw API responses are available on request from sales@protecto.ai.

FAQ

Is tokenization better than masking?

Neither is better in the abstract. Tokenization is correct when a downstream system must recover the original value under control. Masking is correct when nothing needs the original back and the record still has to be readable. The data type decides, which is why classification comes first.

Can we skip classification and mask everything that looks sensitive?

You can, and the output will look protected. What you lose is policy selection. Retention, regional handling, and access rules are written against data types. A generic label gives the pipeline nothing to select on, as the payroll example shows.

Does masking break AI output quality?

It can, when the replacement removes context the model needed. If a service prompt says the April invoice is wrong and the protection layer replaces “April” with a generic person tag, nothing leaked and the answer is still wrong. Context-preserving replacement exists to avoid that.

Read next

The full four-layer breakdown, the scorecard by data family, and every scope limit are in the pillar post, OpenAI PrivacyFilter vs Protecto: what 150 prompts and 344 annotations told us about classification. If you are evaluating tools rather than designing a pipeline, the sibling post PII redaction software: a buyer’s checklist for AI pipelines turns these findings into tests you can run in a POC.

To see how Protecto classifies and transforms your own prompt patterns, schedule a demo. For the underlying data, write to sales@protecto.ai. More at www.protecto.ai.

DeepSight PII and PHI Detection Datasheet
Free datasheet
DeepSight PII and PHI Detection Datasheet

How DeepSight identifies sensitive data by meaning, with results from Protecto’s internal test across five multilingual input sets.

Classify first, then transform

See how Protecto classifies and transforms your own prompt patterns, or request the prompt set and scoring rules.

About the author

Amar Kanagaraj

Founder and CEO of Protecto

Amar Kanagaraj is the Founder and CEO of Protecto, a company focused on securing enterprise data for LLMs, AI agents, and agentic workflows. He is a second-time entrepreneur with 20+ years of experience across engineering, product, AI, go-to-market, and business leadership. Before Protecto, Amar co-founded FileCloud and helped scale it to over $10M ARR as CMO. Earlier in his career, he worked at Sun Microsystems, Booz & Company, and Microsoft Search & AI. He holds an MBA from Carnegie Mellon University and an MS in Computer Science from Louisiana State University.

Share article

Table of Contents

DeepSight PII and PHI Detection Datasheet
Free datasheet
DeepSight PII and PHI Detection Datasheet

Turn these challenges into your next AI advantage.

Talk to a solutions engineer about securing your data privacy, governance, and compliance.

New: run any AI model without exposing real values.
See how it works