Insight · November 10, 2026
Data that
never happened.
Synthetic data is generated to stand in for the real thing. It is a real tool with a real limit, and the two are easy to confuse.
01 · The short answer
What synthetic data is.
Synthetic data is artificially generated data that imitates the statistical shape of real data without containing any real records. A model or a simulator learns how a real dataset behaves, then produces new examples that keep its patterns. Teams use it to train and test systems when real data is scarce, private, or missing the rare case they most need to cover.
IBM frames it plainly: a stand in for real data, used mainly to train machine learning models, and a way to work with the shape of sensitive data without exposing the people inside it.
02 · Why it exists
The real data is often the thing you cannot get.
Most useful data is hard to use. It is locked behind privacy rules, it is expensive to label, or the events you care about are rare by definition. A fraud team has millions of clean transactions and almost no fraudulent ones. A medical model has plenty of the common case and too little of the one that kills. You cannot train well on a case you barely have.
Synthetic data answers three of those gaps at once. It fills scarcity by generating more of the rare case. It handles privacy by learning the pattern of sensitive records rather than copying them, so the output can be shared where the originals never could. And it balances a skewed dataset by manufacturing the minority class until a model has enough of it to learn. IBM lists the same four jobs: scarcity, imbalance, privacy, and bias.
The one line to keep
“Synthetic data is only ever as honest as the real data it learned from.”
03 · How it gets made
Five ways to generate data that never happened.
01
Generative models
A GAN pits a generator that invents records against a discriminator that judges them, and the two train against each other until the output is hard to tell from the real thing. Variational autoencoders learn the same distribution a different way.
02
Simulation
A rules based or physical model produces the data outright. A driving simulator renders road scenes and sensor readings that never happened, so a system can meet the rare case before it meets it on the road.
03
Statistical sampling
Fit a distribution to a real dataset, then draw new rows from the fit. The synthetic table keeps the shape and the correlations of the source without carrying any of its actual records.
04
Oversampling the minority
When one class is rare, a method such as SMOTE builds new examples of it between the ones you have, so a fraud or defect model has enough of the case it is meant to catch.
05
Large model generation
A capable model writes the training material for a smaller one, in the shape a task needs. This is how a small model can be taught to reason without scraping ever more of the open web.
04 · Not a free copy
It inherits the source, and that cuts both ways.
Synthetic data is not neutral. It learns whatever was in the real data, including the bias. If the records you train the generator on under represent a group, the synthetic set will under represent it too, and now the skew looks like fresh, abundant data. Generating more of a flawed pattern does not fix the flaw. It launders it.
Privacy has a matching trap. A synthetic record that looks like a real person is not automatically a leak, and a record that looks invented is not automatically safe. The protection comes from the method, not the resemblance. The standard guarantee, differential privacy, binds the generating mechanism so that no single real record can swing the output much, which is what keeps a real person from being reconstructed from the synthetic set. Without a guarantee like that, a generator can memorize and quietly hand back the very data it was meant to protect.
05 · One example that shows it working
A small model taught from textbooks that were written for it.
Microsoft trained its Phi family on a deliberate mix: heavily filtered public web data plus synthetic data written to read like a textbook, generated to teach subjects such as reasoning, math, and coding. The Phi-3 technical report frames the goal as a data optimal regime, choosing what the model is fed rather than only how large it is, and reports a small model performing well above what its size alone would predict.
The lesson is not that synthetic data replaced the web. It sat beside it. Filtered real data built the general knowledge, and the generated textbooks sharpened the reasoning. That is the pattern that holds up: synthetic data earns its place when it fills a gap the real data leaves open, not when it stands in for the real data entirely.
dataset: real: filtered public web data # the ground truth synthetic: textbook style examples # fills the reasoning gap rule: accumulate, never replace # keep real data in the mix check: provenance: record how each synthetic set was generated evaluation: score the model on real, held out data only
A hybrid recipe · real data is the floor, synthetic fills the gap
06 · How to use it well
Accumulate, do not replace. Then write down what you did.
The single most useful result in this area is also the most practical. A 2024 study asked whether model collapse is inevitable and found that it is not, on one condition: keep the real data in the mix. When each new round of synthetic data accumulates alongside the original real data, the error stays bounded. When synthetic data replaces the real data round after round, the error grows. The authors saw the same split hold across model sizes and architectures.
So the operator rules are short. Generate to fill a specific gap, a rare class, a privacy wall, a scenario you cannot collect, and keep real data as the floor the model is still anchored to. Evaluate on real, held out data, never on more synthetic data, because a generator grading its own work tells you nothing. Document how every synthetic set was made and what it was checked against, so the provenance travels with the data. This is the same discipline that makes fine tuning and the data under retrieval trustworthy: know where every example came from.
07 · The catch
A model fed its own output forgets the edges of the world.
In 2024, researchers published a result in Nature with a blunt name: model collapse. Train a model on data produced by earlier models, generation after generation, and the model degrades in a way that cannot be undone. The rare parts of the distribution, the tails, go first. Then the variance narrows, until the output converges on something that barely resembles the world it started from. In one run, text that began on medieval architecture had drifted into a list of jackrabbits by the ninth generation.
The mechanism is simple and unforgiving. A model slightly overweights what is common and underweights what is rare. Sample from it to make the next dataset, train on that, and the small error compounds every round. The rare case, the exact thing synthetic data is often reached for, is the first thing to vanish. The authors draw the obvious conclusion: genuine human generated data becomes more valuable, not less, as model output fills the web.
None of this makes synthetic data a mistake. It makes it a tool with a known failure mode. Used to fill a gap, anchored to real data, and checked against the real world, it does real work. Used as a cheaper replacement for the data underneath it, it quietly eats the thing it was supposed to protect.
Closing
Fill the gap. Keep the ground.
Find the one case your real data is too thin to cover. Generate only that, keep every real record you have, and grade the result on data the generator never saw. Synthetic data is a way to reach the case you are missing. It is not a way to stop needing the real one.
Written by · November 10, 2026
Donny Smith · ECD, Founder, Bttr.
Over the past 15 years, he has led creative teams and contributed to products used by millions of people worldwide, working with companies including Apple, Opendoor, JP Morgan, GE Aerospace, Pepsi, and Alterra Mountain Company.
LinkedInFacts current as of November 2026 · Nature · AI Models Collapse When Trained on Recursively Generated Data · July 2024 · arXiv · Is Model Collapse Inevitable? Breaking the Curse of Recursion by Accumulating Real and Synthetic Data · April 2024 · arXiv · Phi-3 Technical Report, Microsoft · April 2024 · IBM · What Is Synthetic Data? · 2026
Share this perspective
More insights
Adjacent perspectives.
Bttr. Field Brief
The brief Bttr. writes for senior buyers.
Monthly. One signal worth your time on Brand Operating Systems, AI search visibility, and the infrastructure buildout. No filler.