Real data vs synthetic data for AI agents
The short answer
Real data vs synthetic data for AI agents divides the labor. Synthetic data handles edge cases and privacy. Real data grounds the model and tests it. Labs buy app clones for about $300,000, but no public study shows real company data beats a clone.
Everyone in the public sources has a stake here. Troveo and micro1 gain when AI labs want real data. Clone vendors gain when they do not.
Where does synthetic data win?
Troveo's guide says synthetic data "earns its place in four situations":
- Edge cases. These are rare events, such as crashes a driving system rarely sees.
- Privacy-sensitive domains. Generated stand-ins avoid personal records.
- Labeling. Synthetic data arrives with labels attached.
- Volume balancing. It fills classes that have too few examples.
For agents, Epoch describes environments that are simulated websites or app clones. Each task has a prompt and an automated grader. A clone is synthetic by design.
Where does real data win?
The same Troveo guide says synthetic data is risky in bulk and names model collapse, where models trained on model output get worse. Troveo also says that testing on generated scenarios grades the generator and not the world. Its view is that real data does the grounding, and held-out real data does the testing.
micro1's CFO made the agent version of the point in an undated recorded forum talk. Built environments imitate the world. Real company data shows how a real problem got resolved (self-reported).
A second micro1 speaker said training so far has leaned mostly on synthetic data built with experts' help. These setups do not copy the real world, and a narrow ask yields a fixed artifact. The new data aims to capture "what is really going on behind the scenes in the workplace" (self-reported, about 17:30 to 18:00).
What does Epoch add?
Epoch's FAQ reports that exclusive deals for environments cost roughly four to five times non-exclusive ones. One lab researcher it quotes said many clones are buggy and not useful. It also says labs such as Cursor can use user data to build tasks. It does not say whether real company data makes better environments than clones.
What does this mean for a company with records?
Real company data is a bet. The bet is that agents learn more from a real chain of work than from a built one. The vendors say yes. No independent comparison is public. Read my page on the reasons labs want real data and the RL environment explainer next.
My free fit check and rights check start from what you hold.
micro1 may pay us a referral fee if a company we introduce goes ahead with it. How we are paid
Apply to micro1's data partnership (paid link)micro1's own data-partnerships pagePaid link. How we are paid
Vaultfish helps you run the process. We are not lawyers and this is not legal advice. If you need counsel who knows this area, we can point you to one. Ask for a referral
