Thoughts

·

5 min read

Synthetic customers in product development: where they help, where they mislead

Synthetic customers are used as respondents, and as respondents they are weak. Used for what-if questions across the customer lifecycle, anchored in your own data, they show a team where to look.

Winding river through green marshland, seen from above

In short

Synthetic customers make a poor jury and a good what-if machine. Anchor them in your own customers, move them along dimensions you choose, and use the runs to decide what to ask real people.

01 The pull

Cheap, instant, always available and always polite.

02 The flaw

As a jury they care about everything and drift to the average.

03 The use

Sliders on the journey: where personas overlap, build. Where they split, ask.

Synthetic customers are everywhere this year, and most teams use them the wrong way round. They ask them for a verdict, which is where they are weakest. Used for what-if questions across the whole customer lifecycle, and anchored in your own data, they are fast and useful. The decision this changes is where the research budget goes: real customers get the three questions that matter, not thirty that might.

Two camps, one habit

The large consultancies are in. McKinsey describes customer digital twins that predict a customer’s next purchase and the risk of losing them. Bain layers synthetic customers on top of real feedback to run experiments that used to be out of reach. BCG presents synthetic panels as a faster and cheaper route to insight, and says plainly that they cannot replace research on a radically new product.

Design research pushes back. IDEO published The case against AI-generated users. Nielsen Norman Group ran the same studies with synthetic and real participants and found that real people care about some things more than others, while synthetic users seem to care about everything.

Both camps share one habit. They treat synthetic customers as respondents: you ask, they answer. The argument is about how good the answers are.

Tempting, and a poor jury

They answer at two in the morning. They never cancel, need no incentive and never say a concept is boring. For a team under pressure that is hard to resist, and in many companies it is the only customer contact that fits into a sprint.

As a jury they have habits that do not help. They rate too many things as important. They are too agreeable. And they pull towards the middle: in a study in Science Advances, stories written with AI ideas were rated more creative and were also more similar to each other. The university’s summary of the study puts that at 10.7% for writers who used one AI idea. Each answer gets better and the set gets narrower. A jury like that approves too much and notices too little.

Exhibit 1

Two of four flaws matter less once you compare runs instead of asking for a verdict.

Four documented habits of synthetic customers, what each does to a verdict, and whether it still hurts when you compare two runs of the same model.

What it means. A bias that sits in both runs mostly cancels when you look at the difference. Sameness and the pull of the past do not cancel. They need anchors from real customers and a spread you set on purpose.

Sources and method

Sources: Nielsen Norman Group (2024) on priorities and positivity; Doshi and Hauser (2024, Science Advances) on similarity; BCG (2026) on radically new products. The right column is my reasoning, not a measurement.

Ask what if, not what next

Give the same model a different job. Stop asking what customers think. Ask what changes when something on their journey changes.

Personas were meant to be living tools long before AI. Pruitt and Adlin’s The Persona Lifecycle treated them in 2006 as something a team plans, raises and retires. Synthetic customers make that literal and stretch it across the customer lifecycle: order, set-up, first use, daily use, the first problem, renewal or goodbye.

Then come the sliders. Take a connected product for the home and run four synthetic personas through the same journey: choosing it, installing it, connecting it, first use, daily use and the first problem. Each gets a comfort curve, the way a journey map records how people feel at each step. Then set one slider, say the product runs in automatic mode by default, and let them walk through it.

Lay the curves on top of each other and the steps sort themselves. Where all four sit in their comfort zone, the team has shared ground to build on. Where they split, the run has found a question for real people. In the invented run below, automatic mode suits everyone once the product is running. Installation and the first problem divide them, and the tenant who needs a landlord’s permission is unsure before the product is even switched on.

Exhibit 2

Where the personas overlap, build. Where they split, ask.

Four synthetic personas run through the same journey, with the product in automatic mode by default. Each line shows how comfortable a persona is at each step. The lower row counts how many of the four sit inside the comfort zone.

Run: automatic mode on by default Illustrative

Hands-off owner wants it to just workTinkerer wants controlTenant needs the landlord’s yesOccasional user uses it now and then
Shaded comfort zoneDot inside itRing outside itYellow all four inside
COMFORT AT EACH STEPHappyFineUnsureStuckChooseInstallConnectFirst useDaily useFirst problemWHERE THEY OVERLAP3 of 42 of 43 of 44 of 44 of 41 of 4AskBuildAskCOMFORT AT EACH STEPHappyFineUnsureStuckChooseInstallConnectFirst useDaily useFirst problemWHERE THEY OVERLAP3 of 42 of 43 of 44 of 44 of 41 of 4AskBuildAskCOMFORT AT EACH STEPHappyFineUnsureStuckInstallFirstProblemWHERE THEY OVERLAP3/42/43/44/44/41/4AskBuildAsk

What it means. First use and daily use are shared ground: all four are comfortable, so the team can build there. Installation and the first problem split them. That is also where synthetic personas are least reliable, because they spread less than real people do. Those two steps go to real customers.

Sources and method

Invented values to show the method. Not measured data and not a client. On the narrower spread of synthetic samples: Bisbee and others, Political Analysis (2024).

The shared ground also looks larger in a run than it is. In a study in Political Analysis, ChatGPT matched the averages of a large American election survey, yet its answers varied less than those of the real respondents. Put the same steps in front of a few real people and the overlap shrinks.

Exhibit 3

Real people spread wider, so the shared ground is smaller than the run suggests.

The same six steps. At each step the left pill shows where the middle 8 in 10 synthetic variants land, the right pill the middle 8 in 10 of a small real sample. The yellow column marks the shared ground from Exhibit 2.

Same run, checked against real people Illustrative

Synthetic run middle 8 in 10 variantsReal people middle 8 in 10 of a small sampleShaded comfort zoneYellow shared ground from Exhibit 2
COMFORT RANGE AT EACH STEPHappyFineUnsureStuckReal people reach belowChooseInstallConnectFirst useDaily useFirst problemShared ground in Exhibit 2COMFORT RANGE AT EACH STEPHappyFineUnsureStuckChooseInstallConnectFirst useDaily useFirst problemShared ground in Exhibit 2COMFORT RANGE AT EACH STEPHappyFineUnsureStuckInstallFirstProblemShared ground

What it means. Treat the overlap as a shortlist, not a decision. In the run, first use and daily use sit inside the comfort zone. Real people reach below it. Put five of them through those two steps before you build.

Sources and method

Invented values to show the method. Not measured data and not a client. That synthetic samples vary less than real ones: Bisbee and others, Political Analysis (2024); Li, Li and Qiu (2025, preprint).

Nobody should decide on a run like this. But the team walks into the next workshop knowing which interview to book.

When the pressure is on cost

The same set-up answers a harder question. The budget is cut and something has to go: where does it hurt least? Remove the onboarding call, shorten the support hours or drop a feature, and watch which groups fall off at which step. The run does not tell you the saving. It gives you a ranked list of where to look before you cut.

This is hypothesis management more than research. Every assumption about a persona, a feature or a step lives in one model, and a new one can be plugged in and run against all the others. In my experience, in many firms those assumptions sit in PDFs and in spreadsheets with many tabs, which is why testing a single one takes a week.

What each company sets for itself

The tools are generic. The set-up is yours, and it comes down to six decisions.

Exhibit 4

Six decisions turn a generic tool into your own what-if machine.

The set-up page of an invented team. The numbers are starting values to change, not norms.

What it means. The model is the same for everyone. The anchors, the axes and the record of what the runs got right are yours, and a competitor cannot buy them.

Concept page with my own starting values. Not measured data.

One dial is not enough for the spread. A 2026 study of persona simulations shows that a synthetic population can look widely spread and still be flat, because all its variety runs along a single axis. Spread people along several independent dimensions, and check the spread against a small real sample.

Limits

Synthetic agreement is not demand: a persona that likes a feature has not paid for it. The anchor contains real people and deserves the care of any customer data. And a model nobody maintains drifts away from its customers, the way a design system does. That is the subject of Design system drift.

Real customers decide what is true. Synthetic ones help decide what is worth asking them.

What would prove me wrong

If what-if runs lead teams to the same questions they would have asked anyway, the sliders are decoration. A fair test: before each run, write down the three questions the team would take to customers. Compare them with the three it takes afterwards. If the lists match for a year, stop.

Next

Set up like this, a team produces a large number of well-formed variants. The next problem is choosing among them without swiping through hundreds. That is a later study.

Sources (12)
Consultancies
McKinsey on customer digital twins, 2024Bain on synthetic customers, 2025BCG on synthetic panels, 2026
Design research
IDEO, The case against AI-generated users, 2025Nielsen Norman Group, Synthetic Users, 2024
Studies
Doshi and Hauser, Science Advances, 2024University of Exeter on the studyBisbee and others, Political Analysis, 2024Li, Li and Qiu, homogenisation in silicon samples, preprint 2025The Chameleon’s Limit, persona collapse, preprint 2026LLM-based social simulations require a boundary
Method
Pruitt and Adlin, The Persona Lifecycle, 2006
Aerial view of marshland with winding waterways
Aerial view of a river delta with brown marshland

Create a free website with Framer, the website builder loved by startups, designers and agencies.