Last spring I talked with the CFO of a mid-market manufacturer that supplies the transportation industry. His big complaint was that too many inventory tools treat all inventory the same. In his business, some parts have to sit on the shelf for a long time because of contract commitments for warranty work. That little nuance in shelf life decides whether he trusts an inventory recommendation, whether it comes from a consultant or a piece of software.
I didn’t think much of it during the call. It was only later, when I went back to my notes and worked through them alongside a panel of synthetic CFOs, that it hit me. The synthetic CFOs had raised the exact same concerns: which inventory to hold, how long to hold it, and what shelf life does to cash flow.
I talked with three real CFOs, and every one of those conversations mirrored the synthetic ones. I was blown away by how congruent the two groups were.
Stick figures, again
Back in 2012 I wrote that provisional personas are stick figures. They’re founded in intuition, not observation, and they crumble the first time someone pushes on them with an opinion. Personas built from watching and talking to real people are photographs.
Had you handed me a synthetic panel a few years ago, I would have expected stick figures.
This one held up because somebody did the research first. A human-centered design team led by Seth talked with about fifteen CFOs, synthesized those conversations into five personas, and built the panel from them. I didn’t build it. I just got to use it.
I was using it on a suite of AI strategy assistants for manufacturers. We’d spot something in a manufacturer’s data that looked like an insight, take it to the panel, and see whether it held up or fell apart. The panel didn’t get anything wrong. In a few places, especially around capital flow, it pushed us to dig deeper.
So should you use synthetic users?
Yes, if someone talked to real people first. If your synthetic users came from a demographic profile and a prompt, you’ve built a stick figure that talks back.
I’ve watched personas like that bite a team a couple of times. They deal in generalities, so you never design anything specific enough to stand out or move the needle. (A Prolific study of 996 people found that adding demographic personas to a model roughly tripled its error.)
And to be fair, three conversations isn’t a study. It’s three CFOs who sounded like the panel.
All research has a shelf life
I’d argue that all research has a shelf life. People keep changing, and so does everything they’re responding to. New regulations, capital getting easier or harder to come by, a shift in the market: all of it changes how a CFO answers, and it ought to change the synthetic panel too. In my work, access to capital and regulation move fast enough to demand a fresh look every six to nine months, let alone what’s happening with AI and robotics.
So if you’re thinking about cutting your researchers and buying a synthetic panel instead, stop. Who’s going to have the next fifteen conversations?
I still use the panel when I’m brainstorming insights that could help a manufacturer improve performance, raise EBITDA, that sort of thing. But the real people are still vital. They’re how we keep up with what’s changing in manufacturing across the markets we serve.
That CFO wouldn’t trust an inventory tool that treated every part on the shelf the same. I don’t trust a synthetic panel whose research nobody has checked the date on.
