The Customer in the Simulation: What Determines Whether AI Roleplay Reaches the Live Floor
-
IntouchCX Team
By Michael Baehr, Director, Training Solutions, IntouchCX
Whether AI belongs in agent training is no longer up for debate. AI roleplay has moved from pilot to standard practice across most contact center onboarding programs. The harder question is whether that practice holds up when a real customer takes the call somewhere nobody scripted.
That is where most programs stall, and the technology is rarely the reason why. They stall in design.
The friendly classmate problem
When roleplay meant pairing up with a classmate, a fellow trainee played the customer. They would watch the new agent search for the right screen, notice the struggle, and suddenly remember their order number. A minute later the problem fixed itself, everyone laughed, and the class moved on. Then the new agent took a first live call, and the customer on the other end had no reason to help.
AI removed the friendly partner, but it will play whatever customer you describe to it. Describe that customer vaguely, and you get one who is too easy, too predictable or too familiar to prepare anyone for a real call. Research backs this up: in a 2026 study of LLM-simulated shoppers, simulated users were polite in 39.2% of their turns, compared with 19.9% for real people. A realistic customer has to be designed on purpose.
Adoption has moved faster than measurement
Contact centers track abandonment, average handle time and quality closely. ICMI’s 2025 survey shows 85%, 84% and 77% of centers measuring each. Yet no published contact center study has connected roleplay scores, completions or repetitions to those live outcomes.
For a brand, that gap carries real risk. A training program can report strong simulation numbers that never reach the customer.
Meanwhile, the bar keeps rising. Three shifts are making practice matter more than ever:
- The calls reaching people are harder. In Verint’s 2026 survey of 1,000 agents, 61% expect AI to leave them with more complex, technical work. The customer who reaches a person has often already explained the problem to a bot that could not help.
- Practice became unlimited. AI removed facilitator time as a constraint. Once repetition is free, the quality of what agents repeat becomes the deciding variable.
- Real-time guidance created a new skill. Agent assist tools prompt agents mid-call, and new agents who do not yet know the terminology can freeze when a suggestion appears. Receiving in-call guidance is a skill, and roleplay is where it can be practiced safely.
Four design decisions behind roleplay that transfers
Every AI roleplay program makes four design decisions, deliberately or by default. Programs that reach the live floor make all four from the customer’s side of the call. Programs that stall make them from the QA checklist.
1. Who the customer is. The persona drives the exercise. A customer with real motivations, real impatience and a reason to push back gives the agent a genuine chance to get the call right or wrong. Variety matters as much as realism: one refund scenario played by five different customers teaches more than five rehearsals with the same one.
2. Which calls get practiced. Libraries drift toward the dramatic, because the hardest escalations are memorable. The better rule is to rank scenarios by the operation’s own contact-driver volume, cover the most frequent calls first and add rare, difficult ones as agents progress.
3. What surrounds the conversation. Live calls involve knowledge bases, CRMs and in-call prompts. Practice that skips those steps produces agents who sound confident and still stumble on the floor.
4. What gets measured and coached. AI can evaluate almost any behavior that can be described precisely, and very little that cannot. “Measure empathy” has to become specific, observable criteria before the first scenario runs.
What the programs revealed
Across documented AI roleplay programs in several industries, two patterns stood out.
At a national same-day delivery service, agents who practiced with personas built from real customers handled calls 21 seconds faster than non-users and sustained 92% CSAT. Even more telling, performance variance across the practicing cohort was 3.1 points, against 10.7 for the control group.
The programs also showed that practice pays off most when pressure is highest, and that speed and quality improve on different timelines. That second finding has a direct consequence: a pilot judged too early can miss the result that matters most.
The Transfer Test
Until the industry has better evidence, each program has to prove for itself that practice carries over to live calls. The full whitepaper proposes the Transfer Test, a six-question scorecard that judges AI roleplay by live results across a full onboarding cycle, starting with a simple question: does practice cover the real contact mix?
Download the full whitepaper
The complete paper covers:
- The evidence behind each of the four design decisions, including results from car sharing, professional services and global cosmetics programs
- The full Transfer Test scorecard, with how to measure each question and what good looks like
- Where roleplay programs lose their footing, from data concerns to scenario ownership
- Why training is starting to look like the first level of a video game, and what that means for trainers and team leads
Download the full whitepaper