Resources

Are AI Roleplay Programs Preparing Agents for the Right Calls?

AI roleplay gives agents the opportunity to practice customer conversations before they reach the live floor, but the value of that preparation depends on how closely those exercises reflect the calls they will actually handle. A training program can produce strong simulation scores while leaving new hires unprepared for situations that fall outside a limited set of scenarios.

Through its work supporting agent training programs, IntouchCX has observed how scenario libraries can become disconnected from the types of calls agents handle in production. This creates an opportunity to rethink how organizations design AI roleplay.

Why Are Agents Practicing Calls That Don’t Reflect Live Demand?

Training scenarios are frequently selected based on the situations that stand out most to trainers and supervisors. Difficult escalations and emotionally charged conversations receive attention because they expose skills that agents need to develop. Yet these situations can account for a relatively small portion of the calls handled each day.

As Michael Baehr, Director, Training Solutions at IntouchCX, explains, “Libraries drift toward the dramatic. The hardest escalations are memorable, so they get built first, even when they make up a small share of volume.” The result is that new hires spend their rehearsal time on exceptional situations and reach live operations with little preparation for the requests that fill most of their day.

Baehr has observed this pattern in programs that relied on four or five familiar scenarios. Agents performed well on those practiced calls, while quality scores and satisfaction results were weaker on contact drivers that the training library had not covered. Handle times on those unpracticed calls were also less favorable. A 2026 study performed by McKinsey & Company identified the call types with the longest handle times and had new hires practice them in AI simulations modeled on real customer concerns. Positive interactions rose by roughly 33 percentage points, and interruptions during simulations fell by roughly 28percentage points. These results followed from practice that mirrored the calls the help desk received.

Scenario selection therefore needs to begin with actual contact-driver data. Insights into call volume and recurring service issues allow training teams to prioritize the situations agents are most likely to face, while reserving more complex cases for later stages of development.

IntouchCX approaches this challenge through Akin, its fully customizable AI roleplay and simulation platform, which helps agents build proficiency and reach agent readiness faster. The technology creates on-brand AI personas for unscripted customer conversations across voice, chat and email, allowing agents to practice situations grounded in real service demand. Real-time AI evaluations and manager insights also help training teams identify where agents need additional coaching and refine practice accordingly.

What Makes a Simulated Customer Realistic Enough for Training?

A realistic simulated customer has a clear reason for calling and responds according to their own motivations, even when the agent struggles to resolve the issue. The AI persona should maintain its character throughout the conversation, requiring agents to interpret different reactions and adapt their responses as they would during a live call.

This level of realism becomes especially important when agents practice the same type of request repeatedly. If every simulated caller responds predictably, trainees can memorize a successful sequence without learning how to handle variations in the conversation. As Michael Baehr states, “One refund scenario played by five different customers teaches more than five rehearsals with the same one.”

IntouchCX applied this principle in a program for a global beauty brand, building AI-generated personas around the program’s actual top call drivers instead of generic scenarios. After each practice conversation, Akin scored the agent on resolution quality and on the empathy and clarity of the responses. Agents trained with Akin handled calls 42 seconds faster than the comparison cohort and raised CSAT by 6%, more than three times the gain of their peers.

The AHT and CSAT results show what realistic practice builds in new hires: the adaptability to stay effective when a familiar request arrives with an unfamiliar reaction. Because each simulated caller responds from personal motivations, trainees learn to interpret the emotional cues on the line and adjust their approach in the moment. That skill carries into operations, where it shortens calls and resolves issues faster. 

How Does Realistic Practice Translate Into Live Performance?

Realistic practice prepares agents for live performance by recreating the conditions in which they must resolve customer issues, including the need to navigate systems while maintaining a conversation. When training incorporates these demands, agents can develop the skills required to apply their knowledge without interrupting the flow of a call.

During production, agents consult knowledge resources and customer records while responding to the person on the line. They also need to interpret real-time guidance without allowing a prompt or system lookup to interrupt their response. As Baehr explains:

“Practice that skips those steps produces agents who sound confident and still stumble on the floor.”

To prepare agents for these situations, simulations should incorporate the same resources they will use in live operations. For example, when a simulated caller provides an order number, that information should match the record available in the training system. Agents can then practice locating the correct information while keeping the conversation moving, rather than rehearsing responses in isolation.

The effectiveness of this preparation ultimately needs to be evaluated through live results. Training leaders should examine whether agents perform better on the types of calls they practiced and track those outcomes throughout onboarding. Comparing performance across practiced and unpracticed contact drivers provides a clearer understanding of which scenarios are contributing to agent readiness and where additional training is needed.

How Should Training Leaders Decide What to Practice Next?

A scenario library should evolve alongside the service environment. Contact patterns change as brands introduce new policies and customers encounter different problems. Training content that remains unchanged eventually prepares agents for situations that no longer reflect current demand.

A practical way to manage this process is to connect scenario planning with live performance data through four decisions:

  1. Prioritize by contact volume: Identify the most frequent reasons customers reach support and ensure those situations receive sufficient practice before expanding into less common escalations.
  2. Introduce variation within each scenario: Create personas with different motivations and responses so agents develop adaptability instead of memorizing a fixed conversation.
  3. Increase complexity as proficiency improves: Begin with clearly defined requests, then introduce situations that require greater judgment and more demanding system navigation.
  4. Use production results to refine the library: Compare quality scores and satisfaction across practiced contact drivers, then direct additional training toward areas where performance remains weaker.

Strong scores during simulation provide useful coaching information, but their relationship with the outcomes needs to be established before they influence certification decisions. As agents progress through onboarding, quality assessments and performance insights can reveal recurring difficulties that require targeted coaching or adjustments to the scenario library. Measurement should extend across the onboarding cycle because improvements in efficiency and service quality do not necessarily appear at the same time. 

Tracking performance at both the individual and program level gives training teams the visibility to decide where coaching will have the greatest effect. 

What Does It Take to Build AI Roleplay That Prepares Agents for Live Calls?

Building effective AI roleplay requires training teams to stay connected to the conditions agents face in production. Scenario design needs input from people who understand the service environment, while performance data should inform how the library changes as new demands emerge.

This connection between training and frontline performance guides how IntouchCX works with enterprise brands to develop AI-based practice programs. Drawing on experience across different industries, its training teams design scenarios around actual contact drivers and develop simulated personas that reflect the decisions agents need to make during live conversations.

Human trainers remain central to this process. AI enables repetition and immediate evaluation, while experienced coaches interpret performance patterns and help agents apply what they have learned. As Baehr explains:

“However many gold stars the AI hands out, people still need someone to tell them they did it right.”

For training leaders, the objective is to develop agents who can apply practiced skills when a conversation takes an unexpected direction. Achieving that requires scenarios grounded in real service demand and evidence that the preparation improves performance where it matters.

Read Michael Baehr’s whitepaper, The Customer in the Simulation: What Determines Whether AI Roleplay Reaches the Live Floor, to explore the design decisions and measurement framework behind AI roleplay that prepares agents for live calls.