Building, testing and releasing an AI agent into customer service may feel like the project has been completed successfully, but the harder work begins when that agent starts handling real customer conversations, making decisions, triggering workflows and representing the brand without a human sitting behind every interaction.
At that point, AI agents become part of the customer service operating model and customer service leaders need to consider how they will run it over time.
As Csaba Tamas, Chief Product Officer at Parloa, told CX Today, businesses risk “agent drift” if they fail to actively manage them.
“An AI agent was built for a particular business context with a particular service and product definition, but over time, the context is changing. So that means that you have new conditions, business conditions, new policies, new add-ons in the business, and those changes might not be captured in the AI agent.”
The risk increases over time. A stable business with limited change may not feel it quickly. A more dynamic business may see problems appear sooner. “Essentially because the agent is still behaving like the business was behaving 18 months before, and that might or might not be relevant,” Tamas said.
The One-Time IT Project Mindset Creates Risk
Many enterprises still treat AI agent deployment as a one-time IT project, Tamas noted. “You find a budget, you bring together an IT team, they build the agent, they test the agent and they walk away.”
This creates another operational problem. If the first agent works, the business will usually want to scale more agents across more processes.
“If this is successful, then you would want to do more AI agents because in the business you have hundreds of business processes,” Tamas said.
The result is a whole host of agents requiring Once an AI agent enters production, enterprises need to make regular improvements after the enter production to prevent drift.
This is why Parloa argues that AI agent building needs to move closer to the business. As Tamas advised:
“Ultimately, the AI agent building shouldn't be an IT project; it should be a business project. It should be something that the subject matter experts in the business are building.”
Enterprises Underestimate the Cost of Change
Tamas said buyers often underestimate the marginal cost of each agent modification and the speed at which changes can be made.
Early in the project, when the priority is getting the first agent working, those questions can feel secondary. But live customer conversations expose cases that testing did not cover.
“Pre-production, let's assume that you did extremely good testing via simulated conversation,” Tamas said.
“You run evaluations, and you run a couple of hundred different perspectives and you are proud, but guess what? In real life, there will be a 101st and 102nd corner case. Or a 500th case.”
Each new case requires a decision. “Do I need to again secure a budget? Bring a new project team together to make a small modification?” Tamas said. “Do I wait for five to 10 issues to accumulate, or am I doing a little adaptive work every time I'm experiencing a new issue?”
The alternative is a self-service model where an operator can adjust the agent more directly.
“Compare this with the opportunity where the subject matter expert can use a self-service interface to just click and rework the agent code configuration, and it's done,” Tamas said.
The First Months After Deployment Are Critical
Post-launch optimization can have a significant impact, especially during the first few months, Tamas said.
“What we observed is that right after you go to production, there will be yet another substantial improvement in production in the first three to six months.”
The practical question is how quickly the organization can react to performance gaps.
“Do you need one hour, one day, one week, one month, or one quarter to react to these gaps?”
