the voice of customer experience technology
Front page
NewsService Operations1h · 12:01 BST · 12 min read

AI Agent Observability Just Changed Again, What Next?

Elastic and Grafana are pushing observability beyond traces, using AI investigation, OpenTelemetry, and incident workflows to connect technical faults with customer impact. For contact-center teams, the test is whether a broken interaction reaches the right owner, gets fixed, and stays fixed.

AI observability

I’ve spent the last few weeks watching observability vendors make increasingly ambitious claims about watching AI agents. We've got better traces, more evaluations, and plenty of promises about automated investigations. I'm still wondering who gets the call when something goes wrong between the systems.

Elastic has taken a decent shot at answering that. At ElasticON New York on October 8, it introduced nightshift AI SRE, an always-on investigator that can catch problems conventional alerts miss, test possible causes, and recommend fixes with human approval. It's in private preview on Elastic Observability Serverless, with broader third-party access still to come.

Elastic's Observability General Manager, Baha Azarmi, says nightshift learns from engineers' investigations and reviews its findings after incidents close:

“That is how it keeps up with production without running up a token bill nobody wants to see.”

I'd certainly welcome a smaller bill. But I'd be more impressed if an investigation could follow a failed customer call through the carrier, CCaaS platform, and CRM, then work out which team needs to fix it. That's where a lot of the real frustration sits.

New Relic and Honeycomb have made their own moves in recent weeks, and Grafana's ObservabilityCON 2026 is next on October 19–21. I've got five questions for Grafana, and quite a few of them have less to do with AI than the people who'll be left dealing with the incident.

TL;DR

  • Elastic's nightshift AI SRE looks promising, with some important limits. Its October 8 launch brings investigation and proposed remediation closer together, although broader third-party integrations remain ahead.

  • Grafana has some convincing demonstrations lined up for ObservabilityCON. LexisNexis Risk Solutions plans to show how Grafana IRM connects investigations, ServiceNow tickets, and on-call paging. The question is how well that works when the incident crosses platform boundaries.

  • Someone needs to explain the costs and the backup plan. Grafana's metered AI evaluations, the migration from OnCall OSS to Cloud IRM, and its recent SMS notification problems raise questions about what service teams will pay and how reliably they'll be reached.

What Did Elastic Announce at ElasticON New York?

Elastic’s October 8 launch gave us something concrete to shine a spotlight on. Nightshift AI SRE brings a Context Engine, Detection Engine, Investigation Engine, and Remediation Engine to incident work. Elastic says its own SREs already use it in production, although the product remains in private preview.

According to Elastic’s technical account, investigations can start from an alert, test competing explanations, and flag missing evidence. The Remediation Engine can then suggest a workflow or connector action, subject to human approval. I'd welcome that over another automatically generated incident summary.

But broader third-party observability and data integrations are still planned, and contact centers rarely have the luxury of dealing with one supplier.

IDC Senior Research Director Shannon Kalvar put his finger on the problem in Elastic's launch announcement:

“These agents need access to a broader context, particularly heterogeneous estates.”

If a customer call breaks between a carrier, CCaaS platform and CRM, can nightshift follow the evidence, identify the right incident owner and confirm the service has recovered? Or does an engineer still have to chase three suppliers?

ElasticON New York promised “real-time AI agents to accelerate incident and threat response,” alongside ES|QL and AI-integration previews. Two days earlier, Elastic was asking, “Who approved an agent’s last action?” in its AI control gap session.

Back in London in February, the emphasis was on giving agents better context. By October, the conversation had shifted toward approval and control.

That's progress, but I'd like to see where Elastic's investigation ends and another team's work begins. A proposed fix is helpful. Knowing the customer can actually get through again would be better.

Why Is AI Agent Observability Becoming a CX Reliability Issue Now?

AI agent observability is turning into a CX reliability issue because agents can complete a workflow cleanly and still leave the customer with a bad result. That failure won’t always trigger the kind of signal an operations team is trained to chase.

Honeycomb's September 29 research found that 71% of respondents were deploying AI agents faster than they could manage or govern them. Meanwhile, Enterprise Management Associates research reported on October 8 found that enterprises typically use six to eight observability tools.

Of the 356 IT professionals surveyed, 41% said switching between tools slowed investigations, while 44% blamed integration and API complexity. That's an awful lot of detective work before anyone can start fixing things.

There's a cost to that delay. Axis Intelligence's six-incident analysis found that, on average, 27% of customer-impact time elapsed before the cause was identified. It's a limited sample, but the finding deserves attention.

And demand is growing. Gartner's October 1 AI Hype Cycle reports that only 17% of organizations currently have agents deployed, while more than 60% expect to within two years.

Service teams could soon be investigating far more incidents where the system reports success and the customer has a rather different story. I'd want both versions in the incident record.

How Do Enterprises Monitor AI Agents in Production?

I wouldn't fancy investigating an AI agent failure with six different telemetry formats and half the conversation missing. Yet that's a reasonable prospect for engineers trying to piece together model requests, tool calls, and handoffs across different platforms.

GitHub made one pragmatic move on September 22, adding enterprise-managed OpenTelemetry export for Copilot sessions, model requests, and tool execution. Prompts and responses are excluded by default. Fair enough for privacy, but investigators still need a way to establish what an agent actually told someone without collecting more personal information than they should.

New Relic's October 6 announcement tackles the format problem. Its “Normalize on Read” approach brings open-source agent traces and native application monitoring data together. AWS's CloudWatch Omni is also generally available, covering evaluations across different models and runtimes.

For contact centers, I'm particularly interested in Operata's September Genesys Cloud AVA integration. It puts AI exchanges, telephony events and human-agent handoffs on one timeline. That's a record an engineer might actually enjoy opening when someone's disputing what happened during a transfer.

New Relic has more coming, too. Its AI Evaluation, due for public preview in November, links quality scores with traces, while its Ground Truth CLI aims to preserve investigation and recovery evidence.

Its Service Architecture Intelligence and Infrastructure 360 announcement interests me more. It promises visibility into service ownership, infrastructure dependencies, and changes, including gateways and load balancers. Infrastructure 360's public preview is scheduled for October 13.

I'd want to see that working during an actual incident, especially when the engineer who finds the fault isn't the person authorized to fix it.

What Should ObservabilityCON 2026 Deliver for Service Teams?

Grafana has set itself a fairly ambitious agenda for ObservabilityCON 2026 on October 19–21. Under the banner “AI for Observability. Observability for AI,” it's promising agent evaluations, OpenTelemetry pipelines, and closer links between investigations and incident management.

I'm particularly interested in the customer demonstrations. DriveWealth says its support team now self-serves 25% of questions through Grafana Assistant, while its SRE team uses Assistant and IRM to investigate alerts before paging an engineer.

LexisNexis Risk Solutions has an even more interesting workflow on the agenda. An alert creates an incident, opens a ServiceNow ticket, triggers an investigation, and pages the on-call team. Grafana's Knowledge Graph is supposed to help connect that activity with service ownership.

I'd like to see how much of that actually works without someone filling in the gaps manually. Deutsche Telekom IT GmbH is also lined up to discuss World Cup-scale reliability, which should give us another useful operational perspective.

What Five Questions Must Grafana Answer at ObservabilityCON?

There are promising demonstrations lined up, but they haven't happened yet. And I've got a few questions I'd like Grafana to answer before we start handing out marks.

1. Does portability end when the trace arrives?

Grafana's “any agent anywhere” pitch sounds attractive. OpenTelemetry helps move data, but can evaluations, approval decisions, and remediation history follow it into the right ITSM record? I'd like to know what survives when an investigation crosses suppliers.

2. Can Grafana tell whether the customer outcome was actually good?

The agenda promises conversations, tool calls, and evaluations. Fine, but can Grafana spot a failed customer transfer when every individual API call succeeded? I'd like to see that demonstrated.

3. When does AI agent observability become control?

Grafana's IRM agenda insists that “only humans decide what findings to trust and how to act on them.” Good. If an automated investigation recommends changing call routing or network configuration, who approves it, and how quickly can someone reverse the decision?

4. Who pays for all this monitoring?

Grafana began metering agent observability on October 1. Published Pro rates start at $1.50 per 1,000 generations and $2 per million evaluation or guard tokens, with conventional telemetry charged separately.

Interestingly, Grafana also backed OllyGarden's October 9 funding round, supporting technology designed to reduce unnecessary telemetry. I'd like to hear how Grafana recommends deciding what's worth collecting.

5. What happens when the incident system fails?

Grafana OnCall OSS was archived in March, leaving teams migrating to Cloud IRM to check escalation routes and integrations.

Grafana's own October 5 SMS incident affected notifications to Austrian, Thai, and Bangladeshi numbers. Its October 9 update still recommended alternative channels.

Can Grafana demonstrate that a backup paging route actually reached someone? I'd put that fairly high on the list.

Who Is Responsible When a Customer-Facing AI Agent Fails?

If an AI agent starts sending customers to the wrong queue overnight, who's allowed to switch it off? I'd hope someone has worked that out before the complaints start arriving.

Dataiku’s September 24 CIO study suggests plenty of organizations haven't. It found that 81% of respondents had lost complete oversight of agents created outside approved systems, while responsibility was scattered across IT, AI teams, security and other departments.

Dataiku CEO Florian Douetteau put it rather well:

“Monitoring tells you an agent is running. Managing tells you whether it’s earned the right to keep running.”

I suspect the arguments over ownership get particularly interesting when disabling an agent means interrupting customer service. Who can make that call, and who explains the decision to the contact-center manager?

ServiceNow’s October 6 AI Workflow Factory promises continuous process discovery and improvement. I'd like to know whether that extends to assigning responsibility when an automated process needs to be stopped or rolled back.

PagerDuty tackled another part of the headache on October 7. Its AI-Powered Incident Communications can draft updates from incident calls and channel conversations for human review. Anything that saves engineers from rewriting the same update six times is welcome, provided someone checks what customers are actually being told.

Grafana Cloud IRM's proposed shared incident record could help coordinate those decisions. But someone still needs the authority to make them. I'd want to see that responsibility agreed before the next incident, preferably without a committee meeting.

In Other News

A couple of other beats to watch this week:

  1. Splunk MCP Server was affected by a server-side request forgery flaw, disclosed on October 7 as CVE-2026-76286. The problem is fixed in version 1.2.1, with exploitation depending on specific tool permissions.

  2. New Relic’s Infrastructure 360 is due to enter public preview on October 13, mapping cloud resources, dependencies, and configuration changes into one view and feeding richer context into Autopilot. That’s another sign observability is moving closer to work orchestration, although we should confirm the preview is live before publication.

What I’m Watching Next: Next up is ObservabilityCON 2026 on October 19–21. We’ll grade Grafana against the five questions above after Pier 27.

If you’re running customer-facing agents and have a reliability, cost, or observability story we should test, get in touch.

Grafana’s Exam Has Already Got Harder

Grafana has picked an interesting week to make its case. Since September, AWS has added agent evaluations to CloudWatch Omni, Honeycomb has expanded into fleet-level monitoring, and New Relic is connecting investigations with recovery checks. GitHub and Splunk have made their own moves. There's plenty for buyers to compare before anyone takes the stage at Pier 27.

I suspect Grafana knows that another impressive demonstration of agent tracing won't settle much. I'm more interested in what happens after an alert: whether the right person gets involved, whether they have the authority and evidence to act, and whether anyone checks that the customer's service is working again.

Grafana's Cloud IRM plans suggest it's thinking seriously about those questions. ObservabilityCON is its chance to show how well the pieces fit together.

FAQs

What Is AI Agent Observability?

AI agent observability is the ability to see how an agent behaves once it’s running, including the models it calls, tools it uses, tokens it consumes and actions it takes. For CX teams, the useful part is connecting that record with the customer outcome, especially when a workflow completes without throwing an obvious error.

What’s the Difference Between AI Agent Observability and AI Evaluation?

Observability gives teams evidence about what an agent did. Evaluation asks whether the behavior or output met a quality, safety or business standard. AWS, Grafana and New Relic are now bringing those evaluations into production monitoring, so teams can connect agent behavior with traces, tool calls, guardrails and, increasingly, business impact.

What Role Does OpenTelemetry Play in AI Agent Observability?

OpenTelemetry gives teams a common way to collect and move agent telemetry between systems, including model requests, tool activity and traces. That reduces the risk of every agent platform inventing its own monitoring format. The harder question is whether evaluations, policy decisions and remediation history stay portable once the telemetry reaches an observability platform.

What Does AI Agent Observability Cost?

There’s no useful universal price because cost depends on what gets retained and how often evaluations or guards call another model. Grafana made that concrete on October 1: generation ingestion and LLM-based evaluation and guard usage are now metered, while metrics and traces keep standard telemetry pricing. Sampling still matters.

rate this story
helps rank stories across CX Today
The discussion0 takes · attributed & checked

Does this reflect your experience?

opening the room…
Read nextordered by techtelligence · every pick explained
same beat · Service Operations

The Reliability Stories CX Leaders Can’t Afford to Miss

10 Oct 2026
same beat · Service OperationsOpsgenie End of Life: The Six-Month Countdown Has Started7 Oct 2026same beat · Service OperationsSalesforce Outage Chaos: The Week Uptime Lost Its Impact4 Oct 2026