Last month, Canadian courts ordered Air Canada to pay up after the airline's GenAI-powered chatbot misled a customer.
As reported by CX Today, Jake Moffatt, a bereaved grandchild, paid more than $1600 for a return flight to and from Toronto.
Yet, in accordance with the airline’s bereavement rates, he only needed to pay around $760.
After realizing this, Moffatt sued, and Air Canada tried to use the defense that the bot is a separate legal entity for which it cannot be held liable.
That defense failed, and Marc Benioff, Chair & CEO of Salesforce, suggests the ruling holds great significance. During an earnings call, stated:
Just as they would for a human employee, they were being held liable for a digital employee.
Moreover, Benioff warned that these AI models are "very confident liars, producing misinformation and hallucinations," and suggested: "There's a danger for companies, for enterprises, for our customers, that these are not trusted solutions.
"These [public] models don't know anything about the company's customer relationships and, in some cases, are just making it up.
"Enterprises need to have the same capabilities that are captivating consumers, but they need to have it with trust, and they need to have it with security. And it's not easy."
Thankfully, Benioff believes there are three "essential components" that enterprises can build into their GenAI bot strategy to deliver trusted experiences.
These are a compelling user interface (UI), a world-class AI model, and a huge data set.
Yet, Benioff stresses that the data set must include metadata so the AI understands and delivers the critical insights and intelligence that customers need.
"That's not just some amalgamated stolen public data set… that’s the deep integration of data and metadata,” he concluded. “Oh, and that's what Salesforce does."
