Salesforce has announced the "world’s first" LLM (large language model) Benchmark for CRM.
With the solution, Salesforce customers can rank LLM models on a leaderboard to select the best possible option for their chosen generative AI (GenAI) use case.
The LLM Benchmark enables this by leveraging a "comprehensive evaluation framework", with criteria grouped into four categories: accuracy, speed, cost, and trust & safety.
With scores available across each category, Salesforce hopes businesses can make "smart decisions" when evaluating LLMs and utilize its open Einstein platform to leverage multiple models where they're best fit.
So, for example, a service team may harness one LLM to draft customer replies and another to summarize customer conversations.
However, the LLM Benchmark will only be available across service and sales use cases on release – with Salesforce planning to expand the offering across its CRM apps thereafter.
The CRM leader also promises to continually advance its evaluation of LLMs and - eventually – to rank fine-tuned models alongside iterations of hallmark offerings like ChatGPT, Gemini, and Llama.
Sharing the news, Silvio Savarese, EVP & Chief Scientist at Salesforce AI Research, stated: "Salesforce’s new LLM Benchmark for CRM is a significant step forward in the way businesses assess their AI strategy within the industry.
It not only provides clarity on next-generation AI deployment but also can accelerate time to value for CRM-specific use cases.
"Our commitment is to continuously evolve this benchmark to keep pace with technological advancements, ensuring it remains relevant and valuable."
That commitment is significant as CRM customers have so far lacked a reliable way to analyze the effectiveness of GenAI-powered deployments beyond manual reporting.
Yet, with the LLM Benchmark, businesses can not only monitor the best-performing LLM but gain new insight into the performance of such deployments. The following screenshot underlines this.
[caption id="attachment_61115" align="alignnone" width="850"]
Here's a sneak peek at Salesforce's LLM Benchmark for CRM.[/caption]
With such a dashboard, Salesforce has indeed brought a comprehensive Benchmark to the CRM market. However, other CX vendors have taken similar approaches to match GenAI use cases with the optimal LLMs.
For instance, Zoom uses a federated AI approach. Its researchers uncover the best-placed LLM for use cases across its communications platform, which Zoom then recommends to customers.
Meanwhile, Five9 has a GenAI Studio to test and score LLMs across use cases. Users may also customize and optimize the prompt that powers each GenAI application.
