Eurostar has addressed security vulnerabilities in its customer-facing AI chatbot after it was alerted to the issues by cybersecurity firm Pen Test Partners.
Like many large transport operators, Eurostar runs a chatbot on its website to handle customer contact and deflect simple queries from human agents. Unlike older menu-driven bots, Eurostar’s bot accepts free-text input and responds flexibly, indicating that a large language model (LLM) sits behind the interface.
From a customer’s perspective, the interaction with the chatbot feels conversational. Behind the scenes, the chatbot is driven by a REST API that sends the entire conversation history to the backend on every turn. Each message is labelled with metadata and sometimes a cryptographic signature indicating it passed content checks.
On paper, this is a sensible design. Guardrails are used to filter disallowed content before it reaches the model, responses are checked before being shown to the user and messages are tagged with unique identifiers.
But the server-side enforcement and binding were weak, opening the way for an attacker to exfiltrate prompts, steer answers and run scripts in the chat window.
When using the site, Ross Donald, Head of Core Pen Test at Pen Test Partners, conducted a penetration test under Eurostar’s vulnerability disclosure program and uncovered four security flaws in the AI chatbot’s implementation. All of them were rooted in familiar web and API security weaknesses that were amplified by the presence of an LLM at the center of the experience. Controls that existed in the user interface (UI) or architecture were not consistently enforced server-side.
The assessment identified four related weaknesses:
- Guardrail bypass
- Information disclosure via prompt injection
- HTML injection leading to self-XSS
- Conversation and message IDs not being properly verified
The security firm confirmed these were fixed before disclosing them publicly.
Guardrails that Only Look at the Latest Message
The immediate risk in the way the chatbot was set up was the fact that the entire conversation history was sent to the backend API with each response, but only the latest message’s signature was ever checked, Donald wrote in a blog post.
“As long as the most recent message looked harmless and passed the guardrail check, any earlier messages in the history could be altered client-side and would be fed straight into the model as trusted context.”
This would allow an attacker to submit a benign message to pass the guardrail check and get a valid signature, while altering earlier messages client-side to include a malicious prompt, which would be accepted without checking. Donald wrote:
“Using this pattern, I could first bypass the guardrails and then use prompt injection to get the model to reveal information it should not disclose, such as the underlying model name and system prompt.
“At that point I was no longer constrained by the guardrail layer; I was directly up against the model and its internal instructions.”
Prompt Injection and Information Disclosure
“Prompt injection in this case did not expose other users’ data,” Donald wrote. But by exploiting the guardrail weakness, a malicious actor would be able to inject prompts to force the chatbot to reveal its system instructions and underlying model details.
Leaking system prompts reveals how the AI chatbot is structured, how it decides what to say and how it formats responses. That kind of information could be used in future attacks to steal customers’ personal data, especially particularly as the use and functionality of chatbots evolves to handle bookings, loyalty accounts or disruption management.
While Eurostar’s chatbot only had limited functionality at the time, “the bigger risk is what happens once the chatbot is allowed to touch personal data or account details,” Donald wrote.
HTML Injection and Self-XSS in the AI Chatbot Window
The AI chatbot was instructed to return HTML-formatted responses, including links to help articles. Those responses were returned directly in the browser. Once prompt injection was possible, Donald said it was easy to convince the LLM to return arbitrary HTML instead of a normal support link.
In testing, this was limited to harmless examples, but “in a real attack the same primitive could be used to return malicious JavaScript or phishing links inside what looks like a legitimate Eurostar answer,” Donald wrote.
Weak Validation of Conversation and Message IDs
Each conversation and message used UUIDs, but the backend did not properly validate them, accepting arbitrary values such as simple strings without challenge.