The governor of California has issued an executive order to strengthen guardrails for all contracts connected to generative AI tools in the state.
The aim is to reduce risks such as bias, privacy violations, and misuse while ensuring these technologies are deployed safely and accountably.
For Californian enterprises, this policy may affect how companies design, deploy, and manage AI tools that interact with customers.
However, whilst the policy strengthens procurement standards for AI systems, this leaves gaps in ongoing oversight, real-time monitoring, and how AI information is managed once deployed in public-facing services.
Chris Hood, Head of Digital Business Strategy and Platforms at Google Cloud, says generative AI is raising expectations for fast, personalized government services, but also increasing the risk that confident errors will quickly damage trust.
“Generative AI is raising the floor on responsiveness. Citizens now expect government to communicate with the clarity, speed, and personalization they get from commercial services," he said.
“But that creates a dangerous gap: the moment a chatbot gives a confident wrong answer about benefits eligibility or permit requirements, trust collapses in a way it never would from a long hold time.
“Newsom's directive is a step in the right direction, but contract language only gets you so far.”
Newsom’s Approach to AI Risk and Accountability
This new executive order framework focuses on how California’s government buys and uses AI systems, leveraging state contracts to set localized standards.
At the end of March, Gavin Newsom, Governor of California, spoke out on the concerns of risk, trust, and accountability in AI across the country.
“California’s always been the birthplace of innovation. But we also understand the flip side: in the wrong hands, innovation can be misused in ways that put people at risk,” he explained.
“California leads in AI, and we’re going to use every tool we have to ensure companies protect people’s rights, not exploit them or put them in harm’s way.
“While others in Washington are designing policy and creating contracts in the shadow of misuse, we’re focused on doing this the right way.”
As a result, companies that want contracts with California will need to show that they use AI responsibly and safely, meet strict privacy and security requirements, and demonstrate safeguards against misuse, setting a higher bar for vendors.
This means developing new certification requirements for AI vendors to assess risks such as harmful or illegal content, bias, discrimination, and civil rights impacts, and determining with whom the government can work.
By expanding the responsible use of AI inside government, this order aims to encourage ethical and effective use of AI in public services and improve how the government operates while managing risks.
Increasing Compliance and Regulation
Newsom’s policy can help shape CX in California by changing how AI is used in public-facing services.
Implementing stricter requirements will mean AI systems will have to be tested more thoroughly and carefully monitored for errors and bias, meaning customers are likely to experience more reliable interactions with fewer incorrect answers and more consistent service quality.
These tighter privacy rules will also require customer data to be managed more carefully in line with state expectations, reducing the risk of data misuse and increasing trust in digital services.
