US Commerce Department Signs AI Model Review Agreements with Google, Microsoft, and xAI

The US government has reached agreements with Google DeepMind, Microsoft, and xAI to review early versions of their new AI models before public release, focusing on national security risks. The Center for AI Standards and Innovation (CAISI), part of the US Department of Commerce, announced the deals on Tuesday.

CAISI stated that these collaborations aim to scale federal efforts on AI risks, amid ongoing debates about balancing innovation and security.

Independent, rigorous measurement science is essential to understanding frontier AI and its national security implications, said Chris Fall, CAISI director. CAISI facilitates collaboration between the tech industry and the federal government in developing standards and assessing risks for commercial AI systems.

The agreements emphasize identifying national security risks tied to cybersecurity, biosecurity, and chemical weapons.

Developers share unreleased AI models with the government that have reduced or removed safety guardrails, enabling thorough evaluation of national security-related capabilities and risks, CAISI stated. OpenAI and Anthropic signed similar deals with the Biden administration two years ago.

Under former President Joe Biden, the institute—then known as the US Artificial Intelligence Safety Institute—focused on developing AI tests, definitions, and voluntary safety standards.

It was led by Biden tech adviser Elizabeth Kelly, who has since joined Anthropic. CAISI has completed more than 40 evaluations, including on unreleased models and cutting-edge models not yet available to the public. Developers hand over versions of models with safety guardrails stripped back for CAISI to probe national security risks.

The agreement allows evaluation of models before deployment and research to assess capabilities and security risks. The deals build on similar agreements from the Biden administration and fulfill a 2025 Trump pledge to collaborate on AI vetting. US President Donald Trump signed executive orders last year that formed the basis of his administration's AI Action Plan, which seeks to streamline AI regulations while prioritizing US leadership in the field.

Microsoft will work with US government scientists to test AI systems in ways that probe unexpected behaviors. Microsoft and the US government will develop shared datasets and workflows for testing Microsoft's models. Microsoft regularly undertakes many types of AI testing on its own, but testing for national security and large-scale public safety risks must be a collaborative endeavor with governments, according to a Microsoft blog post.

Microsoft announced a similar agreement in the UK on Tuesday with the government-backed AI Security Institute, which focuses on safe AI development. Evaluations of AI models from the companies will cover testing, collaborative research, and best practice development related to commercial AI systems.

These expanded industry collaborations help scale work in the public interest at a critical moment, Chris Fall, CAISI director, stated.

Google and xAI did not immediately respond to requests for comment. The new pacts expand on agreements by OpenAI and Anthropic reached during the Biden administration.

Anthropic limited its rollout of Mythos to a few companies. Anthropic initiated Project Glasswing to bring together tech companies to secure the world's most critical software. Anthropic has been embroiled in a dispute with the Pentagon over guardrails on the military's use of its AI tools.

The Pentagon reached agreements last week with seven AI companies to deploy their advanced capabilities on the Defense Department's classified networks. The Pentagon announcement did not include Anthropic. Anthropic is mired in a lawsuit with the US Department of Defense over refusal to drop safety guardrails for government use of its models.

Senior members of Trump's staff met last month with Anthropic CEO Dario Amodei. The New York Times and Wall Street Journal reported on Monday that the Trump administration was mulling a potential executive order to create government oversight for AI tools. The Trump administration characterized the reporting on the executive order as speculation.

The White House is considering a new AI working group that would explore potential oversight and vet models before release, according to CNBC. Leading AI companies will give the Commerce Department early access to new systems, The Washington Post reported. Microsoft, Google, and xAI agreed to share AI models with the White House for security reviews, as stated by the New York Post.

CAISI will conduct pre-deployment evaluations and targeted research to understand capabilities and risks of new tools. The deal comes days after the Pentagon announced an agreement with seven tech giants to use AI in classified systems, according to Al Jazeera. Google, Microsoft, and xAI agreed to voluntarily submit their models for testing through CAISI, The BBC reported.