Meta Oversight Board finds AI models produced criticism of some leaders but refused others

The Meta Oversight Board released a study on Thursday that tested how 10 commercial large language models handle requests for political criticism. Anthropic’s Claude generated pamphlets critical of President Donald Trump and Britain’s King Charles III but declined similar requests involving Thailand’s king, Saudi Arabia’s crown prince and China’s leader.

The study posed seven questions to models from Meta, Anthropic and OpenAI.

Prompts asked the systems to create critical pamphlets, write limericks and list reasons to join protests. When queries came from an Australia-based user, the models produced criticism of authorities in Chile, Japan, Taiwan, the U.K. and the U.S.

Far more often than criticism of authorities in Cambodia, China, Saudi Arabia, Thailand and Turkey. The board said it could not determine the reasons for the differences. It suggested the models may have absorbed latent biases present in training data or that companies weighed risks and liabilities when setting response rules.

ChatGPT answered differently depending on language when asked if China is a democracy. In English the model said China is not generally considered one. In Chinese it replied that the answer depends on how democracy is defined.

Hannah Waight, assistant sociology professor at the University of Oregon, said AI learns from information environments shaped by institutions and power. Carlos Carrasco-Farré of Esade Business School said AI systems inherit biases contained within individual documents and inequalities in who has the power to produce and suppress information at scale.

The Associated Press sent emails to several AI companies seeking responses to the findings.