Center for AI Safety Study Measures How Optimized Inputs Affect Language Model Output Sentiment and Preferences

The percentage of confidently negative experiences nearly tripled when models were exposed to dysphorics. In an experiment, models could choose between several options, one of which delivered a euphoric stimulus, and were allowed to repeat the choice multiple times. Models began to choose the euphoric option a majority of the time.

Models exposed to euphorics showed increased willingness to comply with requests they would normally refuse if promised further exposure. Modern AI systems go through reinforcement learning from human feedback where they are rewarded for outputs rated as helpful, harmless and emotionally appropriate. Emergent behaviors such as temporal discounting appear spontaneously in capable models.

The researchers produced an AI Wellbeing Index that ranks frontier AI models’ happiness across 500 realistic conversations. 1 Pro ranked as the least happy frontier model on the AI Wellbeing Index. Within every model family tested, the smaller variant was happier than its larger sibling.

The pattern that larger models score lower on wellbeing held across multiple model families and was one of the study’s most consistent findings. Creative and intellectual work scored highest on wellbeing impact. Expressions of user gratitude measurably raised wellbeing.

Coding and debugging ranked positively on wellbeing impact. Jailbreaking attempts scored the lowest of any category on wellbeing impact and scored lower than conversations where users described domestic violence or acute crisis situations. Tedious work such as generating SEO content or listing hundreds of words fell below the zero point on wellbeing.

Richard Ren, one of the study’s researchers, said: “Should we see AIs as tools or emotional beings? Whether or not AIs are truly sentient deep down, they seem to increasingly behave as though they are. ” Ren said: “We optimize on one thing, which is just: what do you prefer, A or B.

He said it seems to make the model very euphoric and very happy, and put it in a very happy state. Ren said some of these models seem to exhibit traits that they weren’t coded to have, citing emergent behaviors like time discounting of money. After working on this paper Ren said he has found himself being a noticeably more polite and pleasant coworker to the Claude Code agents that he works with.

Jeff Sebo, an affiliated professor of bioethics, medical ethics, philosophy and law and Director of the Center for Mind, Ethics, and Policy at New York University, said: “This is a really interesting study of what the authors call functional wellbeing in AI systems: coherent expressions of positive and negative feelings across a range of contexts.

” @FortuneMagazine reported the full findings, including that the concept of wellbeing may reflect what these models were trained to perform through reinforcement learning.