ai
Microsoft AI Chief Says Distilled Models Fall Short on General Tasks
Microsoft AI chief Mustafa Suleyman said companies using distilled models built from frontier lab data will eventually lag on broad tasks. He said Microsoft is training its models with zero distillation. Demand for frontier models has risen faster than demand for open-source versions.
May 29, 1:20 PM·1m read1 source
SemaforAudio version
Tap play to generate a narrated version.
Developing·Limited corroboration so far. This page will refresh as more sources emerge.
Microsoft AI chief Mustafa Suleyman said companies relying on distillation from frontier models are taking a shortcut that leads to a dead end. “You’ve stuffed your model full of somebody else’s knowledge,” Suleyman said in an interview. He added that Microsoft is building its models with zero distillation.
Suleyman said frontier labs do not release the large data sets used to train their biggest models. Without that information, it is impossible to know what priorities shaped those data sets. Distillation can produce small models for narrow tasks, but models built on distilled data fall behind when used for general-purpose work, he said.
Amazon recently removed its AI leaderboard after tokenmaxxing drove up costs. Those rising costs have increased interest in Chinese open-source models such as DeepSeek. Suleyman said cheap distilled models have not taken over the market. Demand for the most advanced frontier models has grown faster than demand for open-source versions.
Microsoft restructured Copilot leadership last month to allow Suleyman to focus on new model development, including what he described internally as Superintelligence efforts.
