OpenAI Unveils Custom AI Inference Chip Built With Broadcom

OpenAI unveiled its first custom-built inference processor on Wednesday in partnership with Broadcom. The new chip, named Jalapeño, was designed specifically for OpenAI’s inference workloads. The company said its own AI models assisted in the chip’s development. Early test results show significantly better performance-per-watt than current alternatives, according to OpenAI.

The collaboration was formally announced in October, though OpenAI’s plans for custom silicon had been rumored for months as a strategy to lessen reliance on Nvidia GPUs. Google and Amazon have previously developed similar AI accelerators for their own systems.

OpenAI president Greg Brockman discussed the company’s chip strategy on an internal podcast shortly after the partnership was revealed. “We have a deep understanding of the workload,” Brockman said. OpenAI highlighted the chip’s low operating cost when handling real-time coding models.

Pre-training tasks are expected to continue using Nvidia hardware, while inference cost reductions could improve the company’s economics. The announcement noted that OpenAI is designing chip architecture, kernels, memory systems, networking, scheduling, and deployment systems to make its models faster and more affordable.

““OpenAI is not only developing frontier models or building products on top of them; it is designing the infrastructure underneath them: chip architecture, kernels, memory systems, networking, scheduling, deployment systems, and product experience.””

The New York Times reported that OpenAI plans to deploy enough chips to consume 10 gigawatts of electricity, an amount that could power millions of households.