ZML Releases LLM Inference Server Compatible With Multiple Chip Types

ZML released ZML/LLMD, an LLM inference server that runs open-source large language models on Nvidia, AMD, Google TPU, Apple Metal and Intel Arc chips. TechCrunch reported the launch. ZML founder Steeve Morin said the goal is to break existing silos and make different chips available for AI use cases at maximum speed.

The Paris-based company previously released an inference-focused ML framework in 2024 that was updated in March. ZML/LLMD launched as a free product. It is not open source. Morin said the approach lets enterprises and clouds mix chips that may cost less or use less energy.

The startup has a team of 20 people and raised $20 million from investors including 20VC, >commit, AALVC, Drysdale Ventures, Kima Ventures, Kindred Capital, LocalGlobe and Puzzle Ventures. Morin previously served as VP of engineering at Zenly, which Snapchat acquired for nine figures in 2017.

Investors also include Solomon Hykes of Dagger and Docker, Clément Delangue and Julien Chaumond of Hugging Face, and Yann LeCun of AMI Labs.

ZML maintains a relationship with Nvidia and competes with Baseten, Inferact and RadixArk. Morin said the company is co-designing silicon.