OpenAI has published first benchmark results for Jalapeno, its custom inference chip.
The company says Jalapeno can serve more AI work per unit of power while returning responses more quickly. It frames the chip as part of a full-stack strategy: models, products, serving software, chips, memory, networking, and systems designed together around real AI workloads.
That is the main story. Jalapeno is not just another accelerator claim. It is OpenAI arguing that the economics and latency of future AI products will depend on controlling more of the infrastructure stack.
OpenAI is measuring useful work per watt
OpenAI says it tested Jalapeno on InferenceX, a public benchmark from SemiAnalysis that measures the full process of serving an AI request. The company says it compared Jalapeno with leading commercially available AI systems across operating ranges from high-throughput serving to interactive, low-latency use.
Across GPT-OSS 120B, DeepSeek R1 670B, and Kimi K2.5 1T, OpenAI says Jalapeno delivered 1.5x to 1.9x more AI work per watt at peak throughput and 1.7x to 3.6x lower end-to-end latency than comparison systems.
OpenAI also says Jalapeno’s advantage widened in internal testing on frontier OpenAI models, though those results are not independently visible in the post.
The company’s preferred metric is performance per unit of power, not performance per chip. That framing is important because energy availability, rack density, and data-center operating cost are becoming hard constraints for AI deployment.
Agent latency changes the chip problem
OpenAI explicitly links the chip to agentic workloads. Agents often complete many steps in sequence, so a delay at each model call compounds across a task. Faster end-to-end latency can therefore matter more than peak throughput alone.
The chip is designed around the different phases of language-model inference. Prefill is compute intensive, while token-by-token decode is more constrained by memory bandwidth. Communication delays between chips or cores can also leave processing units idle.
OpenAI says Jalapeno minimizes data movement by keeping model state, including KV cache, local while activating the right compute, memory, and networking resources for each inference phase.
That is the kind of optimization a general-purpose accelerator may struggle to match if language-model serving keeps changing quickly.
AI helped build the AI chip
OpenAI also says its models accelerated Jalapeno’s development. According to the company, earlier generations helped with chip design and bring-up, while newer models are helping optimize and program the system.
The post says the team used Codex with GPT-Astra to bring three open-weight models that were not part of Jalapeno’s original production plan to high performance within two months. For selected GPT-OSS attention and mixture-of-experts blocks, OpenAI says AI-generated implementations ran 1.5x to 1.8x faster than existing human-expert-written implementations.
Those figures apply to selected blocks, not full-model serving. Still, they point to a development loop that matters: AI systems helping design and optimize the infrastructure that will serve later AI systems.
Deployment is planned by year end
OpenAI says it plans to begin deploying Jalapeno within its compute infrastructure by the end of 2026. It also says Gen 2 is deep in development and Gen 3 is taking shape.
The company is not saying it will stop using outside accelerators. It says it will continue deploying NVIDIA and other partner hardware for training and inference. The strategic point is diversification and leverage: getting more useful work out of power and hardware while reducing dependence on any single serving architecture.
