OpenAI has published the first measured results for Jalapeño, its custom AI inference chip, and the headline is not simply “more speed”. The company says its purpose-built silicon can process more AI work per unit of power while cutting the delay users experience. If those gains hold at production scale, the OpenAI Jalapeño chip could affect the responsiveness, capacity and economics of AI services.
Background: why OpenAI is building an inference chip
Training a large model demands enormous computing power, but running that model for millions of prompts creates a different challenge. This second stage, known as inference, must balance throughput, latency, memory bandwidth, networking and electricity use. Interactive agents make the problem harder because a small delay can recur across dozens of sequential steps.
OpenAI unveiled Jalapeño with Broadcom in June 2026 as its first custom “Intelligence Processor”. Broadcom contributed silicon implementation and networking expertise, while Celestica supported boards, racks and system integration. OpenAI said the chip moved from initial design to manufacturing tape-out in nine months, with AI tools assisting parts of design and optimisation.
The move does not mean OpenAI is abandoning commercial accelerators. The company says it will continue deploying NVIDIA hardware and systems from other partners. Instead, custom silicon gives it another option designed around the workloads behind ChatGPT, Codex, the API and future AI agents.
What the first Jalapeño benchmarks show
OpenAI tested Jalapeño with InferenceX, a public benchmark from SemiAnalysis that measures the end-to-end process of serving an AI request. Tests covered three open-weight models: GPT-OSS 120B, DeepSeek R1 670B and Kimi K2.5 1T. The comparisons were made across operating points ranging from high-throughput batching to low-latency interactive use.
According to OpenAI, Jalapeño delivered 1.5 to 1.9 times more AI work per watt at peak throughput and 1.7 to 3.6 times lower end-to-end latency than the comparison systems across those models. In highly interactive workloads, the reported performance advantage ranged from 2.1 to 4.1 times.
Power rating and sustained use
The processor has a published rating of 700 watts. OpenAI says measured sustained power stayed at or below 550 watts during the tested workloads. The company normalised benchmark results using each accelerator’s published chip power rating, arguing that work per unit of power is more useful than a simple per-chip score.
Results varied by model
On GPT-OSS 120B, OpenAI reported roughly 1.9 times higher peak mixed throughput per kilowatt and 1.7 times lower end-to-end latency than the listed GB200 comparison. On DeepSeek R1, it reported about 1.7 times higher peak throughput per kilowatt and 3.6 times lower latency than the GB300 comparison. For Kimi K2.5, the figures were approximately 1.5 times and 3.4 times respectively.
These are OpenAI’s results rather than an independent production audit. Hardware configuration, software optimisation, model precision, prompt lengths and service-level targets can all change benchmark outcomes, so the numbers should be read in their stated context.
How the custom AI inference chip works
Large language model inference has two major phases. During “prefill”, a system processes the user’s prompt and is generally compute-heavy. During “decode”, it generates output token by token and often becomes limited by memory bandwidth. Moving model state between processors can create extra delays.
Jalapeño was designed around the chip, memory, network, serving software and rack-scale system as one platform. OpenAI says model state, including the key-value cache used during generation, can be placed and kept local. A large connected network domain is intended to reduce data movement and keep processors from sitting idle.
This full-stack approach is especially relevant to agentic AI. An agent may plan, call tools, inspect results and revise its work many times. Lower token latency at each stage can produce a noticeably faster completed task, not merely a quicker first response.
Why Jalapeño matters to users and businesses
Consumers should not expect a new device containing the chip. Jalapeño is data-centre infrastructure, so any benefits would arrive through cloud services. OpenAI says it plans to begin deploying the processor in its compute infrastructure by the end of 2026.
Potential practical effects include faster ChatGPT responses, more responsive coding agents, improved service capacity during peak demand and better economics for API workloads. For developers, lower inference costs could make multi-step agents, real-time assistants and high-volume content analysis more viable. Businesses may also benefit if more predictable latency improves customer-facing automation.
Energy efficiency matters beyond speed. AI data centres face power constraints in many regions, and a chip that completes more useful work per kilowatt can increase capacity without requiring a proportional increase in electricity. However, efficiency can also stimulate additional demand, so it does not automatically mean total energy use will fall.
Risks, limitations and open questions
The main limitation is that these are vendor-reported benchmark results from a system still undergoing production qualification. Real services include networking, storage, routing, reliability overhead and changing traffic patterns that a controlled benchmark may not fully capture.
Jalapeño also requires model-specific software work. OpenAI says new model families need fresh kernels and optimisation, although it used Codex with GPT-Astra to bring three unplanned open-weight models to high performance within two months. AI-generated implementations ran 1.5 to 1.8 times faster for selected attention and mixture-of-experts blocks, but that result applied to individual blocks, not complete models.
There is also no guarantee that infrastructure savings will immediately become lower retail prices. OpenAI could use improved economics to reduce prices, increase margins, offer faster service tiers or fund more infrastructure.
What to watch next
The next milestones are production qualification, large-scale deployment and independently reproducible comparisons. Watch for API pricing changes, new latency tiers, capacity improvements and details about how much Jalapeño traffic enters everyday OpenAI products.
OpenAI says the processor starts a multigenerational roadmap: a second generation is deep in development and a third is taking shape. The longer-term question is whether custom silicon becomes a specialised complement to NVIDIA systems or takes a substantial share of OpenAI’s inference workloads.
Conclusion
The first OpenAI Jalapeño chip benchmarks are significant because they target the two constraints users feel most directly: waiting time and limited capacity. Reported gains of up to 1.9 times peak work per watt and up to 3.6 times lower end-to-end latency are promising, particularly for multi-step AI agents. The evidence is still company-reported and production deployment has not begun, but working first-party silicon gives OpenAI more control over the cost and speed of serving its models.