NVIDIA has published early performance results for Vera Rubin NVL72, its next-generation rack-scale AI platform, claiming up to 30 times more agentic AI throughput per megawatt than its GB300 NVL72 system. The company also says token costs can be up to 35 times lower in the tested scenario.
Those are striking figures, but they need context. They come from NVIDIA measurements using a specific agentic coding workload and model, and the Vera Rubin results are still pending review by benchmark developer SemiAnalysis. Even so, the announcement shows where AI infrastructure is heading: away from judging a chip in isolation and towards optimising an entire data-centre system for long-running AI agents.
What is NVIDIA Vera Rubin NVL72?
Vera Rubin NVL72 is a rack-scale computing system designed for demanding AI training and inference. Rather than treating GPUs, CPUs, networking and software as separate products, NVIDIA is building them as one coordinated platform.
The broader Vera Rubin architecture includes Rubin GPUs, Vera CPUs, sixth-generation NVLink switching, BlueField-4 data-processing units, ConnectX-9 networking and Spectrum-X infrastructure. NVIDIA also pairs the hardware with software such as TensorRT-LLM and Dynamo to manage how inference jobs are divided, routed and cached.
The “NVL72” name refers to a scale-up domain that connects 72 accelerators with high-bandwidth, low-latency links. This matters for large mixture-of-experts models and long-context agents because data, model components and cached context must move quickly across the system.
What NVIDIA announced
On 24 August 2026, NVIDIA released measured Vera Rubin NVL72 results using SemiAnalysis AgentX, a benchmark designed to replay production-style coding-agent sessions. Unlike a simple prompt-and-response test, AgentX preserves growing context, tool-call delays, sub-agent activity and varying request lengths.
In NVIDIA’s test with the DeepSeek V4 Pro model, Vera Rubin NVL72 delivered up to 30× higher throughput per megawatt than GB300 NVL72 at the stated interactive-performance target. NVIDIA separately claimed up to 35× lower cost per million tokens.
The comparison is especially notable because GB300 is itself a modern Blackwell-based rack system. NVIDIA says GB300 NVL72 can provide up to 15× the throughput per megawatt of its older Hopper architecture on the same DeepSeek workload.
Why agentic AI changes the infrastructure challenge
A conventional chatbot may answer one question using a relatively short prompt. An AI agent can search databases, call tools, write and run code, delegate work to sub-agents and repeatedly inspect its own results. Context accumulates at every step.
NVIDIA cites OpenRouter data indicating that agentic requests can consume 15 times as many tokens as ordinary chat requests. That makes power consumption, memory movement and context reuse central economic issues, not secondary technical details.
Throughput per megawatt matters
Many data centres are constrained by available electricity long before they run out of demand. If a platform completes more useful agent work inside the same power envelope, a cloud provider can support more customers without securing an equivalent increase in grid capacity.
For Australian businesses, the impact will probably arrive through cloud pricing and service quality rather than direct rack purchases. More efficient inference could make coding agents, research assistants and automated support systems faster and cheaper to operate.
Long context needs smarter caching
Reprocessing an agent’s entire history after every tool call wastes compute. Vera Rubin’s software stack uses distributed key-value caching, cache-aware routing and separate resources for prompt processing and token generation. The aim is to send a request to hardware that already holds useful context and avoid unnecessary recomputation.
Practical impact for developers and businesses
The immediate benefit is not a 30-fold improvement for every application. Results will vary with the model, context length, concurrency, latency target, software and total system configuration. However, the platform points to several practical changes:
- Cloud developers: lower inference costs could make persistent agents and multi-agent workflows more viable.
- Enterprises: faster long-context processing may improve coding, document analysis, customer service and research automation.
- AI providers: higher output per megawatt can improve capacity planning where power and cooling are scarce.
- Infrastructure teams: performance decisions increasingly require evaluating networking, caching, orchestration and energy use—not GPU specifications alone.
Developers should still benchmark their own workload. A short customer-support exchange, a code agent working across a large repository and a research system coordinating several tools place very different demands on infrastructure.
Risks, limitations and unanswered questions
The headline numbers are vendor-reported preview results. NVIDIA states that they were measured with the AgentX workload but are pending SemiAnalysis review. The results also do not yet include Vera CPU performance for tool calling.
The 30× figure applies to a particular DeepSeek V4 Pro test and comparison point; it should not be read as a universal speed-up. Cost estimates also depend on assumptions about system pricing, utilisation, power and operations.
Greater efficiency can reduce energy used per task, but cheaper inference may increase total AI use—a rebound effect that could keep overall electricity demand rising. Data-centre water use, grid connections, hardware supply and electronic waste remain important concerns.
Finally, more capable agent infrastructure does not solve application-level risks. Businesses still need access controls, human oversight, audit logs, data-governance rules and safeguards against agents taking incorrect or unauthorised actions.
What to watch next
The most important next step is independent validation. Watch for SemiAnalysis review, broader model coverage and tests that compare end-to-end agent completion rather than raw token output alone. Real cloud availability and pricing will show whether NVIDIA’s efficiency claims translate into customer savings.
It will also be worth tracking how Vera CPUs and NVIDIA’s Groq 3 LPX inference accelerator contribute to tool execution and low-latency token generation. Together, these components indicate that specialised processors may handle different stages of an agent workflow.
Conclusion
NVIDIA Vera Rubin NVL72 is a sign that agentic AI is reshaping the data centre. NVIDIA’s claim of up to 30× more throughput per megawatt is promising, particularly as power becomes a hard limit on AI expansion. But it remains an early, workload-specific result awaiting external review.
For users and businesses, the real test will be simpler: do AI agents become faster, more affordable and more reliable in production? Vera Rubin’s system-level approach makes that outcome more plausible, but independent benchmarks and real-world cloud deployments will provide the proof.