NVIDIA says Groq 3 LPX is now in full production as part of its Vera Rubin inference platform, positioning the system as a low-latency decode layer for long-context AI agents.
The company published the announcement on August 24 alongside new Vera Rubin efficiency claims. NVIDIA says Groq 3 LPX delivered 3,400 output tokens per second on an Artificial Analysis benchmark using Gemma 4 31B for 100,000-token long-context use cases. It also says the result was four times faster than the nearest alternative platform in that benchmark.
The important shift is architectural. NVIDIA is not pitching agentic inference as one faster chip. It is dividing the work: Rubin GPUs handle large-scale context processing, while LPX accelerates latency-sensitive token generation. That split matches the shape of agent workloads, where long context, tool calls, subagents, and repeated response generation make decode latency visible to users and expensive for operators.
Nebius is the first AI cloud named as adopting Groq 3 LPX. NVIDIA also says CoreWeave has deployed Spectrum-X Multiplane to connect Vera Rubin racks, and SpaceXAI plans to use NVIDIA Vera CPUs for agent orchestration, tool use, code execution, data processing, and simulation.
Agent inference is becoming a power budget problem
NVIDIA’s companion efficiency post makes the economic argument more explicit. The company says agentic AI workloads consume 15 times more tokens than a simple chat request, citing OpenRouter data. That is plausible for multi-step agents because each intermediate result becomes part of the next prompt, but it also means model serving can stop looking like ordinary request-response inference.
NVIDIA says Vera Rubin NVL72 delivers up to 30 times higher throughput per megawatt than GB300 NVL72 on the DeepSeek V4 Pro model in the SemiAnalysis AgentX workload. It also claims up to 35 times lower cost per million tokens than GB300 NVL72. The company notes that the Vera Rubin figures are early and that the results are pending SemiAnalysis review.
Those caveats matter. This is vendor-published performance data, not a neutral buyer benchmark. But the shape of the claim is still useful: the bottleneck NVIDIA wants customers to measure is agentic work per megawatt, not only single-request latency or peak throughput.
For power-constrained AI factories, that metric is not academic. If agents run continuously across coding, research, customer service, and internal operations, the limiting resource becomes how many useful multi-step tasks a data center can serve inside fixed power, cooling, and network budgets.
The decode layer is now a product surface
The LPX announcement is also a sign that decode is becoming independently productized. In classic LLM serving, operators tuned one stack for prefill and generation together. In agentic workloads, those phases behave differently. Prefill can involve large accumulated context. Decode determines how quickly the system returns the next action, command, or answer.
NVIDIA’s read is that pairing Rubin GPUs with LPUs can reduce the tradeoff between speed and throughput. It describes a rack-scale Groq 3 LPX deployment with 256 LP30 accelerators connected through direct chip-to-chip links, designed as a deterministic inference engine for token generation.
That makes the agent stack more modular, but not simpler. Buyers will need to evaluate model behavior, serving topology, network fabric, cache strategy, power management, and token economics together. A fast model on a poor serving path can still be slow. A strong serving stack can still be wasted if the agent harness burns tokens without better task completion.
The practical test is whether cloud customers can see these gains in real workloads, not only in vendor-chosen examples. Coding agents, research agents, and multi-agent customer workflows are good tests because they combine long context, variable output lengths, repeated tool calls, and user-visible latency.
