Nvidia DGX Spark Boosts Open‑Weight LLM Performance, Setting New Benchmarks

The first public benchmark that demonstrated the DGX Spark’s ability to run 200‑billion‑parameter models in a single‑box configuration appeared on October 14, 2025, when Georgi Gerganov posted a detailed performance thread for llama.cpp on the platform. He measured both prefill (pp) and generation (tg) throughput across a range of context lengths and batch sizes, using the llama.cpp CUDA builds together with the llama‑bench and llama‑batched‑bench utilities [report]. His methodology quickly became the de‑facto standard for community testing, establishing a reproducible baseline that other contributors could verify.

By early 2026 the community had formalized its workflow to address the “partial‑flags” problem that had plagued early reports. A shared runtime image, orchestration scripts, and a unified recipe format were codified, culminating in the launch of Spark Arena on February 11, 2026 [report]. Spark Arena now hosts a live leaderboard that aggregates results from multiple nodes and software stacks, allowing direct comparison of inference speed under identical conditions. The current top‑of‑the‑board decode rates are listed in the arena’s public table: gpt‑oss‑120b running on vLLM with MXFP4 precision across two nodes achieves 75.96 tokens per second; Qwen3‑Coder‑Next on SGLang with FP8 precision on two nodes reaches 60.51 tok/s; a single‑node vLLM MXFP4 run of gpt‑oss‑120b records 58.82 tok/s; and NVIDIA’s own Nemotron‑3‑Nano‑30B‑A3B on vLLM with NVFP4 precision on a single node delivers 56.11 tok/s [report].

These figures represent a dramatic leap over the performance of prior desktop‑class AI boxes, which typically required multi‑node clusters to approach 30 tok/s for 120‑billion‑parameter models. The DGX Spark’s unified memory architecture—combining up to 1 TB of high‑bandwidth HBM2e with NVLink‑connected GPUs—eliminates the data‑movement bottlenecks that limited earlier systems, according to the benchmark methodology described by Gerganov. Because the entire model resides in a single address space, the latency of tensor swaps is reduced to a few microseconds, allowing the vLLM and SGLang runtimes to sustain higher decode throughput even at FP8 and MXFP4 precision levels.

The platform’s speed gains are reinforced by NVIDIA’s broader hardware roadmap. Recent Reuters reports note that the company is developing a new AI chip based on the Blackwell architecture, which is expected to further improve FP8 and tensor‑core performance for large language models [Reuters – Exclusive: Nvidia working on new AI chip for China]. While the current DGX Spark leverages the existing Blackwell‑derived GPUs, the upcoming Rubin AI chip—promised to integrate GPU, CPU, and networking functions on a single die—could tighten the compute‑to‑memory pipeline even more, according to NVIDIA CEO Jensen Huang’s comments on the firm’s positioning for the AI shift [Reuters – Nvidia CEO Huang says chipmaker well positioned]. If these next‑generation silicon advances materialize, the token‑per‑second ceiling set by Spark Arena could be pushed well beyond the 80 tok/s range observed today.

The community’s rapid convergence on reproducible benchmarking also signals a maturing ecosystem for open‑weight LLM deployment. Whereas earlier attempts at local inference were hampered by fragmented tooling and inconsistent measurement practices, the Spark Arena framework now provides a transparent, open‑source reference that developers can use to validate performance claims before scaling to production workloads. This transparency is especially valuable as enterprises evaluate the trade‑off between on‑premise inference—offering data‑privacy and latency benefits—and cloud‑based offerings that still dominate the market. In the short term, the DGX Spark’s ability to deliver near‑state‑of‑the‑art throughput for 120‑billion‑parameter models on a desktop‑sized chassis positions it as a compelling bridge for organizations seeking to experiment with large models without committing to multi‑node clusters.