Nvidia launches NVFP4 to boost low‑precision inference efficiency and accuracy

NVFP4 arrives as the first 4‑bit floating‑point format that actually scales with the size of modern models. NVIDIA’s Blackwell Tensor Cores apply a two‑level micro‑block scaling scheme: each 16‑value block receives a fine‑grained E4M3 factor, while a single FP32 scalar is applied to the whole tensor. According to the NVIDIA Technical Blog, this “high‑precision scale encoding” lets the format retain the dynamic range of FP8 while using only a quarter of the memory of FP16, cutting memory bandwidth demands by up to 4×. The result is a quantization pipeline that avoids the “noticeable accuracy drop” that typically plagues ultra‑low‑precision inference, especially for larger language models that are sensitive to rounding errors.

The architecture builds on earlier 4‑bit schemes such as FP4 (E2M1) and MXFP4, but adds a shared FP8 scale per 16‑value block—a detail highlighted in NVIDIA’s comparison table. While FP4 and MXFP4 both rely on a single power‑of‑two scale per 32‑value block, NVFP4’s per‑micro‑block scaling reduces the quantization error that can accumulate across a tensor. The blog notes that the format’s value range spans roughly –6 to +6, with representable steps like 0.0, 0.5, 1.0, 1.5, 2, 3, 4, and 6, giving developers a predictable numeric envelope for inference workloads.

Performance gains are evident across NVIDIA’s GPU generations. Figure 1 in the blog shows that Blackwell’s fifth‑generation Tensor Cores deliver the highest peak throughput for 4‑bit formats, outpacing Hopper and Ampere on both dense and sparse kernels. By supporting FP4, MXFP4, and now NVFP4, Blackwell gives developers a menu of precision‑cost trade‑offs, but the blog positions NVFP4 as the sweet spot for “best accuracy at ultra‑low precision.” In practice, this means that inference pipelines can run at FP4 bandwidth while preserving the quality of FP8‑level results, a claim that could reshape how cloud providers price AI services.

Beyond raw numbers, NVFP4’s design reflects a broader shift toward “micro” floating‑point formats that are purpose‑built for inference rather than training. The Technical Blog stresses that quantization remains the most common model‑compression technique because of its post‑optimization flexibility and framework support. However, the traditional FP4 format has been hampered by a “risk of noticeable accuracy drop compared to FP8,” a drawback that NVFP4 mitigates through its dual‑scale architecture. By delivering “lower risk of noticeable accuracy drop particularly for larger models,” NVIDIA aims to make 4‑bit inference viable for production‑grade workloads that previously required at least INT8 or FP8.

Industry observers have already linked NVFP4 to NVIDIA’s upcoming RTX 50‑series launch, which TechCrunch reports will feature the Blackwell Ultra GPU family. While the Verge notes that the RTX 5090 and RTX 5080 announcements were delayed, the underlying hardware roadmap remains focused on expanding low‑precision capabilities. If NVFP4 lives up to its promise, developers could see a new generation of AI services that run faster, cost less, and consume a fraction of the memory footprint of today’s FP16‑based deployments—potentially accelerating the rollout of on‑device AI and edge inference where bandwidth and power are at a premium.