… NVIDIA GTC 2026: "GB NVL72 Inference King" slide Unpacking the GTC benchmark At GTC, NVIDIA's cost-per-million-token benchmark used FP4, MTP=3, and March 7 data on DeepSeek 1k/1k: each choice favors NVIDIA's result. …
… At 129 tok/s/user interactivity, the MI355X GPU delivers inference at $0.173 per million tokens with 2,378 tok/s/GPU - 2.9% lower cost than Nvidia B200 TRT-LLM and 1.22x higher throughput per GPU than Nvidia B200 SGLang. …
… Thanks to these solutions, organizations can move faster and get more value from generative AI. …
… Even so, the result is impressive for a first-time AMD effort on a brand-new workload category: The AMD Instinct MI355X platform achieved 93% of NVIDIA B200 single-node performance and 87% of NVIDIA B300 single-node performance in Single Stream. …