In addition to Muon, NVIDIA also supports many other optimizers for the research community to explore, including: The ultimate form of orthogonalized optimizer MOP (Momentum Orthogonalized by Polar decomposition) An advanced SOAP variant that updates eigen basis per step with eigen decomposition plus KL correction in REKLS
Advancing Emerging Optimizers for Accelerated LLM Training with NVIDIA Megatron | NVIDIA Technical Blog…It supports open source inference engines including SGLang, TensorRT™ LLM, and vLLM and simplifies the complexities of distributed serving by disaggregating the various phases of inference across different GPUs, intelligently routing requests…
…Intelligent workload placement across distributed sites The NVIDIA AI Grid reference design provides a unified framework for building geographically distributed, interconnected, and orchestrated AI infrastructure. Figure 1 shows how existing network assets…
…GMS restore path with pluggable backends (GDS, UCX, etc), currently gated on pending CUDA driver patch TensorRT-LLM support Multi-GPU and multi-node support via quiesce/resume hooks for PyTorch, NCCL…
…distributions directly. NVIDIA plans to keep working with third parties on bringing AIConfigurator to more systems and tools. AIConfigurator is actively welcoming contributions, including performance data for new hardware, additional backend support…
…Tech Blog Accelerated and Distributed UPF for the Era of Agentic AI and 6G NVIDIA AI Aerial A key enabler for 6G and AI-RAN is the distributed user plane function (dUPF…
…Both LLM training and large-scale video generation have clear long-tail distributions in sequence length. A small fraction of ultra-long samples accounts for a disproportionately large share of the computational…
…The SDK supports a range of operating systems and distributions and includes drivers, libraries, tools, documentation, and example applications. DOCA-Host is the DOCA package for host installation and includes several installation…
…Additionally, CUDA Tile programming is now supported on Compute Capability 9.0 (NVIDIA Hopper) GPUs in addition to all other supported GPU architectures. We are also releasing CUDA Python 1.0, solidifying…
…Slurm 25.11 supports TopologyParam=BlockAsNodeRank with TopologyPlugin=topology/block , ensuring allocations are sorted so applications can discover segments by node rank. Distributed training jobs achieve full NVLink bandwidth across node boundaries…
…The AI framework delivers low-latency, high-throughput, distributed inference for production-grade multi-node AI deployments. Dynamo supports leading open source inference engines, including SGLang, NVIDIA TensorRT LLM, and vLLM. It…