Optimizing Communication for Mixture-of-Experts Training with Hybrid Expert Parallel | NVIDIA Technical Blog
… Low-precision native support for FP8/ BF16 dispatch operators and BF16 combine operators. …
Search
People also ask
What other optimizers does NVIDIA support for research?
In addition to Muon, NVIDIA also supports many other optimizers for the research community to explore, including: The ultimate form of orthogonalized optimizer MOP (Momentum Orthogonalized by Polar decomposition) An advanced SOAP variant that updates eigen basis per step with eigen decomposition plus KL correction in REKLS
Advancing Emerging Optimizers for Accelerated LLM Training with NVIDIA Megatron | NVIDIA Technical BlogHPC-X
… Key Features Offloads collectives communications from MPI onto NVIDIA Quantum InfiniBand networking hardware Multiple transport support, including Reliable Connection RC , Dynamic Connected DC , and Unreliable Datagram UD Intra-node shared memory communication Receive-side tag matching Native suppo… …
Advancing Emerging Optimizers for Accelerated LLM Training with NVIDIA Megatron | NVIDIA Technical Blog
… In addition to Muon, NVIDIA also supports many other optimizers for the research community to explore, including: The ultimate form of orthogonalized optimizer MOP Momentum Orthogonalized by Polar decomposition An advanced SOAP variant that updates eigen basis per step with eigen decomposition plus… …
Scaling Autonomous AI Agents and Workloads with NVIDIA DGX Spark | NVIDIA Technical Blog
… Future releases of cuTile will support fully automated cross-platform autotuning. …
Enhancing Distributed Inference Performance with the NVIDIA Inference Transfer Library | NVIDIA Technical Blog
… UCX is a community-driven networking library and is widely tested internally. …
Powering AI Factories with NVIDIA Enterprise Reference Architectures | NVIDIA Technical Blog
… Maximize uptime and optimize performance with Enterprise Support. …
Achieving Peak System and Workload Efficiency on NVIDIA GB200 NVL72 with Slurm Block Scheduling | NVIDIA Technical Blog
… The software infrastructure needs to evolve to support this new paradigm. The Slurm topology/block plugin provides the foundation and the commitment to continue working with the Slurm community to make it easier to deploy, understand, optimize, and operate at scale. …
Removing the Guesswork from Disaggregated Serving | NVIDIA Technical Blog
… How the SGLang community is contributing Mooncake: Initial SGLang support in AIConfigurator AIConfigurator initially supported only TensorRT LLM, reserving interfaces for SGLang and vLLM without full implementation. …
NVIDIA Isaac Lab-Arena
… Access community benchmarks on common core. …
NVIDIA Jetson Software
… Partner Supported Linux NVIDIA partners offer Enterprise-grade Linux with long-term support for Jetson. …