In addition to Muon, NVIDIA also supports many other optimizers for the research community to explore, including: The ultimate form of orthogonalized optimizer MOP (Momentum Orthogonalized by Polar decomposition) An advanced SOAP variant that updates eigen basis per step with eigen decomposition plus KL correction in REKLS
Advancing Emerging Optimizers for Accelerated LLM Training with NVIDIA Megatron | NVIDIA Technical BlogDeveloper Tools & Techniques CUDA 13.2 Introduces Enhanced CUDA Tile Support and New Python Features Mar 09, 2026 By Jonathan Bentz Discuss (0) Discuss (0) L T F R E AI-Generated…
…The llama-nemotron-embed-vl-1b-v2 model supports three input modes—text-only, image-only, and combined image and text—allowing you to index everything from plain documents to slide decks…
…support for more distribution functions (DF) (spot EDF, measured EDF, measured BSDF) has been added to the PTX, Native x86 and LLVM IR backends. For a complete list of changes and fixes…
…DROID Distributed Robot Interaction Dataset: a large real-world manipulation dataset with more than 50k demonstrations across varied tasks, collected using Franka Panda robot arms. See the DROID paper . RoboArena A distributed…
…If Codex can’t resolve a model ID into the intended profile, fallback defaults may change truncation, search-tool availability, verbosity controls, reasoning-summary support, and parallel tool-call support before Dynamo…
…The latest features of the open source NVIDIA TensorRT-LLM inference serving software and the NVIDIA Dynamo open source distributed inference serving framework were used to support the newly added and more…
…Yifei focuses on end-to-end training optimization, distributed model parallelism, and MLIR compiler infrastructure for deep learning systems. View all posts by Yifei Song View all posts by Yifei Song About…
…Nemotron Personas Datasets Fully synthetic, privacy-safe personas are grounded in real-world demographic, geographic, and cultural distributions. Part of NVIDIA’s growing global collection for Sovereign AI development, featuring datasets for…
…Enterprise-grade security controls and container signing for deployments Enterprise support: NVIDIA support and maintenance for production deployments NIM standardizes the deployment layer, but maximizing GPU utilization requires intelligent orchestration. This is…
…Open source reference implementations and custom CUDA kernels (dl-lowlat-infer) provide reproducible, architecture-agnostic low-latency inference pipelines for financial time series workloads, supporting deployment in both traditional data centers and…