Tracked topic
…9 MIN READ Feb 02, 2026 Optimizing Communication for Mixture-of-Experts Training with Hybrid Expert Parallel In LLM training, Expert Parallel (EP) communication for hyperscale mixture-of-experts (MoE) models is…
…9 MIN READ Feb 02, 2026 Optimizing Communication for Mixture-of-Experts Training with Hybrid Expert Parallel In LLM training, Expert Parallel (EP) communication for hyperscale mixture-of-experts (MoE) models is…
…9 MIN READ Feb 02, 2026 Optimizing Communication for Mixture-of-Experts Training with Hybrid Expert Parallel In LLM training, Expert Parallel (EP) communication for hyperscale mixture-of-experts (MoE) models is…
…9 MIN READ Feb 02, 2026 Optimizing Communication for Mixture-of-Experts Training with Hybrid Expert Parallel In LLM training, Expert Parallel (EP) communication for hyperscale mixture-of-experts (MoE) models is…
…9 MIN READ Feb 02, 2026 Optimizing Communication for Mixture-of-Experts Training with Hybrid Expert Parallel In LLM training, Expert Parallel (EP) communication for hyperscale mixture-of-experts (MoE) models is…
…Dynamo, NVFP4, optimized CUDA kernels, advanced parallelization techniques, and beyond. Build with NVIDIA GPU-accelerated endpoints Developers can start building with DeepSeek V4 through NVIDIA GPU-accelerated endpoints on build.nvidia.com…
…optimized model architectures and utilities that simplify the implementation of distributed training (both data parallel and domain parallel). By abstracting away the details of training models at scale, PhysicsNeMo enables developers and…
…It allows multiple virtual GPUs (vGPUs) and, consequently, virtual machines (VMs) to run in parallel on a single GPU, while providing the isolation guarantees that vGPUs offer. The capabilities provided by MIG…
…A multi-agent harness fans out work across parallel subagents with short, independent contexts. Default round-robin routing is blind to both patterns — it cannot account for cache locality, request priority, or…
…Retrieve and act on the solution cuOpt executes the optimization on the GPU using massive parallelism to evaluate the solution space faster than traditional CPU solvers. Once a solution is found, the…