Advancing Emerging Optimizers for Accelerated LLM Training with NVIDIA Megatron | NVIDIA Technical Blog
… Because whole layers vary in size, each GPU needs to collect differently sized parameter updates from different GPUs through all gatherv . …
Search
Post-Training Quantization of LLMs with NVIDIA NeMo and NVIDIA TensorRT Model Optimizer | NVIDIA Technical Blog
… All the quantized variants of the Llama 3 70B model can be served using only one NVIDIA H100 GPU while the baseline FP16 precision requires at least two GPUs. …
LLM Inference Benchmarking: How Much Does Your LLM Inference Cost? | NVIDIA Technical Blog
… He has contributed to production applications of LLMs covering RAG systems, optimization of inference servers, pretraining of LLMs from scratch, custom evaluation of LLMs, or quantization using FP8 formats. …
MLOps – NVIDIA Technical Blog
… 13 MIN READ Feb 27, 2026 Maximizing GPU Utilization with NVIDIA Run:ai and NVIDIA NIM Organizations deploying LLMs are challenged by inference workloads with different resource requirements. …
How Small Language Models Are Key to Scalable Agentic AI | NVIDIA Technical Blog
… Fine-tuning agility also plays a major role: adding a new skill or fixing a behavior can be done in a few GPU hours on an SLM, compared to days or weeks of fine-tuning for LLMs. …
Automating Inference Optimizations with NVIDIA TensorRT LLM AutoDeploy | NVIDIA Technical Blog
… On a single NVIDIA Blackwell DGX B200 GPU, AutoDeploy performed on par with the manually optimized baseline in TensorRT LLM Figure 4 . …
Pruning and Distilling LLMs Using NVIDIA TensorRT Model Optimizer | NVIDIA Technical Blog
… Distillation takes 8 hours with 96 nodes, each having eight NVIDIA H100 GPUs 6K GPU hours . …
Build a Retrieval-Augmented Generation (RAG) Agent with NVIDIA Nemotron | NVIDIA Technical Blog
… Agentic RAG goes a step further by leveraging autonomous systems integrated with LLMs and retrieval mechanisms. …