SLMs are well-positioned for the agentic era because they use a narrow slice of LLM functionality for any single language model errand. LLMs are built to be powerful generalists, but most agents use only a very narrow subset of their capabilities. They typically parse commands, generate structured outputs such as JSON for tool calls, or produce summaries and answer contextualized questions. These tasks are repetitive (up to the differences in prompt payloads), predictable, and highly specialized—well within the scope of specialized SLMs. An LLM trained to handle open-domain conversations is o
How Small Language Models Are Key to Scalable Agentic AI | NVIDIA Technical Blog…Recent results show that GB200 NVL72 delivers significant improvement in performance for all AI workloads, including training ( >2.6x with recent MLPerf training ), across different inference use cases ( real-time inference for…
…Discuss (1) Discuss (1) Tags Agentic AI / Generative AI | General | NeMo | Nemotron | Intermediate Technical | Tutorial | featured | LLMs | Open Source | pandas | Synthetic Data Generation | Training AI Models About the Authors About Alex Steiner…
…Use cases include high-throughput MD simulations of many atomistic systems and agentic AI workflows that require high-throughput asynchronous simulations. Key features of the BMD NIM include: Dynamic batching: Optimize GPU…
…18.75% / 16.6% (unchanged) Compute throughput rebounds to 24.0% Memory throughput improves to 12.9% 3. K-loop split For causal attention , we apply a triangular mask: each query can…
…This overlap dramatically improves GPU utilization and throughput. Scaling even further, driver inference can be batched across many scenes, while multiple rendering processes generate perception inputs in parallel. We provide initial implementations…
…Higher density with overhead Experiment 2 showed that software-level sharing can also improve per-GPU density compared to the baseline, achieving ~0.76 req/s per GPU. However, the CUDA driver…
…We’d love to hear how you’re thinking about disaggregated inference on Kubernetes . Discuss (0) Discuss (0) Tags Agentic AI / Generative AI | Data Center / Cloud | Networking / Communications | General | Cloud Services | Dynamo…
…Enables immediate deployment with ongoing performance improvements over time Turnkey setup : Ships as part of TensorRT LLM with examples and documentation AutoDeploy can be used for: New or experimental architectures : Rapidly deploy…
…Discuss (0) Discuss (0) Tags Agentic AI / Generative AI | Data Science | Edge Computing | Cloud Services | TensorRT | TensorRT-LLM | Advanced Technical | Tutorial | Inference Performance About the Authors About Ruixiang Wang Ruixiang Wang is…
…The remaining 20 GPUs from their quota, plus access to the over-quota pool, are available for post-training jobs when the team occasionally needs to improve their models based on customer…