I tested Nvidia's flagship GPUs for gaming, and the RTX 5090 wasn't the winner
…pay for the extra fps, but with GPU prices being what they are now, the gap is much smaller than it was at launch for the $10,000 Pro graphics card. Related…
Search
Related topics:
LLMs
All sources
xda-developers.com
39
developer.nvidia.com
32
blogs.nvidia.com
14
amd.com
7
theregister.com
3
phoronix.com
2
wccftech.com
2
press.asus.com
2
huggingface.co
2
pcgamer.com
2
extremetech.com
1
cnx-software.com
1
Videos
More videosThis hidden Proxmox setting may sound cursed, but it’s really useful for coding and DIY projects
…But I occasionally need to work with AI-accelerated workloads on my dev VM. Since I’ve already enabled GPU passthrough long ago (which is a lot easier than you think), I…
Home Assistant's local LLM support outperforms Gemini for Home, and Google knows it
…You’re at the mercy of the model Google chooses and wait for an update if it falls short. Running LLMs locally frees you from those constraints — of course, you still need…
I use this local AI tool to turn boring documents into cool narrations
…Sign in to your XDA account I recently started integrating local LLMs with my arsenal of free and open-source tools, and they’ve been a game-changer for my productivity needs…
Top stories
I built my own Googlebook with a Raspberry Pi, local LLMs, and old hardware
1w agoI added a second GPU just for local AI workloads, and it cost less than upgrading my main one
1w ago13 years later, the GTX Titan is still the most important GPU Nvidia ever made
1w agoMy RTX 5090 can't keep up with Apple Silicon on the biggest local LLMs, and I hate to admit it
1w agoASUS Unveils Infrastructure Breakthroughs at SC24
…NVIDIA HGX H100 GPUs, totaling 1,024 GPU cards. Next-Gen Infrastructure Solutions ASUS is advancing digital transformation with HPC and AI-driven server systems for diverse enterprise needs, including the ASUS…
Contemplating Meta’s Homegrown MTIA Compute Engine Roadmap
…The reason why the future MTIAs as well as the current MTIA 300, which has been deployed for R&R training workloads, need to look like GPUs and AI XPUs because they…
Post-Training Quantization of LLMs with NVIDIA NeMo and NVIDIA TensorRT Model Optimizer | NVIDIA Technical Blog
…With PTQ, models can be served more efficiently using fewer GPUs. Summary This post showed you how to use PTQ in NeMo to build efficient TensorRT-LLM engines for LLM deployment. For…
Discussions and forums
My first 10 inch rack with local LLM! No more Spotify, Google Home, Netflix, ChatGPT...
I'm pretty new to homelabbing and this is my first mini rack! Started with the Beelink ME Mini and then just kinda grew from there (it's always the way hey haha). It idles at 70 watts (not too shabby for how much is goin…
Computer build using Intel Optane Persistent Memory - Can run 1 trillion parameter model at over 4 tokens/sec
As the title states, my build is indeed able to run a 1 trillion parameter model (in this case Kimi K2.5) locally at ~4 tokens/second. I thought r/LocalLLaMA would be interested in the build due to that stat line, and al…
80 tok/sec and 128K context on 12GB VRAM with Qwen3.6 35B A3B and llama.cpp MTP
Just wanted to share my config in hopes of helping other 12GB GPU owners achieve what I see as very respectable token generation speeds with modest VRAM. Using the latest llama.cpp build + MTP PR, I got over 80 tok/sec w…
2.5x faster inference with Qwen 3.6 27B using MTP - Finally a viable option for local agentic coding - 262k context on 48GB - Fixed chat template - Drop-in OpenAI and Anthropic API endpoints
2026-05-07 edit: I have updated the hardware based recommendations with more focus on quality. I do not recommend q4_0 KV cache anymore beyond 64k context. After multiple rounds of testing with the different size quants,…
services with actually generous free tiers for open-source projects. my list, what would you add?
Been in the weeds shipping an OSS side project for the past few weeks (social media publishing API). Real launch post is coming, this isn't that. Along the way I kept a list of services that actually have usable free tie…
Deploying Disaggregated LLM Inference Workloads on Kubernetes | NVIDIA Technical Blog
…Better GPU utilization: Separating stages lets each saturate its target resource (compute for prefill, memory bandwidth for decode) rather than alternating between both. Frameworks like NVIDIA Dynamo and llm-d , implement this…
I built a local LLM server I can access from anywhere, and it uses a Raspberry Pi
…Related I ran local LLMs on a "dead" GPU, and the results surprised me My Pascal card may not be ideal for intensive workloads, but it's more than enough for light…
Ollama is still the easiest way to start local LLMs, but it's the worst way to keep running them
…Available VRAM on the GPU D Internet bandwidth Spot on! VRAM is the key bottleneck for local LLM inference. If a model fits entirely in your GPU's VRAM, it runs dramatically…