I replaced cloud LLMs with local models running off a Proxmox LXC, and the performance trade-off was worth it
… Related Your old GPU can still run big LLMs – you just need the right tweaks There's a lot you can do with these models Proxmox LXCs are incredible for hosting llama.cpp With some GPU passthrough wizardry, I can put my old graphics cards to good use Like most LLM-hosting enthusiasts, I started my j… …