768GB of cheap Intel Optane DIMM memory sticks used to run 1-trillion-parameter LLM on a system with a single GPU — local Kimi K2.5 install achieved roughly 4 tokens per second
…Intel Xeon Gold 6246 CPU Tyan S5630GMRE-CGN motherboard Asus Dual GeForce RTX 3060 OC 12GB GPU 6x 32GB Samsung 2666MHz DDR4 ECC DRAM sticks 6x 128GB Intel Optane DCPMM PC4-2666…
