… Salvatore echoed Huang’s argument made at GTC, that Nvidia’s platforms – while expensive – improves token generation. “Increases in token generation or increases in performance basically generate more revenue, they reduce costs, they get you more value from the same infrastructure,” he said. “This … …
… Which arguably should be called the Vera-Rubin-Groq platform, given that Huang said during his keynote that low latency, premium priced token generation should represent somewhere on the order of 25 percent of the compute in an AI cluster. …
… That revenue that OpenAI and Anthropic are chasing requires token chewing and token spitting. And the top brass at Anthropic are wracking their brains trying to figure out how to get the money to get the infrastructure to meet future token demand. …
… This is true given the current state of AI and the current cost of processing tokens. No one said it would remain true forever. …
… Nvidia developers addressed the typical tradeoffs when balancing efficiency and accuracy through architectural changes, including using a latent mixture-of-experts that call four times as many specialists for the same inference costs by compressing tokens before they reach the experts and multi-tok… …
… They care what it costs so they can justify the budget, but they are not afraid to pay a premium as long as it is not too much. This is how IBM mainframes and Power Systems machines still generate maybe $25 billion a year in revenues even in 2026. …
… In any event, in the table above, output speed is expressed in tokens per second, and the time to first token TTFT is expressed in seconds. The two smaller Llama 4 models beat OpenAI’s GPT 4o in terms of smarts by a little bit and cost 21X to 31X less per million tokens processed. …
… The second workload is called decode, and it means instead of chewing on tokens to either understand a query and its context or trying to find the statistical graph of linkages between tokens in a corpus of data, the system is spitting out tokens for the answers to a query in whatever modality the … …
… So far, only eyebrows have been raised. So, SambaNova now has $350 million in cash and a revised architecture that it says can deliver the kind of tokenomics that agentic AI is going to require, besting Nvidia’s GPU behemoths. …
Compute Taalas Etches AI Models Onto Transistors To Rocket Boost Inference Adding big blocks of SRAM to collections of AI tensor engines, or better still, a waferscale collection of such engines, turbocharges AI inference, as has been shown time and again by AI upstarts Cerebras Systems, SambaNova … …