r/pcmasterrace • u/Bobert25467 • Mar 27 '26

News/Article Google's new AI algorithm might lower RAM prices

Post image

https://arstechnica.com/ai/2026/03/google-says-new-turboquant-compression-can-lower-ai-memory-usage-without-sacrificing-quality/

42k Upvotes

92% Upvoted

2.1k comments sorted by

View all comments

Show parent comments

38

u/clyspe Mar 27 '26

Some rough numbers for people who don't run LLMs themselves: on long context, weights are ~5/8 of the memory usage for me, context is ~3/8 (128k context). So the 3/8 is what's going down in size. As we go up in context length, the size required increases linearly, so as we get more capable models, this advantage is going to grow.

1

u/ElementNumber6 Mar 28 '26

Then there's attention span to consider. You can provide large focus, but the llm may not consider everything you give it every time, and may decide to focus more in one area, or another, subsequent times, completely ignoring everything else, leading to very different conclusions.