r/pcmasterrace • u/Bobert25467 • Mar 27 '26

News/Article Google's new AI algorithm might lower RAM prices

Post image

https://arstechnica.com/ai/2026/03/google-says-new-turboquant-compression-can-lower-ai-memory-usage-without-sacrificing-quality/

42k Upvotes

92% Upvoted

2.1k comments sorted by

View all comments

12k

u/Vogete Mar 27 '26

so now we're just gonna get LLMs 6x the size for the same memory usage

4.2k

u/maxneuds Linux Gaming Mar 27 '26

But 8x faster. That's probably what will happen.

2.3k

u/TheHuntedShinobi Mar 27 '26

“16x the detail” -Todd Howard

875

u/Kazu88 Desktop Mar 27 '26

"It just works"

323

u/Crazy_Asylum Mar 27 '26

the more you buy, the more you save.

129

u/tarchival-sage RTX 5090 Aorus Master | 9800x3D | Aorus Master x870E Mar 27 '26

Look at my jacket

64

u/Thee_Sinner R5 3600 4.2GHz, Sapphire 5700XT 2115MHz, 32GB DDR4 3600 CL14 Mar 27 '26

I shipped my pants

8

u/xXBeefSquatch5KXx Mar 27 '26

Ship my pants? Right here!? I JUST SHIPPED -THE BED-

Fun fact the district manager in the show super store is in that commercial and it’s like he’s playing the same guy lol

3

u/ReadyAimTranspire Mar 27 '26

You can afford pants in this economy?

2

u/De4con Mar 27 '26

Shipping your pants costs ~8% extra now.

→ More replies (3)
→ More replies (1)

2

u/UninitiatedArtist Mar 27 '26

And this canvas bag…that you definitely will never get unless you’re an influencer.

→ More replies (2)

60

u/GregTheMad Ryzen 9 7900X, RTX 2080, 32GB Mar 27 '26

"You see that image? You can slop it."

17

u/ReadyAimTranspire Mar 27 '26

6X-8X sloppage

4

u/RemoteButtonEater Mar 28 '26

We're giving you more slop per slop, and that's value passed on to the consumer.

24

u/KernelERROR Mar 27 '26

“WHOS LAUGHING NOW?!!….. yes I was in the chess club 👉👈😳”

14

u/Bignuka Mar 27 '26

Country roads, take me home!

→ More replies (1)
→ More replies (9)

5

u/LucidFir Mar 27 '26

6x8 is like 48... !!!

2

u/omegaweaponzero Mar 27 '26

As much as people quote this as some type of "gotcha", that game did have 16x the detail. Fallout 76 had better draw distances than Fallout 4 and could load in more assets at once.

2

u/SirArkhon Mar 27 '26

It’s the same thing with “it just works”. Todd was talking about the settlement building mechanic with its snapping features and specifically the way players could hook up power.

2

u/I-MnUbee Mar 27 '26

More like, "we're going to charge 16x the retail"

→ More replies (10)

131

u/RUBSUMLOTION 9800X3D | RTX 5080 Mar 27 '26

“12 billion planets! All unique.” - Todd Howard

107

u/cantadmittoposting Mar 27 '26

"12 billion planets! All covered in data centers" - Todd Howard announcing the compute power required to actually release another Elder Scrolls game.

34

u/Mimical Patch-zerg Mar 27 '26

Another Elder Scrolls game

You mean Skyrim 128 bit Fus-Ro-Dah Remastered Edition

2

u/turbospeedsc Mar 27 '26

the way we are going, we are going end like this, instead of saying, hey guys do you notice now there is enough for everyone and we could like enjoy the world.........

Nop, shareholders need their second bathroom renovation for the pool house in their 4th vacation home, last time it was redone was in 2021, over 5 years now!!!

2

u/SvenniSiggi Mar 27 '26

"Wow, 12 billion planets. so...all randomly generated and not having much of interest except on a few of them." -me

→ More replies (2)

93

u/Journeyj012 Year of the desktop Mar 27 '26

it's 33% faster since we scaled up by 6x.

36

u/MrV705 Mar 27 '26 edited Mar 27 '26

Original speed -> X Original size -> Y

{Apply algorithm} New speed -> 8X New size -> Y/6

Make it 6 times bigger New new speed -> 48X New new size -> Y

It's now 4800% what is was before (in the speed department).

Edit: This, of course, assumes many things, among others: that this information is actually true, that the speed keeps the same rate if the model is scaled in size, that the bubble doesn't collapse (sincerely hope it does).

4

u/entropicdrift i7 3770K, GTX 1080, 16GB DDR3 Mar 27 '26

The speed inscrease is at the same number of parameters, as is the size decrease. If we 6x the parameters we slow down significantly, perhaps more than the 8x speedup. The size to speed correlation is not linear when it comes to computer hardware.

→ More replies (1)
→ More replies (18)

716

u/GroundbreakingMall54 Mar 27 '26

honestly yeah thats exactly how it works every time. SSDs got bigger so games went from 50gb to 200gb, monitors got better so we need beefier GPUs... its just the circle of life but for hardware requirements

460

u/I_Dont_Think_Im_AI Mar 27 '26

Yes, but also no. 8k tvs have been being made, but manufacturers basically just said, "No one's buying" and have stopped making them.

LG Stops Making 8K TV Panels As Next-Gen Tech Slowly Fizzles Out | PCMag

There is a point when the gains just don't make sense anymore.

232

u/JarvisIsMyWingman Mar 27 '26

No 8K content, where's the need other than bragging rights.. How did they not see this coming? /s

154

u/Alternative_Wait8256 Mar 27 '26

Streaming services are giving worse and worse quality they won't be providing 8k unless you pay a massive premium I suspect.

No one owns media anymore so good luck buying 8k content.

112

u/theblackyeti Mar 27 '26

I own media. Am I suffocating in a pile of blu-rays and 4ks? Absolutely and I fucking love it.

69

u/DogadonsLavapool 9070XT|7700x and MBP Mar 27 '26

For real. Not having crunchy squares during darker scenes is peak. Ripping to a jellyfin servers is pretty damn easy too.

27

u/nalaloveslumpy Mar 27 '26

Look at Mr. I'm made of SSDs over here....

28

u/DogadonsLavapool 9070XT|7700x and MBP Mar 27 '26

Lmao I was buying that stuff when it was cheap. I've got 20tb of extra space

41

u/nalaloveslumpy Mar 27 '26

Hey, uh, I need your address for completely non-burglary related reasons.

12

u/PaulTheMerc 4790k @ 4.0/EVGA 1060/16GB RAM/850 PRO 256GB Mar 27 '26

Datahoarders call for aid, will you answer?

→ More replies

16

u/EnjoyerOfBeans Mar 27 '26

Modern HDDs are easily good enough to stream movies from.

→ More replies (7)

2

u/Jon_TWR R5 5700X3D | 32 GB DDR4 4000 | 2 TB m.2 SSD | RTX 4080 Super Mar 27 '26

Nah, media servers are almost always spinning rust.

→ More replies (4)

2

u/maxiligamer RTX 3060 12GB, Ryzen 5 5600, 32GB 3200MHz Mar 27 '26

Recently started using Jellyfin and I wish I started this shit earlier. I have to only watch a couple series at a time due to storage, would have liked to start this back when SSDs were almost free

→ More replies (4)

33

u/SaintTastyTaint Mar 27 '26

Even a standard 1080p bluray looks and sounds so much better than streaming to me

9

u/Alternative_Wait8256 Mar 27 '26

Absolutely it does,

2

u/AlphaSpellswordZ Fedora | 32 GB DDR5 | R7 7700X | RX 6750 XT Mar 28 '26

They are objectively better.

18

u/anr4jc Mar 27 '26

People swear by streaming but when you see a true Blu-ray disc with a high bitrate, the difference in picture quality is insane

8

u/Alternative_Wait8256 Mar 27 '26

Agreed, streaming is crap compared to a blu ray

→ More replies (5)

2

u/LimoncelloFellow Mar 27 '26

My pile became more manageable when i ditched all the cases for most of my collection and acquired a few disc binders.

2

u/corndogs88 Mar 27 '26

Physical Media Gang rise up!

→ More replies (2)

35

u/SlideJunior5150 Mar 27 '26

4k streaming compression is like 720p dvd quality. 1080p now looks like 480p, the compression is ridiculous.

25

u/Local_Band299 R7-8700F|32GB-DDR5-7200MTs|RX9060XT-16GB Mar 27 '26

Lossless audio makes a huge difference as well. Compared Pacific Rims 4KBD Atmos to Amazon Primes Atmos. The 4KBD had more depth to it. More bass amd dynamics.

10

u/Dt2_0 Mar 27 '26

Part of the problem is people don't know what they are missing with bass. Everyone things more rumble and shake=better bass. That's not really true. Rumble happens generally between 80z and 120hz. It's the sub bass, everything below 80hz that sounds amazing on a fully uncompressed track. Below 80hz, the sound waves are larger than the space between your ears, so you can't tell where the sound is coming from, this creates a feeling of being engulfed in the sound that you just cannot get with more compressed audio tracks.

2

u/Local_Band299 R7-8700F|32GB-DDR5-7200MTs|RX9060XT-16GB Mar 27 '26

Friend has a Klipsch set up with an Onkyo reciever. It's a 3.0.2 set up. FL, Center, FR, with 2 atmos heights in the Front L&R towers. It fucking booms. He was afraid we would have the cops called in us when we were watching my 4KBD copy of Pacific Rim. He was thinking about getting a sub, but it might be too much bass.

4

u/Dt2_0 Mar 27 '26

If you get a good sub, with isolation feet to keep it away from the floor, you can easily run a sub without shaking the house down. Target a crossover at 80hz. Should give you all the punch you need with none of the super heavy rumble. Do the Sub Crawl to properly position it and let the receiver calibrate the sub.

I'd recommend something like a RSL Speedwoofer 10E or 10S. If they want to be really sure they aren't going to cause any shake, a sealed sub like a SB-3000 Micro from SVS is a great option too.

3

u/nongrammatical Mar 27 '26

TrueHD ftw

3

u/Dt2_0 Mar 27 '26

Atmos (full uncompressed) and TrueHD are the same quality. Atmos is object based and your receiver does a lot of the processing on where the sound actually goes.

TrueHD says A sound plays in B channel.

Atmos says X sound is created by an object at room coordinates X/Y/Z, the receiver goes okay Channel A play sound at 60% volume, Channel F play sound at 100% volume, Channel D play sound at 30% volume.

Its really awesome when you look at how it actually works in a properly calibrated room with a 11 channel 7.x.4 setup. The issue is Atmos implementations. Some are better than others. Soundbars are usually terrible at it, as they try to reflect sound off the walls to emulate speaker placement, and most don't offer proper calibration suites. I find it extremely overrated in headphones as well from personal experience.

Atmos is basically TrueHD, but with 3D sound source positioning. And it is awesome when implemented properly and when playing uncompressed audio tracks.

2

u/ResoluteGreen Mar 27 '26

I...don't think this is true? Atmos is object based audio as you describe, but TrueHD is just the quality or compression of it. You can have Atmos on both Dolby Digital Plus or TrueHD.

→ More replies
→ More replies (7)

2

u/Farranor ASUS TUF A16... 1 year of hell Mar 27 '26

Commercial DVD video is usually 480i, not 720p, with awful MPEG2 compression at around 10Mb/s. 480p in a modern format looks much better than DVD at a fraction of the bitrate. Even YouTube at 480p looks better than DVD most of the time (complex scenes can hit their bitrate cap).

17

u/JarvisIsMyWingman Mar 27 '26

Actually I own physical media. Too many after the fact "edits" with streaming providers, and just random quality levels of streaming. Or the fact that stuff just disappears from all platforms.

5

u/aVarangian 13600kf 7900xtx 2160 | 6600k 1070 1440 Mar 27 '26

8k streaming but with just enough bitrate that it'd look good at 720p

6

u/Cinderstrom Mar 27 '26

H a h a yes. Buying.

2

u/ncocca Mar 27 '26

It's insane that some major sporting events or even regular shows are still being broadcast at 720p.

→ More replies (5)

78

u/zgillet i7 12700K ~ PNY RTX 5070 12GB OC ~ 32 GB DDR5 RAM Mar 27 '26

Even with the content, it's just not worth it until you are nearing theater-size screens.

I've always said the high PPI mobile screens are basically snake oil after a certain point.

15

u/TransBrandi Mar 27 '26 edited Mar 27 '26

My understanding is that a lot of editing for movies is done with 2K masters, so many of the 4K movies are upscalled from 2K. I'd imagine that upscaling all the way to 8K would not look great, and even if this doesn't affect more recent productions older movies will still hit that limit. If they were ever digitized to be edited (rather than splicing film) they would have to be re-edited rather than just rescanning film.

edit: Someone commented by pointing out that 2K masters were fine in the past due to constraints on computing power for sfx and only targetting 1080p. They deleted their comment, so I'm adding this here.

IIRC Blade Runner 2049 was mastered in 2K, so that's a lot of movie history (2017 and backwards) that's stuck in that even if that was the final movie to ever be edited in 2K.

4

u/Ultrace-7 Mar 27 '26

Older movies (35mm), if rescanned specifically for the purpose, can go to 8k digital with stunning results. It takes very efficient scanners and is a time consuming process, which means it would happen rarely unless the studios thought the result would be worth the cost, but it can definitely be done.

→ More replies (1)
→ More replies (2)

5

u/JarvisIsMyWingman Mar 27 '26

Agreed, I just want cheaper and bigger 4K please.. I got a nice theater at home, and almost got my popcorn to Alamo standard to make it perfect!

2

u/Nope_______ Mar 27 '26

How do you do your popcorn?

3

u/JarvisIsMyWingman Mar 27 '26

West Bend Stir Crazy Popper

Yellow Popcorn by Great Northern Popcorn

Golden Barrel Butter Flavored Coconut Oil

Flavacol Popcorn Seasoning Salt

Butter

Main trick is the right amount of the seasoning salt and butter. We use regular cooking oil for when we have people with coconut allergies and adjust the butter accordingly.

I'm amused at what subreddit this is being discussed under :)

2

u/zgillet i7 12700K ~ PNY RTX 5070 12GB OC ~ 32 GB DDR5 RAM Mar 27 '26

I'm drooling onto my desk at work.

Popcorn is a universal language.

2

u/zgillet i7 12700K ~ PNY RTX 5070 12GB OC ~ 32 GB DDR5 RAM Mar 27 '26

4k or even 2K projectors at 60 Hz minimum need to get cheaper. That's when you really need the pixel density.

5

u/Spork_the_dork Mar 27 '26

Yeah I literally cannot see the pixels on my 1440p phone screen even when I try. Anything beyond that is completely pointless to me.

7

u/Alternative_Wait8256 Mar 27 '26

Very true 4k and 8k at 60in and below.. you won't notice it.

17

u/RichtofensDuckButter Mar 27 '26

I don't know what you're saying. You can absolutely notice the difference in pixel density between a 60-in 4K and a 27-in 4K.

9

u/Alternative_Wait8256 Mar 27 '26

Sorry I meant at 8k

3

u/RichtofensDuckButter Mar 27 '26

That makes sense. Definitely diminishing returns there.

2

u/pudgylumpkins PC Master Race Mar 27 '26

At living room viewing distances though? I know my vision isn’t good enough to resolve detail like that.

→ More replies (4)

2

u/F9-0021 285K | 4090 | A370m Mar 27 '26

Maybe if you don't have good eyes, but Apple makes retina displays for a reason. 100ppi might cut it for gaming, but not for all use cases.

→ More replies (3)

2

u/MoistSystem1323 Mar 27 '26

Which exactly what I want it for but without the content there's no point. And I'm not paying over $20k for a screen

2

u/Saedeas Mar 27 '26

Ultra high pixel density matters a lot for VR headsets (many of which just use phone screens), but that's about the only use case I can think of.

3

u/zgillet i7 12700K ~ PNY RTX 5070 12GB OC ~ 32 GB DDR5 RAM Mar 27 '26

This I can agree with. Foveated rendering is the real key to resolution in VR since a massive chunk of the screens aren't being looked at. Not much you can do with a TV multiple people are watching.

→ More replies (1)
→ More replies (5)
→ More replies (17)

32

u/Blaze_Vortex Mar 27 '26

There is also the point when people just aren't buying anymore. 8K TVs are stupid expensive.

15

u/llDS2ll Mar 27 '26

cries in 3D TV

5

u/happyinheart Mar 27 '26

I have a 12 year old 3D TV. I just bought a 3D Blue Ray movie to see if I can get it to play in 3D on my TV through a PS4.

2

u/PaulTheMerc 4790k @ 4.0/EVGA 1060/16GB RAM/850 PRO 256GB Mar 27 '26

with or without glasses?

2

u/Downtown_Recover5177 Mar 27 '26

Crying tears of joy? I’m so glad the 3D fad is finally gone. Fucking nauseating.

4

u/Late-Combination5060 Mar 28 '26

3d or VR? I never once heard 3d glasses to be nauseating 

6

u/OfficialXstasy Mar 27 '26

Yeah, and good luck trying to find content for it.

→ More replies (2)

21

u/funlovingmissionary Mar 27 '26

Yes, but this is not one of those. Bigger models are still better, and we haven't reached a state of "good enough" with ai, like we did with 4k tvs.

→ More replies (6)

35

u/zzazzzz Mar 27 '26

thats more about timing than anything. there is no content in 8k. the internet infrastructure couldnt handle streaming 8k content even if it did exist and then there is no hardware to play any games in 8k either so all in all the usecase is just non existent.

37

u/[deleted] Mar 27 '26

[deleted]

6

u/zzazzzz Mar 27 '26

8k is relevant for massive displays. obviously an 8k phone or home tv is nonsense. but at massive size the human eye can very much tell the difference.

and bitrate is just a streaming issue, bluerays are still so high in bitrate it might as well be raw from a picture quality standpoint. and framerates for movie content is limited by the directors choice not really because of technical limitations. most just want to be at 24frames.

and when we talk about streaming, you will see neither improve greatly just by nature of increased cost. already today most streaming services bitrate/resolution are abysmal worse than even years ago, because its way cheaper and most ppl are watching on their phones either way or dont really notice/care.

2

u/kominik123 Mar 27 '26

Massive display at 8K is great for Linus Tech Tips, not ordinary Joe. Common folk doesn't have a home theater where a 40°+ field of view immerses you into the story.

Bitrate IS streaming issue. Home internet connection speed is increasing while the content provider bitrate is stagnating. Hopefully new codecs will bring higher quality in the same bitrate and infrastructure load.

27

u/dragonbud20 i7-5930k|2x980 SC|32GB DDR4|850 EVO 512GB|W8.1 Mar 27 '26

Honestly, huge portion of people can't even tell the difference between 1080p and 4K.

Are you talking about screens over 30 inches or under? At over 30 inches, I would tell anyone who can't see the difference between 1080p and 4k to go to an optometrist and get their eye checked. I agree with you that the difference quickly becomes irrelevant on smaller screens.

21

u/TransBrandi Mar 27 '26

Distance from the screen is also an important factor.

13

u/kominik123 Mar 27 '26

Screen size is not that much relevant to the situation, because you usually watch the big screen from further away than the small screen. You don't want to watch 65" TV from 1 meter (3ft) - sure it's easy to spot the difference in pixel density, but you'll break your neck and burn your eyes.

Yes, everyone has a different size to distance ratio but for example my mother has 60" at 2,5m (about 8ft) and in that distance, it's hard to spot the difference. Another example: monitor at work. I have 27" at 1440p and believe there's no point in going 4K.

Of course, when you work with visuals, and there are many other usecases, you absolutely want and need higher density. But watching Netflix, like a huge portion of people do? That's why i said "normal size in normal distance".

11

u/froop Mar 27 '26

If everyone watched their TVs at the recommended distance, you might have a point, but in reality most people are watching the TVs they could afford or fit from whatever distance their living room allows. 

→ More replies (1)

3

u/Secana0333 Mar 27 '26

im using a 55inch as my PC screen. When it reverts to 1080p it looks like shit!

9

u/HeGotTheShotOff Mar 27 '26

as your PC screen? then you're likely viewing it much closer than optimal viewing distance.

→ More replies (1)

3

u/kominik123 Mar 27 '26

Me too actually. I set up 4K@120Hz but when watching movie or TV show, i am having hard time telling the difference if the source is FHD or UHD because it's 2,4m (almost 8ft) away from me. It's easy to spot a poor codec/bitrate thou.

→ More replies (2)
→ More replies (6)

3

u/Megneous Mar 27 '26

Honestly, huge portion of people can't even tell the difference between 1080p and 4K.

Those people are honestly fucking idiots though. I thought that I would be wasting my money by getting a slightly larger monitor that was 1440p 144 FPS capable, so I started it off at 1080p (yes, I realize that 1080p on an appropriately sized monitor looks better than on larger monitors than on a slightly larger monitor meant for 1440p, but I figured that a comparison between the two resolutions would still be a fun thing to do). So I looked at 1080p 60 FPS on Warframe. Then I switched it to 1440p 144 FPS. Holy shit, it was fucking beautiful. Never, ever going back to 1080p 60 FPS.

→ More replies (2)
→ More replies (4)
→ More replies (3)

3

u/justeffingpeachy Mar 27 '26

Shit half the streaming services won’t even give you 4K anymore unless you pay for the premium package, what the fuck are you even going to do with an 8k TV?

4

u/Cool_Discipline6838 Mar 27 '26

They would keep increasing except for the fact the limit in this case is the human eye.

At 10 feet 4k and 8k appear identical on a 65" tv

2

u/ytman Mar 27 '26

I'd say 4k is the threshold, and if I'm desk gaming I think 2k is good enough. At that resolution its more about pixel size than anything else.

2

u/jib_reddit Mar 27 '26

I want a 8k monitor but they are ridiculously expensive right now.

2

u/Round-Tradition-3890 Mar 27 '26

8k TV's are irellevant to the discussion. He was talking about gaming PC's.

8k gaming monitors are selling well, because GPU's which support 8k gaming are selling well.

8k TV's are not selling well, because there isn't a single streaming platform or broadcasting service that supports it, and 4k blu-ray is still the best quality physical medium available with Xbox and PS5 having a maximum resolution of 4k.

4

u/HeKis4 Mar 27 '26 edited Mar 27 '26

I mean, 8k is just plain useless, we're beyond human eye limits at any comfortable viewing distance. A 4K 55" TV is beyond our ability to resolve details at around 1 meter already and I don't know anyone that sits this close. For a 28" 1440p screen this limit is at 80 cm which is already smack dab in the "comfortable viewing distance" for them in my experience.

Without mentioning the absence of content, even the absolute highest end cameras used in filmmaking don't support 8k.

3

u/trusty20 Mar 27 '26

Without mentioning the absence of content, even the absolute highest end cameras used in filmmaking don't support 8k.

Why would you say something so completely untrue? Plenty of "the highest end cameras in filmmaking" support 8k, i.e some RED models.

And beyond that, even the lowest end old crusty 35mm film camera in grandpas attic can easily do 8k resolution scans.

→ More replies (4)

2

u/HeGotTheShotOff Mar 27 '26

I mean, 4k is overkill. its better but with ideal viewing distance is hardly necessary. I have access to both 4k tvs and HD and at the proper distance its pretty hard to tell the difference.

→ More replies (4)
→ More replies (22)

28

u/-Altephor- Mar 27 '26

Ah to never need more than 150 MBs again. Those were the days...

5

u/Damienkn1ght Mar 27 '26

I remember my older brother got his first PC and it had 105MB, and it seemed like a dream. How could we ever use that up? Had a 4x CD ROM Drive too. Man it was cookin when we played Master of Orion.

2

u/D4rk4ss4ssin30 Mar 27 '26

Didn’t Bill Gates say nobody needed more than like 74kb back in the day?

2

u/LeMegachonk Ryzen 7 9800X3D - 64GB DDR5 6000 - RX 7800 XT Mar 27 '26

My first computer had a 40MB hard drive, and 10-20MB was more typical at the time (1989). I still have the platters from that drive somewhere. The coating on the platters literally wore off, moreso toward the outer rim, so they have a copper sunburst look to them.

→ More replies (1)

6

u/throwawaycuzfemdom Mar 27 '26

For a long time game sized followed the CD-Dvd-Double Layer DVD-Bluray-Double Layer Bluray and then digital became the king and the sizes exploded :/

→ More replies (6)

2

u/PayZealousideal8892 Mar 27 '26

Oh, yes. NVIDIA is innovating and improving GPU's each generation, because monitor technology is improving so they need to keep up.

→ More replies (15)

120

u/EbbNorth7735 Mar 27 '26

It's context size, so it's short term memory. The amount of stuff it can think about at any given time. The weights aren't affected. Still a big improvement if it's true. Context size ram requirements exponentially grows with more context. It's a big win for large context implementations.

36

u/clyspe Mar 27 '26

Some rough numbers for people who don't run LLMs themselves: on long context, weights are ~5/8 of the memory usage for me, context is ~3/8 (128k context). So the 3/8 is what's going down in size. As we go up in context length, the size required increases linearly, so as we get more capable models, this advantage is going to grow.

→ More replies (1)

49

u/cantadmittoposting Mar 27 '26

That'll be pretty useful, its pretty noticeable when an LLM hits context limits and you start remembering more of a conversation than the model

13

u/pidude314 Ryzen 7800x3D | 9070XT Mar 27 '26

My favorite is when you hit a context size so large that it just completely resets. Gemini has done that for me before. It just fully reset the conversation and couldn't access anything at all from the prior prompts

3

u/cute_spider Mar 27 '26

In Visual Studio, I have copilot write out a big summary of everything we talked about when its brains get too full.

2

u/pidude314 Ryzen 7800x3D | 9070XT Mar 27 '26

I've started doing the same

→ More replies (3)

3

u/Kleenex_Tissue Mar 27 '26

Even with a bigger context size many LLM's start hallucinating or incorrectly recalling data before they even hit the limit.

2

u/Sawses Mar 27 '26

For sure. I run a RAG as a way to quickly look up things in my tabletop games, since the two I play each have dozens of books and it's nice to have the model point me directly to the book and chapter with the information I need.

Because of context length limits, I can really only get accurate answers about 3-5 books at a time. It would be nice to have that go up.

→ More replies (2)

3

u/BenevolentCheese Mar 27 '26

Context size is currently the biggest inhibitor for LLMs in high level usage, you can be damn sure any increasesin RAM availability are going to go straight into increasing context.

2

u/SwagginsYolo420 Mar 27 '26

This so-called "AI" was inevitably going to be optimized to require far less resources.

→ More replies (3)

50

u/TheWombatOverlord Mar 27 '26

This is usually what happens. Its a common enough phenomenon to get its own name: Jevon's Paradox. Efficiency gains of a resource usually leads to increased consumption of that resource.

17

u/cute_spider Mar 27 '26

That's just Induced Demand but for efficiency gains!

3

u/TheWombatOverlord Mar 27 '26

As with cars so too with computers. It actually is a thing that you can see happen in basically every aspect of society and the economy.

2

u/cute_spider Mar 27 '26

It's also why it's so hard to replace fossil fuels in the energy grid. We set up all these solar panels for passive energy and then immediately feed all that extra energy into bitcoin and AI!

2

u/TheWombatOverlord Mar 27 '26

At least many countries have already shifted most of their electricity to green sources. But it has definitely been and will continue to be a slower transition because of induced demand.

3

u/YimmyGhey Mar 27 '26

TIL! I was trying to basically describe this to someone yesterday and didn't know there was a term for it.

2

u/TheWombatOverlord Mar 27 '26

Its been a brain worm for me for the past couple years so glad I could pass it on!

2

u/Unlucky-Equipment999 Mar 27 '26

Looks like I'm one of today's lucky 10,000. I wonder if this phenomenon is appropriate to explain how the existence of upscaling technology will not lead to consumers' GPUs lasting longer, but just a skipping of optimization in gaming.

2

u/TheWombatOverlord Mar 27 '26

I think you can definitely make the argument that this is what has happened with Moore's law and the progression of computer technology as a whole has yielded. Bigger file sizes, less optimizations.

We landed people on the moon in 1969 with room sized computers measured in kilobytes less powerful than a Gameboy. Opening up my calculator app takes up 43MB of memory and uses 5 threads.

Computer scientists have a adage that on wikipedia is called Wirth's law. Worth checking that and the "See Also" section because this is a topic that touches alot of modern life.

51

u/cficare 9800x3d - 5080 Astral - 32GB of $$$ Mar 27 '26

Gemini knows so much more about Tangerines, now! The future is here!

12

u/the5thusername Mar 27 '26

Full glass of red wine soon!

2

u/JesusWasATexan Area51; Ultra9 275HX; RTX 5080; 64GB DDR5; Mar 27 '26

→ More replies (1)
→ More replies (1)

30

u/BadFurDay Mar 27 '26

https://en.wikipedia.org/wiki/Jevons_paradox

More like 10x bigger/more LLM datacenters and RAM prices will keep rising.

11

u/Comfortable_Ebb7015 Desktop Mar 27 '26

No, it will not change anything! It just compresses more the cache, not the model itself! It means that the model will simply be able to keep more context in memory. But the biggest chunk of the memory is still used by the model itself! Investors are dumbass!

→ More replies (2)

7

u/PsudoGravity Mar 27 '26

Better than nothing ngl

→ More replies (1)

2

u/-LaughingMan-0D Mar 27 '26

No. This is just for KV cache compression, the context window of your conversation with it.

2

u/Platypus__Gems Mar 27 '26

I think at some point the amount of training data becomes the limit to how bit LLM gets.

→ More replies (1)

2

u/bitches_love_pooh Mar 27 '26

Like how battery life on cellphones hasn't really gotten better

2

u/Alternative_Cause766 Mar 27 '26

This is objectively correct and exactly how economics work What you are missing is that that will drive out other data centers and cause them to close down.

There isnt an increase in capitalizable demand automatically when the productive power goes up. Its just that the guys who own THOSE more efficient ones get to steal up more market share and push out smaller guys.

2

u/ZeidLovesAI Mar 27 '26

So DLSS5 for RAM?

1

u/Ok-Friendship1635 Mar 27 '26

Diminishing returns. Why spend 6x the amount of energy for only 2.5% gains.

1

u/aeiou403 2060 Super/12400F Mar 27 '26

Exactly they will just stop optimizing the llms

1

u/buffalosoldier221 Mar 27 '26

Maybe I'm coping but that's not exactly how it would work in this case, I think. Just because you have 6X the ram, doesn't mean that you have the GPUs, storage and cooling hardware to process that, right?

1

u/FlingFlamBlam 9800X3D | 9070XT | 32GB @ 6400MHz Mar 27 '26

They wouldn't be able to scale all the other hardware up at a matching ratio though.

More likely what they'll do is reduce RAM buying by some amount and redirect that money to buying other hardware.

So maybe RAM might go down by some amount, but everything else will get more expensive.

Fuck the RAM companies and fuck "AI", but this probably won't save us. Great news for people who already have systems and just need to replace a stick or two though.

1

u/nnomae Mar 27 '26

Yeah, even if this pans out all it does is give google more return per penny spent on RAM. They'll just try to buy even more of it.

1

u/ClearlyCylindrical Mar 27 '26

and they'll be more cost effective so total usage will be higher.

1

u/windmachine2000 Mar 27 '26

They don’t have the electricity to do that

1

u/Individual-Praline20 Mar 27 '26

Yep, so 6 x 8 = 48 times better, right? I already see the AI slop ads everywhere 🤭

1

u/PwanaZana Mar 27 '26

yepp Jevons paradox

1

u/Pixelplanet5 Mar 27 '26

honestly i dont think that is going to happen as we have already found that making LLMs bigger is not increasing their capabilities as much as it does on the lower end of the spectrum.

given that basically all AI companies are bleeding money they gonna focus on making the existing LLMs run more efficiently first and maybe then start making them better again.

1

u/spooky_strateg Mar 27 '26

Better local llms

1

u/Sinsanatis Desktop Ryzen 7 5800x3D/RTX 5070ti/32gb 3600 Mar 27 '26

Yeah cuz a majority of the ram has already been prepurchased

1

u/OphidianSun Mar 27 '26

Still good I guess? I doubt usage is gonna increase 6x so things like total energy consumption should drop no? Or maybe they'll need fewer datacenters? Or maybe I'm tired of a bunch of asshoils with bottomless pockets running around fucking everything up and that's all just cope.

1

u/LosEagle Mar 27 '26

Implying google will not keep the algorithm proprietary.

1

u/Cmdr_Shiara Mar 27 '26

It's good for deploying ai to systems like robots or drones where you can't have as much compute power

1

u/Oracle1729 Mar 27 '26

So 6x as wrong on basic stuff as the current generation. 

1

u/BaconIsntThatGood PC Master Race Mar 27 '26

This is basically what happened as cost per token dropped dramatically. Token usage skyrocketed as a result.

1

u/skyper_mark Mar 27 '26

Yeah, these corps always pull out the same card.

"You've reduced power usage by 10x? Good! Ramp up production by an equal amount!"

1

u/SourceScope Mar 27 '26

Indont even want ai

1

u/generally_unsuitable Mar 27 '26

Jevon's Paradox.

An increase in efficiency causes a price drop which creates an increase in use.

1

u/BobbyTables829 Mar 27 '26

Were starting to have diminishing returns already

1

u/Ironic_Laughter Mar 27 '26

"I'm doing 1000 calculations per second and they're all wrong"

1

u/Thefrayedends 5900X3D/RTX 5070Ti+55"LGOLED. Alienware m3 13" w OLED screen Mar 27 '26

From a local perspective, this is fucking incredible lol. IF, it is true.

But the huge and largest models won't get any bigger. They already have basically the entire ingestible block of possible ingestible data.

There is no more data until humans generate it, and we're not creating it at rates that will be whole number multiplicative in short time frames.

Also much of that data that we do create isn't really 'new' so the rate of data volume increase is even lower than what you might imagine.

But for local models, yes, this could be huge. If this is even true. If it is true, it means someone like myself can go from running a modest 24Billion parameter model to a 150B model.

1

u/Aleashed Mar 27 '26

RAM Co CEOs smashing that consumer RAM switch button

1

u/qrayons Mar 27 '26

Only if memory is still a bottle neck for the models.

1

u/Sorlex Mar 27 '26

Yeah did the crypto spree teach people nothing? When prices go down, they don't scale down they buy more to scale up.

This will result in nothing but a dip in ram prices if anything before they go right back up again.

1

u/EggsceIlent Mar 27 '26

Nah I never understood the freak out over ram prices

Been building my own pcs for decades and ram spikes for awhile every now and then.

It's was bound to pass eventually. Just wait it out.

And here we are.

1

u/TriviPiviP Mar 27 '26

How is that supposed to work? If the LLM would simply get 6x the size we would need 6x the RAM since the whole model has to be on the RAM?

1

u/hudimudi Mar 27 '26

Well the model is still the same size. Kv Cache gets improved. So there are gains but they are also limited. Still good tho!

1

u/Bovronius Mar 27 '26

Yeah, it's like when electricity gets cheaper, or stuff gets more efficient, people just buy MORE refridgerators instead of enjoying the savings. "Well I already got one in the garage so I don't have to see my family between beers, why not one in my man cave next to my unused podcasting setup now!".

1

u/TheComplimentarian Mar 27 '26

When hardware gets expensive, code magically gets efficient.

1

u/Rainbows4Blood Mar 27 '26

Training time still grows exponentially with size, so I'm cautiously optimistic that this won't happen. At least not to the full extent. And I hope that we can get some of our RAM back.

1

u/lavenderthiefs Mar 27 '26

chrome would see that breakthrough and still find a way to eat 12gb with three tabs open

1

u/DyerOfSouls Mar 27 '26

Reluctantly, this.

1

u/uesernamehhhhhh Mar 27 '26

And then they will need 6 times more gpu's and cpu's...

1

u/[deleted] Mar 27 '26

Probably a bit bigger but it also slows down entire process sooo... Not gonna happen.

1

u/HustlinInTheHall Mar 27 '26

Eventually they'll just run locally for all basic tasks. You pay the electricity cost. Same as happened with compute 

1

u/FluffyWuffyy FluffyWuffyy Mar 27 '26

Yup

1

u/DudeDudenson PC Master Race Mar 27 '26 edited Mar 27 '26

No, now they'll use this to double down on the LLM is AI bubble and ram will be even more expensive

1

u/ftgyhujikolp Mar 27 '26

Frankly the context window is one of the big limiting factors for LLMs. 6x bigger is a lot.

1

u/thedarkherald Mar 27 '26

Exactly this is literally what the article says that now you can run more complex models.

1

u/klop2031 Mar 27 '26

Thats what im hoping for

1

u/Salt-Willingness-513 Mar 27 '26

I aint mad if i can run 120b models on cpu with decent speed though. That would be an amazing improvement.

1

u/Olde94 9700X/32GB/4070S + 4800hs/40GB/1660ti Mar 27 '26

Yeah i can now run models locally on my phone with 12gb ram that outperforms the early big models

1

u/vplatt Mar 27 '26

Skynet imminent!

→ More replies (7)