Dieser Bereich kann Inhalte enthalten, die nicht für alle Nutzer geeignet sind. Dazu können unter anderem Texte, Medien oder Diskussionen gehören, die als beleidigend, extremistisch, gewaltbezogen oder anderweitig belastend empfunden werden. Wenn du solche Inhalte nicht sehen möchtest, nutze bitte die jeweiligen Filter- und Meldeoptionen der Plattform oder meide entsprechende Threads/Communities.
I agree, but this will help pop the bubble because local models we can run on our own hardware will crash the demand for data centers
Where do you think those models are trained?
And most personal computing has tended towards only having a smartphone, and a laptop for a real PC. You can’t run much in the way of an LLM on those (at least not with any real speed).
You can if the laptop is extremely expensive. But the laptop is extremely expensive.
I don’t think you’re thinking this through. You can have 1k people running 1 server or 1k people each with their own hardware running models. Which one of those do you think will increase hardware demand?
Local models tend to be more efficient since people will be more likely to run compressed and MoE models.
Also, it’s basically 1 GPU >= 1 request for the most part in data centres, each request is its own LLM. Each time you make a new request after a set timeout, model weights get loaded in VRAM, context gets initialized, the query gets parsed, and it spits out tokens. These frontier models can be 300 GB in size or more, which all needs to be kept in VRAM for best performance, usually distributed across multiple GPUs, and each loaded model can only answer one query at a time.
Compare this to someone like me, trying to cram Qwen 3.6 MoE on the 16 GB RX 6800XT I already have in my own computer, not using up drinking water or prime real estate to cool my PC, powered using hydroelectricity.
It’s so much more eco-friendly and economical, I wish frontier models would just die tbh, or at least only be used for distillation. LLMs offer diminishing returns past a certain point, and you can get 90% of the frontier model with a MoE local model.
What does efficiency have to do with hardware demand?
Higher efficiency = less hardware demand
What? No? How?
Not that I have anything against local use—I do my stuff locally—but people running local models will very probably greatly increase demand for memory relative to cloud-based AI compute (and it’s why I think that in the near term, most AI compute stuff is gonna be in the cloud, because we don’t have the memory to do everything locally).
If you run in the cloud, the hardware is shared. When one person isn’t using it, another can be. That can lead to high rates of capacity utilization, approaching 100%.
If I have local AI compute hardware, if I’m not using it, it’s idle. If 1% of the time, I have it crunching something for me, then I’m only getting 1% capacity utilization of that hardware. That means that to provide the same level of compute capability to everyone for local use, I need 100 times as much hardware.
AI cloud companies have purchased more memory than the rest of the world is buying.
If we did everything locally, we’d need a hundred times more memory than what the cloud AI companies are picking up.
We couldn’t do that without far more memory production capacity. Even if someone started on a buildout of that scale today, a new memory factory takes 4–5 years to get into production.
The people that use it super heavily can do on-prem and benefit and the people using it 1% of the time can use cloud and barely cost any money.
Correct. This isn’t an all or none situation.