Dieser Bereich kann Inhalte enthalten, die nicht für alle Nutzer geeignet sind. Dazu können unter anderem Texte, Medien oder Diskussionen gehören, die als beleidigend, extremistisch, gewaltbezogen oder anderweitig belastend empfunden werden. Wenn du solche Inhalte nicht sehen möchtest, nutze bitte die jeweiligen Filter- und Meldeoptionen der Plattform oder meide entsprechende Threads/Communities.
I should clarify - I do think there are use cases for smaller local models performing very specific tasks - the technology I am specifically referring to as “useless” are the massive data center LLMs.
AI is such a blanket term (intentionally conflated with useful technology by the people poised to profit from said LLMs) that it’s a bit difficult to discuss it as a singular technology. I have no problem with machine learning, or with neural networks in general.
I am, however, having difficulty reading this as a good-faith article
“Lukasz Kaiser, one of the people who invented the Transformer, said on “Unsupervised Learning” in June:”
That man (Lukasz Kaiser) is an employee of OpenAI, which is a massive conflict of interest imo, and disqualifies him from making impartial claims about the technology.
What I find particularly sketchy is the vague use of the word transformer in this context. The transformer was invented in 1886, there are no living persons who helped to invent it. To present this man as the inventor of “the transformer” without clarification/disambiguation is disingenuous. It smells fishy to me.
Kaiser is one of the coauthors of Attention is all you need, the paper that introduced the Transformer architecture, the basis for all major LLMs.
3b1b has a writeup: https://www.3blue1brown.com/lessons/attention/
Attention is the mechanism that lets an LLM use the (correct parts of the) entire context to predict the next word.
Oh yeah, I was able to figure out why the article said that with a couple search queries, but I still think that the phrasing in the article is misleading (and likely intentionally so).
Edit: Also, I think part of what’s confusing people is that they keep using terms like “attention” and “inference” to describe computer processes that may or may not have some kind of underlying similarity to the corresponding human capabilities. I also believe this to be deliberate obfuscation.