Dieser Bereich kann Inhalte enthalten, die nicht für alle Nutzer geeignet sind. Dazu können unter anderem Texte, Medien oder Diskussionen gehören, die als beleidigend, extremistisch, gewaltbezogen oder anderweitig belastend empfunden werden. Wenn du solche Inhalte nicht sehen möchtest, nutze bitte die jeweiligen Filter- und Meldeoptionen der Plattform oder meide entsprechende Threads/Communities.
Have you tried running a local model on a M series Mac?
Yes ofc I ran Gemma 4 for example, but compare that to the speed of Gemini in the cloud the difference is massive.
How much RAM do you have and which version of the model did you run?
Local LLMs can be just as fast as long your device clears the requirements. If you noticed a huge difference, there’s a really good chance that you tried to use a model that requires more RAM than you have
I ran Gemma 4 31 B quantized so it fits in my RAM. The decoding speed was decent, but if you look at the newest models for example Gemini flash 3.5 they have a decoding speed of 280 token per second, they generate an entire page before my Mac locally generates a sentence.
That is a bit too much for your hardware, even the Q4_0. You needed a smaller version (26B likely would suit you better. It would be faster and is a MoE)