Dieser Bereich kann Inhalte enthalten, die nicht für alle Nutzer geeignet sind. Dazu können unter anderem Texte, Medien oder Diskussionen gehören, die als beleidigend, extremistisch, gewaltbezogen oder anderweitig belastend empfunden werden. Wenn du solche Inhalte nicht sehen möchtest, nutze bitte die jeweiligen Filter- und Meldeoptionen der Plattform oder meide entsprechende Threads/Communities.
I really love when I ask the AI to do something, and it gives me 40-50 lines to do it, and I tell it there’s no way something like that should take that much code, and it’s like oh, ya you’re right, here’s a 5 line version using something from from the standard sdk.
I’ve tried to put in its memory to not do things like that, but it keeps doing it.
I watched it remove a JSON library include from the code because it wasn’t used yet, and then in the very next prompt where we needed our again, it attempted to write a basic parser from scratch…
Brilliant.
Ive had it do stuff like that completely ignoring that I already have a library in my app that should handle some sort of task. But removing it first is chefs kiss.
I guess the AI isn’t iterating over its “ideas” and code yet. I come up with overcomplicated and overcomplex code too. But my natural neural network keeps reevaluating the code while I think about it, write it down, test it, and often even come back to it later after I moved on to another part of the codebase.
It looks to me like AI is using the infamous waterfall method to write code. It does the “thinking” phase and does bugfix iterations when compilation or tests fail. But never does it question the design it came up with after it wrote it down. It doesn’t reevaluate whether the last round of fixes made tests work that test irrelevant implementation details. It doesn’t question whether there really need to be four methods sharing 90% of their body for doing the basically the same thing only subtly different.
I hold the suspicion that AI will be as slow as humans when it becomes as reliable because what makes us reliable is the redundancy in our thinking processes. The constant reevaluation in subtly different states of mind helps to catch design flaws and silly mistakes.
Reevaluation of finalized code is probably not something you can just put into memory. You would need to modify the harness to automatically inject a new prompt triggering reevaluation of the design and implementation after each step. This would likely increase token use and execution time by an order of magnitude. And if you do it by spawning fresh agents, there would be a high chance of having the AI go in circles, switching between a few different versions of the same shitty design or implementation. So you need to keep the context, which grows each iteration. So context rot may become a real problem. The AI might deviate from the initial goal just like some human devs tend to do…
And natural neural networks learn on the job. They adapt to the codebase and industry they are working with/in. claude.md and memory text files help a lot. But they are still sometimes ignored. And I don’t know why. Also, human rules are almost always somewhat fuzzy. Long lines are to be avoided - but unit tests and that one massive dict of dicts really are a lot more readable when ignoring that rule. There might be a lack of good training material on what makes code readable for humans.
That actually makes a lot of sense and goes with how ill code something larger as well.
Get a rough version working, sometimes horrible code. See how it functions. Reassess it and optimize it. Go bug hunting, which will often result in more optimizing, then finish all the error handling I probably skipped and only logged. More bug hunting, then review my code like a code review before submitting it for a code review.
Possibly motivated to do more complex problems to spend more tokens?
I can believe this. I’ve been put on slop duty, and had the slopmachine go all
And I’m here thinking, if you’re gonna run the whole suite anyway, why’d you run an individual test? That’s just redundant.
Even without external libraries, it routinely comes up with solutions that are at least 3-4x the complexity and LOC than how I would implement it.
It’s because the average quality of code that they are trained on is like that. Most code that is written by humans is slop, and the models represent that.
Producing overly complicated solutions to uncomplicated problems
This is maybe the main reason why I don’t fuck with AI. “Oh it accesses all of the writing on the public internet!” That never inspired a lot of confidence. 99% of Internet posts are crap.
It often can do it better if you prod it though. Its just so weird that its really hard to get it to do that. It knows how even if it is trained on poor over complicated stuff.
How much time does that prodding take vs how long would it take you to just write the (better) code yourself?
Its usually just 1 or 2 messages saying i dont like this so its only the time it takes to read it and say try again. A minute or two?
Its faster to just ask to do it again in most cases.
The times itd probably be quicker to do it on my own are the more complicated ones where it gets something wrong but it wasnt so blatantly obviously bad that I just tell it to try again, so I spend time with the wrong stuff, see its wrong with a little work, then toss it and try again, or re work it myself from that.