Bringing up information theory and godels incompleteness theorem in regards to ai advancement is like saying there are hard physical limits on shrinking the dye in the 1980s. Yes hard axiomatic limits exist, but within those limits is a huge problem space. Even if we limit it to solving problems on a screen using existing techniques, that problem space is huge and occupying a large amount of time for the knowledge workers who make up ~25% of the labor force.

Unlike shrinking the dye though we are not near that limit, we still have a lot of improvement to make before AI can solve all the problems in that space. I think we can both agree the current LLMs aren’t able to replace all screen work. But it is making big strides in that problem space, such as going from failing basic arithmetic to solving the hardest math problems in a couple years.

Like with shrinking the dye there’s not going to be a singular breakthrough, like recursive self improvement, that’ll throw us up against those limits almost immediately. Progress is done incrementally over years with a bunch of different smaller improvements, like opaque recurrence. This does have some acceleration as the technology helps with the design of the technology, computers help chip architects design better chips, which make better computers, but a human is still required. Same with AI, every lab is probably heavily relying on coding agents to assist them with developing and deploying new techniques. The AI may not be creating these techniques, but it is accelerating the creation of these techniques.