This is basically the same as asking someone who knows a bit about everything to recite obscure information from memory. I doubt most humans would do better.

Sure, but a good web resource or even a good reference book would provide better help faster. Unfortunately it’s getting harder to harder to find those good web resources as search results get overtaken by AI slop.

iterate until the result is correct, then give me a one-page document explaining what I need to know, with cited evidence — far faster than I’d do it

I have gotten it to do this in certain situations, like the other day I wanted to simplify a math formula, so I gave it a loop with a Python script that checked its solution against the original reference version. This worked pretty well.

But I had to write code specifically for that situation. Even with a good skeleton to start with, it’s a non-negligible amount of work to get that set up. I feel like the scenario in which this is useful is kinda narrow: when I have a very good idea of exactly what I want, but some step along the way is a hassle. General software engineering, like making a whole app, is far too open ended, and most of the sub-problems I encounter in software engineering seem either too open ended or too small to benefit from this approach.

That also doesn’t account for the speed of models. My experience is a ~20GB locally hosted model takes like 1-5 minutes to produce a good length response, and the few times I have used online models they are often slower. A few minutes per iteration, accounting for debugging when it goes off track, is not exactly fast or hassle free.

I just feel like the trade off where using AI vs doing it all myself is pretty limited in when AI offers an advantage.