This blog post is LLM generated, but the implied prompt and some of its decompressions are relevant.
If we can make LLMs and surrounding processes produce results as reliably/reproducibly as compilers, the window of abstraction that we will feel obligated to review will shift. (Depending on the context of course. Something like bitcoin warrants a floor to ceiling review window.)
One catastrophic omission is not distinguishing between open weights running locally and blackbox API calls. Maybe it's a thing somewhere, but I don't know anyone that compiles code via an API call to a closed source compiler.
Whoops your customers lost all their money because we downgraded your spec "compilations" to Haiku after you hit your API limits.
I am by no means a LLM or AI expert but making output reproducable would be a massive accomplishment and one I don't see any time soon.
There's certainly not much motivation to do it yet. Most folks are not concerned about reproducibility. It's a long range concern and everyone is still in awe of becoming a "programmer" or "writer" overnight.
Yeah, no doubt