Everyone knows that debugging is twice as hard as writing a program in the first place. So if you’re as clever as you can be when you write it, how will you ever debug it?
This quote is older than I am, but has been shown through scientific studies to still hold today. Anyone that has asked me about my opinion about AI has heard that I worry about the environmental impact. If you haven't asked me you now know also. Just as importantly to me is how Kernighan's Law will apply to the world we are going to experience with the advent of accessible AI tools.
I developed much of my base skills at the time of RAD (Rapid Application Development) was a popular development method. As strange as it may sound to those that haven't been exposed to this practice, RAD is not actually about developing applications quickly, but using tools that allow you to iterate quickly so the final product fits the customer need as closely as possible the first time you deliver it. My favorite RAD tool is the programming language TCL/Tk. TCL is an amazing language that allows for quickly creating almost any kind of infrastructure, and Tk allows for quickly developing user interfaces. Between them you can test if your perceived answer to a problem is plausible and you can show it to your customer to see if they agree. Due to the limitations of requirements gathering there is almost zero chance you got it completely right the first time, and these languages let you quickly make changes (even in real time ask they are talking to you) so you can capture how they want to work.
The second reason I feel that TCL/Tk is a great RAD language is that it is not practical to deliver a product at scale in these languages. This means when I get a mocked up version that proves my algorithms, APIs, and UIs are the desirable method of accomplishing the requirements, management can't just ask me to deliver that as a product despite it not being in a condition that could be maintained. This means I get the opportunity to translate this to the final version of the product and keep the things I learned, and discard the things that didn't work. This is one of the reasons I don't like AI.
AI should be the ultimate RAD tool. it listens to the customer, and doesn't have to hold back when they aren't communicating efficiently, it just sits there with infinite patience. It can iterate with relatively low cost and the customer can try thousands of ways of describing their task. The output of the AI should be the starting point of requirements gathering, but that will never happen.
AI produces something far to close to a final product for customers to be willing to wait for a new version that is implemented by someone that has a deep knowledge of how to do these things. AI has removed the tedium and allows the customer to describe what they want. Engineers can take that output and create something that will withstand the rigors of time, but that engineering process takes time and there is this thing that looks good right there and works good enough. It takes a great deal of will power not to put something you have access to right out into the world, and most people don't have that restraint.
My unpopular opinion is that AI should only deliver images of what is described unless the user can prove that they are clever enough to debug a generated program.