Do you know why AI struggles with making production ready code?

It’s because it has never seen it.

Training Data is Not Production Code

The code on which AI coding assistants have been trained is on things like StackOverflow, Github, and online articles.

That is at best problem solving, which is why AI assistants seem good at some things. Examples include isolated tasks, bug fixing, and even code explanation.

However, real production code is rarely on Github. And by that, I don’t mean that React isn’t open source, I mean that the millions of real and production deployed React projects are not open source.

How many fully open-source games do you know? For example, go search for released Unity games on Github and compare that to the number of games released. I’ve found one list that lists around 50 games in total, mostly small ones.

That is about the same amount that is released on Steam every day.

Production code is vastly different from libraries and examples and things made to showcase a technology. Unlike what you might think if you don’t live in this space, production code is not clean or perfect or stable.

Production code is messy, hacky, and full of bugs. Production code has unused features, partially built features, and features that are really bugs but that can get past QA if we just brand it properly.

The purpose of production code is to produce, not demonstrate or land customers. The product is a solution to a problem or entertainment – it is never the purpose in itself. Nobody writes production code just for the heck of it.

That’s why AI models can’t do it, or do it really poorly. It has never seen, in any extent, real production grade code. That code is hidden in private repositories or does never see sunlight outside of a company’s on-premises servers.

So next time you try to take AI generated code to production, keep in mind that unless you have the experience of putting things into production and maintaining it there, then AI cannot help you.

You need real humans for that.

You Can’t Learn Production Code in School

The second reason why it’s extremely hard for AI to produce production grade code is that there really isn’t anything to learn.

By that, I mean that production code is largely one-off. Each code base is unique, each facing challenges possibly rarely if ever seen before.

It’s muddled up by that cranky dude in operations that insists on backups being automatically restored if the system goes down, despite all advice to the contrary.

Why does he insist on that? Who knows but that’s what production means.

It’s the forced dependency injection where none is needed by that architect who has a hiccup for DI.

Why? Who knows! It doesn’t matter because although DI is cool sometimes, you might not need it, or you do, and that’s what’s unique in your particular situation.

There’s an old saying that every developer wants to start over when they first join a project. That is true if they come back to the project they built two months later as well.

That is rarely a good idea. The reason it’s a bad idea is that hidden in those quirky decisions is a vast and secret knowledge base that caters to unique things with your particular situation, your particular culture, history, language, or people.

AI will never see that and it cannot be taught in schools either because it has to be done for each project, every time.

You need real humans for that.