AI Agents That Fix Code: The Hidden Risk

Imagine you hire someone to replace the lock on your front door. They replace it, test it, it works. Except along the way they left the back window unlocked. And nobody went to check the window, because the request was about the door. That is roughly what happened at Snowflake, a data company with billions in revenue: an AI agent fixed a security flaw and, in the same move, created a path into the company's internal Jira.

The part that stings is that the fix came from a good tool, a respected one, used by plenty of serious people. GitHub's Copilot Autofix suggests automatic patches for vulnerabilities found in code. Someone skimmed the review, approved it, merged it. The team moved on believing the issue was closed. Researchers at Wiz found the hole later.

What exactly went wrong

The target was an automation script in the repository, one of those routines that run on their own when someone opens a pull request or comments on an issue. It is code almost nobody reads carefully. No screen, no client watching, it just does the boring job of connecting things.

The original flaw was a known one: data coming from outside went straight into the script. The AI produced a fix that handled the specific case flagged in the alert. Nice work. Except the script kept accepting untrusted input through another path, and that path had access to the credentials the automation uses to talk to Jira. The result: anyone who knew how to write the right comment in a public place could put Snowflake's bot to work for them on the inside.

Look at the mechanics, because they repeat in places that look nothing like programming:

• The AI answered exactly the question it was asked.
• The question was too narrow.
• Nobody checked the surroundings, because the alert had turned green.

The green alert is the silent villain here. An open problem bothers people, generates meetings, gets chased. A problem marked as solved disappears from the radar forever.

Why this matters to people who do not write code

You might be thinking this is an engineering team topic. It is not. Swap "CI/CD script" for any automation running in your business today.

The agent that answers customer email. The bot that pulls orders from WhatsApp into the ERP. The routine that reads invoices and posts them to finance. The AI that screens résumés and moves candidates through stages. They all share the same anatomy: they take data from outside, they hold some credential, and they act with nobody watching.

I have seen a smaller case with the same face. A support automation that created tickets from the website form. Someone figured out you could write text in the "subject" field that the AI read as an instruction instead of content. The ticket turned into a query against the customer database, and the answer came back to the person in the confirmation email. Nobody broke into anything. The door was open by design, and the AI was too polite to be suspicious.

Automation does not make few mistakes. It makes them fast and in silence.

The blind spot is AI reviewing its own AI

Here is my strong opinion in this article: the most dangerous pattern spreading in 2026 is AI approving AI's work.

The flow has become common. One agent writes. Another agent reviews. A third runs the tests the first one wrote. On paper it looks like rigor. In practice, all three share the same blind spot, because they read the problem through the same frame. It is like asking three people to check the math on the bill when the error is on the menu.

And there is a psychological effect on top. When the patch arrives well written, with a clear explanation and a convincing rationale, humans review less. An AI pull request feels more trustworthy than an intern's, even when it is worse. Trust bought with good writing is the cheapest thing on the market right now.

I am not telling you to drop the tools. I use AI agents every day and they save me real hours. I am saying the review has to come from a different angle than the one that produced the change.

How to use AI agents without this kind of surprise

What works is not complicated, it is tedious. And tedious is exactly what people skip.

Separate who can act from who can only talk. If an agent reads customer requests, it should not carry the same credential that creates users in the system. Two accounts, two levels. That alone solves half the incidents out there.

Give the smallest permission possible, not the most convenient one. Snowflake's automation had broad Jira access because one day it was easier that way. Almost every over-broad credential started as a time saver on a busy afternoon.

Treat external input as suspicious, always. User comment, form, email, WhatsApp message, file name. None of that is an instruction. It is content. If your automation does not make that distinction, it will obey the first polite stranger who shows up.

Ask for the context, not just the fix. When an agent proposes a change, the useful question is not "does this solve it?". It is "what else touches this file, and what does this change now allow that it did not allow before?".

Put a human at the money and sensitive data points. Not everywhere. Transfers, supplier registration, mass sending, deletion. The rest can run on its own.

Review what is already green. Once a quarter, open what automation marked as resolved and look again. It is the cheapest audit there is and nobody does it.

The right amount of distrust

None of this means fear. It means calibration.

Snowflake is not a sloppy company. It has a security team, it has process, it has good tooling. Even so, an automatic patch created a new door. If it happens there, it happens in your 20 person operation, with the difference that there someone found it before it became bad news.

The healthy way to think about an AI agent is as a new hire: fast, tireless, literal. It will do exactly what you asked, on time, without complaining. And that is precisely why the care has to live in the request and in the access limits, not in the hope that it will use common sense. Common sense is the one thing it does not have.

Good automation is not the one that does the most on its own. It is the one that knows where to stop and call someone.

If you are building AI automations in your business and want someone to look at the design before it turns into a problem, take a look at how I work. Half an hour of conversation usually saves a lot of headache later.

A Snowflake AI agent fixed a security flaw and, in the same commit, opened the door to the company's internal Jira.

The patch was correct. The problem is that the question was too narrow.

The alert turned green, nobody looked around it, and a problem marked as solved disappears from the radar forever.

This is not just an engineering team topic. Swap "CI/CD script" for the bot that reads your WhatsApp and drops orders into the ERP. Same anatomy: takes data from outside, holds credentials, acts with nobody watching.

The most dangerous pattern of 2026 is AI approving AI's work. Three agents checking the math when the error is on the menu.

An AI agent is a new hire: fast and literal. Common sense is the one thing it does not have, so the care has to live in the request and in the access limits.

If you are building AI automations in your business, it is worth looking at the design before it turns into a problem. Reach out.

#ArtificialIntelligence #Automation #InformationSecurity #Technology #Business