What AI's defeat in Go teaches about business

Have you ever approved a report that AI wrote without checking the numbers? I have. It went wrong once, and that was enough to change the way I work. This article is about the moment AI fails and nobody notices, and about a Go match that explains it better than any whitepaper.

In July, South Korean player Shin Jinseo, the world number one in Go, beat KataGo, one of the strongest Go programs in existence, playing with a two stone handicap. A handicap, in Go, is a head start. He got the head start and he won. This is not a human "beating the machine" on equal terms, and anyone selling that story is stretching it. But it is an interesting data point, because since 2016 the narrative was different: AI passed humans in Go and never looked back.

What actually happened in that match

It is worth separating the fact from the headline.

KataGo is still stronger than any human being in a game without a head start. That has not changed. What changed is that the best players in the world spent nine years studying the AI's games, absorbing its style and learning where it hesitates. Shin did not win by brute force. He won because he knows his opponent.

And there is an even stranger precedent. In 2023, a researcher named Kellin Pelrine, an amateur player at a decent but nothing exceptional level, beat top versions of KataGo in fifteen out of sixteen games. Not because he played better. Because a research group trained an adversarial program purely to look for holes in the AI, found a specific pattern it could not see, and Pelrine executed that pattern by hand.

An amateur beat a superhuman AI using a three move trick. The AI did not play worse that day. It always had that hole. Nobody had looked for it.

Why this matters to people who have never played Go

Go is the cleanest environment there is for testing artificial intelligence. Fixed rules, complete information, objective outcome. No client changing their mind, no badly filled spreadsheet, no supplier who answers on WhatsApp.

If AI has blind spots in that environment, imagine what it has in yours.

Your company is not a board. It is a place where the registration has a mistyped tax ID, the salesperson writes the order in a notebook, and the discount rule has an exception only Márcio from finance knows about. Every AI you drop in there will operate on top of that chaos.

AI does not fail where you are looking. It fails where nobody thought to look.

The blind spot is always the same: the average against the edge

An AI model is a pattern machine. It learns what happens often and gets excellent at it. The problem lives in the rare case.

I saw this in practice on an order classification project. The system sorted orders by service type, and it got it right 96% of the time. A beautiful number to present. Except the remaining 4% were not random: they were almost all large orders, with mixed scope, from the clients that brought in the most revenue. The hard cases were exactly the expensive cases.

Nobody had noticed because the dashboard showed the global accuracy rate. 96% looks great until you discover the errors are concentrated exactly where it hurts.

That is the lesson from Go. KataGo had an astronomical win rate against humans. And at the same time it had a structural flaw an amateur could exploit. Both things were true together.

"But won't AI just get better and fix this?"

It will get better, yes. And that is exactly where the trap is.

Every new version patches the known holes. After Pelrine's exploit went public, the KataGo team trained the model against that specific attack and it stopped working. Great. Except the same research group came back later and found new flaws, different ones.

This is not a bug you fix once. It is a characteristic of how these systems learn. You close one door, another one opens.

That is not a reason to avoid AI. It is a reason to stop treating "AI is better now" as if it answered "how do I know it is right in this case". Those are different questions.

What to do in practice in your company

You do not need a governance committee or a 40 page policy. You need four habits.

Sort decisions by the cost of being wrong. If AI gets an email subject line wrong, you rewrite it and move on. If it gets a credit approval wrong, or classifies an invoice wrong, or answers an angry client wrong, the damage is different. Automate the first group generously. In the second, keep a human in the path.

Measure error where it hurts, not on the average. Stop looking only at overall accuracy. Break it down by segment: by client size, by order type, by value. That 96% project became useful the day we started measuring orders above ten thousand separately.

Let the system say it does not know. The worst AI is the one that answers with total confidence about something it does not understand. A good design has an exit: when confidence drops below a threshold, the case goes to a human queue. It is simple to implement and it saves more headaches than any model fine tuning.

Attack your own system once a quarter. Take twenty genuinely weird cases, the ones the support team talks about over coffee, and run them through the AI. That is what FAR AI did with KataGo. You do not need a research lab, you need two hours and someone who knows the exceptions in the business.

The detail almost nobody mentions

There is a part of Shin's story I find more interesting than the win itself.

He has trained with AI for years. His style changed because of it. Several moves that are standard among professionals today came from machine games nobody understood in 2017. In other words: the human who beat the AI is a human the AI made stronger.

That is the realistic version of the future, and it is far less dramatic than the two extremes you read about. The machine did not replace the player, and the player did not ignore the machine. He used the machine until he knew it well enough to know where it stumbles.

That is how I think when I build automation for a client. AI handles the volume, the repetitive work, the first draft. The person handles the judgment, the crooked case, the decision that has someone's name attached to it. Whoever knows where to draw that line wins on both sides. Whoever hands everything to the machine ends up discovering the blind spot the most expensive way: with a client complaining.

If you are putting AI into a process in your company and you are not quite sure where it can betray you, it is worth talking before you find out the hard way. Take a look at how I work.

I have approved an AI written report without checking the numbers. It went wrong once, and once was enough.

In July the world number 1 in Go beat KataGo with a handicap. Before that, in 2023, an amateur won 15 of 16 games against the same AI using a three move trick. The AI did not get worse that day. The hole was always there, nobody had looked for it.

I saw this in a real project: 96% accuracy classifying orders. Beautiful number. Except the 4% of errors were exactly the big orders, from the clients that brought in the most revenue.

AI does not fail where you are looking. It fails where nobody thought to look.

Measure error by segment, not by average. And once a quarter, take twenty genuinely weird cases and throw them at your system. Two hours of work, a lot of headaches avoided.

If you are putting AI into a process and you do not know where it can betray you, talk to me before you find out from a client complaint.

#ArtificialIntelligence #Automation #Management #DigitalTransformation