Predictive surveillance with AI: the risk that lands on you

You hired an AI to summarize meetings. It listens to everyone who speaks, transcribes, and stores. Now imagine the same kind of system, in someone else's hands, hired to predict who is going to protest before they protest. This is not fiction. It is what The American Prospect reported about Anthropic building a predictive surveillance system to monitor activists under a contract with US security agencies.

Predictive surveillance with AI is exactly what the name says: pile up mountains of data about people and ask the model to point out who will probably do something. Not what they did. What they are going to do. And if that sounds far from your business, stay with me, because the distance is smaller than it looks.

What actually changed in this story

State surveillance is nothing new. What changed is the barrier to entry.

Before, cross referencing data on thousands of people took a big team, a budget, and months of work. Today it takes an API, a good database, and a well written prompt. The same technology that turns your 40 page contract into three paragraphs can read 40 thousand messages and build a behavior profile.

Anthropic, it is worth saying, is the company that sells "safe AI" and "constitutional AI" harder than anyone. It is the one that writes manifestos about existential risk. That this is the company at the center of an activist monitoring contract says something important: a usage policy written on a website guarantees nothing. A big contract changes policy.

I am not saying Anthropic is the villain and the others are saints. I am saying the opposite: if the most careful one does this, stop treating vendor reputation as a security layer.

Why this hits you even if you never work with government

Three practical reasons.

The first is that you already do predictive surveillance without calling it that. A credit score is a behavior prediction. A system that flags a customer as "likely churn" is a behavior prediction. An HR tool that ranks résumés is a behavior prediction. The difference between that and the story in the news is scale and consequence, not technique.

The second is regulation by contagion. When a case like this blows up, lawmakers do not reach for a scalpel, they reach for a sledgehammer. Brazil's LGPD already has article 20, which gives people the right to request review of an automated decision that affects their interests. If you have any system that decides something about a person with no human in the middle, you are already in the crosshairs. A tightening rule will not ask whether your intentions were good.

The third is the most annoying one: your data may be on the other side. If you use a tool that sends customer data to an AI API, you do not know what happens next. You know what the contract says. Those are not the same thing.

A privacy policy is a promise. Architecture is a guarantee.

The one question test

I use this with clients and it settles 80% of the doubts. The question is:

If this database leaked tomorrow, and someone ran a model on it to predict the behavior of the people listed, what would come out?

I ran this exercise with a clinic. They had a simple scheduling system, nothing fancy. Except the "notes" field held years of free typing from the front desk. Things like "patient came in crying", "husband cannot know", "always pays late". Nobody designed that as a sensitive database. It became one by accumulation.

There was no AI anywhere in that system. But if someone pointed a model at that field, they could build a psychological and financial profile of 3 thousand people in one afternoon.

The fix was not a six month project. It was splitting the free text field into closed categories, archiving the old history in a separate database with restricted access, and setting a deletion deadline. Two weeks of work. The risk dropped from "catastrophic" to "annoying".

What to do in practice

Nothing here needs an ethics committee or a half million dollar consultancy.

• Inventory what you send out. Every AI tool you use: what does it receive? Full name? Tax ID? Customer conversation content? Write it in a spreadsheet. Most companies cannot answer this, and that is the whole problem.
• Turn on zero retention where it exists. OpenAI, Anthropic, and Google offer a no data retention mode on API and enterprise plans. It is usually a setting, not a new contract. Plenty of people pay for it and never switch it on.
• Minimize before you send. If the AI needs to classify the sentiment of a ticket, it does not need the customer's name. Strip it. Replace it with an identifier. That is half an hour of work and it changes the nature of the risk.
• Humans decide, AI suggests. In any decision that affects a person (credit, hiring, cancellation, pricing), the AI delivers a suggestion with a rationale and a human signs off. That puts you on the right side of article 20 and, as a bonus, catches model errors before they turn into lawsuits.
• Set a deletion deadline and keep it. Data that no longer exists does not leak, does not get subpoenaed, and does not feed any prediction. Keeping everything forever is the most expensive decision there is, and nobody makes it on purpose.
• Write down what you will not do. One page. "We do not use AI to predict individual customer behavior outside of X." It works as a brake when someone excited pitches the idea in a meeting.

The objection that always shows up

"Diego, this is going to slow me down. My competitor will use everything and get ahead."

Maybe. For a while.

Except the math is incomplete. A system that decides about people with no trace of justification is a delayed action bomb. When it goes wrong, it goes wrong in a way you cannot explain, because not even you know why the model decided that. I have watched a company spend three weeks trying to reconstruct the logic behind an automatic denial in order to answer a consumer protection complaint. Three weeks of senior team time. The "delay" they would have avoided was two days.

And there is the commercial side. Corporate clients already ask where their data travels. Whoever answers clearly closes the contract. Whoever answers "I think it stays in the cloud" loses. Data governance became a sales argument in B2B, not just a compliance checkbox.

The point that matters

The news about predictive surveillance is not about Anthropic. It is about the fact that the power to predict human behavior at scale got cheap and accessible, and nobody agreed on the rules first.

You do not control what an AI company does with a government contract. You control what leaves your company, how long it stays stored, and who signs the decisions. Is that a little? Yes. But it is the piece that is yours, and most companies have not even taken care of that.

The work is unglamorous: inventory, minimization, deletion deadlines, human in the loop. None of it makes a pretty LinkedIn post. It makes you sleep well when the next headline drops.

If you use AI in your business and cannot say which data walks out the door, this is the moment to look. Tell me what you are using and we map it together, without turning it into a year long project.

Anthropic, the company that talks the most about "safe AI", showed up in a predictive surveillance contract to monitor activists.

And the lesson is not about Anthropic. It is that predicting human behavior at scale got cheap: it used to take a team and months, now it takes an API and a well written prompt.

If you think this does not touch you, try the one question test: if your database leaked tomorrow and someone ran a model on it, what would come out?

I did this with a clinic. The "notes" field had years of free typing from the front desk. Nobody designed that as sensitive data. It became sensitive by accumulation. Two weeks of work took the risk from catastrophic down to annoying.

A privacy policy is a promise. Architecture is a guarantee.

If you use AI in your business and cannot say which data walks out the door, it is time to look. Tell me what you are using and we map it together.

#AI #DataProtection #Privacy #DataGovernance #Technology