Local AI: the AI agent that runs inside your company

You open the AI API invoice and get a shock. Three thousand reais in a month where nobody "used AI" on purpose. It was the little bot classifying email overnight, the script summarizing tickets, the flow checking orders every ten minutes. Each call costs cents. Cents times a hundred thousand calls turn into a salary. That is exactly where local AI starts to make sense: a smaller model, running on a machine of yours, on all day, with no meter spinning.

Meta released Muse Glimmer in June, an open 30 billion parameter model built for exactly this: agents that stay on all the time, on your infrastructure. It is not the smartest model in the world. It does not need to be. It was built for something else.

What "always on" means in practice

There are two kinds of AI use inside a company, and people mix them up.

The first is occasional use. Someone asks for a text, an analysis, a proposal. It happens ten, twenty times a day. It needs to be good. It costs little because the volume is low.

The second is background use. The agent that reads every incoming email and decides whether it is a quote request, a complaint or spam. The one monitoring inventory. The one transcribing and summarizing every sales call. That runs all the time, with nobody watching, and the volume is absurd compared to the first case.

The common mistake is using the expensive model for both. It is like hiring a senior lawyer to stamp paperwork. It works, but you are paying lawyer rates to stamp paper.

Big models are for thinking. Small models are for working.

Why a 30B model changes the math

Thirty billion parameters sounds like a lot. In 2023 it was data center territory. Today it runs on a single graphics card with around 24 GB, the kind an architecture studio already has in its rendering machine.

That changes three things at once.

The cost becomes fixed. You buy or rent the machine and that is it. It does not matter whether the agent ran a thousand or a million times that month. A client of mine had a flow classifying WhatsApp messages: about 40 thousand a month. In the cloud, it came close to R$ 900. Moved to a local model on a VPS with GPU at R$ 400 a month, there was room left to stack three more automations on top without touching the price.

The data stays put. Accounting firm, clinic, law office. If the agent reads client documents, someone will ask where that document went. "It stays on the company server" is a much easier answer to give than a paragraph about the retention policy of some vendor abroad.

Latency disappears. No round trip to a server on another continent. Answers in under a second, every time. For an agent that takes twenty steps on a task, that is the difference between two seconds and thirty.

Where local AI does not fit

I will be honest, because almost nobody writes this part.

Small models make more mistakes on open ended tasks. If you ask "analyze this contract and tell me the risks", it will give a reasonable answer and you will think everything is fine. It is not. On that kind of task the gap to a big model is glaring, and expensive to find out about too late.

Local models also do not have the last few months of the internet in their head. They do not know about the law that changed last week. They need access to your data to be useful, and that is integration work, not magic.

And there is the obvious part: somebody has to take care of the machine. If your company has nobody who knows how to bring up a server and it goes down on a Saturday, you traded an invoice for a problem.

The rule I use is simple. If the task is repetitive, has a clear format and an occasional mistake breaks nothing, go local. If the task is open ended, rare and expensive to get wrong, use the good model and pay the cents.

An example you can copy

Parts distributor, six people in sales. About 300 orders a day came in over email and WhatsApp, all in free text: "send me 20 of that filter I bought last month".

The old flow was one person reading and typing into the ERP. Four hours a day of that person.

What we built:

• A local agent reads the message and extracts three things: client, product, quantity.
• It checks the client history in the database to resolve "that filter".
• If confidence is high, it creates the draft order in the ERP.
• If there is any doubt, it drops it in the human queue with what it understood already filled in.

No "autonomous AI" here. The agent is a fast intern who fills out forms and raises a hand when it does not know. Around 70% of the orders go through with no human touch. The other 30% arrive half done, so even those move faster.

The task is boring, repetitive and structured. A 30B model handles it easily. And since it runs locally, nobody counted API calls.

How to test this without burning money

Do not buy a GPU. Seriously. That is the classic mistake: the owner gets excited, spends 15 thousand on a machine and it turns into furniture.

Do it in this order:

1. Pick one task only. The most repetitive one you have. If you cannot explain in two sentences what the agent decides, it is not ready yet.
2. Run it in the cloud first, with a good model. For two weeks. It is expensive, and that invoice is exactly what will tell you whether migrating is worth it. It is also how you find out whether the task works at all.
3. Measure the hit rate. Compare it to the human doing the work. If the human gets 97% right and the AI gets 91%, decide whether that gap hurts. Sometimes it hurts. Sometimes it is irrelevant.
4. Only then test it locally. Rent a machine with a GPU by the month, run the same test, compare the numbers. If it drops a lot, go back.
5. If it held up for three months straight, then you can think about buying hardware.

That path costs a few hundred reais to find out whether the idea is any good. Much cheaper than finding out later.

The part that matters to you

The relevant news is not "another model came out". A new model comes out every week and six months from now Muse Glimmer will be old.

The point is that the quality needed for routine automation is no longer exclusive to whoever has a contract with a cloud giant. A one developer studio can build, today, an agent running on the client's infrastructure, with data that never leaves it, at a fixed monthly cost that fits a ten person company.

That changes the kind of question worth asking. It is no longer "how much will this AI cost per month". It is "which task in my operation is repetitive enough to justify an agent taking care of it".

If you already have a task in mind and want an honest opinion on whether automating it pays off, reach out for a chat. Half the time my answer is that it is not worth it yet, and that is useful information too.

Someone sent me a screenshot of an AI API invoice: 3 thousand reais in a month where nobody "used AI" on purpose.

It was a little bot classifying email overnight. Cents per call, a hundred thousand calls, a full salary.

The mistake is not AI. It is using an expensive model for a dumb task. It is hiring a senior lawyer to stamp paperwork.

Big models are for thinking. Small models are for working. And today a small model runs on your own machine, with your data, at a fixed cost.

At a parts distributor, 70% of the orders land in the ERP with no human touch. Monthly cost: the rent on the machine, and that is it.

If you already have a repetitive task in mind and want to know whether automating it pays off, reach out. Half the time I say it is not worth it yet, and that is an answer too.

#ArtificialIntelligence #Automation #LocalAI #SmallBusiness #Technology