Samsung

Your system is slow and the server bill keeps climbing. You already changed plans, already upgraded the machine, and the problem comes back. I want to tell you why that happens using an unlikely example: a Samsung memory chip that learned to do math on its own.

At Hot Chips, the conference where manufacturers show off their new chips, Samsung presented the evolution of PIM, short for Processing-in-Memory. In plain English: computation inside the memory. And the idea behind this technology explains a lot about what makes a SaaS expensive and slow. Stick with me and I'll translate everything.

The problem Samsung is attacking

Every computer works like this: data sits in memory, and the processor does the math. But memory and processor are separate chips. To calculate anything, the data has to travel from one chip to the other, and then back.

That trip is the bottleneck. The industry calls it the "memory wall". The processor got absurdly fast over the last few decades. The road between it and the memory, not so much. The result: the processor spends a good part of its time idle, waiting for data to arrive.

And there's a detail few people outside the field know: moving the data burns more energy than doing the math. The mathematical operation itself is cheap. The transport is what costs. With AI this exploded, because AI models are basically mountains of data being moved back and forth all the time.

Samsung's solution is almost obvious in its simplicity: if the trip is the problem, eliminate the trip. They placed small compute units inside the memory chip itself. The data no longer travels. The math happens where the data lives. Less energy, less waiting, more results per watt.

The expensive part was never the math. It's the transport.

What this has to do with your SaaS

Everything. Because your system suffers from exactly the same illness, just at a larger scale.

Think about the path of a common screen in your system, like a sales dashboard. The customer's browser requests the page. The server receives it. The server queries the database, which is sometimes on another machine, sometimes in another data center. The database returns 50 thousand rows. The server processes those rows, sums, groups, filters. Sends the result to the browser. The browser draws it.

See how many trips? Each of those arrows is data on the move. And every move costs time and money. Cloud providers charge for outbound traffic, charge per request, charge for processing time. When the data travels too much, you pay for the trip.

The classic mistake is this: fetching 50 thousand rows from the database to sum them on the server and show a single number on the screen. The database knows how to sum. It does it better and faster than any code of yours. Asking for the ready-made sum instead of the raw rows is the software version of what Samsung did in hardware: bring the math to the data, instead of dragging the data to the math.

Signs your system is paying too much freight

You don't need to read code to get suspicious. Some symptoms show up in the daily life of the business:

• The cloud bill grows faster than the customer count.
• Reports that take several seconds to open, or that "freeze" at the end of the month.
• The tech team always fixes slowness the same way: upgrading the machine.
• Screens that load everything at once, even when the user just wanted the summary.

That last one is treacherous. I've seen a system that downloaded a customer's entire history just to show the name and the balance. It worked fine with 200 customers. With 20 thousand, it became a moving truck delivering an envelope.

Upgrading the machine fixes the symptom for a few months. It's like buying a bigger truck because you insist on hauling the entire household every time. The right question is not "how much does the bigger truck cost". It's "why are we carrying all of this".

Three changes that follow the same logic as PIM

I'm not suggesting you buy a Samsung chip. I'm suggesting you steal the principle: bring the processing closer to the data. In practice, three fronts solve most cases.

Let the database do the work. Sums, averages, groupings, filters: the database does all of that natively. If your server is receiving giant lists to process, math is being done in the wrong place. A well-written query replaces hundreds of lines of code and cuts the data's trip at the root.

Store the result, don't redo the math. If the sales dashboard is viewed 300 times a day and the numbers only change every hour, calculate once per hour and serve the ready-made result. That's called caching, and it's probably the best-returning performance move that exists. Samsung eliminated the data's trip. Caching eliminates the repeated math. Same family of idea.

Serve content close to whoever uses it. If your customers are in Brazil and your server is in Virginia, every click crosses the continent twice. CDNs and server regions closer to the user are the geographic equivalent of PIM: shortening the distance between the data and whoever needs it.

None of these three requires rewriting the system. They are architecture adjustments, almost always cheaper than a server upgrade. And, unlike the upgrade, the effect is permanent: you stop paying the freight, instead of buying a bigger truck every year.

Why this will matter more from now on

The reason Samsung is investing heavily in this is AI. Language models move absurd amounts of data, and the energy cost of that became a billion-dollar problem. When the world's biggest manufacturers redesign hardware to shorten the data's path, it's a sign that transport has become the dominant cost of computing.

That flows down the entire chain to you. Data centers burning more energy means more expensive cloud. More expensive cloud means a wasteful system becomes an expense that shows up in your bottom line. In the next few years, the difference between a well-architected SaaS and a poorly architected one won't be just speed. It will be margin.

My opinion, and it's a firm one: most systems I see out there don't need more server. They need less travel. The machine upgrade became the industry's default painkiller, and it works precisely because it postpones the hard conversation about architecture.

Where to start without becoming a hostage to tech jargon

You don't need to understand chips to act. You need to ask the right questions to whoever takes care of your system:

• What are the three slowest screens, and how much data do they load to show what they show?
• What are we calculating every single time that could be calculated once and stored?
• Which line of our cloud bill is growing: processing, traffic, or storage?

If the answers come back vague, that's the diagnosis. A healthy system has specific answers to those questions.

I work exactly at this layer: looking at a company's system, finding where the data is wandering around for no reason, and shortening the path, with automation and lean architecture. Without selling a bigger server, because a bigger server is almost never the answer. If your cloud bill is climbing faster than your revenue, tell me how your system is doing and I'll tell you where to look first.

Is your cloud bill growing faster than your customer count? The problem is not the server.

Samsung just showed a chip that does calculations inside the memory itself. Why? Because moving data costs more than processing it.

Your SaaS suffers from the same illness: it fetches 50 thousand rows from the database to show one number on the screen. The data travels too much, and you pay for the trip.

Upgrading the machine is buying a bigger truck so you can keep hauling the entire household every single time. The right question is: why are we carrying all of this?

Most systems don't need more server. They need less travel.

If your cloud bill is growing faster than your revenue, reach out and I'll tell you where to look first.

#SaaS #Architecture #CloudComputing #Performance #Tech