Vertical vs Horizontal Scaling

Congratulations! You've made it, the SaaS you've been working on for the past year (I hope it's not a ChatGPT wrapper) is now finally viral. Before you start dreaming about the Villa you're about to buy, you have a problem to tackle. Your current code is deployed on your own laptop and when thousands of requests start coming in your laptop is going to be on fire.

Why? You may ask. Because there are certain physical limitations for a computer on how fast it can process information, the RAM size, network bandwidth and a lot more and you can't bypass the laws of physics just because you want to earn more money. So now you have to figure out how to make sure you cater to all the requests from users. Thankfully some people in the past had the same problem and they found 2 options.

Vertical Scaling

Your laptop might have let's say 16GB of RAM, so you figure maybe 32GB of RAM and a faster CPU would be good enough so you go ahead and buy a new laptop. You're able to handle a couple more requests and then again your SaaS goes viral(YAY! Or NAY!) and your 32GB of RAM is unable to handle the new customer's request, you go out to buy an even stronger laptop. Let's say 128GB and an even faster CPU, it can handle the traffic for the time being, but if you're lucky enough you might get more paying customers so you upgrade again. Do you see a cycle? The more requests are coming in, you are buying stronger hardware to cater to that cycle but there's a limit to this, AWS offers a machine with 192 vCPUs and 24TB of RAM, yes TB. But now what? If you receive more traffic, what else can you do? This is the maximum you can upgrade a computer's hardware, you have to figure something out.

Also there's another problem to fix, let's say your extremely strong machine dies for a few minutes. Then what? No requests receive a response; each one of your paying customers and future customers are seeing a loading spinner.

So do you see 2 issues? First there is a physical limitation on upgrading a computer. Second, your one device becomes a single point of failure; if it goes down, so does your revenue.

Horizontal Scaling

So we are again back to the scenario where your 16GB of RAM is useless for the traffic it is receiving. You call your friends(I'm assuming you have friends), get their computer, deploy your code on their computer and whenever a request comes in, it is going to be sent to your computer or one of your friends' computers. Now there is a piece of software sitting in front of these computers which routes the request to the appropriate computer which is called a Load balancer, but we'll discuss that in a separate article. Now did you see what happened, your system alone was unable to cater to all the requests so rather than upgrading your system you added more systems to handle the request and the available computer processes the request and sends the response. If the traffic grows you just have to buy new laptops(or make more friends) and make it part of the system so they can start processing the requests.

This seems like the perfect solution right?

Sadly this is real life we're talking about, nothing is perfect. If you notice there is no guarantee that requests from the same user are going to be served by the same computer. Say a user logs in and their session gets stored in memory on your laptop. The next request from that same user hits your friend's laptop. Your friend's laptop has no idea who that user is. They're effectively logged out.

To scale horizontally, your servers need to be stateless; any machine must be able to handle any request, with no memory of what happened before.

And then there's the problem of databases and how to scale them as you scale your system horizontally and frankly that is a more difficult problem to solve. Want to find out -> Here's the article.

Which one should you pick?

Always start with Vertical Scaling. No, seriously.

Most of the time, your problem is not that your server is too small. It's a slow database query, a missing index, an inefficient loop somewhere in your code. And we all know, not every project you create is going to go viral. Don't be prepared for the problem that's going to occur way down the line. Ship what you have, tackle the problems as they come in. Because you can never be prepared for what's about to come (Philosophy 101).

Move to horizontal scaling when:

• You've hit the limit on what a single machine can handle
• You need your app to stay up even if one server crashes
• Your traffic spikes unpredictably and you need to auto-scale up and down
• Your app is already stateless and containerized

Most mature systems use both. App servers scale horizontally. Databases typically scale vertically.

Vertical scaling is simple. Horizontal scaling is resilient. Most systems eventually need both.