How Databases Scale

Would you like to time travel? I know exactly how. Scale a database for a live application and you'll age 10 years in 2 months.

Ask any software engineer about the most difficult problem they've tackled and, after trying to find a life partner, they'll surely say it is understanding how to scale a database. But don't you worry, we'll understand how databases scale and what exactly makes them a difficult problem to tackle.

The Obvious Step

Your app is growing. More users, more traffic, more revenue (cha ching). But there's a slight problem: your database, which used to breeze through requests, is now struggling.

The first thing you'll do is get a bigger machine. More RAM, more CPU, faster disk. And this works great. For a while.

This is vertical scaling, and if you've read my article on Vertical vs Horizontal Scaling, you already know there's a ceiling. AWS will happily rent you a monster machine with 24TB of RAM, but eventually that isn't enough, or your one giant database becomes a single point of failure.

So what do you do? The obvious answer is just add more machines, right? Well, this trick works great for app servers but needs some tweaking for databases.

Why You Can't Just Add Another One

Your app servers are generally stateless, so a request for user A can be served by any server instance. Sadly your database cannot be stateless. It is the state (some Walter White vibes detected).

You can't start a new database instance and expect it to serve half the traffic. Go ahead, think about it.

The reason is this: if an entry for a user is in instance A of the database and an update query comes in and you add that entry to instance B, how do you determine which instance has the correct data?

Note

You might want to suggest that during reads we take the entry with the latest lastUpdated timestamp. But that would require us to read from both instances. If we had 10 instances, a single read would mean querying all 10. At that point I would respectfully say bye bye to my cha ching.

So how do we tackle database scaling? There are a few patterns we follow, and we pick whichever suits our case best. We'll cover the common approaches briefly here, and each one gets a detailed article of its own so you understand how beautifully we tackle these complex problems.

Read Replicas (Master Slave)

This is where most people start, and the idea is pretty simple. We have one instance called the master (or primary) which accepts all the write requests. Then we have slaves (or secondaries), which are just copies of the master and remain in sync. Writes are taken care of by the master, and all the read requests are served by these secondary instances.

The problem of finding the "correct data" is sidestepped here, because only one instance is getting the writes. How up to date the reads are depends on the syncing strategy of the replicas, but that is a separate discussion in itself.

Surprisingly (or not, idk) this is the most commonly used approach, because most applications are read heavy. Think about it: how much time do you spend on TikTok and Instagram scrolling rather than posting? You go through hundreds if not thousands (I'm here) of reels but end up posting at most once or twice a day. Reads massively outnumber writes almost everywhere, and master slave is built exactly for that imbalance.

What we won't cover here is how the slaves are kept in sync, or what happens when a master goes down. Both are excellent questions, and you can expect excellent explanations soon.

Multiple Masters

This is the one which actually ages people.

Master slave solves the read problem beautifully, but sadly doesn't do well for writes, because we still have just one machine accepting them. If your system is write heavy you need more than one master, and this is where things stop being simple.

The moment you allow writes to go to multiple instances, how do you make sure all the masters are in sync and holding the correct data? We don't want one instance to say you have a balance of 100 rupees and another to say 1 crore (although I would love this). Now you need a way to keep them in sync, and a way to decide who wins when they disagree.

There are multiple ways companies tackle this, and the most famous of them is using CRDTs. We'll go through that in a different article.

Sharding

This frankly deserves a separate article in itself and you'll get that soon (YAYYY!), but the idea is actually very simple. Rather than copying the data across multiple instances, you split your data into smaller chunks called shards. Let's say one chunk holds data for users who follow @AskMaddyy on X, and another holds the ones who don't (why are you doing this??). Each chunk holds a slice of the whole data and handles its own reads and writes independently.

Theoretically this sounds like the best approach, but how do you decide which user goes to which chunk? How do you handle queries that span different chunks? What happens when only one chunk is getting the majority of the traffic, do you split again? We'll get into this properly in the detailed article.

A lot of people also think sharding and partitioning are the same thing and use them interchangeably. They are related but not the same, and that will also be covered in the dedicated article.

While you wait for me to drop another banger (not a song, I'm talking about another article) I want you to think: can we apply the master slave technique to a shard?

The Real Difficulty

To be honest, the concepts of database scaling are not hard. You go through them for a couple of minutes, or just read my articles (self promo doesn't hurt), and you'll understand what they are.

The main problem is that they have to be applied to real and live data. We can't afford to be wrong for even a second and lose something.

Most of you reading this won't need anything beyond master slave for a long, long time. OpenAI is still running on one master Postgres instance and is still able to handle your request to count the number of 'r's in strawberry.

But now you understand the full situation, and why everyone who has been through database scaling is either going to brag about it for the next 10 years or go completely silent when you bring up the topic.