How I Think About Designing Software That Can Survive Growth

Growth is one of those words that sounds simple until you have to design for it.

A product has 100 users. Then 1,000. Then 10,000. Orders increase. The database grows. Reports become slower. Background jobs start competing with customer requests. A tenant that was small last year becomes responsible for most of the workload.

At that point, teams often say, "We need to scale."

I think that is usually the wrong starting point.

The better question is:

What assumption did we make when the system was small that is no longer true?

That question changes how I design software.

I do not design for the biggest number first

I rarely start a project by designing for millions of users just because the business might eventually reach them.

That creates a different kind of problem. The system becomes complicated before the business has earned that complexity.

Instead, I want to understand the expected workload.

How many users will exist?

How many requests will arrive per second?

Which operations are read heavy?

Which operations are write heavy?

How large can the important tables become?

Will one customer be significantly larger than the others?

Which operations must be fast and which can take a few seconds?

What happens during the busiest hour of the day?

These questions give me constraints. Architecture should respond to constraints, not imagination.

I identify the dangerous growth dimensions

Not all growth is the same.

A system can grow in users without growing much in data. Another system can have relatively few users but generate enormous amounts of data.

An ecommerce platform might grow from 10,000 to 1 million orders. A healthcare system might have fewer users but retain years of clinical records. A multi tenant SaaS product might have hundreds of customers while one customer produces more traffic than all the others combined.

So I look at growth across several dimensions:

Traffic

More requests can expose CPU, memory, connection pool, and network limits.

Data

Large tables can change query plans, index size, backup time, storage cost, and maintenance behavior.

Concurrency

Many users performing operations simultaneously can expose locking, transaction, and connection problems.

Background work

Queues, reports, exports, notifications, and scheduled jobs can quietly consume the resources needed by customer facing requests.

Tenant concentration

In a multi tenant system, averages can be misleading. One large tenant can create a noisy neighbor problem for everyone else.

The architecture should acknowledge these dimensions explicitly.

I design the database around access patterns

One of the most expensive mistakes is treating database design as an exercise in creating tables and relationships without thinking about how the application will actually use them.

I care about the queries first.

If the application frequently asks for a customer's recent orders, the schema and indexes should make that access pattern efficient.

If it frequently filters by tenant, status, and date, I think about those dimensions together.

If a table will eventually contain hundreds of millions of rows, I ask whether a single logical table is still the right physical storage strategy.

That can lead to techniques such as careful indexing, partitioning, archiving, read replicas, workload isolation, or eventually sharding.

But I do not introduce those techniques simply because they are available.

The query pattern and workload should justify them.

Multi tenant systems need special attention

Multi tenancy changes the scaling problem.

Imagine a SaaS product with 500 customers. For years, every customer behaves roughly the same.

Then customer number 501 arrives and generates ten times the traffic and data of the average customer.

The architecture has a choice.

Allow that tenant to compete for the same database resources as everyone else, or introduce boundaries that allow the workload to be isolated.

Those boundaries might exist at the query, connection, queue, cache, database, or infrastructure level.

The important idea is not a specific technology.

It is workload isolation.

When one workload grows disproportionately, the system should have a way to prevent that workload from becoming everyone else's problem.

I separate customer facing work from expensive work

A request that a user is waiting for should not compete unnecessarily with a report that can run for two minutes.

This is why I like explicit background processing for expensive operations.

Generating a large report, exporting thousands of records, processing uploaded files, sending bulk notifications, rebuilding search indexes, or running analytics should usually not block the request that started the operation.

Instead, the request can create a job and return quickly.

A worker processes the job independently.

This gives the system another scaling boundary.

Customer traffic can scale separately from background work.

I treat observability as part of architecture

You cannot scale what you cannot see.

A production system should make it possible to answer questions such as:

Which endpoints are slow?

Which database queries consume the most time?

Which tenants generate the most traffic?

Where are connections being exhausted?

Which queue is growing?

Which background job is failing repeatedly?

How much of the database workload is reads versus writes?

Without those signals, scaling becomes guesswork.

And guesswork in production is expensive.

I prefer gradual scaling boundaries

A healthy architecture gives you places where you can introduce more capacity when the workload requires it.

Start with a well designed relational database.

Add the right indexes.

Measure real query behavior.

Introduce caching where repeated reads justify it.

Move expensive work to queues.

Add read replicas when read pressure requires them.

Partition large datasets when their access patterns justify partitioning.

Isolate heavy tenants when noisy neighbors become a real problem.

Consider sharding when a single database has become a genuine bottleneck that other techniques cannot reasonably solve.

The order matters.

I would rather have a simple system with clear scaling boundaries than a complicated system with every distributed systems pattern installed from day one.

I design migrations as carefully as features

Growth often exposes another problem: changing the database safely.

Adding a column is easy.

Changing a huge table with hundreds of millions of rows is not necessarily easy.

At scale, I think about migrations as production operations.

Can the change run without blocking important traffic?

Can old and new application versions coexist during deployment?

Can data be migrated incrementally?

Can the change be rolled back safely?

Can indexes be created without unacceptable impact?

Can the migration be monitored while it runs?

The architecture is not finished when the schema looks correct in development.

It is finished when the system can evolve safely in production.

The biggest scaling mistake is often organizational

Sometimes the technical architecture is not actually the main bottleneck.

A team can have excellent infrastructure and still struggle because nobody owns performance.

Nobody knows which queries are expensive.

Nobody has defined acceptable response times.

Nobody knows which customer workloads are exceptional.

Nobody has decided what happens when the database reaches a certain threshold.

Scaling needs engineering ownership as much as it needs infrastructure.

I want teams to know their important limits before production teaches them the hard way.

My rule of thumb

I do not ask, "How do we build this so it can handle a billion users?"

I ask:

What will grow first?

Then:

What will become the bottleneck when it grows?

Then:

What boundary can we introduce before that bottleneck becomes a customer problem?

That leads to much better architecture decisions.

The goal is not to predict the future perfectly.

The goal is to build a system that can evolve without forcing the team to rebuild everything when reality changes.

That is what scalable software means to me.