Capacity Planning Before Architecture Diagrams

Architecture diagrams are persuasive because boxes and arrows make a system feel concrete. They are also easy to draw before the problem is understood. A queue, cache, database, and fleet of workers can look reasonable at ten requests per second and collapse at ten thousand. Capacity planning is the discipline that turns “the system should scale” into explicit assumptions that can be challenged.

The goal is not to predict an unknowable future with perfect precision. The goal is to identify which variables dominate the design, calculate a defensible operating envelope, and leave enough margin for failures and uncertainty.

Start with a workload, not a technology#

Use a hypothetical notification service. Product requirements say:

• 2 million monthly active users.
• 20% are active on a typical day.
• An active user triggers 8 notifications per day.
• A product launch may create a peak 12 times the daily average.
• Each notification is written once and attempted on two delivery channels.
• Delivery history is retained for 90 days.

These are assumptions, not facts. Write them down with an owner and a confidence level. A number supplied by measured production traffic is high confidence. A launch estimate from a product forecast is not. The distinction determines how much headroom is sensible.

The dependency chain is straightforward:

flowchart LR
    A[Product assumptions] --> B[Traffic model]
    B --> C[Concurrency and bandwidth]
    B --> D[Storage growth]
    C --> E[Compute and connection budgets]
    D --> F[Retention and partitioning]
    E --> G[Failure capacity]
    F --> G
    G --> H[Architecture choices]

Convert daily activity into arrival rate#

Daily active users are:

2,000,000 monthly users × 20% = 400,000 daily active users

Notifications accepted per day are:

400,000 users × 8 notifications = 3,200,000 notifications/day

The average request rate is:

3,200,000 / 86,400 seconds ≈ 37 requests/second

An average of 37 requests per second sounds small. It is also the least useful number in the exercise because traffic is not uniform. Applying the stated 12× peak factor gives roughly 444 accepted notifications per second. Round assumptions in the conservative direction: use 500 requests per second as the design peak.

This is the first useful result. It is specific enough to test and small enough that a single well-sized service could probably accept the traffic. Nothing in this estimate requires microservices, global active-active deployment, or a distributed database.

Separate ingress from downstream work#

One accepted notification causes two delivery attempts, so downstream delivery demand peaks near 1,000 attempts per second. Retries make this a range rather than a point estimate.

Suppose 2% of attempts fail transiently and each failed attempt is retried at most twice. The additional steady-state load is approximately:

1,000 × 0.02 × 2 = 40 attempts/second

That arithmetic is deliberately simple. Correlated failures are not. If a provider is unavailable, nearly every attempt to that provider may fail, and retries can amplify the incident. Capacity planning must therefore include two modes:

The second row often matters more than the first. Google’s SRE guidance emphasizes graceful overload handling because a service that queues unlimited work tends to convert overload into high latency, memory pressure, and cascading failure.

Use latency to estimate concurrency#

Arrival rate measures how often work begins. Concurrency estimates how much work is in flight. A useful approximation is Little’s Law:

concurrency ≈ arrival rate × average time in system

If the ingress API receives 500 requests per second and its average end-to-end time is 80 milliseconds:

500 × 0.080 = 40 concurrent requests

Designing only for the average is unsafe. At a 400 millisecond tail latency, the same arrival rate implies 200 concurrent requests. Connection pools, thread pools, memory limits, and downstream concurrency should be based on the latency distribution and overload behavior, not only on the mean.

For delivery workers, assume each provider request takes 250 milliseconds at the design percentile:

1,000 attempts/s × 0.250 s = 250 concurrent provider calls

If a provider quota permits only 150 concurrent calls, adding workers will not solve the bottleneck. The system needs per-provider admission control and a queue that can delay excess work.

Estimate storage as a growth function#

Assume the stored notification record—including identifiers, timestamps, routing state, and database overhead—averages 1.5 KB. Two delivery-attempt records add 0.75 KB each.

notification: 1.5 KB
attempts:      2 × 0.75 KB
total:         3.0 KB per accepted notification

Daily logical growth is:

3,200,000 × 3 KB ≈ 9.6 GB/day

At 90-day retention, the primary logical dataset is about 864 GB. Then account for indexes, replication, write-ahead logs, backups, and maintenance headroom. A rough 3× physical multiplier produces a 2.6 TB planning figure.

The multiplier must eventually be measured on the chosen database. At this stage it reveals two architectural facts:

1. Retention policy is a major capacity lever.
2. Time-based partitioning or archival will matter before exotic database technology does.

Reducing retention from 90 days to 30 days saves more storage than many implementation-level optimizations. Capacity planning exposes product decisions disguised as infrastructure decisions.

Estimate bandwidth at every boundary#

Payload size affects network, serialization, logging, and queue cost. If an accepted request averages 2 KB:

500 requests/s × 2 KB = 1 MB/s ingress

That is modest. The system may nevertheless generate more internal traffic:

• Queue envelope and metadata: 3 KB × 500/s = 1.5 MB/s.
• Two provider requests: 2 KB × 1,000/s = 2 MB/s.
• Database writes and replication: implementation dependent, often several times logical payload size.
• Logs and traces: potentially larger than application payloads if emitted without sampling.

Bandwidth is rarely a single number. Calculate it per boundary so an innocent-looking fan-out or observability policy does not become the dominant cost.

Design for a failure domain#

Suppose the service runs across three availability zones with equal capacity. If the design peak is 500 accepted requests per second, a fleet sized for exactly 500 fails its objective as soon as one zone is lost.

For N+1 zone capacity, the remaining two zones must carry the full peak:

required capacity per zone = 500 / 2 = 250 requests/s
total provisioned capacity = 3 × 250 = 750 requests/s

The topology becomes a consequence of the failure objective:

flowchart TB
    C[Clients] --> L[Regional load balancer]
    L --> A[Zone A: 250 req/s]
    L --> B[Zone B: 250 req/s]
    L --> D[Zone C: 250 req/s]
    A --> Q[Durable bounded queue]
    B --> Q
    D --> Q
    Q --> W[Provider-scoped worker pools]
    W --> P1[Email provider]
    W --> P2[Push provider]

This does not guarantee survival of every incident. It states a clear design contract: the service should sustain the modeled peak after losing any one zone. Larger correlated failures require degradation, traffic shedding, or a multi-region strategy with substantially higher complexity.

Turn uncertainty into scenarios#

A single spreadsheet value hides risk. Use at least three scenarios:

The expected case sizes routine operation. The high case informs near-term headroom. The stress case defines what should degrade first. It is often unreasonable to provision the complete system for the stress case, but it is reasonable to require bounded queues, explicit rejection, and recovery without data corruption.

Make the model testable#

Every important number should connect to a test or production signal:

• Arrival rate → load-test request profile and production requests/second.
• Latency distribution → p50, p95, and p99 histograms.
• Queue growth → queue depth and oldest-message age.
• Provider capacity → quota utilization, throttles, and error rate.
• Storage multiplier → measured bytes per logical record and daily growth.
• Failure headroom → zone-loss or instance-drain exercise.

Load testing should validate a ratio such as “one instance sustains 120 requests per second at the target p99,” not merely report that a test passed. Ratios can be placed back into the capacity model when code, payloads, or infrastructure change.

A compact planning worksheet#

Before drawing the final architecture, record:

1. Users or producers and their activity rate.
2. Average and peak arrival rate.
3. Fan-out and retry amplification.
4. Payload size and bandwidth per boundary.
5. Latency percentiles and derived concurrency.
6. Logical daily storage growth and physical multiplier.
7. Retention and archival policy.
8. Quotas for every dependency.
9. Required failure domain and remaining capacity.
10. Degradation and rejection behavior beyond the envelope.

The model will be wrong. That is expected. A visible, measurable model can be corrected; an architecture chosen from intuition alone cannot explain why it should work.

Reconcile the estimate with production evidence#

A capacity model becomes useful when it participates in an operating loop. After launch, compare each assumed quantity with an observed one: requests per active user, payload percentiles, service time, cache hit rate, write amplification, and daily retained bytes. Revise the model when a product change alters workload shape. A new media format, another fan-out consumer, or a longer retention rule can invalidate the original conclusion without changing request count.

Do not compare only daily totals. A service can match its daily forecast while failing during a concentrated ten-minute event. Record peak windows and distributions. The p95 payload size is often more useful for bandwidth planning than the mean, while a high latency percentile can dominate concurrent work. Re-run the failure calculation using slow-path service time because degraded dependencies often make requests live longer precisely when capacity is reduced.

Load tests should challenge specific claims. If the model assumes one instance sustains 400 requests per second below its latency objective, test that claim with representative payloads and downstream behavior. Increase load until the objective fails, identify the bottleneck, and preserve a safety margin between that limit and admission control. A test that achieves headline throughput while violating latency or correctness does not validate usable capacity.

Assign an owner and review triggers: a material shift in peak traffic, a new synchronous dependency, a retention change, or sustained consumption above planned headroom. Capacity planning is not a diagram prerequisite performed once. It is a maintained argument connecting demand, evidence, and an operable failure posture.

Sources and further reading#

• Google SRE: Production Services Best Practices
• Google SRE: Handling Overload
• Google SRE: Addressing Cascading Failures
• AWS Well-Architected: Performance Efficiency
• AWS Builders' Library: Using load shedding to avoid overload