CONCEPT · 60 minutes
A repeatable six-step framework for any system design question. Learn to clarify requirements, estimate capacity, define APIs, choose a data model, plan for scale, and discuss trade-offs — all while narrating your reasoning aloud.
The first real design I led on my orchestration framework, I made the mistake almost every engineer makes: I started drawing boxes immediately. The task was a job-scheduling subsystem, and within an hour I had database tables, worker queues, and retry logic sketched out — and I had not pinned down a single number about how it would actually be used.
The number that undid me came later: how many jobs per second did this need to handle? My answer turned out to be off by an order of magnitude, and the architecture I had committed to could not absorb the gap. The rebuild cost far more than the questions I should have asked at the start would have.
That was the lesson: the constraints — not your instincts, and definitely not your diagrams — drive every structural choice. The skill is taking an underspecified problem, surfacing the constraints that matter, designing something that meets them, and being able to explain why it is shaped the way it is.
Think of it like designing a building. An architect does not pour the foundation before knowing how many floors there are, whether the ground is stable, and what the building codes require. The design work happens on paper first, and the constraints determine every structural choice. Software design follows the same logic — and it is just as expensive to get the foundation wrong.
What actually matters in open-ended design: Can you take an underspecified problem, identify the constraints that matter, design a system that meets those constraints, and articulate the trade-offs clearly? A polished diagram matters far less than structured reasoning you can defend.
After this module, you will have a repeatable six-step framework you can apply to any design problem — regardless of the domain. You will also know how to communicate your reasoning so collaborators can follow your thinking, and how to work through the trade-off discussion that determines whether a design holds up.
The examples in this module come from systems I have built: the python-orchestration-system (FastAPI + Celery DAG scheduler), the rag-engine-rs (Rust RAG backend with pgvector and streaming), and the workflow-engine-rs (typed-node graph execution engine). These are real systems with real constraints — using them as examples means the trade-offs I describe are ones I have actually faced.
Every system design question, regardless of the domain, can be answered with the same six steps in the same order. The order is not arbitrary — each step feeds the next.
Before you draw anything, spend two to three minutes asking questions. You are almost never handed all the constraints up front — surfacing them is part of the job.
Functional requirements — what must the system do?
Non-functional requirements — under what constraints must it do it?
Write these down visibly. Whoever you are designing with will correct you if you have misunderstood — and catching it now is far cheaper than building on a wrong assumption.
The most common skip: Engineers skip non-functional requirements because they feel abstract. But the difference between "100 QPS" and "100,000 QPS" determines whether you use a single Postgres instance or a sharded cluster. Never skip this step.
With requirements clear, do order-of-magnitude math. You are not looking for precision — you are sizing the problem so you know what kind of infrastructure you are designing.
The mental model: a back-of-envelope estimate answers "is this a Postgres problem or a distributed-systems problem?"
A rough formula for read-heavy systems:
For storage:
State your assumptions out loud. "I will assume 10 million daily active users, each making about 10 requests per day. That gives roughly 1,200 average RPS and maybe 6,000 at peak with a 5× peak multiplier."
Now define the interface between clients and your system. This is the contract that everything else must satisfy.
List the three to five most important endpoints. For each one, name the HTTP method, the path, the key inputs, and the key outputs. The API definition forces you to be concrete about what the system must do before you commit to any storage or infrastructure choice — this is the step most people skip because it feels like detail work, but it is where you discover whether your requirements were complete.
With the API defined, design the data model that supports it. Name the entities, their key fields, and the relationships between them.
For each entity, ask: what queries does the API require? The query pattern determines the index strategy and often the database choice.
Design rule: The redirect path (GET /:code) is the hot read. Keep it to a single index lookup. Push analytics writes to an async queue so they do not add latency to the redirect.
Now that you have a working design, identify where it breaks under load.
Ask yourself: at the scale you estimated in Step 2, which component becomes the bottleneck first?
Common bottleneck patterns and their solutions:
| Bottleneck | Signal | Solution |
|---|---|---|
| Read QPS | High DB CPU, slow reads | Read replicas, read-through cache |
| Write throughput | Write queue depth grows | Write-ahead log, async writes, sharding |
| Storage volume | Disk filling | Archival strategy, time-partitioned tables |
| Latency | P99 exceeds SLA | CDN for static, edge caching, connection pooling |
Name the bottleneck, explain why it is the limiting factor at your estimated scale, and then propose the solution. The solution always introduces new complexity — name that too.
Here is the complete framework as a sequence — each step only makes sense after the one before it:
flowchart TD
A["Step 1\nClarify Requirements\n(functional + non-functional)"]
B["Step 2\nCapacity Estimation\n(order-of-magnitude math)"]
C["Step 3\nAPI Definition\n(the client contract)"]
D["Step 4\nData Model\n(serves the API's query patterns)"]
E["Step 5\nScale Strategy\n(name the bottleneck first)"]
F["Step 6\nTrade-off Discussion\n(what you chose and why)"]
A --> B --> C --> D --> E --> F
style A fill:#3b82f6,color:#fff
style F fill:#10b981,color:#fffThe arrows carry meaning: the API (Step 3) determines what queries the data model (Step 4) must support; the data model determines where the bottleneck appears under load (Step 5); the bottleneck determines which trade-off you must defend (Step 6). Skipping any step — especially Step 1 — breaks the chain, and you end up defending choices that were never grounded in actual constraints.
Every architectural choice is a trade-off. The last step is to make the key ones explicit.
The strongest trade-off statements follow this pattern:
"For the write path, I chose [choice] over [alternative] because [reason given the constraints]. The cost of this choice is [what you give up]. At [threshold], I would reconsider and [alternative]."
This pattern shows you understand why your design is shaped the way it is, not just that you can draw boxes. It also shows you know the design has limits — which is far more credible than claiming it scales infinitely.
Before continuing, verify you can answer these without looking back:
If any of these are fuzzy, re-read that step before moving on.
The biggest differentiator between engineers at the same technical level is not what they design — it is how they communicate the design as they build it.
An engineer who draws a correct diagram in silence gives a reviewer almost no signal. A teammate cannot tell whether you understand why the design works or just reproduced a template.
Narrate every decision as you make it, framing it as a consequence of your requirements.
Instead of: (silently drawing)
Say: "I am putting a cache in front of the read path here because the redirect use case is almost entirely read — and reads will be the bottleneck at 6,000 RPS. I will use a TTL of 24 hours for most links, shorter for links that are about to expire."
Instead of: "I would use Postgres."
Say: "I am choosing Postgres over a key-value store here because the analytics query needs joins across users and click events. A key-value store would force me to denormalize, which makes the data model harder to maintain. If write throughput became the bottleneck, I would reconsider."
The formula is: what + why given the constraints + what you gave up.
Why narration matters: In a design review, the engineer who explains their reasoning as they go is far easier to trust than one who presents a finished diagram in silence — even when the two designs are identical. The narration is the evidence that the design is understood, not copied, and it lets a teammate catch a flawed assumption before it becomes code.
If you are thinking, say so. "Let me think about the write path for a moment." This is always better than silence.
If you are unsure, say that too. "I am less confident about the sharding strategy here — I know that Postgres supports hash sharding, but I would want to look at the actual query patterns before committing to a shard key." An honest statement of uncertainty is more credible than a confident wrong answer.
The trade-off discussion is where senior engineers separate themselves from mid-level ones.
A complete trade-off statement has four parts:
Suppose you designed a URL shortener with a single Postgres instance for the redirect path, backed by an in-memory cache.
Poor trade-off statement: "I would add more caching to improve performance."
Strong trade-off statement: "The current design uses a single Postgres instance because the write volume — roughly 10,000 new links per day — is well within what a single instance can handle. If this system were used for high-traffic events where a single link might receive millions of clicks in minutes, I would move the cache to a distributed layer like Redis with a write-through pattern, and I would use a short TTL with background refresh to avoid thundering-herd on cache expiry. That adds two new failure modes: Redis downtime and cache inconsistency during refresh. I would handle Redis downtime with a circuit breaker that falls back to Postgres, accepting higher latency rather than unavailability."
The difference: the second answer names the specific failure mode that would trigger the change, names the exact alternative, and describes the new failure modes the alternative introduces.
Think about a system you have built or worked on closely.
Pick one architectural decision in that system — a database choice, a queuing strategy, a consistency model. Walk through the six steps for it as if you were guiding a new teammate through the design. What trade-off would you name in Step 6? What is the threshold at which you would revisit that decision?
Write a three-sentence answer before moving to the quiz. Applying the framework to a real system you know deeply is the most effective way to make it automatic when you later face an unfamiliar one.
Apply the full six-step framework to a real design problem. The goal is not a perfect answer — it is a structured walkthrough you can defend.
The task: Design a notification delivery system. Users can subscribe to events (order shipped, payment received). When an event fires, subscribed users receive a notification via email and push. The system must handle both low-volume steady state and sudden spikes (a flash sale fires thousands of notifications in seconds).
Use the starter below to sketch your design. Work through each step before revealing the hints.
<CodeExample title="Design Prompt Starter — Notification System" language="markdown" code={`## Step 1: Requirements Functional:
Non-functional:
Apply the six-step framework to realistic system design scenarios
1. You are asked to design a URL shortener. What should you do first?
Correct Answer: B
Step 1 is always clarification. Scale, latency requirements, and feature scope determine every architectural choice downstream. Jumping to a schema before knowing expected QPS or whether analytics are needed will send you down the wrong path.
2. During capacity estimation for a read-heavy system, you have no daily-active-user number to work from. What is the correct move?
Correct Answer: B
Capacity estimation is an exercise in order-of-magnitude reasoning, not precision. State your assumptions explicitly — what matters is the reasoning process, not a perfect number. Saying '10M DAU, which gives roughly 120 RPS at peak' shows structured thinking and gives everyone a number to challenge.
3. Why define the API before choosing a data model?
Correct Answer: B
The API defines what the system must do from the caller's perspective. Once you know the exact inputs, outputs, and latency requirements for each endpoint, the data model follows naturally. Going database-first often leads to premature normalization or sharding choices that don't match the actual query patterns.
4. Someone jumps straight from the data model to 'I'd shard the database across 100 nodes.' What is wrong?
Correct Answer: C
The scale strategy step requires identifying the bottleneck first — is it read QPS, write throughput, storage volume, or latency? The solution follows from the bottleneck. Jumping to 'shard' without naming the bottleneck signals pattern-matching rather than understanding.
5. Asked 'what would you change about your design?', what is the strongest answer pattern?
Correct Answer: C
The strongest answer names the specific trade-off baked into the current design, the threshold at which it breaks down, the alternative you would adopt, and the new failure modes that alternative introduces. This pattern demonstrates that you understand your design rather than having memorized a template.
6. In the six-step framework, what immediately follows capacity estimation?
Correct Answer: C
The order is: clarify requirements → capacity estimate → API definition → data model → scale strategy → trade-off discussion. API comes before data model because the API contract determines what queries the data model must support.