System Design Core Concepts

Key Points

• Scalability: A system's ability to handle increased load by adding resources — vertical (bigger machine) or horizontal (more machines).
• Latency vs Throughput: Latency = time to complete one request; Throughput = number of requests handled per second. Optimise both independently.
• CAP Theorem: A distributed system can only guarantee 2 of 3 — Consistency, Availability, Partition Tolerance. Networks always partition, so you trade C vs A.
• Load Balancer: Distributes incoming traffic across servers using algorithms like Round Robin, Least Connections, or IP Hash. Removes single points of failure.
• Horizontal Scaling (Scale-out): Add more servers behind a load balancer. Preferred for stateless services. Requires session management (e.g., sticky sessions or centralised session store).
• Caching: Stores frequently accessed data in fast memory (Redis, Memcached) to reduce DB load. Strategies: Cache-aside, Write-through, Write-back, Read-through.
• CDN (Content Delivery Network): Serves static assets from edge servers geographically close to users. Reduces latency for images, JS, CSS.
• Database Sharding: Splitting a DB table horizontally across multiple instances (shards) using a shard key. Improves write throughput but complicates joins and transactions.
• Database Replication: Copying data to one or more replicas. Primary handles writes; replicas handle reads. Types: synchronous (strong consistency) and asynchronous (eventual consistency).
• Message Queue: Decouples services using async messaging (RabbitMQ, Kafka, SQS). Producer publishes events; consumers process independently. Improves resilience and scalability.
• Consistent Hashing: Distributes keys across nodes in a ring structure. Adding/removing a node affects only adjacent keys — minimises cache misses during scaling.
• Idempotency: An operation that produces the same result whether called once or multiple times. Critical for retry logic in distributed systems (e.g., payment APIs).
• Rate Limiting: Controls the number of requests a client can make in a time window. Algorithms: Token Bucket, Leaky Bucket, Fixed Window, Sliding Window.
• SQL vs NoSQL: SQL = structured, ACID transactions, vertical scale (Postgres, MySQL). NoSQL = flexible schema, horizontal scale (MongoDB, DynamoDB, Cassandra). Choose based on access patterns.
• Eventual Consistency: Nodes in a distributed system may temporarily diverge but will converge to the same state given enough time and no new updates.