Systems Architecture — Cheat Sheet¶

A living quick-reference. Add entries as you study. The goal is a document you'd actually open when making a design decision — not a glossary.

Table of Contents¶

• CAP Theorem
• Distributed Systems Patterns
• Messaging Delivery Semantics
• Microservices Communication Patterns
• Data Consistency Patterns
• Resilience Patterns
• Observability Quick Reference
• Architecture Decision Checklist

CAP Theorem¶

The CAP theorem states that a distributed data store can guarantee at most 2 of 3 properties simultaneously during a network partition:

[!IMPORTANT] In a real distributed network, P is non-negotiable — partitions happen. The real choice is: during a partition, do you prioritize C (consistency) or A (availability)?

CAP Placement of Common Databases¶

PACELC Extension¶

PACELC extends CAP to also consider the latency/consistency trade-off during normal operation (no partition):

If Partition → C or A?
Else (normal operation) → Latency or Consistency?

Example: DynamoDB is PA/EL — Available during partitions, Low latency (not strongly consistent) during normal operation.

Distributed Systems Patterns¶

Complete Pattern Reference Table¶

Messaging Delivery Semantics¶

[!NOTE] Exactly-once is often achieved via at-least-once delivery + idempotent consumers rather than true exactly-once transport. "True" exactly-once (Kafka Transactions) is available but adds latency and complexity.

Kafka-Specific Quick Reference¶

Producer → Topic (partitioned) → Consumer Group
              │
              └── Each partition consumed by one consumer in a group
              └── Multiple consumer groups = independent processing

• Offset: position in a partition log; committed by consumer on successful processing
• Consumer Group: enables horizontal scaling; Kafka guarantees each partition to one consumer per group
• Replication Factor: how many broker copies; typical production: 3
• acks=all: wait for all in-sync replicas to acknowledge → durability guarantee

Microservices Communication Patterns¶

Sync vs Async Decision Matrix¶

gRPC vs REST for Internal Services¶

Data Consistency Patterns¶

The Dual-Write Problem¶

Application
    │
    ├──► Database (write succeeds)
    │
    └──► Message Broker (CRASH — write fails)
         → Event never published
         → Downstream services never notified
         → DATA INCONSISTENCY

Solutions in order of preference: 1. Outbox Pattern — write both in one DB transaction; relay publishes 2. Change Data Capture (CDC) — Debezium reads DB WAL; publishes events 3. Event Sourcing — the event IS the source of truth; no dual-write

Saga vs 2PC Comparison¶

Resilience Patterns¶

Circuit Breaker State Machine¶

           failures >= threshold
Closed ─────────────────────────► Open
  ▲                                  │
  │  probe succeeds                  │ cooldown expires
  │                                  ▼
  └─────────────────────────── Half-Open
         (sends one probe request)

Retry Backoff Formula¶

# Exponential backoff with jitter
import random

def backoff_delay(attempt: int, base: float = 1.0, max_delay: float = 60.0) -> float:
    # exponential: 1s, 2s, 4s, 8s, ...
    delay = min(base * (2 ** attempt), max_delay)
    # full jitter: random between 0 and delay
    return random.uniform(0, delay)

Rule of thumb: Always add jitter to prevent thundering herd (many clients retrying at the exact same moment).

Observability Quick Reference¶

The Three Pillars¶

Distributed Trace Anatomy¶

TraceID: abc123
  └── Span: API-Gateway (200ms total)
        ├── Span: Order-Service (150ms)
        │     ├── Span: DB query (40ms)
        │     └── Span: Inventory-Service call (80ms)
        │           └── Span: DB query (30ms)
        └── Span: Auth check (20ms)

• TraceID: one per request, propagated in HTTP headers (traceparent in W3C standard)
• SpanID: one per operation within a trace
• Parent SpanID: links child spans to parent

OpenTelemetry Trace Propagation Header (W3C)¶

traceparent: 00-{traceId}-{spanId}-{flags}
Example:
traceparent: 00-0af7651916cd43dd8448eb211c80319c-b7ad6b7169203331-01

Architecture Decision Checklist¶

Use this when evaluating any architectural option:

• What problem does this solve? — stated as a problem, not as "we want to use X"
• What are the alternatives? — at least 2 other options considered
• What does this cost? — operational complexity, learning curve, infrastructure, latency
• What do we give up? — the explicit trade-off stated
• Is this reversible? — one-way door or two-way door?
• What is the failure mode? — how does this break, and what happens when it does?
• Who owns it? — which team is responsible for this component?
• How do we know if it's working? — what metrics/alerts prove this is healthy?