System Design · Guide

High Availability

Redundancy, failover, multi-AZ, RTO/RPO.

— min read System Design

Theory

High Availability: Redundancy, failover, multi-AZ, RTO/RPO.

High availability minimizes downtime through redundancy and automatic failover. Expressed as "nines": 99.9% allows ~8.7 hours downtime per year; 99.99% allows ~52 minutes.

RTO (Recovery Time Objective) is max acceptable downtime; RPO (Recovery Point Objective) is max acceptable data loss. Active-passive pairs standby capacity; active-active serves from multiple nodes simultaneously with conflict handling.

Deploy across multiple availability zones within a region for infrastructure redundancy. Multi-region adds disaster recovery at the cost of replication lag and cross-region consistency complexity. Automate failover with health-checked load balancers or leader election.

Production rollouts require idempotent automation, peer review, staged apply, and documented rollback — treat changes as production code.

Interviewers want STAR stories linking High Availability to measurable outcomes: fewer outages, faster deploys, lower cost, or reduced toil.

Architecture Diagram

Users / clients
         |
  High Availability
         |
  Core services
         |
  Data + observability

Examples

bash
# High Availability
# Redundancy, failover, multi-AZ, RTO/RPO.
# Validate in staging before production rollout.

Interview Questions

What problem does High Availability solve?

It addresses the core use case described in production architecture — map features to reliability, scale, or velocity outcomes.

Key components of High Availability?

Identify inputs, outputs, control plane, data plane, and failure domains — interviewers want structured decomposition.

Common production pitfalls?

Misconfiguration, missing observability, no rollback path, and scaling bottlenecks under peak load.

How do you test changes safely?

Staging parity, canary/gradual rollout, automated health checks, and documented rollback.

Metrics to prove success?

Error rate, latency percentiles, throughput, cost, and toil reduction — pick one primary SLO.

Beginner vs advanced concern?

Beginners focus on setup; advanced teams focus on blast radius, security boundaries, and operability at 10× scale.

Best Practices

  • Treat High Availability config as code with review and CI validation.
  • Define SLOs and dashboards before production cutover.
  • Document rollback and ownership for on-call.
  • Use least privilege for credentials.

Common Mistakes

  • Adopting High Availability without measurable success criteria.
  • No staging environment mirroring production constraints.
  • Missing rollback path during incidents.
  • Undocumented on-call expectations.

Trade-off Analysis

High Availability improves redundancy, failover, multi-az, rto/rpo. but adds operational and cognitive complexity — justify with load and team size.

Favor simplicity until metrics (p99 latency, error rate, cost) prove the pattern necessary.

Every redundancy layer trades capital/operational cost for availability — align with explicit SLO targets.

Document accepted inconsistency windows and recovery behavior before production cutover.

Cheat Sheet

HighRedundancy, failover, multi-AZ, RTO/RPO.
SLOService level objective
RollbackRevert to last known good
CanaryLimited blast-radius rollout
RunbookIncident steps

Practical Exercises

High Availability sandbox

Stand up High Availability locally or in free tier; document commands and failure recovery.

Failure drill

Introduce misconfiguration; practice detection and rollback under time limit.