Engineer replacing a drive in a server rack in a dim data center, illustrating disaster recovery design

RTO and RPO for Web Applications: A Disaster Recovery Design Guide

•10 min read

Most teams can quote an uptime number, but far fewer can say how much data they would lose or how long they would be down if their primary database disappeared this afternoon. Those two numbers, recovery point objective (RPO) and recovery time objective (RTO), are the engineering core of any continuity plan. This guide covers how to set RTO and RPO for web applications component by component, then design the backups, replication, failover and degradation paths that actually meet them. If you need the business side first (critical processes, people, communication), start with our business continuity plan for small business in 8 steps. This article is about the system underneath.

What RTO and RPO actually measure

  • RPO is the maximum amount of data, measured in time, you accept losing. An RPO of 15 minutes means that after recovery, at most the last 15 minutes of writes may be gone.
  • RTO is the maximum time from the start of an incident until the service is usable again. It includes detection, decision, restore and verification, not just the time the restore command runs.

Two consequences engineers often miss:

  1. RPO is a property of each data store, not of the application. Your Postgres cluster, your S3 bucket of uploads, your Redis queue and your third-party CRM each have their own RPO. The application's real RPO is the worst of them for any given user workflow.
  2. RTO is dominated by humans and dependencies. A restore that takes 20 minutes is irrelevant if nobody notices the outage for an hour, or if the restored app cannot start because its secrets lived in the account that was compromised.

Setting RTO and RPO for web applications component by component

Start with an inventory, but make it a data model, not a spreadsheet nobody updates. Keep a recovery catalog in the repository, next to your infrastructure code, so it gets reviewed in pull requests:

# recovery-catalog.yaml
workflows:
  checkout:
    rto: 1h
    rpo: 5m
    depends_on: [postgres-main, payments-provider, object-store-invoices]
  reporting:
    rto: 24h
    rpo: 24h
    depends_on: [postgres-replica, warehouse]

components:
  postgres-main:
    kind: database
    backup: pitr            # base backup + WAL archive
    copy_location: separate-account
    restore_runbook: runbooks/postgres-pitr.md
    last_restore_test: null # filled by the restore drill job
  object-store-invoices:
    kind: blob
    backup: versioning + cross-account replication
  payments-provider:
    kind: external
    degraded_mode: queue-and-retry

This structure lets you answer concrete questions automatically: which workflows depend on a component that has never been restore-tested, or which component's backup method cannot meet the strictest RPO of the workflows that use it. A small CI script that fails when rpo targets are tighter than what the backup method can deliver is cheap and catches drift.

Set targets per workflow with the business owner, not per server. Checkout and login usually deserve tight targets; internal reporting can often wait a day. Tightening a target always costs money and complexity, so make every tight number justify itself.

Choosing a recovery pattern: the trade-offs

AWS's disaster recovery whitepaper describes four patterns that are useful vocabulary on any cloud: backup and restore, pilot light, warm standby and multi-site active/active. It notes that if your definition of disaster includes losing a whole region, or regulation requires it, you should consider pilot light, warm standby or multi-site active/active.

  • Backup and restore. Cheapest, slowest. You rebuild infrastructure from code and restore data. RTO depends on how fast you can provision and how big your data is. Fine for most internal tools and many small SaaS products, provided the infrastructure is fully codified.
  • Pilot light. Data is continuously replicated to the recovery location, but compute is off or minimal. Failover means starting and scaling application servers. Good middle ground when RPO must be tight but RTO can be tens of minutes.
  • Warm standby. A scaled-down but fully working copy runs all the time. You can send test traffic to it, which makes it far easier to trust. Failover is mostly scaling and traffic routing.
  • Multi-site active/active. Both sites serve traffic. Lowest RTO, but it forces you to solve write conflicts, data residency and cross-region latency. It changes your data model, not just your infrastructure.

The honest trade-off: every step up reduces RTO, but also increases the number of moving parts that can fail during normal operations. For many small teams, a well-tested backup and restore with infrastructure as code beats a poorly tested warm standby.

The database is where RPO is decided

For PostgreSQL, nightly pg_dump gives you an RPO of up to a day. For anything tighter, use continuous archiving and point-in-time recovery (PITR): a base backup plus the stream of write-ahead log (WAL) files. The PostgreSQL documentation on PITR is the reference, and one detail matters for RPO: the archive command is only invoked on completed WAL segments. On a quiet database a segment can take a long time to fill, so set archive_timeout to force regular switches:

# postgresql.conf
wal_level = replica
archive_mode = on
archive_command = 'test ! -f /mnt/wal_archive/%f && cp %p /mnt/wal_archive/%f'
archive_timeout = 300   # force a segment switch at least every 5 minutes

In production, replace the cp with a tool that ships WAL to object storage in another account (pgBackRest and WAL-G are common choices) and alert when archiving fails. Each forced switch still produces a full-size segment file, so very short timeouts increase storage use.

To recover to a moment just before a destructive mistake, you restore the base backup, set a target and, per the docs, create a recovery.signal file in the data directory before starting the server:

restore_command = 'cp /mnt/wal_archive/%f %p'
recovery_target_time = '2025-01-15 14:02:00+00'
recovery_target_action = 'pause'

Pausing lets you inspect the data before promoting. The same docs suggest temporarily restricting pg_hba.conf so ordinary users cannot connect until you are sure the recovery succeeded.

Replication is not backup. A streaming replica reduces RTO for hardware or zone failure, but it faithfully replays DROP TABLE, a buggy migration or ransomware encryption. You need both: replicas for availability, PITR for recoverability. Managed databases (RDS, Cloud SQL, Azure Database) offer PITR as a setting, but check the retention window and whether snapshots can be copied out of the account.

Everything that is not the database

  • Object storage. Enable versioning so overwrites and deletes are recoverable, and replicate to a bucket in a separate account with its own credentials.
  • Queues and caches. Decide explicitly whether they are state or not. If a Redis queue holds jobs that exist nowhere else, it has an RPO and needs persistence or a durable source of truth. Prefer designs where the queue can be rebuilt from the database.
  • Secrets and configuration. Your recovery environment needs DNS credentials, TLS certificates, API keys and the backup decryption keys. Store a sealed copy outside the primary account, or you will discover during the incident that the keys to your backups were inside the blast radius.
  • Infrastructure. If production was created by hand in a console, your RTO is however long it takes someone to remember every click. Terraform, Pulumi or CloudFormation turn rebuilds into a reviewed, repeatable step.
  • Third-party SaaS. Email providers, payment gateways and CRMs will have outages you cannot fix. Export what you can on a schedule, and design the application to degrade instead of failing.

Designing for degraded mode

When a dependency fails, the goal is to keep the critical workflow alive with reduced function. Two patterns do most of the work.

Circuit breaker. Stop calling a failing dependency for a while so requests fail fast instead of piling up on timeouts:

class CircuitBreaker {
  private failures = 0;
  private openedAt = 0;
  constructor(private threshold = 5, private cooldownMs = 30_000) {}

  async call<T>(fn: () => Promise<T>, fallback: () => T): Promise<T> {
    const open = this.failures >= this.threshold &&
                 Date.now() - this.openedAt < this.cooldownMs;
    if (open) return fallback();
    try {
      const result = await fn();
      this.failures = 0;
      return result;
    } catch (err) {
      this.failures++;
      if (this.failures >= this.threshold) this.openedAt = Date.now();
      return fallback();
    }
  }
}

Transactional outbox. Instead of calling the email or webhook provider inside the request, write the intent to your own database in the same transaction as the business change, and let a worker deliver it with retries:

BEGIN;
INSERT INTO orders (id, customer_id, total) VALUES ($1, $2, $3);
INSERT INTO outbox (id, topic, payload, status)
VALUES (gen_random_uuid(), 'order.confirmation_email',
        jsonb_build_object('order_id', $1), 'pending');
COMMIT;

If the email provider is down for two hours, orders still succeed and confirmations go out when it returns. Make the worker idempotent, because retries will deliver some messages more than once.

Treat restores as a scheduled job, not a hope

A backup you have never restored is a hypothesis. Automate a restore drill that runs on a schedule, restores the latest backup into an isolated environment, runs sanity checks and records how long it took:

#!/usr/bin/env bash
set -euo pipefail
start=$(date +%s)
./scripts/provision-restore-env.sh            # infra as code, isolated network
./scripts/restore-latest.sh --target restore-env
psql "$RESTORE_URL" -c "SELECT count(*) FROM orders WHERE created_at > now() - interval '1 day';"
./scripts/smoke-tests.sh "$RESTORE_APP_URL"
end=$(date +%s)
echo "restore_seconds=$((end-start))" | ./scripts/report-metric.sh
./scripts/destroy-restore-env.sh

Compare the measured restore time and the age of the newest restored row against the RTO and RPO in your catalog, and update last_restore_test. Then, at least occasionally, run a full game day where someone who did not write the runbook follows it.

Common mistakes

  • Backups stored in the same account, with the same credentials, as production.
  • Only nightly dumps while the business assumes no data loss.
  • A replica treated as the backup strategy.
  • WAL archiving failing silently for weeks because nobody alerts on it.
  • Restore runbooks that depend on the system being restored (a wiki hosted on the same cluster, for example).
  • Measuring RTO as restore duration and ignoring detection and decision time.
  • Synchronous calls to email, SMS or CRM providers inside critical requests.

Checklist

  1. Recovery catalog in the repo with RTO and RPO per workflow and dependencies per component.
  2. Every stateful component has a backup method that can meet its strictest RPO.
  3. Database PITR enabled, with archive_timeout or equivalent sized to your RPO, and alerts on archive failures.
  4. Backup copies in a separate account with separate credentials; object storage versioned.
  5. Secrets, DNS access and decryption keys recoverable outside the primary account.
  6. All infrastructure defined as code and rebuildable from scratch.
  7. Circuit breakers and an outbox around external providers in critical paths.
  8. Automated restore drills with measured times, plus periodic human game days.

Getting this right is mostly careful design rather than expensive tooling, and it is much easier to build in from the start than to retrofit. If you are planning a new system or hardening an existing one, our team builds custom software designed for recovery from day one.

Frequently asked questions

What is the difference between RTO and RPO?

RPO is how much recent data you can afford to lose, expressed as time. RTO is how long the service can be unavailable. A system can have a tight RPO (continuous WAL archiving) and still a long RTO (manual rebuild), so you need to design for both separately.

Is a read replica enough for disaster recovery?

No. A replica helps you recover quickly from a server or zone failure, but it copies destructive changes immediately. You still need point-in-time backups stored outside the primary account.

How often should we test restores?

Often enough that a broken backup is found before you need it. Automating the drill as a scheduled job makes frequent testing cheap; tie the frequency to how often your schema, data volume and infrastructure change.