# Dapr on AKS recovery drill checklist

Record evidence for every item. A recovery is not proven by a successful command alone; verify the application from the user's point of view.

## Drill record

- Date and change ticket:
- Operator and reviewer:
- Source and recovery regions:
- Git revision and container digests:
- Declared RTO:
- Declared RPO:
- Actual RTO:
- Observed data-loss window:

## Before the drill

- [ ] Confirm the drill cannot route production traffic accidentally.
- [ ] Export the current AKS, Dapr extension, and application versions.
- [ ] Confirm the recovery cluster has enough quota and capacity.
- [ ] Confirm GitOps can reconcile namespaces, identities, Dapr resources, and applications.
- [ ] Verify secrets can be resolved in the recovery region without copying secret values into Git.
- [ ] Record the backup or replication position for every stateful dependency.
- [ ] Confirm DNS or global-routing changes have a tested rollback.

## Recover the platform

- [ ] Bring the AKS recovery stamp to the approved configuration revision.
- [ ] Verify the Dapr control plane is healthy with `dapr status --kubernetes`.
- [ ] Verify mTLS and certificate expiry with `dapr mtls --kubernetes` and `dapr mtls expiry`.
- [ ] Reconcile Dapr Configuration, Component, Resiliency, and Subscription resources.
- [ ] Reconcile application Deployments only after their Dapr resources are ready.
- [ ] Verify every expected pod contains the approved `daprd` image.

## Recover data and messaging

- [ ] Promote or restore each state store according to its documented runbook.
- [ ] Promote the messaging namespace or endpoint according to the selected Service Bus recovery model.
- [ ] Validate consumer groups, topics, subscriptions, rules, dead-letter queues, and retained messages.
- [ ] Decide explicitly whether Dapr Scheduler data is restored or recreated.
- [ ] Check that actor placement and reminders converge after applications start.

## Validate before traffic

- [ ] Run direct health checks against the recovery ingress.
- [ ] Complete one state write/read/delete transaction.
- [ ] Publish and consume a uniquely identified test message.
- [ ] Invoke one service through Dapr and exercise the expected retry/circuit-breaker policy.
- [ ] Exercise one actor method and, where used, one workflow or scheduled job.
- [ ] Confirm traces, metrics, logs, and alerts arrive from the recovery region.
- [ ] Confirm duplicate delivery and replay behavior is safe.

## Shift and observe traffic

- [ ] Move a small percentage of synthetic or production traffic first.
- [ ] Observe latency, error rate, sidecar restarts, throttling, and queue depth.
- [ ] Increase traffic only after the agreed observation window.
- [ ] Record the time at which the recovery service became usable.

## Close and improve

- [ ] Test or document failback; do not assume it is the reverse of failover.
- [ ] Capture gaps, owners, and target dates.
- [ ] Update RTO/RPO evidence and the runbook.
- [ ] Remove drill-only resources and confirm normal routing.
- [ ] Schedule the next drill.