HyperOS Systems All articles
Operational Resilience

Expiry Is a Failover Event: Proving TLS Trust From the Recovery Estate

HyperOS Systems
Expiry Is a Failover Event: Proving TLS Trust From the Recovery Estate

Photo: Ana Las Heras, CC BY-SA 4.0, via Wikimedia Commons

Failover plans tend to treat TLS as a property of the application rather than a service in its own right. The runbook confirms that servers boot, databases mount and DNS answers. It rarely confirms that the chain presented to a customer, partner or internal service will be valid, complete and trusted from the recovery estate. In a UK regulated estate, a recovered service that presents an expired, untrusted or incomplete chain is not recovered. It is reachable but not usable.

Trust Is a Recovered Service

A certificate is not a decorative artefact attached to a web tier. It binds identity to endpoint and carries expiry, issuer and revocation semantics. When the recovery estate answers, those semantics must still hold. The certificate must chain to a root the relying party trusts. The private key must be available to the recovered service. The revocation path must be reachable from the recovery egress. The clock must agree with the validity window. If any of those conditions fails, the failover has produced a service that looks alive on a network diagram and fails in a browser, an API gateway or a partner integration.

This is why certificate readiness belongs in operational resilience evidence, not only in a security team backlog. The question is not whether certificates are managed. The question is whether their management survives the failover boundary.

Inventory the Chain, Not Just the Leaf

Certificate inventories often capture leaf certificates and expiry dates. That is insufficient for recovery attestation. The inventory must record the full chain: leaf, intermediates, roots and the trust stores that rely on them. It must distinguish public certificates from internal CA certificates, partner-facing mTLS certificates and service-to-service certificates. It must record subject alternative names, key type, signature algorithm and chain order.

For each certificate, record the private key location: HSM, cloud KMS, software store or escrow. Record the access policy, the break-glass route and the restoration procedure. A certificate whose key sits in an HSM that does not fail over is not recoverable, however valid its dates. A cloud KMS key bound to a region that is not part of the recovery design is a single point of trust failure. If an internal CA issues the chain, prove that the issuing CA and its revocation distribution points are available in the recovery estate.

Treat each chain as a configuration item with an accountable owner and a technical owner. Map it to the service, the failover group and the recovery time objective. That mapping is the difference between a spreadsheet and an evidence pack.

Test the Paths the Estate Will Use

Certificate readiness is path-dependent. A renewal that works from the primary site may fail from the recovery site because egress policy, DNS challenge routing or CAA records behave differently. Test ACME challenges from the recovery estate, including HTTP-01 and DNS-01 where used. Confirm that the ACME account credentials are available and that outbound access to the certificate authority is permitted. Test DNS propagation for challenge records and check CAA records against the recovery estate.

Revocation checking is equally path-dependent. Test CRL and OCSP reachability from recovery egress. If OCSP must-staple is used, confirm that stapling still succeeds. Some recovery designs block outbound revocation traffic, which may be acceptable only if the application policy explicitly soft-fails. Record that decision. Test trust store updates from the recovery image, not from a golden image that has drifted. Test time synchronisation: expired and not-yet-valid conditions depend on the clock, and a recovery estate using different NTP sources can misjudge validity windows.

For partner-facing mTLS, test from recovery IP ranges. Allowlisting is a common failure point: the partner trusts the primary egress but not the recovery egress. Record the partner trust store update lead time. A chain is only as strong as the slowest counterparty.

Align Expiry With the Failover Timetable

Expiry is not merely a date. It is a constraint against the failover timetable. For each certificate, compare the notAfter date with the recovery time objective plus the expected operating window. A certificate that expires during a declared incident is a latent failure. Build a watchlist for certificates expiring inside the recovery window, certificates with manual renewal, certificates owned by teams without out-of-hours cover and certificates with short lifetimes.

Automated renewal is valuable only if the automation survives failover. If the ACME client runs on a server that is down, renewal stops. If renewal depends on a CI pipeline that is unavailable, the certificate enters the recovery window with less headroom. Record renewal ownership, escalation and the evidence that renewal was tested from the recovery estate. For UK regulated firms, certificate expiry should appear in operational resilience mapping and incident severity models, not only in a monitoring dashboard.

Record Ownership and Custody

Every certificate needs an accountable owner, a technical owner and a documented renewal route. Ownership must survive staff changes. Record private key custody in detail: which HSM or KMS, which access policy, which break-glass process and which restoration test. If keys are escrowed, test restoration into the recovery estate. If keys are generated in a cloud service, prove that the recovery region can present them. Include certificate signing requests, approval records and change tickets in the evidence pack.

Custody also applies to trust stores. Record who can add a root, who can remove one and how trust store changes are approved. A recovery estate that trusts the wrong root or lacks an intermediate is not trusted by its consumers.

Evidence That Survives the Exercise

An evidence pack should be reproducible. Include the inventory export, chain validation output from the recovery site, ACME and revocation test logs, time synchronisation checks, key custody attestations, owner sign-offs and expiry-versus-RTO calculations. Store it alongside the failover runbook, not in a separate spreadsheet that nobody opens during an incident.

During exercises, deliberately stop renewal automation and observe. Deliberately block OCSP and record the application behaviour. Deliberately restore a key from backup and validate the chain. These are not destructive tests; they are evidence of readiness. The recovery estate can answer on the network and still fail the trust test. UK technology leaders should treat certificate chains as recoverable dependencies with owners, paths and expiry constraints. Inventory the chain, test from the recovery site, align expiry with the timetable and record custody. That is how a failover avoids becoming a certificate incident.

All Articles

Related Articles

Rebuild, Don't Merely Restore: Attesting the CI/CD Path to Recoverable Software

Rebuild, Don't Merely Restore: Attesting the CI/CD Path to Recoverable Software

Proving the Way Back: A Failback Evidence Regime for UK Enterprises

Proving the Way Back: A Failback Evidence Regime for UK Enterprises

The Witness Nobody Names: Attesting Remote-Hands Work Inside the Colocation Cage

The Witness Nobody Names: Attesting Remote-Hands Work Inside the Colocation Cage