HyperOS Systems All articles
Enterprise Integration

Counting What Survived: Reconciliation Evidence for Post-Failover Integration Estates

HyperOS Systems
Counting What Survived: Reconciliation Evidence for Post-Failover Integration Estates

Failover invocation is usually treated as a moment of relief: the service is back, the dashboard is green and the incident bridge begins to empty. For integration estates, however, that is precisely when a quieter and more consequential problem begins. Systems may be available whilst the data moving between them is incomplete, out of sequence or duplicated. The gap between service restoration and data trustworthiness is where reconciliation controls must live.

Availability Is Not Accuracy

Recovery objectives tend to be expressed in time: how quickly a service can be restored, how much data loss is tolerable, how long an important business service can remain disrupted. Those measures matter, but they do not answer a more practical question for UK technology leaders: after failover, how do we know that the records we depend on are the records we expect? A payment instruction that exists in one system but not another, a customer update that has been applied twice, or a batch sequence with a missing middle are all failures that service availability metrics will not detect.

Reconciliation should therefore be designed as a control, not commissioned as a cleanup exercise. It needs an owner, a defined method, thresholds for escalation and a documented evidence trail. In regulated UK estates, where operational resilience expectations focus on important business services and impact tolerances, reconciliation is part of demonstrating that the service is genuinely fit for continued operation, not merely reachable.

Start With a Reconciliation Contract

Before a failover event, each integration flow should have a reconciliation contract. This is a short, testable definition of what reconciled means for that flow. It should state the authoritative source, the expected record count or count range, the sequence or ordering rule, the duplicate detection method and the acceptable exception rate. It should also name the business owner who can accept residual risk and the technical owner who operates the checks.

The contract prevents a common failure mode in recovery testing: teams verify that messages are flowing, but not that the right messages have flowed once and only once. For asynchronous integrations, a green connector is weak evidence. The contract forces the question of completeness and correctness, and it gives auditors something concrete to inspect.

Counts, Sequences and Duplicates: Three Tests That Earn Their Keep

Record counts are the simplest reconciliation test and still one of the most useful. For batch files, compare source and target totals at the appropriate level of granularity. For streaming or event-driven flows, compare counts over a defined window and reconcile late arrivals separately. Counts alone will not prove correctness, but they quickly expose gaps.

Sequence checks address ordering and completeness. Where integrations rely on monotonic identifiers, batch numbers or sequence fields, a post-failover check should identify gaps, regressions and out-of-order processing. This is particularly important where downstream systems assume chronological processing, such as financial postings, claims handling or inventory movements.

Duplicate detection is the third essential test. Failover and retry logic can produce repeated delivery, especially where acknowledgement was lost during the transition. Idempotency keys, natural business keys and hash comparisons can all be used, but the method must be agreed in advance and applied consistently. A duplicate that is invisible to the integration layer may become a visible error in a customer account or a regulatory return.

The Exception Queue Is a Register

Exceptions are often treated as operational noise: items in a queue to be cleared by the integration team. For evidence purposes, the exception queue should be a register. Each exception should carry an identifier, the flow concerned, the detection method, the time of detection, the owner, the disposition and the rationale for closure. Where an exception is accepted rather than corrected, the acceptance should be time-bound and approved by the appropriate business owner.

This register becomes one of the most persuasive artefacts in a post-incident review. It shows that reconciliation was performed, that exceptions were triaged and that residual risk was consciously managed. It also supports internal audit and risk committee scrutiny, because it connects technical anomalies to business consequences and accountable decision-making.

Packaging the Evidence

A reconciliation evidence pack should be assembled whilst the recovery is still fresh, not reconstructed weeks later. It should include the reconciliation contract for each in-scope flow, the flow inventory and business service mapping, the checks performed and their results, exception reports and dispositions, timestamps for detection and resolution, and sign-off from technical and business owners. Where reconciliation was not performed, the pack should say so and explain the compensating control.

For UK regulated firms, the pack should be mapped to important business services and impact tolerances. It should show whether data integrity remained within tolerance during the disruption and, if not, what remediation and customer or counterparty impact assessment followed. This is the difference between an incident report that says recovery succeeded and an evidence set that shows the business can rely on what survived.

Designing for Repeatability

Reconciliation is most credible when it is exercised before it is needed. Recovery runbooks should include reconciliation steps, not just infrastructure checks. DR tests should include partial failover, failback and split-brain scenarios, because these are the conditions most likely to produce duplicates and sequence gaps. Evidence capture should be automated where possible, with immutable logs and retention aligned to the organisation's audit and regulatory obligations.

In 2026, integration estates are more distributed and more asynchronous than the recovery plans many UK firms inherited. That makes reconciliation a permanent control rather than a post-incident project. The ledger after failover is not a finance ledger; it is the record of what was counted, what was compared and what was accepted. Technology leaders who build that ledger into their resilience regime will be better placed to demonstrate that restored services are also trustworthy services.

All Articles

Related Articles

The Consent Layer Is a Control Layer: Auditing AI OAuth Grants in the SaaS Estate

The Consent Layer Is a Control Layer: Auditing AI OAuth Grants in the SaaS Estate

Open Paths, Closed Loop: Retiring Firewall Exceptions on a Quarterly Cycle

Open Paths, Closed Loop: Retiring Firewall Exceptions on a Quarterly Cycle

Recovery on the Clock: Benchmarking Restore Throughput for Defensible RTOs

Recovery on the Clock: Benchmarking Restore Throughput for Defensible RTOs