Jason Scholes of SynApps Solutions highlights the risks of underestimating NHS data migration complexity, and draws on current best practice to explain how trusts can expedite critical system transitions.
This article was kindly supported by SynApps Solutions.
Moving patient records between old and new systems is always a big deal. But even the most experienced digital programme teams can be caught out by the scale of the challenge. By the time a trust has committed to a new EPR deployment and agreed contract timelines, the migration may have become a secondary concern. Projects plans may have been built on assumptions rather than evidence, and timelines set accordingly.
When issues arise later (and they will) the impact is a delayed go-live, inflated budget, an exhausted team and/or (most seriously) gaps in clinical information. Fortunately, the problems that derail migrations are almost always knowable in advance. To help avoid them, adopt these three principles that consistently differentiate successful large-scale migrations from those that run into problems.
1. Start with the data, not the deadline
A significant biggest predictor of a difficult migration is source data that hasn’t been properly assessed. Applications that have been extended and modified over two-three decades will have encountered bespoke configurations, inconsistent categorisation, non-standard identifiers and all manner of workarounds. All of this is manageable if understood early enough.
Running a structured data profile before any technical migration work begins — assessing volumes, quality, format, identifier consistency and the gap between what exists and what the target system expects — can transform the project plan from an estimate into a reliable forecast. It will also surface often-unmeasured constraints (e.g. transfer bandwidth for cloud-hosted targets, API access limitations and/or the actual scale of imaging volumes). Such constraints are straightforward to plan around if identified early. Six months into delivery, they will become programme risks.
The need to anticipate obstacles also applies to vendor relationships, especially if trusts are migrating away from proprietary systems whose outgoing vendors have limited commercial incentive to cooperate. Extraction costs can be significant, and export tooling may be poorly documented (a trust could spend months in negotiation for access to records it has always owned). Far better to identify potential issues before project timelines are fixed than trying to tackle them mid-delivery.
2. Phase the migration, not just the go-live
A single cutover might appeal because of the potential to align with contract termination dates, avoid prolonged parallel running and keep things simple for the Board. But a big-bang approach risks concentrating years of accumulated data complexity into a single point of failure, without capacity to absorb anything that hasn’t come up in testing. This can heap pressure on clinical and IT staff. That’s aside from the potential reputational and patient safety consequences of getting any of this wrong.
Treating the cutover weekend as the end of a much longer process is far less risky. Historical, stable data (records unlikely to be needed before go-live) can be migrated weeks or even months in advance in managed batches, with validation taking place at each stage. By the time the final cutover happens, the active data pool is a fraction of what it would have been otherwise.
This means treating migration as something with its own schedule. Trusts will also need to maintain continuous read-only access to legacy data throughout the transition. Since an independent clinical viewer or vendor-neutral archive (VNA) is an important factor for clinical safety, access cannot be interrupted for any clinician who may need to see a patient’s history.
3. Make it a clinical programme, not an IT project
Although IT teams will understand data types and database schemas, they won’t necessarily appreciate which data elements are critical for patient safety at go-live, and which (if incomplete) can be recovered retrospectively without clinical risk.
Bringing clinical safety officers, CCIOs and frontline operational leads into project governance from the outset, as co-decision-makers, changes the way workstreams are prioritised – and therefore what is caught. The minimum viable dataset for go-live needs to be defined by the people who will depend on it clinically. Similarly, mapping rules need clinical validation, and user acceptance testing needs clinical sign-off.
Emerging frameworks such as openEHR offer a more rational long-term foundation — a vendor-neutral architecture that separates the data layer from any particular application, so that future system changes won’t require a migration. Although investing in open architecture may not have been the starting point for many projects underway at trusts today, it is a scenario worth building towards.
What good looks like: A case in point
The recent experience of a major multi-site acute trust illustrates the advantages of adopting these recommended practices. It needed to decommission a legacy clinical document management system holding 15 million+ historical patient records, while deploying a new enterprise-wide EPR. The Trust faced a strict contractual deadline to exit its legacy vendor agreement, but a single cutover of that volume represented too high a risk to daily clinical operations.
The Trust took a phased approach from the outset. First it deployed a VNA as a permanent open repository. Months before the EPR go-live, it began migrating historical inactive records in background batches. An independent clinical viewing layer was embedded directly into the Trust’s existing portal throughout, so that clinicians could access older patient histories at any point — unaware that the underlying data was moving.
By the time the cutover weekend came around, more than 90 per cent of the data had been migrated, validated and signed off by clinical safety leads. The final migration involved only the active records from the previous 48 hours. The Trust met its decommissioning deadline, maintained full clinical continuity and avoided any data bottlenecks. All of this was possible because the migration was treated as a clinical programme as much as a technical one. Also, the analytical groundwork was done up front, and the scope wasn’t allowed to shrink under the pressure of contract deadlines.
The NHS drive to accelerate digital transformation and beat budget constraints can make migration an easy target for compression. But this is a false economy, as emerging issues often cost more to fix later than to plan for upfront. Remember, too, that the teams absorbing that cost are those expected to deliver the next programme.
Jason Scholes is Chief Technology Officer at SynApps Solutions