5. Identifying a preservation approach

There is unlikely a ‘one size fits all’ solution for preserving the records in a line of business system – the approach should be informed by the information you have gathered in steps 2-4 of this guidance, the staffing/budget resources available and the organisation’s appetite for risk.

The Electronic Document and Records Management Systems (EDRMS) Preservation Toolkit, created by the Digital Preservation Coalition’s EDRMS Preservation Task Force, identifies four different preservation approaches to preserving records from an EDRMS or other recordkeeping systems. The toolkit provides a summary of each approach and reasons why you might choose this approach which are also broadly applicable to records in line of business systems. The four approaches are:

  • Leave in current system and manage ‘in situ’
  • Migrate records to a new system
  • Transfer records to a digital archive
  • Emulate the system or application that holds the records

In this section, we briefly outline each approach and discuss the key considerations in the context of records in line of business systems.

Option 1: Leave in current system and manage ‘in situ’

This might be considered when the system is supported, still in use, and the records have relatively short retention periods. Tim Gollins’s talk, Preserving Semi-Current Records: EDRMS Task Force – Reflections, presented at Preserving Semi-Current Records: A Digital Preservation Coalition Briefing Day, provides a useful framework (see Figure 1 below) for thinking about when or whether to use Preservation ‘in situ’.

0 to 7 Years 5 to 15 Years 10 to 50 Years 40 Years to Forever
Full functionality No special preservation action Migrate Business System (Extract, Transform and Load (ETL)) Migrate Business System (ETL) and move to archive? ???
Modification/update No special preservation action Migrate Business System (ETL) Migrate Business System (ETL) and move to archive? ???
Extension only (such as new versions) No special preservation action Migrate Business System (ETL) and move to archive? Migrate Business System (ETL) and move to archive? ???
Read-only No special preservation action Migrate Business System (ETL) or move to archive Move to archive Move to archive
Figure 1 – copy of Tim Gollins’ ‘A Framework’. Reproduced courtesy of the National Records of Scotland

Option 2: Migrate records to a new system

If the records are left in the current system, then it is likely migration to a new system will take place in the future. Procurement of systems and the subsequent migration are relatively common in the public sector. In these cases, archive practitioners may be able to influence the procurement specification to ensure systems have recordkeeping and preservation functionality (e.g. ability to extract/export data). ISO/TS 16175-2:2020 ‘Information and documentation — Processes and functional requirements for software for managing records’ provide detailed guidance but may be difficult to implement in practice. In terms of digital preservation functionality, the Digital Preservation Coalition’s Digital preservation requirements for procuring IT systems is a good resource.

The following are also helpful:

In addition, archive practitioners may be able to influence migration plans to ensure records or their metadata are not lost during the migration. The National Archives’ Migrating information between records management systems (PDF, 295 KB) is aimed at migrating information between record management systems including EDRMS, Enterprise Content Management or Content Management Systems, but the principles will be relevant to line of business systems.

For example, this includes:

  • Defining your business requirements for the records you are considering migrating, including understanding what records the originating system holds and what may need migrating
  • Identifying what you require of your records for them to be found, opened, worked with, understood, and trusted
  • Testing for continuity, for example by using The National Archives’ Testing for Continuity Checklist (PDF, 43 KB)

Archive practitioners may find that IT colleagues may use migration methods with legacy line of business systems. Kevin Bolton and Sarah Wickham, in The Welsh Vital Digital Information Project, (2021) found that in Welsh local government “there are projects underway to manage out legacy systems and harmonise systems onto a single platform.”

Option 3: Transfer records to a digital archive

This approach could include a wide range of methods such as:

  • Exporting the records/data and metadata into an open format (e.g. XML, CSV)
  • Exporting the records using some type of report function (e.g. PDF) along with any metadata in an open format
  • For databases, capturing the native database files (e.g. SQL files) or using the SIARD Suite, a free software developed by the Swiss Federal Archives to archive databases and used in over 50 countries around the world. It extracts content from relational databases and stores it in the SIARD format. If the database includes links to externally stored records consideration should be given to transferring them along with the database (Artefactual Systems and the Digital Preservation Coalition, 2021). For more information about preserving databases see the Digital Preservation Coalition’s Preserving databases (PDF, 517 KB).

Other considerations include:

  • Ensuring any documentation about the system and the records that provide important context are preserved
  • Identifying whether you are preserving all the records in the system or simply some of them
  • Determining the frequency of transfers

The National Archives of Australia has produced some good criteria to determine the frequency of transfer (PDF, 2.2 MB). In a UK public sector context, the following points will be useful to consider:

One-off transfer

  • System no longer active or used
  • Organisation/department/function closed
  • System not supported or out of license
  • System is about to be migrated to a new system or combined with other systems

Periodic snapshot

  • System is still active and in use
  • System deletes or overwrites records

Some practical examples of UK archive services preserving records in line of business systems using this approach include:

  • The Archives First consortium was able to export records and metadata from the system that Gloucestershire County Council’s democratic services team use for managing Council minutes & agendas (Civica’s Modern.Gov). See Archives First: digital preservation – Further investigations into digital preservation for local authorities (Viv Cothey, 2020 (PDF, 622 KB)).
  • A group of local authorities in Wales worked with a Council’s democratic services team and a commercial digital preservation software supplier to ingest minutes, agenda papers and metadata from Civica‘s Modern.Gov in an automated way.
  • The National Records of Scotland have received digital records from depositors that were kept in business systems, including the National Confidential Forum (NCF) which is briefly described in Retour – The Newsletter of the Scottish Records Association, Issue 37 (Spring 2022), pp.15-16. It describes how “A common technical challenge to digital archiving is the successful export of data from depositor business systems in a format which can be preserved and fully understood. The system used by NCF was not designed with export in mind, and NRS staff needed to work closely with the depositor over several months to ensure the eventual transferred data are complete, accurate, and reliable.”
  • Crossrail Ltd extracted information from multiple systems and added them into a single Crossrail Archive using human-readable JavaScript Object Notation (JSON) objects. See Archiving Crossrail’s data (Isao Matsumoto, 2018).

Some practical examples from Australia include:

Option 4: Emulate the system or application that holds the records

Many archive practitioners in the United Kingdom, particularly in local government, may find this approach is impractical or not feasible, unless:

  • There is a strong justification for preserving the system/application because of its inherent historical value or evidential value
  • The records cannot be preserved without preserving the system, and/or
  • The archive service/organisation has a strong track record in emulation

As the Public Record Office Victoria (2016) outlines in the context of database management systems “maintenance of all original software, or software designed to mimic the original system…is not scalable as the archive quickly becomes complex as Relational Database Management System (RDBMS) products and versions are accumulated. Even if the complexity and cost of maintaining a diversity of platforms could be managed, retaining staff with the breadth of product and version knowledge required would make their administration unsustainable.”

However, archive practitioners may find that colleagues in IT and departments are using some type of emulation for legacy systems. For example, Peter Lehane (2013) outlines in the context of the State of New South Wales in Australia that “During our pilot migrations we’ve found that emulation is a practical solution that agencies themselves are using to manage legacy business systems, for example by keeping old software (such as older versions of Lotus Notes) running in virtual machines.”