The application of technology-assisted review to born-digital records transfer, Inquiries and beyond: research report
Born-digital records pose many challenges for government departments. These include high volumes of records and a lack of structure of born-digital record collections. This affects not just the appraisal, selection and sensitivity review processes when transferring records to The National Archives but also poses challenges for departments responding to Inquiries and Freedom of Information requests. There are also broader information management and security concerns for born-digital record collections.
The National Archives conducted trials of technology-assisted review (this is sometimes referred to as TAR or computer-assisted review) to examine these challenges and explore the potential solutions. We also looked at additional research to test how these tools and processes could help meet the challenges of born-digital records. Technology-assisted review describes a process or processes involving expert document reviewers using a combination of computer software and tools to automatically classify records. This report summarises the key lessons learned from that work.
The report concludes that technology-assisted review using eDiscovery software can support government departments during appraisal, selection and sensitivity review as part of a born-digital records transfer to The National Archives. This support also extends to responding to Inquiries and Freedom of Information requests, as well as information management and information security. We summarised these findings into eight lessons learned:
- Understanding born-digital collections at a high level
- Reducing the amount of information to review
- Extracting meaning
- Identifying personal information
- Procurement
- User interface
- Collaboration with other teams
- Confidence in technology-assisted review
There is no completely automated solution; human input is still required at all stages. But technology-assisted review offers ways to understand, value, prioritise and reduce the volume of born-digital records needing to be manually reviewed. The report ends by setting out further research The National Archives will conduct and the support it plans to give to government departments to help them manage their born-digital record collections. As such we will continue to work with Cabinet Office and the Government Digital Service to provide solutions that are tested with government departments. In addition, we will continue collaboration with other centres of expertise within government and beyond to enhance methods and tools.
Contact us on GovernmentTransfers@nationalarchives.gov.uk if you have any questions about this report.
The digital landscape in government 2014-2015: business intelligence review
The Digital Transfer Project launched in September 2013, to prepare The National Archives and government departments for large scale born-digital record transfers, starting in 2016 under The Public Records Act. The aim of the project was to develop a scalable process for the transfer, preservation and presentation of born-digital records with long-term value; enabling them to be held securely while closed, and be accessible to the public when open.
As part of the project, we undertook a business intelligence review to understand the current digital landscape and digital challenges of government departments in the UK. In particular to understand:
- the volumes and types of digital information being created and managed in government
- the capabilities and capacity of government departments to manage digital records, both current and legacy (those records not currently in business use)
- the ability of government departments to prepare for the transfer of digital records to The National Archives
This snapshot of the digital landscape in government helps The National Archives and government departments to plan resources and prepare for the future transfers of born-digital records. The findings will also assist in determining areas where government departments may need support during the transfer process enabling potential challenges to be identified and addressed at an early stage.
An expert team at The National Archives experienced in digital records management undertook the research and writing of this report. It was undertaken prior to the publication of Sir Alex Allan’s review of government digital records and seeks to complement the review’s work.
The digital landscape in government 2014-15: business intelligence review
Contact us on GovernmentTransfers@nationalarchives.gov.uk if you have any questions about this report.