Digital transformation in government has brought an increase in the scale and variety of public records along with a reduced emphasis on organising and structuring data. Traditional processes designed for paper records cannot handle the volume, diversity, complexity and distributed nature of departmental digital records.
The project by The National Archives explored the potential of Artificial Intelligence (AI) tools to assist with this challenge. Five AI vendors applied their tools to classify a dataset supplied by The National Archives. The tools and platforms evaluated were Adlib Elevate, Amazon Web Services, Microsoft Azure, InSight by Iron Mountain, and Records365 by RecordPoint. Promising results were obtained overall with no tool or approach consistently outperforming the others across all tasks.
The project found that while AI cannot replace the expertise of Records Managers, commercially available AI tools and pipelines can be successfully applied to aid the task of records selection in semi-structured and unstructured collections. These products are still evolving and this a good time to engage with suppliers and influence the direction of development.
The document ‘Using AI for Digital Selection in Government‘ outlines the key findings of this project along with guidance for government departments looking to implement their own AI solution. The guidance is written for UK government officials who are familiar with the appraisal, selection, review and transfer of records to The National Archives for permanent preservation.
As part of the project an experienced market researcher was contracted to undertake an analysis of the current AI commercial tools available. They completed a report explaining the reasons for the selection of certain products. The suppliers each provided a report at the end of the project. Links to these are included below. As part of the project, The National Archives also developed a ‘benchmarking tool’; this report provides detailed information about this tool.
Guidance and key findings of the project
Using AI for Digital Records Selection in Government (PDF, 0.6 MB)
Additional project outputs
Report of First Phase Market Research (PDF, 2.7 MB)
AI for Selection Evaluation Template (XLS, 0.1 MB)
The National Archives Benchmarking Tool (PDF, 1 MB)
Supplier Reports
Adlib Elevate by Adlib (in conjunction with Deloitte) (PDF, 17 MB)
Amazon Web Services (AWS) used by Kainos (PDF, 39 MB)
InSight by Iron Mountain (PPT, 2 MB)