Good information management underpins good information security. This is increasingly the case in the digital era. The Government’s Security Policy Framework states:
The effective management of information is critical to safeguarding it. Government organisations will consider good information management practice as the basis for their information security arrangements.
Having good policies, procedures and training in place, as well as the right technical environment, to understand and manage your information and data assets will therefore help you to manage your security-related information risks. See our information management guidance and standards pages for general information management guidance. For guidance on information security, the following sources of information are useful.
Policies
Security Policy Framework, Cabinet Office
Cyber security
For further guidance on information assurance and cyber security see the following resources:
10 steps to cyber security, BIS
Information Risk Management Guidance, CESG
Security top tips, BCS
Cyber research and guides, CPNI
HoMER tool, CPNI
Risk and information risk guidance
Guidance and tools for assessing risks to your information.
The Orange Book, HM Treasury
Cyber risk, The Insititute of Risk Management
Risk guides, The Institute of Risk Management
Everyone relies on the integrity of digital information, from the citizen to heads of government. It is essential that this information is preserved for future generations, just as traditional records have been preserved for us on paper and parchment.
Our approach to digital preservation
Digital preservation encompasses the long term storage of and continued access to digital records. The digital records preserved by The National Archives fall into three broad categories:
- ‘Born-digital’ – records produced in digital form such as spreadsheets, emails, websites, databases and video
- ‘Digitised records’ – scanned copies of paper or parchment records that are accessioned in place of the paper record
- ‘Digital surrogates’ – paper or parchment original is retained as the record and a digital image is used as an access copy
The records are ingested into our digital preservation system, the Digital Records Infrastructure (DRI). Records currently ingested into DRI include First World War diaries, Second World War Home Guard service records, UK Supreme Court video recordings and records from The Al Sweady and Leveson Inquiries.
The National Archives undertakes a parsimonious approach to digital preservation, focusing on preserving the original files. This is achieved by making sure they are virus free, undertaking fixity checks to ensure the integrity of the files over time and by “knowing what we’ve got“ (identifying all the file formats in our collections). We achieve file format identification by using the tools described in the section below.
We are increasingly turning our minds towards the issues of scale and speed of transfer, and delivery of digital records to end users, which we believe to be the current challenges of digital preservation at The National Archives.
Our approach to digital preservation is summarised in the paper below:
Parsimonious preservation paper (PDF, 0.04Mb)
Digital preservation tools developed by The National Archives
PRONOM is an online database containing details of more than 1,300 different digital file formats. Along with DROID, our file format identification tool (which utilises the signatures within the database), PRONOM enables digital archivists, records managers and anyone using the tool to find out what files they have and in which formats. DROID also outputs other useful information such as the last modified date and the size of the files scanned.
The DROID tool can scan a computer, hard drive or collection of digital files and identify file formats either through its file extension (for example .doc for Word files) or by matching the file’s internal signature with specific entries in the PRONOM database. Internal signatures are based on recurring patterns in the byte sequence in the headers and footers of file formats and as such are a far more accurate way of identifying file formats, as extensions can be easily changed, deleted or used by different formats or versions.
Currently around 70 percent of entries in the PRONOM database have internal signatures. The more signatures there are in the database, the more accurate the DROID tool is at identifying files, and the greater its use to the digital preservation community. In light of this we endeavour to keep PRONOM updated based on our own incoming digital collections, and file format research provided by the wider digital preservation community.
You can download our latest version of DROID for free. For previous versions go to http://droid.sourceforge.net/. For more information, see our PRONOM resource.
The CSV validator is a CSV validation and reporting tool. The National Archives receives metadata files alongside new digital records, currently this is received in CSV format. The CSV validator takes a CSV Schema file and a CSV file. It will verify that the CSV Schema itself is syntactically correct and then assert that each rule in the CSV Schema is upheld in the CSV file.
Bookmark this page, or subscribe to the feed, for news on developments in public sector information within the UK, Europe and beyond, including:
- notice of surveys and consultations
- news on reports and articles of interest
We hope this will be of use and would welcome feedback on the features and news presented here. Email us at psi@nationalarchives.gov.uk
Find out who to contact at The National Archives for advice or information.
You can also listen to our podcasts, read our blogs and share your knowledge with other information management professionals over on our social media channels.
If you wish to contact The National Archives for advice or information in any of the following areas, the most appropriate contacts are listed below. For general enquiries or if you have a records, research or record copying enquiry, please visit our Contact us page.
Contacting us by email
| Crown copyright, licensing and public sector information psi@nationalarchives.gov.uk | For Crown copyright, licensing and public sector information |
| Legislation Services legislation@nationalarchives.gov.uk | For legislation publishing enquiries |
| Official Publishing Enquiries +44 (0) 20 8392 5218 official.publishing@nationalarchives.gov.uk | For enquiries about official publishing of Parliamentary papers (including Command and House of Commons Papers), and requests for Command Paper numbers from government departments |
| Information management information.management@nationalarchives.gov.uk | For queries relating to information management, including the selection and transfer of records to The National Archives |
| Information Management Assessment programme information.management@nationalarchives.gov.uk | For queries relating to the IMA programme |
| Digital preservation digitalpreservation@nationalarchives.gov.uk | Conserving and preserving digital information. Developing and researching suitable systems to handle technological changes in digital formats |
The digital preservation team is currently undertaking research in the following areas:
Safeguarding the nation’s digital memory
The nation’s digital heritage is rich, complex and fragile. It is under threat from rapidly evolving technology, outdated policies and a skills gap across the archives sector. To preserve this heritage for future generations, we must understand and navigate a vast and ever-shifting risk landscape. This is a challenge which no single archive is currently equipped to address.
We propose a collaborative approach to managing digital preservation risk, bringing established statistical risk management methods into the digital heritage sphere. Project participants will create a structured evidence base, pooling our collective experience to map and explain an interconnected network of risk events, actions and impact on heritage. This will build a holistic understanding of risk, enabling archivists to prioritise threats and choose the most effective actions to combat them.
This approach will help us capture, analyse and share our experience and reasoning to improve transparency and accountability. It can flex to accommodate the diverse contexts and priorities of our organisations, communities and collections. The project will improve access to robust methods and tools and our events will raise awareness of the approach, to help archives make better decisions for the future of our digital heritage.
We are working with five partners across the archives sector in England, academic partners from the Applied Statistics & Risk Unit (AS&RU) at the University of Warwick, and the Digital Preservation Coalition will provide independent evaluation and support for training events. See the project page for more detailed information and background.
This project is support by a grant of £93,500 from the National Lottery Heritage Fund, and AS&RU’s postdoctoral research assistant is supported by an EPSRC Impact Acceleration Account award.
AI for Digital Selection
Digital transformation in government has brought an increase in the scale and variety of public records along with a reduced emphasis on organising and structuring data. Traditional processes designed for paper records cannot handle the volume, diversity, complexity and distributed nature of departmental digital records.
The project explored the potential of Artificial Intelligence (AI) tools to assist with this challenge. Five AI vendors applied their tools to classify a dataset supplied by The National Archives. The tools and platforms evaluated were Adlib Elevate, Amazon Web Services, Microsoft Azure, InSight by Iron Mountain, and Records365 by RecordPoint.
See the project page for more detailed information and background.
PRONOM – file format research
PRONOM enables digital archivists, records managers and anyone using the tool to find out what files they have and in which formats. The digital preservation department carries out file format research in order to make sure that PRONOM covers the most important formats for The National Archives and the digital preservation community. The first priority is being able to identify all file formats transferred to us by UK Government departments. The second driving force of our research comes from the digital preservation community. We actively encourage organisations or individuals dealing with digital records to help provide information on the formats they encounter.
Contributors include The British Library, National Library of New Zealand, The Church of Jesus Christ of Latter-day Saints and many more.
See the wide spread of regular contributors from around the world who have played a vital part in expanding the coverage of the signatures included in PRONOM.
Plain text file format identification
The aim of this project is to support DROID, The National Archives’ file format identification software. At present, DROID recognises binary file formats only, but we aim to find a way of identifying the type of text files from their contents. An AI based methodology was used to build the prototype, which recognised five file types (.py, .java, .txt, .tsv and .csv) successfully, using prediction based classification to recognise file formats.
Two blogs – How to correctly identify the file type of a text file from its contents and Motivation to undertake file format identification research for plain text files – provide further insight.
Earlier research projects
For research projects carried out before 2020 see the UK Gsovernment web archive.
The National Archives is tackling the challenges of digital preservation, ensuring continued access to digital information in the future.
We play an active role in storing and preserving digital material created by UK government.
These pages provide guidance on the selection and transfer of records. Bodies transferring records to The National Archives or to places of deposit under the Public Records Act 1958 should follow this process.
This process applies to records in all formats and media, including paper and digital records.
Thorough planning is essential for good information management. You need to secure support from senior management for this work and ensure that you have the right people involved across your organisation. You should also have a clear understanding of your business requirements and compliance requirements under legislation such as the Public Records and Freedom of Information Acts.
The Tithe Commutation Act, 1836 (6&7 Will.IV, c.71) allowed payment of tithes in kind to be substituted by money payments. In most cases the principal records of the commutation of tithes in a parish under the Tithe Act 1836 is the Tithe Apportionment, altered apportionment and the tithe maps.
The Tithe Act, 1936 (26 Geo. V and 1 Edw. VIII. C.43) abolished all tithe rent charges. Responsibility for tithe documents created under the tithe acts (1836, 1837, 1839, 1860, 1891) were placed under the charge of the Master of the Rolls, who has the authority to transfer them to an approved place of deposit. This responsibility is exercised by The National Archives: Historical Manuscripts Commission. The Master of the Rolls has issued Tithe (Copies of Instruments of apportionment) Rules 1960 (SI 1960/2440), as amended by the Tithe (Copies of Instruments of Apportionment) (Amendment) Rules 1963 (SI 1963/977)] concerning the care, custody, access to, and definition of tithe documents.
The tithe acts provided for the making of an original and two copies of every confirmed instrument of apportionment. The originals are now in The National Archives. The two copies were deposited with the registrar of the diocese and with the incumbents of and churchwardens of the parish. In many cases the copies and subsequent altered apportionments are now deposited in the relevant local record office.
Further information about the history of tithes is in our research guide on tithe records.