If you have digital records to transfer to The National Archives, please get in touch with your Information Management Consultant (IMC) or email our Information Management Team.
Currently we can accept: digital records that are due for transfer under the 20-year rule; records of inquiries; and records of public record bodies that are closing. If you want to discuss other circumstances that might warrant early transfer, please contact us to discuss.
From 2014 we will also accept early transfer of digital records, provided that they meet our criteria. Please see below for an overview of the digital transfer process.
For digital records, the following actions will need to be carried out during the selection and transfer process. We will work with you to identify when the actions need to be brought into that process. Sharing the guidance below with your ICT supplier or colleagues can support the coordinated effort.
- Assessing your IT/records management system
- File format evaluation
- Applying for closure on transfer
- Metadata and preparation
- Delivery of digital records to The National Archives
Assessing your IT/records management system
For digital records, one of the first things we will ask is how the records you intend to transfer are stored and managed. For example, are they in an electronic document and records management system (EDRMS), on shared drives, tape or another kind of specialist system? Perhaps the transfer will include records from multiple systems? During the process of appraisal you will become aware of the systems that are being used to store your records so inform your Information Management Consultant of this at the time.
You should have a documented process for exporting the records out of your system(s) so that they can be transferred. We will ask you for a sample export of some files and their associated metadata early in the transfer process. This will allow you to test your export method and consult with your IT team/service provider if assistance is needed, prior to the actual transfer taking place.
Your test should include checking that the exported records are not corrupted or changed by the process. For example the date of records (a vital piece of metadata) might be reset to the current date if records are copied over to a holding area prior to export. Similarly when you appraise digital records, take care that this work does not unduly alter the metadata of the records. Moving records around can alter the context of the records by removing them from their original folder structure. And in turn this can make the process of reviewing them for sensitivity much harder (see Applying for closure on transfer below).
Transfer size
For digital records it is important for us to know the size of the transfer - both the volume (usually in gigabytes) and the total number of individual files and folders.
File format evaluations
If a potential transfer contains formats which are not on our suitable file formats for transfer list, please discuss this with us. It could affect the final selection. This list can be found in our guidance and forms.
Likewise if some of your records are duplicated in different formats we will discuss with you which format we would prefer to transfer (for purposes of preservation and access). To do this, we need to know what file formats the transfer will contain. This is particularly important where bespoke applications have been used.
The National Archives has developed its own open source file format identification tool, DROID, which is free to download. We publish guidance on how to use this tool and we can arrange to visit your department for a face-to-face demonstration.
In addition to reporting on file formats, DROID will also generate all the mandatory metadata fields we require with the exception of copyright and closure status which cannot readily be generated automatically (see Metadata and preparation below). We will also carry out our own file format evaluation of any sample data you send us. For this reason it would be particularly useful if the sample you send is representative of the range of formats that will be present in the final transfer.
Applying for closure on transfer
All digital records must be reviewed for sensitivity before they can be transferred. Unless all records are open on transfer you will be asked to fill out a closure on transfer application form for digital records to send to us prior to transfer (see our guidance and forms page). Different series can all go on this form, there does not need to be a new form for each series. However you must only list one series, folder or file per line.
Closure should be asserted at the highest level appropriate - for example, if all records within a series are closed under the same Freedom of Information (FOI) exemption for the same number of years then that should be reflected on this form by referencing the top level folder(s). However if one file within a folder has a different closure period and/or FOI exemption number than the rest of the files within the folder, all files in that folder must have their closure stated.
Bear in mind that the information you enter on this form, specifically the identifier (URI filepath) and date must match the metadata provided for the same records in the Metadata and preparation stage. Once closures have been agreed you must include an up to date copy of this form saved as a csv (comma-separated values) file, along with your digital records when they are transferred to us (see Delivery of digital records). This will fulfill our metadata requirements for the closure status of digital records.
If you have redacted any records, you need to send us the un-redacted (original) versions too. These will be released or made available for you to review on the date calculated from the 'closure period' on the closure on transfer form. See the Guidance and forms section for information on redaction.
The National Archives is unable to accept the transfer of digital records above RESTRICTED/IL3. If such records are selected for permanent preservation, they must be retained within the department until the sensitivity diminishes below this level..
Metadata and preparation
For digital records, along with the files themselves, we require a csv file of metadata per series of records to be transferred. This provides contextual information about the records. Please refer to our Metadata factsheet, on our guidance and forms page, for more information on why metadata is important.
Records management systems should include a function to allow you to export the metadata for the records contained within it. As this varies from system to system we will ask you for a sample metadata export to see if the fields contained are sufficient. If there are options for you to choose for export we can discuss with you the best fit for our metadata requirements.
If your records are kept on shared drives we have a macro enabled spreadsheet which we can send you, along with instructions on how you can use it to scan the files you wish to transfer. In conjunction with the closure on transfer form this will provide us with all the mandatory metadata fields.
The seven mandatory metadata fields required for digital records at folder and file level are listed below. An export from our DROID tool will provide metadata fields 1-5. We also accept any additional metadata that will add further context to the records. Please ensure that identifiers and dates match any referenced in the closure on transfer form.
1. Title
This should be a meaningful folder or file name
2. Identifier
This is not a system-generated ID number but the file path represented as a URI - it supplies context for the record
3. Date
This should not be the date the record was copied to its current location. For records generated in a Windows operating system the date last modified is often the most accurate
4. Folder or file
Specify whether the digital object is a folder or a file
5. Checksum
This is generated using the SHA-2 (256) algorithm and it checks that the file has not changed during transfer
6. Copyright
Only required if the copyright is anything other than Crown copyright
7. Closure status
Please see Applying for closure on transfer - the closure on transfer form covers this metadata field
You also need to send us checksums for the metadata file itself and the closure on transfer form. These can be simple plain text files containing just the checksums.
Final preparatory steps for digital transfer
Once the broad details of the digital transfer have been agreed there are some final checks that you should make:
- Check that you have removed any unnecessary duplicate files from the transfer, and in the case of hybrid transfers, remove any digital files that will be transferred in their original paper format
- Check for and remove any temporary (.tmp) files from the transfer
- Check that you can open all of the files and view them fully without needing to enter a password or supplying a cryptographic key
- If you remove any files at this stage you must edit the metadata file(s) to reflect these changes or regenerate the metadata export
- Scan the files for viruses
A sample export
A sample export of files and the associated metadata for these files will allow you to trial the process of export, and enable us to check the following:
- what file formats your collection is comprised of
- whether the metadata you have automatically generated contains our mandatory fields
- that you have exported the files and the metadata in such a way that they correspond with each other
For ease of handling, your sample records and metadata should be open. We will securely dispose of the sample once we have assessed it, it will not form part of a transfer of records to The National Archives.
Delivery of digital records to the Archives
You must fill out a transfer form (see our guidance and forms page) and email it to us before sending us any digital records. Once we have confirmed both that we have received this form and that the information contained within it is correct, you can then send us your records.
Transfer media
We accept digital records on encrypted DVDs or USB hard drives, and a secure FTP service is in development. For smaller transfers we will send you pre-encrypted DVDs for you to copy your records onto. Please write the series reference and the current date (dd.mm.yyyy) onto the disks themselves using a DVD marker. If you are sending back multiple disks please also number them consecutively (1 of 2, 2 of 2). For larger volumes of data we will send you an encrypted USB hard drive with a pre-set volume ID.
All media must be sent via secure courier and the keyfile, and passphrase to decrypt the media will be sent to you separately via email. We use free and open-source encryption software which you will need to install on the computer you are copying the records from. We will provide you with detailed instructions for this at the time.
Copying your records onto the transfer media
When you copy the records onto the media for transfer, do not delete the original records until you have received confirmation via email that we have safe custody of your records. When you are copying your records, please copy each series into a separate folder on the media, and label the folders with the series references. If you are not sure about the series references please contact us. Each of these folders should contain the files which comprise the series, the associated csv metadata file, and a text file containing the checksum for the metadata csv.
Currently we supply hard drives which have a 2TB capacity. If a series is too large to fit on one piece of media please divide the records logically between two pieces of media. You also need to divide the metadata for the files between the two pieces of media so that each set of files is stored with its associated metadata (and the checksum for the metadata file). You might find it easier to generate the metadata for the two parts of the series separately rather than dividing the file itself.
If any records are closed on transfer you must also include a copy of the closure on transfer form saved as a csv file (and a text file containing a checksum for the form). This does not need to be series specific so it can be copied to the root of the volume.
Please fill out a delivery form (on our guidance and forms page), email it to us and enclose a paper copy of it within the transfer package itself. This will allow us to identify the records easily upon receipt.
