This step is the most technical part of the process. It is about physically moving the records and their metadata from the government department to The National Archives.
To do this the following stages should be followed:
- Prepare
- Move your records into a preparation area, ensuring that the original structure of the records is maintained (this will help end-users find the data, browse and search).
- Use Teracopy to indicate errors in the copying process and to avoid missing or corrupting files. Teracopy is a free software designed to be used to move or copy computer files. There is other software available that does a similar job, if you are considering using different software then please consult The National Archives to ensure that it meets the necessary requirements.
- Agree on the mode of transfer: SFTP (Secure File Transfer Protocol) or in most cases an encrypted HDD (Hard Disk Drive), which The National Archives can provide. SFTP can only be used for open records and smaller collections.
- Generate and validate metadata
You will transfer not only the records but their metadata as well. This metadata, when open, will appear on Discovery so that end-users can search, browse and find relevant material. Completing this step accurately will ensure that we capture the information we need to add value to the records. Generating the metadata is in essence the equivalent of populating a cataloguing template in the paper environment.
Generating metadata:
- The cataloguing template is called a ‘metadata csv’ file
- It is similar to an Excel file – good Excel skills are vital to the process. In order to make the most out of the metadata.csv your team members will need to know how to copy/paste, sort, filter and replace in Excel (the use of macro, functions, vlookup are not necessary)
- How is it generated?
- DROID generates:
- File path (identifier)
- File name (title)
- Date last modified
- File/Folder
- Checksum
- Manual input is needed for:
- Closure type
- Closure period
- Closure start date
- Freedom of Information (FOI) exemption code
- FOI decision date (exemption asserted)
- Title public
- Title alternate
- Description public (only required if you are providing a description)
- Description alternate (only required if you are providing a description)
- Copyright
- Legal status
- Held by
- Optional metadata for which manual input is needed (depends on whether or not this applies to your records)
- Description
- Former reference
- Language
- Other language title
- Start date
- End date
- Date range
- DROID generates:
Validation of metadata:
- The metadata.csv will need to be validated using another software, CSV Validator. CSV Validator is a mandatory software required to transfer records to The National Archives. It has been developed by The National Archives and is free and easy to download.
Typical errors identified at this stage include:
- blank spaces
- unrecognised symbols or characters
- incorrect information from manual generation of metadata
This stage of the process can be quite time-consuming, but it is vital to the overall success of the transfer. Desk instructions to complete this stage of the process can be found here. If you have any questions, or are in need of any assistance, contact your Digital Transfer Adviser.
Note: The records and the metadata should not be deleted until they are displayed on our catalogue, Discovery, even if successfully transferred to The National Archives.
- Proceed to transfer
If using SFTP, we will agree a date and transmission window. If using an HDD, the drive should be sent via trusted courier service. We can provide you with a list of secure courier services.
- Confirm transfer
The National Archives will confirm receipt of the records once they have been uploaded via SFTP or once the HDD has been delivered. Please do not delete the records at this stage. The National Archives will confirm with you when this can be done (after step 5 is completed).