Internal audit
Following the Practice Recommendation issued by the Information Commissioner’s Office, The National Archives responded by committing to a Freedom of Information (FOI) Action Plan.
As part of the Action Plan, The National Archives committed to undertake an Internal Audit to provide assurance on the extent to which the Action Plan is being delivered as intended, and whether associated risks to deliver the action plan are being managed.
This Lessons Learned report focuses on several areas to support The National Archives in improving its performance in relation to FOI compliance activity.
Lessons Learned report – overview
Timeliness compliance for Freedom of Information (FOI) requests at The National Archives and subsequently HM Government has been negatively impacted by the transfer of the Ministry of Defence (MOD) Service Personnel Records, a collection consisting of over 9.7 million records which are being transferred to The National Archives over a 6-year period. The completion of this transfer will see the number of orderable records at The National Archives almost double in number. So far, we have transferred approximately 7 million records from this collection – more information is available at our project page.
The transfer of these much-anticipated service personnel records is an exciting and vital step in preserving and making available this uniquely rich collection. That said, the complexity and scale of the project is unprecedented, and we have identified the following four key areas of learning to support our ongoing FOI compliance recovery activity: responding to risk; resourcing; data capture; and wider service transformation.
Lessons Learned
1. Responding to risk
What is the challenge?
The organisational impact of the MOD Service Personnel Records transfer project is unprecedented. The volume of material that we are bringing into the national collection along with the storage, preservation and access provisions for these records present unique challenges and risks for the institution.
What have we learned?
We have a good Governance & Risk Management framework in place, which includes: project reporting mechanisms; workstream subgroups that focus on key aspects of delivery; and a Steering Group that oversees the strategic direction of the project and feeds in to the standard escalation and reporting process to The National Archives’ Executive Team and Board.
Consequently, risks are well sighted and acknowledged. What we have learnt is that we need to improve how we mobilise ourselves to provide a more effective and impactful cross-organisational response to escalating risk.
2. Resourcing the demand
What is the challenge?
The key organisational challenge of the MOD Service Personnel Records transfer project is one of volume (over 9 million records) and how to make this vast collection accessible to the public.
To provide an access service to such a large collection requires organisation-wide resourcing, prioritisation and planning.
Within the Access Service Team (who facilitate these requests) we have continually committed to increasing resource as demand grows, as evidenced in our action plan: The Access Service Team expanded to three in 2021/22, from three to eight in 2022/23, and from eight to eighteen permanent members across 2023/24. The current financial year (2024/25) has seen our biggest ever expansion, from eighteen to thirty-eight members of staff, supported by specific additional funding from HM Treasury. In addition, we have increased capacity in both our document production and cataloguing teams.
What have we learned?
Demand has dramatically outstripped the pace of investment in extra resource. In 2022 our annual Freedom of Information (FOI) requests for access to the MOD service personnel records were just shy of 8,000. In 2023 the annual figure was over 17,000, and at the mid-point of 2024 the volume had already exceeded 16,500 requests.
In response, we continue to develop a well-resourced, scalable plan that ensures:
- an agile response to changing priorities
- e.g. data driven plan on current and future resourcing levels
- clarity of prioritisation and messaging (both internally and externally)
- technology and facilities solutions that keep pace with recruitment expansion
- appropriate technological systems that support continuous process improvement and efficient service delivery.
3. Data Capture – Forecasting and analytics
What is the challenge?
The organisation has made data a priority and The National Archives has invested in our strategic insights capability to provide focussed support to this project, including dynamic techniques to predict and respond to resourcing demands. However, building this capability takes time and will need to continually evolve as our ability to capture and understand the data around the project becomes more sophisticated.
What have we learned?
Data collection and reporting requires coordination and communication across multiple teams and directorates within The National Archives.
We need to invest in proactive monitoring and more regular analysis of our data to build the required knowledge for successful service delivery.
We need to expand our data analytics and financial modelling skills by improving the overall data culture relating to the project. This includes the creation of dedicated resource to take principal responsibility for enabling robust practices of data capture, collation, interpretation, and insights.
4. Service Transformation
What is the challenge?
Providing an appropriate access service to these records does not easily equate to other operations activity that we traditionally see elsewhere within The National Archives’ own business, or elsewhere in government. The challenge encompasses policy, legal, logistical, technical, technological, communications, recruitment, commercial, cultural, reputational, and financial considerations that are mostly unprecedented in terms of scale and complexity.
Good governance and strong collective leadership are key to ensuring The National Archives can meet both the demands and the opportunities that this project presents.
What have we learned?
We have learnt that marginal gains and continuous improvement practices are an important aspect of everyday activity. We outlined in our Action Plan how and when we need to make decisions to utilise mechanisms to resolve issues with our processes, such as, logging and triaging.
We have also learnt that our leadership capacity has been too narrow and have taken steps to bolster our leadership function to help us to transform our access service in the short to medium term.
This is a major programme of work, involving significant investment and cross-organisational collaboration. The transformation of this service is therefore explicitly prioritised and owned, collectively, at the highest levels of the organisation.
Next steps
1. Resourcing and budget planning:
a. Finalise the 2024/25 expansion
- Prioritise necessary recruitment and onboarding.
- Regular tracking of operational progress.
b. Budget Planning 2025/26
- Agree what further resources are required to assist with the wider strategic direction of the project – specifically around service transformation.
- Include an estates plan to support team expansion – to facilitate closer working, exchange of knowledge and cohesion/consistency of approach.
2. Service transformation:
- Project Planning to include sequencing of activity to ensure successful transformation and return to legal compliance.
3. Improve Data Culture:
- Recruit dedicated resource to enable and refine robust practices of data capture, collation, interpretation, and insights.
January 2025