The science of recordkeeping systems – a realist perspective

A doctoral thesis submitted in partial fulfilment of the requirements for the award of Doctor of Philosophy of Loughborough University (2023).

The mass adoption of email systems in the 1990s onwards brought with it a revolution in communications. For the first time in human history most written communications were sent by digital rather than analogue means. This communications revolution had a huge impact on recordkeeping theory and practice. It stimulated a mini renaissance of records management theory but was also disruptive and contentious.

Theoretical models developed in the 1990s that focused on recordkeeping within an organisation were utopian in nature, assuming that it would always be possible for an organisation to both capture records routinely and aggregate them according to the business activity that they arose from. At the time of this study there had been no major developments in records management theory since the 1990s.

Based on a review of recordkeeping theory, and the evaluation of rival policies towards email, this study develops two new models of how an organisational record system works:

  • the record system matrix – which ‘zooms-in’ Upward’s record continuum model to show the record system of an organisation as an open system embedded in a recordkeeping society. The matrix depicts the sixteen step implementation chain of an organisational record system, presented as four by four matrix, which shows the actions needed from social actors (policy makers, practitioners, and end-users) for an organisation to be able to conduct communication transactions, capture records of those transactions, aggregate those records and apply rules to those records.
  • the record system trade-offs model – which shows the features a record system must possess to be seen to be reliable, efficient, precise, and predictable enough to meet the needs of an originating organisation and its stakeholders; and which shows the 5 trade-offs that organisations are faced with in circumstances where it is not possible to optimise a record system to meet all four of these imperatives.

Taken together these two models will help originating organisations and their stakeholders to understand the likely consequences of the various (imperfect) records management approaches available to them.

The study uses the two models to assess the likely impact on recordkeeping of the possibilities offered by advances in artificial intelligence (AI).

Insights from the two models developed in this study, and from the AI specific thought experiments conducted in this study, are used to chart an incremental route whereby AI capabilities could be applied within email accounts with a view to progressively improving the precision and efficiency with which emails are managed, whilst minimising the risk of acting counter to end-user expectations.

This study made a separation between:

  • the scientific question as to what is a record (and what is a record system); AND
  • the policy question (for an organisation) as to what types of record (in what types of system) they should protect with their records retention regime.

The study further found that:

  • records are a commonplace (rather than a special or unusual) feature of this universe;
  • record systems exist in the natural world as well as in the social world;
  • it is possible to arrive at a definition of record systems that encompasses such systems in both the natural and social worlds;
  • predictability is a core feature of record systems in both the natural and social worlds.

A key opportunity for further research arising from this study concerns the possibility of the development of a science of record systems that embraces both the natural and the social worlds. Such a field of study could draw inspiration from the establishment by Claude Shannon of the discipline of information theory in the 1940s.