Statistics producers should be creative and motivated to improve statistics and data, recognising the potential to harness technological advances for the development of all parts of the production and dissemination process.
V4.1 Statistics producers should keep up to date with developments that can improve statistics and data. They should be transparent in conducting their development activities, and be open about the outcomes and longer-term development plans.
V4.2 Statistics producers should consider testing and releasing new official statistics initially as experimental statistics, under the guidance of the Chief Statistician/Head of Profession for Statistics.
V4.3 Users should be involved in the ongoing development of statistics and data, exploring and testing statistical innovations, so that the statistics remain relevant and useful.
V4.4 Statistics producers should seek to collaborate with other producers, including within the UK and internationally, when developing their statistics, overcoming practical obstacles, and sharing best practice.
V4.5 Statistics producers should keep up to date with developments that might improve methods and quality. They should assess the added value of potential improvements and consider the likely impact on the statistics, including in relation to comparability and coherence.
V4.6 Producers should commit to improve data presentation, enhance insight, and better meet the needs of different types of users and potential users in the dissemination of their statistics and data.
V4.7 New and innovative ways to engage users, potential users and other stakeholders should be considered and adopted as appropriate.
The Reproducible Analytical Pipeline (RAP) is an innovation initiated by the Government Digital Service (GDS) that combines techniques from academic research and software development. It aims to automate certain statistical production and publication processes – specifically, the narrative, highlights, graphs and tables. Tailor made functions work raw data up into a statistical release, freeing up resource for further analysis. The benefits of RAP are laid out in the link above, but include:
- Auditability – the RAP method provides a permanent record of the process used to create the report, moreover, using Git for version control producers have access to all previous iterations of the code. This aids transparency, and the process itself can easily be published
- Speed – it is quick and easy to update or reproduce the report, producers can implement small changes across multiple outputs simultaneously. The statistician, now free from doing repetitive tasks, has more time to exercise their analytical skills
- Quality – Producers can build automated validation into the pipeline and produce a validation report, which can be continually augmented. Statisticians can therefore perform more robust quality assurance than would be possible by hand in the timeframe from receiving data to publication.
- Knowledge transfer – all the information about how the report is produced is embedded in the code and documentation, making handover simple
- Upskill – RAP is an opportunity to upskill individuals by giving them the opportunity to learn new skills or develop existing ones. This also upskills teams by making use of underused coding skills that may exist within their resource; coding skills are becoming ubiquitous nowadays with many STEM subject students learning to code at university
RAP therefore enables departments to develop and share high-quality reusable components of their statistics processes. This ‘reusability’ enables increased collaboration, greater consistency and quality across government, and reduced duplication of effort.
In June 2018, the Department for Transport (DfT) published its RAP debut with the automation of the Search and Rescue Helicopter (SARH) statistical tables. This was closely followed by the publication of Quarterly traffic estimates (TRA25) produced by DfT’s first bespoke Road Traffic pipeline R package. RAP methods are now being adopted across the department, with other teams building on the code already written for these reports. DfT have begun a dedicated RAP User Group to act as a support network for colleagues interested in RAPping.
DfT’s RAP successes have benefitted from the early work and community code sharing approach of other departments, including:
- Department for Digital, Culture, Media & Sport first published statistics using a custom-made R package, eesectors, in late 2016, with the code itself made freely available on GitHub.
- Department for Education first published automated statistical tables of initial teacher training census datain November 2016, followed by the automated statistical report of pupil absence in schools in May 2017. DfE are now in the process of rolling out the RAP approach across their statistics publications
- Ministry of Justice, as well as automating their own reports, have made a huge contribution with the development of the R package xltabr which can be used by RAPpers to easily format tables to meet presentation standards. Xtabr has also been made available to all on the Comprehensive R Archive Network.
The incorporation of data science coding skills with the traditional statistical production process, coupled with an online code sharing approach lends itself to increased collaboration, improved efficiency, and creates opportunities for government statisticians to provide further insights into their data.
|An OSR online article on how collaboration can lead to trusted, high quality statistics that provide public value, when it supports producers to improve and innovate in different ways. It draws on examples from producers working in Children, Education and Skills statistics and how they have collaborated to drive improvements.||'Collaboration is key' online article||OSR|
|An OSR online article on innovation in the dissemination of statistics, focusing on positive features of the development process. It draws on examples from producers working in Children, Education and Skills statistics and how they have developed alternative outputs to supplement their main statistical publications.||‘Engaging the audience – exploring statistics beyond the pdf’ online article||OSR|
|A blog by Louisa McCutcheon (OSR) on some of the challenges faced by producers when innovating in the development and presentation of statistics.||‘Overcoming barriers to change’ blog||OSR|
|A resource for official statistics producers to develop their knowledge and understanding of the broad range of methodological approaches used across the Government Statistical Service (GSS).||GSS methodology webpage||GSS|
|National Statistician’s guidance on interpreting and implementing the principles and practices of Code which relate to the quality and methodological basis of their official statistics. This document is scheduled for revision in 2018 to reflect version 2.0 of the Code.||National Statistician’s Guidance: Quality, Methods, and Harmonisation (2009)||GSS|
|A webpage with links to a series of guidance documents on harmonisation, including what harmonisation is and its aims, the Harmonisation Handbook and the GSS Harmonised Principles.||Harmonisation within the GSS webpage||GSS|
|Guidance that sets out the UK Statistics Authority (UKSA) policy on experimental statistics. Experimental statistics are a subset of new developed or innovative official statistics undergoing evaluation. The guidance sets out when to use the experimental statistics label, when to introduce experimental statistics, and removal of the experimental statistics label.||Guidance on Experimental Statistics||GSS|
|A GSS blog about the work by the Department for Work and Pension’s (DWP) Client Statistics Branch to develop a variety of formats for releasing their statistics, to cover a wide range of users' needs.||GSS data blog: Transforming the release of statistics in DWP||GSS|
|Guidance on using social media to disseminate statistics. It is designed for departments who want to get started on social media, but may also be helpful for departments already using social media.||Guidance on using social media in the GSS||GSS|
|Government Digital Service (GDS) guidance on using social media in the public sector. It aims to raise awareness of the digital transformation of government services and to share GDS best practice.||Social Media Playbook||GDS|
|Guidance on the UK Government's design principles and examples of how they've been used, from starting with user needs to designing with data.||Government design principles||UK Government|