V4: Innovation and improvement


Statistics producers should be creative and motivated to improve statistics and data, recognising the potential to harness technological advances for the development of all parts of the production and dissemination process.

 

V4.1 Statistics producers should keep up to date with developments that can improve statistics and data. They should be transparent in conducting their development activities, and be open about the outcomes and longer-term development plans.

V4.2 Statistics producers should consider testing and releasing new official statistics initially as experimental statistics, under the guidance of the Chief Statistician/Head of Profession for Statistics.

V4.3 Users should be involved in the ongoing development of statistics and data, exploring and testing statistical innovations, so that the statistics remain relevant and useful.

V4.4 Statistics producers should seek to collaborate with other producers, including within the UK and internationally, when developing their statistics, overcoming practical obstacles, and sharing best practice.

V4.5 Statistics producers should keep up to date with developments that might improve methods and quality. They should assess the added value of potential improvements and consider the likely impact on the statistics, including in relation to comparability and coherence.

V4.6 Producers should commit to improve data presentation, enhance insight, and better meet the needs of different types of users and potential users in the dissemination of their statistics and data.

V4.7 New and innovative ways to engage users, potential users and other stakeholders should be considered and adopted as appropriate.


Case studies

The National Travel Survey (NTS) team at the Department for Transport (DfT) has implemented a series of innovations and improvements during 2018/19. Some of these have been simple to implement but have a significant impact, while others have provided opportunities for the team to learn new skills that will provide long-term quality and efficiency benefits, for example, learning to use R Studio to automate data processing methods.

Making efficiencies has freed up analytical resource to make improvements in other areas, leading to a positive snowball effect. A user-first approach has been adopted, with all innovations being about how to further meet users’ needs.

Recent NTS innovations and improvements include:

  • Improving the NTS questionnaire following a feedback exercise, to check the relevance of NTS questions and the burden placed on respondents. As many NTS questions are still required by users, to make space for new topics, questions are rotated so that they are asked every other year. This ensures the survey length is not extended whilst still meeting user needs. New questions undergo extensive cognitive and panel testing to ensure participants understand them and that they collect the data users want
  • Setting up an innovative NTS Panel, consisting of NTS participants who agree to be contacted for follow-up research. This allows additional, smaller pieces of research to be conducted while not making the full NTS interview longer. The panel can target a sub-section of the population (e.g. people who cycle) where it would be disproportionately burdensome to ask everyone in the full NTS. Panel responses can also be linked back to original NTS responses, to greatly enhance the utility of the data
  • Collaborating with other analysts, including those outside of Government, to produce NTS analytical reports, demonstrating the breadth of information available in the NTS. By making the dataset accessible via the UK Data Service, and the ONS Secure Research Service, far more analysis can be undertaken than could be done by the NTS team alone
  • Advance letter and incentive experiments investigating how to boost response rates
  • Methodological improvements to collect walking data more accurately
  • Conducting a Discovery to explore whether developing a digital NTS diary could reduce respondent burden and increase data quality
  • Designing interactive tables and revising the data table categories so that it is easier for users to find the data they are searching for on GOV.UK
  • Publishing ad-hoc analyses, so they are accessible to all and enable the reuse of NTS data
  • Using R Studio to provide regular standard errors and confidence intervals for NTS statistics and ad-hoc analyses
  • Producing a user-friendly quality report to inform users about the quality of the NTS data, including sampling, methodology, quality assurance procedures and confidentiality
  • Making efficiency improvements to NTS data processing methods to greatly increase levels of automation using R, SQL and more advanced Excel functions

These improvements have led to increased engagement with a range of NTS stakeholders:

  • The publication of ad-hoc tables has drawn interest from academics and transport planners who have used the data as the basis for conducting further analysis in collaboration with DfT
  • The analytical reports produced in collaboration with external authors have provided a fresh look at what the NTS can provide and received mainstream and specialist press coverage
  • The NTS Panel has resulted in new demand from policy teams, with the team now looking forward to exploring these new research topics

The team is also testing the use of MailChimp as a new way to keep users up-to-date with NTS statistics and developments through a regular newsletter. The team hopes that this will increase its engagement with NTS users even further.

This example shows how the NTS team keeps up to date with developments that might improve NTS statistics for users, is transparent about its forthcoming development plans, and engages with users to get their feedback on plans to better meet their needs. It also shows how the NTS team collaborates with expert analysts to enhance value and insight, creates efficiencies by innovating methods and quality processes, and seeks to improve users’ experience by finding new ways to engage with them and enhancing the range of statistics that it makes available. The NTS team have also published a blog so that others can learn about these developments.

The Reproducible Analytical Pipeline (RAP) is an innovation initiated by the Government Digital Service (GDS) that combines techniques from academic research and software development. It aims to automate certain statistical production and publication processes – specifically, the narrative, highlights, graphs and tables. Tailor made functions work raw data up into a statistical release, freeing up resource for further analysis. The benefits of RAP are laid out in the link above, but include:

  • Auditability – the RAP method provides a permanent record of the process used to create the report, moreover, using Git for version control producers have access to all previous iterations of the code. This aids transparency, and the process itself can easily be published
  • Speed – it is quick and easy to update or reproduce the report, producers can implement small changes across multiple outputs simultaneously. The statistician, now free from doing repetitive tasks, has more time to exercise their analytical skills
  • Quality – Producers can build automated validation into the pipeline and produce a validation report, which can be continually augmented. Statisticians can therefore perform more robust quality assurance than would be possible by hand in the timeframe from receiving data to publication.
  • Knowledge transfer – all the information about how the report is produced is embedded in the code and documentation, making handover simple
  • Upskill – RAP is an opportunity to upskill individuals by giving them the opportunity to learn new skills or develop existing ones. This also upskills teams by making use of underused coding skills that may exist within their resource; coding skills are becoming ubiquitous nowadays with many STEM subject students learning to code at university

RAP therefore enables departments to develop and share high-quality reusable components of their statistics processes. This ‘reusability’ enables increased collaboration, greater consistency and quality across government, and reduced duplication of effort.

In June 2018, the Department for Transport (DfT) published its RAP debut with the automation of the Search and Rescue Helicopter (SARH) statistical tables. This was closely followed by the publication of Quarterly traffic estimates (TRA25) produced by DfT’s first bespoke Road Traffic pipeline R package. RAP methods are now being adopted across the department, with other teams building on the code already written for these reports. DfT have begun a dedicated RAP User Group to act as a support network for colleagues interested in RAPping.

DfT’s RAP successes have benefited from the early work and community code sharing approach of other departments, including:

  • Department for Digital, Culture, Media & Sport first published statistics using a custom-made R package, eesectors, in late 2016, with the code itself made freely available on GitHub.
  • Department for Education first published automated statistical tables of initial teacher training census data in November 2016, followed by the automated statistical report of pupil absence in schools in May 2017. DfE are now in the process of rolling out the RAP approach across their statistics publications
  • Ministry of Justice, as well as automating their own reports, have made a huge contribution with the development of the R package xltabr which can be used by RAPpers to easily format tables to meet presentation standards. Xtabr has also been made available to all on the Comprehensive R Archive Network.

The incorporation of data science coding skills with the traditional statistical production process, coupled with an online code sharing approach lends itself to increased collaboration, improved efficiency, and creates opportunities for government statisticians to provide further insights into their data.


Guidance and resources

DescriptionLinkSource
An OSR online article on how collaboration can lead to trusted, high quality statistics that provide public value, when it supports producers to improve and innovate in different ways. It draws on examples from producers working in Children, Education and Skills statistics and how they have collaborated to drive improvements.'Collaboration is key' online articleOSR
An OSR online article on innovation in the dissemination of statistics, focusing on positive features of the development process. It draws on examples from producers working in Children, Education and Skills statistics and how they have developed alternative outputs to supplement their main statistical publications.‘Engaging the audience – exploring statistics beyond the pdf’ online articleOSR
A blog by Louisa McCutcheon (OSR) on some of the challenges faced by producers when innovating in the development and presentation of statistics.‘Overcoming barriers to change’ blogOSR
A resource for official statistics producers to develop their knowledge and understanding of the broad range of methodological approaches used across the Government Statistical Service (GSS).GSS methodology webpageGSS
National Statistician’s guidance on interpreting and implementing the principles and practices of Code which relate to the quality and methodological basis of their official statistics. This document is scheduled for revision in 2018 to reflect version 2.0 of the Code.National Statistician’s Guidance: Quality, Methods, and Harmonisation (2009)GSS
A webpage with links to a series of guidance documents on harmonisation, including what harmonisation is and its aims, the Harmonisation Handbook and the GSS Harmonised Principles.Harmonisation within the GSS webpageGSS
Guidance that sets out the UK Statistics Authority (UKSA) policy on experimental statistics. Experimental statistics are a subset of new developed or innovative official statistics undergoing evaluation. The guidance sets out when to use the experimental statistics label, when to introduce experimental statistics, and removal of the experimental statistics label.Guidance on Experimental StatisticsGSS
A GSS blog about the work by the Department for Work and Pension’s (DWP) Client Statistics Branch to develop a variety of formats for releasing their statistics, to cover a wide range of users' needs.GSS data blog: Transforming the release of statistics in DWPGSS
Guidance on using social media to disseminate statistics. It is designed for departments who want to get started on social media, but may also be helpful for departments already using social media.Guidance on using social media in the GSSGSS
Government Digital Service (GDS) guidance on using social media in the public sector. It aims to raise awareness of the digital transformation of government services and to share GDS best practice.Social Media PlaybookGDS
Guidance on the UK Government's design principles and examples of how they've been used, from starting with user needs to designing with data.Government design principlesUK Government

Please help us improve our website by answering a few quick questions.