Case studies of voluntary application


The case studies below serve as examples of organisations that are applying the Code pillars to their data, statistics and analysis (that are not official statistics), to enhance public confidence. They were written by the producer organisation and explain why and how the organisation voluntarily applied Trustworthiness, Quality and Value.

 

Rationale for applying the Code

Financial Lives is our largest tracking survey of adults and their finances. It is a survey of nearly 13,000 consumers and their use of financial services. The 2017 survey was based on nearly 13,000 face-to-face and online interviews, and reveals a wealth of information about different types of consumer and their experiences of financial products and services. It is a key source of information that underpins our consumer protection work and also provides valuable insights for any organisation focusing on consumers and finance.

The second iteration of the survey is taking place in 2019. It is our ambition to have the statistics produced from the survey to be recognised for the rigour, robustness and accuracy with which they are produced. By voluntarily applying the Code our hope is that this will give confidence to those using the statistics and aid our ambition for the survey results to be the used across the whole sector by a variety of organisations including Government, Consumer Groups and Firms.

Understanding the Code and its impact on our current practices

Our initial investigation on how the Code would apply to our business involved looking at the various statements produced by other organisations and understanding how the Code would apply to our working processes. Fortunately, we work under very tight procurement practices and research standards and much of the Code was applied as part of our standard research process.

The Financial Conduct Authority (FCA) is committed to being transparent in how we work. As such we publish all out survey results and are open in how we conduct research. We saw the Code as a useful way to bring together in a single place a clear and succinct document that outlines our process for working with agencies and the decision-making process for how we select certain methodologies and outputs to achieve our research goals and value for money.

Getting working level buy-in

With other commitments and priorities it was important to secure the time to think through the code and its implications, as well as write the statement of compliance (PDF, 0.40MB).

The Office for Statistics Regulation (OSR) was helpful in being open to having discussions how it would affect our business while we saw the benefits:

  • To users of the findings – our results are used by other regulators, industry and consumer groups. As well as being used in our regulatory responses to Government and in policy documents.
  • To respondents to the survey – providing reassurance on how we treat personal data.

Getting senior staff support

It was necessary to set out the benefits and risks to the FCA senior leadership team. It was decided to apply the code to the Financial Lives Survey and once implemented we would look for opportunities to apply the code to other surveys and data collection exercises. The FCA produces a number of statistics both from consumer research and firm regulatory returns (such as complaints data).

Drafting and finalising our statement of voluntary compliance

We sought the support of OSR when writing our statement of compliance. This involved initial telephone conversations and a presentation by OSR to the FCA research team and Data Protection experts.

With help from OSR we finalised our statement of compliance which was signed off and ready for publication.

Applying the Code in practice

The voluntary statement of compliance was published in advance of fieldwork for wave 2 of the Financial Lives Survey. The statement of compliance sits on the FCA website alongside our survey privacy statement and message for consumers. Together we feel that these provide reassurances to respondents and users of data.

Why does the Department for Work and Pensions (DWP) voluntarily apply the Code?

DWP doesn’t just publish official statistics. We also produce a wide range of “other” data and analysis that are important in designing and evaluating policy. Core activities in DWP involve using data to: make decisions, track policies, forecast spending, and much more.

Our data are collected, reused, and analysed by many other groups and organisations outside government – for example, our interactive tool Stat-Xplore enables users to delve into the data and produce bespoke analysis.

The mix of data we use, produce and publish provides a rich range of data that contribute to the evidence base available to individuals and decision makers – information that can ultimately affect each person in Great Britain. So, we want to ensure that any use of statistics in our communications, from a tweet to writing a press notice or creating a presentation, should align with the Code, to ensure messages are clear, measured and appropriate for the story.

What does voluntary application of the Code mean?

It means that we have publicly committed to demonstrating trustworthiness, reassuring users about the quality of our information, and ensuring that the information serves the intended purposes.

Confidence and trust in our data and statistics are fundamental in how the DWP Strategy is delivered:

  • From understanding and managing the performance of the business and helping inform the direction of economic and commercial activities;
  • Allowing the formulation of better public policy and the effective measurement of those policies, including anticipation of risks and pressures on the system;
  • Enabling the public to hold to account all organisations that spend public money, and informing democratic debate and public understanding of our society.

Ensuring that our data and statistics reflect user needs, support better decisions and gain public trust is a shared responsibility for everyone working in DWP. It is essential that those of us who are actively engaged in this area understand our roles and responsibilities.

We have an aspiration to further embed voluntary application of the Code, with the aim of having a shared aspiration across all analytical areas in the Department around when and how we demonstrate our commitment to trustworthiness, quality and public value.

How do you do this – and is it easy?

We apply the four voluntary application stages that aim to help produce analytical outputs that are high quality, useful for supporting decisions, and well respected.  The stages are Understand, Review, Consider and Publish. The commit stage (Publish) is publishing the statement about why users can be reassured that the product achieves Trustworthiness, Quality and Value (TQV).

In leading up to that decision, it is helpful to understand the three pillars (TQV), review the approach to producing and publishing statistics/analysis in relation to the pillars, and consider if there are ways of improving practice. By publishing this statement, we are being transparent.

Our key is to think through each of the pillars:

  • How are we trustworthy? Should people trust what we provide?
  • What is the quality of the data? Is what we are doing the best estimate? Is it misleading? Could people make the wrong decision?
  • How do we provide value? Is it helpful? Can users make decisions using the information?

We have had conversations across all analytical areas to raise awareness. We shared guidance and given advice to analysts who have voluntarily adopted TQV. We also obtained senior backing for applying TQV in the department – our chief analyst said:

“I am really keen for us to be as ambitious as possible in applying the ‘Voluntary Compliance …’ approach to as many outputs as appropriate from the Analytical Community.”

Here are some examples of where we have applied the Code beyond official statistics

Benefit Expenditure tables: We have added a statement of voluntary compliance with the Code to our release of the expenditure tables showing historical and forecast benefit spend. Ministers see these tables well in advance of publication as they’re part of Budget and Spending Review negotiations – so we can’t achieve all the requirements for official statistics. Instead, our statement sets out why we think this is high quality analysis, why it is trustworthy and how it adds value to the debate.

Social Research Reports: Voluntary application has started to be used in DWP research reports, as agreed with the Head of Professional for Social Research, for example, the GSR Automatic Enrolment Evaluation Report (see page 116).  The report brings together the latest evidence and new analysis to show what has happened to workplace pension membership and contributions since automatic enrolment began.

Management Information (MI) is used for operational delivery and the running of the business. We have now begun to apply the Code pillars in our preparation and release of the MI, for example ESA Underpayments and Support for Mortgage Interest MI.

What do you think are the benefits of applying TQV?

It gives you an opportunity to highlight your commitment to demonstrating trustworthiness, reassuring users about the quality of your information, and ensuring that the information serves the intended purposes.

It provides a quick win to improve users’ understanding of the data presented. It also demonstrates a common-sense approach to sharing numerical information and prompts you to review your own processes.

“I found the voluntary Code helpful. The headings (TQV) made sense and it was a useful structure, in fact it reassured us as the producers as much as anything. We also enlisted an analyst from another team to quality assure the peer review of our analysis, which again proved valuable”.

[DWP statistician]

Ultimately, it builds confidence and trustworthiness.

The Scottish Fiscal Commission is responsible for producing independent forecasts of tax revenues, expenditure on social security and on-shore Gross Domestic Product (GDP) for Scotland. It is not considered an official statistics producer; however, it has made an active choice to apply the Code wherever possible.

It reviewed its approaches to producing and publishing analysis against the three pillars of Trustworthiness, Quality and Value (TQV) and considered how it was already meeting the principles within the Code and what could be improved upon.

The majority of Code principles were already being followed within the Commission, in line with its responsibilities as an independent fiscal institution. However, setting out its commitment to voluntarily applying the Code in a clear statement on its website, using the TQV framework, allowed it to demonstrate its independence. Furthermore, by being transparent about its methods and quality assurance, as well as the limitations and the judgements made within its forecasts, it has encouraged trust in the value of its outputs. Building this trust is essential to meet its users’ need for independent scrutiny of the Government’s budget and to inform debate. You can find out more about the Commission’s approach in this blog.

The Commission included a commitment to continue to develop its approaches in line with the best practice outlined in the Code. It has already taken additional steps through starting to pre-announce forthcoming publications up to a year in advance, ensuring all data are released in a reusable format, and by publishing its approach to corrections and revisions.

Please help us improve our website by answering a few quick questions.