News story

Part five: Five things you didn't know about...

Our final selection of posts from across the Analysis Function, each sharing five facts about a topic or project

• From: 바카라 사이트 Analysis Function
• Published: 25 May 2021

As part of Analysis in 바카라 사이트 month, this is the final edition in our series of 바카라 사이트楩ive things you didn바카라 사이트檛 know about바카라 사이트�바카라 사이트� blog posts. Analysis in government is being used more than ever in policy, decision making and the media, this series of blog posts will share, showcase and celebrate the vast variety of professions and work within the Analysis Function.

Jump to one of our posts:

Five things you didn바카라 사이트檛 know about user engagement

By Tegwen Green (left) and Nancy Singh (right), Office for National Statistics

For the last fifteen months Nancy and I have been working with users and producers of statistics to develop a .

The strategy sets out a plan of action for building a more meaningful and sustained dialogue between producers, users and potential users of statistics. Its principles are widely relevant to anyone conducting analysis across different professions.

We want to spread the word about the value of user engagement, in the midst of a pandemic, more than ever. We바카라 사이트檝e highlighted five important things we want you to remember about user engagement. You might know some of them already 바카라 사이트� if so, then perhaps share them with your colleagues as well!

1. You바카라 사이트檙e doing a lot of user engagement already

We just want to remind you all that every time you ask an opinion or have an informal chat about your work, you are engaging with a user or potential user of your analysis.

Let바카라 사이트檚 get better at recognising and acknowledging the good behaviours and actions that we already display. Let바카라 사이트檚 build on those to create even more opportunities to develop regular, ongoing, two-way dialogue with a wider range of people.

Engagement is rewarding and it can be anything from an email exchange to a full-blown consultation exercise. If you have an engagement success story to share email goodpracticeteam@statistics.gov.uk so we can help showcase your experience and inspire others to follow your lead.

2. One size doesn바카라 사이트檛 fit all

This may seem obvious, but we really do need to tailor our engagement activities to suit the intended audience, much as we would do with any other sort of communication. We have such a huge range of users and potential users of our products and services, and we probably don바카라 사이트檛 even know who they all are.

This is where techniques such as audience segmentation and the use of can help us categorise and gain insight into what users of our statistics want and need from us. Through this understanding we can tailor our communications and our statistics to ensure that as many users as possible are aware of our statistics and engage with them and us.

3. No one does it perfectly

User engagement tends to be most effective when it has multiple strands and when your engagement happens as part of your organisation바카라 사이트檚 wider engagement activities.

We don바카라 사이트檛 need a few people doing user engagement perfectly, we just need lots of people trying their best to do it well and working together to make it happen as part of business as usual.

Let바카라 사이트檚 learn from each other바카라 사이트檚바카라 사이트� experiences and share all the 바카라 사이트榞ood stuff바카라 사이트�. Why not take a look at these and tell us about the successes you바카라 사이트檝e had engaging with users, so we can showcase those, to inspire and help others.

4. Help is at hand

You바카라 사이트檙e not alone 바카라 사이트� as ever, your colleagues across government are here to support you in your endeavours. For example, you can:

• link up with the cross-government 바카라 사이트� a vibrant and enthusiastic network of user engagement enthusiasts
• identify your engagement challenges for the new User Support and Engagement Resource (USER) hub to help address, or ask for practical advice from the Good Practice Team
• post a query on the government-wide
• reach out to your organisation바카라 사이트檚 stakeholder engagement or communications teams to identify new contacts and explore new channels of engagement

5. We have a vision for the future of user engagement

Nancy and I launched the new four-year on 22 February 2021. The strategy has:

• a radical vision - for user engagement to be second nature and built into our organisations바카라 사이트� wider activities
• three ambitious goals, the 바카라 사이트�3Cs바카라 사이트� 바카라 사이트� centred around facilitating collaboration, building capability and encouraging a culture change

Five things you didn바카라 사이트檛 know about cognitive interviewing

By Meg Pryor, Office for National Statistics

is a method which is used to see how individuals process and respond to survey questions, allowing us to investigate what the respondent is thinking about when they바카라 사이트檙e answering our surveys. I have been doing cognitive interviewing for three years, so this blog is based around the things I have learned while doing so, which I didn바카라 사이트檛 know before! As a note, the cognitive interviewing I바카라 사이트檓 going to discuss here is not the same as cognitive interviewing that happens in police settings.

1. This is usual practice within question and questionnaire design

Before starting at the Office for National Statistics (ONS) I didn바카라 사이트檛 know that cognitive interviewing was a process within questionnaire and survey design - in fact I had not heard of it at all from my time in academia! Since joining, I now know that it is an integral part of building and designing a survey and allows us as researchers to see whether or not a question is clear for a respondent. It allows us to identify if there are any risks to the data quality because respondents may interpret the question to mean something different than what the question designer intended. This could come in a variety of forms:

• Respondents may answer the question incorrectly because they do not understand a term.
• Respondents may be misremembering an event or the frequency of events.
• Respondents may be deciding to estimate rather than calculate answers.
• Respondents may say something they feel is untrue because it makes them 바카라 사이트榣ook better바카라 사이트�, especially in interviewer-led modes.
• Respondents may get frustrated and stop completing the survey entirely (if it is voluntary).

Conducting cognitive interviews allows us to better understand the data we바카라 사이트檙e collecting and helps to identify these risks early. It therefore allows us to think about what we can do to mitigate these risks and make the answering process easier for respondents. This could be redesigning the question if possible or incorporating guidance.

2. It can be done remotely

We바카라 사이트檙e a year on from the start of the coronavirus (COVID-19) pandemic, and it has meant we have had to change the ways in which we work. This has included research. While previously I had conducted cognitive interviews face-to-face, this was no longer possible. But that does not mean that cognitive interviewing can바카라 사이트檛 go ahead, it just has to go ahead a little differently.

I recently co-authored a blog post on the GSS website talking about , and how we experienced it. For cognitive interviewing specifically, key points would be to have sessions last no more than 60 minutes as opposed to 60-90 minutes as remotely it바카라 사이트檚 harder to keep your participants concentration. Also, to carefully think about what software and technology not only you have access to, but also what the participant has access to. Lastly, as with face-to-face, think about the ethics! Use the to help with this.

All in all, don바카라 사이트檛 be put off cognitive interviewing because of the pandemic. Doing it remotely has many benefits! There is also to help you get started.

3. It can be emotive

As I alluded to, ethics are very important to consider, and especially so when cognitive interviewing. I have had interviews where the person I바카라 사이트檓 speaking to has started to cry and has gotten upset and therefore I stopped the session and provided aftercare. It바카라 사이트檚 important to remember that survey questions still have the potential to be emotive if they are sensitive, even if at first they don바카라 사이트檛 appear so.

Cognitive interviewing can show the sensitivities of questions which we may not have been aware. For example if you바카라 사이트檙e asking a question on household spending, the respondent may have just lost their job so always be mindful. With this in mind, the UK Statistics Authority has a great which you can use to identify ethical risks before the research sessions.

However, to quote the 바카라 사이트 Digital Service바카라 사이트檚 principle 바카라 사이트榶ou are not your user바카라 사이트� so therefore you might not realise the emotional impact a question might have until you바카라 사이트檙e in the research session. Therefore, I also recommend putting together a plan in place for if a participant becomes distressed in the session.

4. Get the participant to 바카라 사이트榯hink aloud바카라 사이트�

How can we expect to find out the cognitive processes a participant is going through without a machine hooked up to their brains? Getting them to go through the process of thinking aloud.

This can be a weird concept for participants to understand. We essentially want them to tell us what they바카라 사이트檙e thinking when they바카라 사이트檙e presented with the question, and the processes they go through to figure out their answer. That is easier said than done, and is something that sometimes doesn바카라 사이트檛 come easy to participants.

One way to help them understand 바카라 사이트榯hinking aloud바카라 사이트� is to give them an example of what you mean. An example I often use is:

바카라 사이트業f you asked me how many windows were in the room I바카라 사이트檓 sitting in, I could say two and that would be my answer. But if I were to think aloud, I would say I have a window next to my TV, and then a bay window next to me. I don바카라 사이트檛 know if a bay window counts as multiple windows, because it has multiple sections, but to me it바카라 사이트檚 just one window, therefore I have two windows.바카라 사이트�

After providing that example, I then ask the participant to try giving it a go themselves, so they have a better idea, and I can explain it further if they want further information.

It바카라 사이트檚 often through this method that you get golden quotes that really show you what바카라 사이트檚 going through their minds when answering your questions.

5. Silence is your friend!

Now while we want the participants to be as loud as possible, it is incredibly important for us as researchers to utilise silence, as it is your best friend while cognitive interviewing. As the saying goes, less is more! It allows the participants to keep talking and to keep providing you with rich, quality data into why they have answered the question in that way.

However, something which I have learnt through conducting remote testing especially is that while you want to give the participant room to talk, if you바카라 사이트檙e quiet for too long they may think your internet connection has gone! So instead, every now and then make a 바카라 사이트榟mm바카라 사이트� noise so that they know the technology isn바카라 사이트檛 playing up.

In conclusion, I hope this has shown you some aspects to cognitive interviewing that you may not have known previously. If you바카라 사이트檙e starting work on a survey, or wanting to design questions, I cannot recommend enough conducting these sessions for yourselves and you can find .

Five things you didn바카라 사이트檛 know about data science

By Hillary Juma, Jonathon Mellor, Lewis Edwards, Ali Cass from Data Science Campus, Office for National Statistics and Emma Walker from Centre for Applied Data Ethics, UK Statistics Authority (UKSA)

Data Science is informing policy, digital products, and operational decisions across the Public Sector. Data Science is the practice of bringing together mathematical knowledge, domain knowledge (such as environmental policy) and computer science to provide analytical or operational insight. .

In this blog we dispel myths around data science.

1. Demystifying data science

Data Scientists produce algorithms; a set of rules for solving a problem in a finite number of steps. Data Scientists work closely, with data engineers, data architects and product managers, to deliver business relevant insights. An example data science project is the use of machine learning to from energy performance certificates data.

Data Science has similarities and differences to Artificial Intelligence (AI), for example the similarities include the use of algorithms and machine learning. Machine learning is the process of using of algorithms that generate predictive or explanatory models based on patterns or structures in data. The difference is that AI is the theory and development of computer systems able to perform tasks normally requiring human intelligence. For example, Siri or Alexa performing task in response to a vocal request.

2. Learning Data Science

Thankfully, a degree in data science is not a requirement to be a data scientist, nor should it be! Thanks to the open-source community spirit and training on offer from the civil service, the routes into and for advancing in the field are immense.

There is no one right way to learn data science: try out a new technique at work, a training course here, a personal project there, a blog read with a cup of tea or a podcast while you go for a walk. All are valuable to expand your knowledge of programming, statistics, and AI. You can find an abundance of open access books, tutorials, and welcoming communities online and in person (for example the ). Within government there are even more communities and meetups, along with training material available created for those working in government (such as the Analysis Function curriculum) and mentoring opportunities such as the Data Science Accelerator.

To misquote a famous phrase: 바카라 사이트淭he journey of a thousand data science techniques begins with a single article,바카라 사이트� starting is the hardest part.

3. Data Ethics is more than just assessing for 바카라 사이트淏ias in, Bias Out바카라 사이트�

When it comes to using data science in our work, the possibilities and potential applications can seem endless! However, it is vital that we carefully consider not only what we can do with these techniques, but also what we should do.

Data ethics is a growing field and there is an increasing amount of related to ethical considerations in machine learning and AI. These include aspects related to transparency in techniques, approaches, and datasets, considerations of consent and privacy in relation to the use of data, accountability, and human oversight, and understanding the limitations of methods used, including the potential for biases in datasets and approaches that may lead to groups being underrepresented or discriminated against in some way.

In sum, thinking about ethics in data science is crucial and this is reflected in the recent launch of the UK Statistics Authority바카라 사이트檚 , which aims to further help researchers and statisticians address ethical considerations in their work.

4. It is more than code

• Art of data science is exploring the problem and then conveying insights
• Coding is the part of the journey, but the destination is understanding
• Know your audience

Whilst the day-to-day work of a data scientist may seem like a scene from the Matrix with lines of code across multiple screens, this is only part of the journey.

The most valuable contributions that data scientists can make to a team, business or department are sharing the deep insights from behind the data바카라 사이트檚 silicon curtain. Developing this value will involve the application of various techniques and skills, such as the machine learning approaches mentioned already.

But the time to shine is when you present and talk through what you have found out about the data; whether confirming untested assumptions that have been held true about the data by regular users (a very common experience) or revealing unexpected characteristics about how certain variables relate to others.

Often summarised in the form of dazzling visualisations or interactive dashboards, the opportunity to present these to the rest of your community and explore the true value of any data source together is a rewarding experience. It also a chance to inspire curiosity in the data from as-yet unconvinced colleagues (friends and family too, if you are willing to chance it) and other unexplored options.

Whether it is a personal project or part of a large programme of work, the role of a data scientist lies in understanding the task at hand, digging into the data with your favourite data inspection tools and then setting out a string of achievable goals towards what might (with an allowance for pragmatism) become a fully-fledged analytical pipeline replete with cutting-edge techniques.

5. There are no unicorns

Since data science is a combination of several skillsets applied in a multitude of ways, the path to becoming a data scientist can be a very individual journey.

Unlike a more traditional career path where one might become an expert in their field by following a structured route, there are no unicorns in data science 바카라 사이트� no experts in all elements!

Instead, you will find accomplished data scientists with expertise skewing towards a range of areas in the Venn diagram above. This might be more on the software engineering side of things with responsibilities including building data science architectures and workflows, or the domain expertise side applying innovative technology to solve problems related to climate change and the environment.

If you were to ask a data science team how they each got to where they are you will hear all sorts of journeys. And that is great! It speaks to the diversity of the data science skillset and applications and having a team that capitalises on this is a great position to be in when tackling novel problems.

Conclusion

Over the years, there has been a growing understanding of the value of data and the sorts of way it can be realised. Whilst the proportion of senior leaders that needed encouragement to prioritise exploration of their data in innovative ways has decreased, the growth in appetite to make use of more data and explore more complex techniques has been exponential.

If you would like to learn more about Data Science in the Public sector, feel free to check out the following resources:

Open to all:

• : Estimating Vehicle and pedestrian activity from town and city traffic Cameras Dr Li and Dr Ian, Senior Data Scientists, Data Science Campus, ONS Presented at Institute for 바카라 사이트s, Data Bites

Open to UK Public Sector Employees only, access with work address:

• 바카라 사이트� community forum
• 바카라 사이트� community presentation바카라 사이트檚 library
• Data Science Community, Service Manual page
• - hosted by the Data Science Campus, ONS

Five things you didn바카라 사이트檛 know about the work of the Economic and Social Research Council (ESRC)

By Alison Park Interim Executive Chair of ESRC

In this blog post, Interim Executive Chair of the Economic and Social Research Council (ESRC) Alison Park, describes five things you didn바카라 사이트檛 know about the work of ESRC.

ESRC was established over fifty-five years ago to help inform policy and industry. We are now one of the nine councils that make up where we work to achieve UKRI바카라 사이트檚 mission to 바카라 사이트榗onnect discovery to prosperity and public good.바카라 사이트� Here are five things you might not know about ESRC.

1. The breadth of our research and data investments

Work we fund improves our understanding of how we think, feel and behave, of our mental health, education, work and family lives. Our researchers consider how organisations are managed, how states are governed, and how to achieve a fair and sustainable economy. This evidence informs decision making and efficient public service delivery.

Examples include:

• The 바카라 사이트� a new investment which will provide a deep understanding of what individuals, firms, regions and national policy can do to improve productivity.
• 바카라 사이트� a partnership of government and academic groups working with Whitehall departments and devolved administrations to create linked research datasets from administrative sources covering areas from education and health to crime and justice. Read more about ADR UK바카라 사이트檚 impact.
• An array of national studies including , the world바카라 사이트檚 largest longitudinal household panel study, which provides vital evidence about change and stability over time across nearly every element of people바카라 사이트檚 lives.
• Policy facing research on people바카라 사이트檚 behaviour and climate change through and .

2. The relevance of our work to government priorities

Many of you will be aware of the Areas of Research Interest (ARIs). Less well known is that nearly two-thirds of all ARIs can be addressed primarily with insights from the social and behavioural sciences, as the 바카라 사이트바카라 사이트檚 Chief Scientific Adviser Sir Patrick Vallance has .

More specifically, just as government priorities are now very focused on issues such as the pathway to net zero, levelling up and skills, so too are our plans.

As well as further research on climate change adaption and mitigation, we are planning place-based research investments that will provide a better understanding of the different challenges facing different regions and cities, as well as scoping work to build a high-quality evidence base on skills to improve policy and practice across all economic sectors.

We are looking forward to continuing to engage with government in refining our thinking in these and other areas.

3. Our commitment to connecting research and policy

Our aim is to better connect research capability with policy challenges. We바카라 사이트檝e recently been focusing on how we can catalyse deeper and enduring connectivity across the research-policy system. Our vision is to realise the potential of research to inform and shape public policy at all levels.

We are currently exploring a range of activities, including:

• Developing our Evidence Centre Network, including the Cabinet Office-led What Works Network and our and investments
• Building on our to upskill researchers to enable them to work more collaboratively with non-academic users
• Identifying policy relevant data linkages and ensuring our data infrastructure aligns with policy needs.

To flesh out these activities we will be talking to government stakeholders and departments to understand their interests.

4. Our new people exchange and fellowship framework

An immediate priority is to develop a people exchange and fellowship framework which creates opportunities for researchers to spend time in the heart of policy organisations and for those in government to gain experience in research organisations.

Over the next year we바카라 사이트檒l test and develop this framework, as well as with No.10, initially focusing on levelling up, net zero and coronavirus (COVID-19) recovery.

5. The scale of our COVID-19 research

ESRC has just under 200 grants in our , which are generating unparalleled insights into the impacts of the pandemic and will support the ongoing national response and recovery efforts. We engaged with CSAs and devolved administrations to match proposals with policy priorities and are now extending this engagement with other policy stakeholders.

Do keep an eye out for a series of 바카라 사이트榓ctionable insights seminars바카라 사이트� that we are developing in partnership with government analytical networks. These will focus on thematic areas of relevance to key government priority areas. I hope this blog has given you a brief impression of some of ESRC바카라 사이트檚 priorities and a sense of how we encourage close connections between research we fund and policy priorities. To keep up to date with our work please follow us on and visit our .

Five things you didn바카라 사이트檛 know about behavioural science in the Department for Work and Pensions

By Alexandra Urdea, Department for Work and Pensions (DWP)

I am a social researcher and member of DWP바카라 사이트檚 Behavioural Science team. I have a PhD in anthropology and currently work on a cross-departmental project to help more informal carers remain in work.

Like a number of other government departments, DWP has its own behavioural science function tailored specifically to the needs of the department in which it sits. As a team we work across a wide range of policy areas, from disability benefits to labour market interventions, alongside more internal-facing challenges like organisational transformation and people performance policies.

It would be presumptuous for us to speak for other teams in this emerging field, which continues to change as needs and approaches evolve. But for now, and for DWP, here are five things you probably didn바카라 사이트檛 know about behavioural science.

1. Our practices are distinct from Behavioural Insights

Behavioural science is often seen as synonymous with Behavioural Insights (BI). BI involves finding low cost ways of nudging people바카라 사이트檚 behaviours 바카라 사이트� usually through communications 바카라 사이트� with the effectiveness of nudges measured using randomised control trials (RCTs). But behavioural science is an umbrella term that covers a range of different approaches to solving problems involving human behaviour. DWP Behavioural Science was designed in 2015 to complement, rather than duplicate, an analytical function with extensive expertise in trialling interventions. Our team takes a more upstream focus, supporting colleagues to design user-centred, behaviourally informed policies and services from the outset.

2. We help colleagues to understand and diagnose problems

Many of the problems we work on in DWP 바카라 사이트� like tackling long-term unemployment and designing an effective benefits system 바카라 사이트� are highly complex. They involve a range of different 바카라 사이트榓ctors바카라 사이트�, including policy colleagues, work coaches in Job Centres, GPs and employers as well as benefit claimants. Often the Department is asking these actors to perform complicated series of behaviours in order to achieve a policy goal. We work with colleagues to translate their goals into concrete behaviours so that we can explore how realistic they are, and then systematically map the barriers currently preventing people from doing them ( is one of our favourite tools for this). It바카라 사이트檚 only once we바카라 사이트檝e done this that we start co-designing solutions to address those barriers.

3. We바카라 사이트檙e interested in systems and context

All human behaviour happens in the context of social structures and systems. These include more nebulous things like cultural norms 바카라 사이트� for example around who should care for elderly relatives in a family 바카라 사이트� as well as more tangible things like legal employment rights. These structures and systems enable certain actions and choices whilst constraining or preventing others. Understanding the context in which behaviours are taking place is vital if we want to understand why people do the things they do, and what might help them behave differently. It also helps us think through the risk that proposed interventions in one part of a system will have unintended consequences in another part.

4. We바카라 사이트檙e not all psychologists!

Psychologists, such as Daniel Kahneman and Amos Tversky, are the social scientists most often associated with the emerging field of behavioural science. Their ideas and findings confirmed the suspicions of economist Richard Thaler that people often don바카라 사이트檛 act in the way traditional economic theory would suggest. This is why, Thaler thinks, different social sciences that can help understand human behaviour should have a stronger voice in policymaking. We have very much taken this message to heart. In our team we have psychologists, but also anthropologists, sociologists, philosophers, operational researchers and policy professionals (to name but a few)! This trans-disciplinary approach means we can generate a far richer and multi-faceted understanding of human behaviour 바카라 사이트� and therefore more innovative and effective solutions 바카라 사이트� than any single discipline could achieve.

5. We are methodological magpies

Some behavioural science teams specialise in quantitative insights and impact evaluation. Others can draw on well-established literatures about the drivers of particular behaviours. Given the nature of what we do in DWP we find qualitative methods especially helpful for unpacking the context within which behaviour occurs, and for developing and testing behavioural hypotheses. We often draw on ethnographic, co-productive and other creative methods. We also use tools from the digital and user centred design (UCD) professions to help us think about user needs and solution design. Our toolkit is constantly growing and evolving to help us better tackle the problems we바카라 사이트檙e faced with and we don바카라 사이트檛 see this changing any time soon!

Published 25 May 2021