Research and analysis

Drivers of technological needs project: Delphi

Published 29 August 2024

바카라 사이트 Office for Science (GO-Science)

What was the challenge? 

The GO-Science Foresight project, Drivers of Technological Needs, was part of GO-Science work to build a cross-government picture of the technologies that are most critical to the UK. The aim was to create a consistent methodology for mapping technologies to four long-term strategic goals covering 바카라 사이트˜sustainable environment바카라 사이트™, 바카라 사이트˜health and life sciences바카라 사이트™, 바카라 사이트˜national security and defence바카라 사이트™ and 바카라 사이트˜a digital- and data-driven economy바카라 사이트™.

To manage the challenge of directly mapping individual technologies to 바카라 사이트˜big picture바카라 사이트™ strategic goals and improve the transparency of results, an intermediate concept of 바카라 사이트˜technical capabilities바카라 사이트™ was introduced (for example, 바카라 사이트˜the capability to deliver effective health services in homes and communities, regardless of income바카라 사이트™). Technical capabilities are the practical steps which are needed to support the delivery of each overarching goal and are themselves underpinned by key technologies (alongside other enablers, such as skills and institutions).  

What was the approach? 

Using a modified Delphi process, the GO-Science project team worked with a diverse group of 60 leading experts from the UK government, business, and academia to identify technical capabilities and assess their importance to achieving the four strategic goals. A high-level overview of the project바카라 사이트™s methodology is provided further down.

Quality assurance was inbuilt throughout the process. Experts were invited to review the curation of capabilities during phase 1. The results of the phase바카라 사이트¯2 scoring process were returned to experts during the workshops in phase 3. Finally, phase 4 aimed to check the judgements that had been derived from the means and standard deviations of their scores and clarify any discrepancies.

A key principle for the Delphi process was that responses gathered from individual experts were anonymous; participants could know who else was involved, but not what they said. Anonymity ensured that opinions were heard independently without bias and helped to avoid groupthink.

Overview of the Drivers of Technology Needs Project Process

Phase 0 바카라 사이트“ Pre-survey work 

• Desk research, exploration of strategic goals from policy documents and development of survey approach. 

• Expert panel identification and recruitment 바카라 사이트“ one panel of approximately 15바카라 사이트¯experts recruited from 바카라 사이트, business and academia, per strategic goal. 

Phase 1 바카라 사이트“ Survey to identify technical capabilities 

• Experts from each panel submitted lists of technical capabilities they considered necessary to achieve a strategic goal. 

• GO-Science project team curation of technical capabilities (merging/re-wording similar capabilities) into lists of between 10바카라 사이트“20 capabilities for each strategic goal including identification of cross-cutting capabilities (relevant for more than one goal). 

• Expert reviews of curated capabilities and finalisation of the technical- capability list for each goal (including common wording for cross-cutting capabilities).   

Phase 2 바카라 사이트“ Survey to score technical capabilities and prioritise 

• Expert scoring of technical capabilities according to their importance to achieving strategic goals. Capabilities were scored on a scale ranging from 5 (essential) to 1 (not directly relevant). 

• GO-Science project team process scoring- survey submissions, calculating the mean score and standard deviation for each capability. 

Phase 3 바카라 사이트“ Workshop to explore scoring and rescore technical capabilities 

• Four expert workshops (one per goal). 

• Expert discussion and rescoring of the ten technical capabilities with the greatest variation in initial scoring, for final prioritisation. 

Phase 4 바카라 사이트“ Outputs 

• Scores finalised, averages and standard deviations calculated for technical capabilities, to produce a final list of 68 technical capabilities. 

• To visualise results from the Delphi exercise, boxplots were produced to show capability rank and the spread of scores. Pre- and post-box-plot workshop figures provided visualisation of how the spread in scoring changed following consensus workshop discussions.

This approach enabled the project team to collect both quantitative assessments for the importance of technologies and capabilities as well as qualitative rationale and context for these assessments. Using this methodology ensured  the project was able to recognise the value of informed, expert judgements, but also that people can change their mind once they hear different perspectives on a complex issue.

What was the impact? 

This project was a significant input into the UK government바카라 사이트™s first attempt to compare its strategic interests in 50 technologies. The analysis was referenced in the Science and Technology Framework as underpinning the choice of critical technologies. Multiple teams have since used the work to consider their own interests in technologies.  

Written in 2023.