No news is good news? Talking to the public about the reliability of assessment: Summary
Published 16 May 2013
Applies to England, Northern Ireland and Wales
© Crown copyright 2013
This publication is licensed under the terms of the Open 바카라 사이트 Licence v3.0 except where otherwise stated. To view this licence, visit or write to the Information Policy Team, The National Archives, Kew, London TW9 4DU, or email: psi@nationalarchives.gov.uk.
Where we have identified any third party copyright information you will need to obtain permission from the copyright holders concerned.
This publication is available at /government/publications/reliability-of-assessment-compendium/no-news-is-good-news-talking-to-the-public-about-the-reliability-of-assessment-summary
1. Investigating public perceptions of assessment reliability
Ipsos MORI undertook two workshops, each with 36 members of the public, including teachers, students and parents. For these workshops the facilitators provided stimulus material about issues relating to the reliability of assessments, on which they attempted to gauge the group members바카라 사이트™ views. A major finding from this work was the workshop participants바카라 사이트™ apparent distinction between inevitable sources of variation in assessment marks and grades, and what they saw as preventable errors in marks and grades. Overall, participants seemed able to cope with the idea that 바카라 사이트˜test-related errors바카라 사이트™ arise and are to some extent inevitable; they were less tolerant of assessment errors arising from blatant human mistakes. The use of the word 바카라 사이트˜error바카라 사이트™ itself was also seen as problematic, especially when used to cover all aspects of unreliability - because it tends to produce negative reactions and assumptions of culpability on the part of assessors and awarding organisations.
2. Communicating with the public about assessment reliability
Blue Rubicon바카라 사이트™s research (conducted exclusively with Ofqual staff) attempted to address the issue of how Ofqual could communicate most effectively about assessment reliability issues with the general public. Blue Rubicon is a 바카라 사이트˜communications messaging consultancy바카라 사이트™, and so is skilled at helping organisations frame communications strategies. In Ofqual바카라 사이트™s case the question of helping the public understand the concept of reliability was seen as an important, yet problematic, challenge. A major recommendation from this strand of work was that the word 바카라 사이트˜variation바카라 사이트™ was a more constructive way of communicating about the uncertainties surrounding assessment results than either 바카라 사이트˜reliability바카라 사이트™ or 바카라 사이트˜error바카라 사이트™. This constituted one of a number of recommendations on how Ofqual staff and representatives might communicate about this issue with members of the public.
3. The implications of the research
Rather than give an exhaustive account of these two specialist strands of work within the Ofqual reliability programme, the conference paper reflects both on the implications of this work when viewed alongside the outcomes from other aspects of the programme, and also on the public바카라 사이트™s current perception of assessment results in the UK. The media are responsible for a large part of the communication in this area, and in recent years they have thrived on so-called crises and disasters, whether these are to do with the annual production of assessment results at both GCSE and A level or with national curriculum assessments. Clearly, the media바카라 사이트™s concerns are different from those of awarding organisations, naturally tending to raise doubts about assessment results, whereas awarding organisations will generally seek to increase the public바카라 사이트™s confidence in the assessment results 바카라 사이트“ which the agencies are responsible for producing.
The paper outlines the considerable technical knowledge about the limitations of assessment scores and grades as precise measures of achievement, and the very real difficulties faced by those who seek to communicate such insights to the public. It reiterates the view that awarding organisations should take central responsibility for estimating the degree of variability in assessment results. The paper concludes, positively, by asserting that a wide variety of media and analogies could be used to get the message of the complexity of the technical data across, and by holding out the hope that following such a strategy could make the UK public 바카라 사이트˜more sophisticated바카라 사이트™ users of assessment results.
Had the presentation returned to its provocative 바카라 사이트˜No news is good news?바카라 사이트™ title, it might have concluded that there are ways of conveying news about the limitations of assessment results that are not necessarily 바카라 사이트˜bad news바카라 사이트™. The presentation does, however, suggest that Ofqual is determined to 바카라 사이트˜out바카라 사이트™ this issue, which has, for many, been a 바카라 사이트˜skeleton in the cupboard바카라 사이트™ for far too long.