BCN SCIENCE CORNER

L’interès pel compromís públic amb la recerca és cada vegada més gran. El Higher Education Funding Council for England ha definit aquest concepte com a «Specialists in higher education listening to, developing their understanding of, and interacting with non-specialists. The ‘public’ includes individuals and groups who do not currently have a formal relationship with an HEI [higher education institution] through teaching, research or knowledge transfer» (HEFCE, 2016). Cada cop més països consideren que el compromís públic amb la recerca és un mitjà per garantir que la investigació científica contribueixi al bé comú i tingui un impacte positiu per a la societat. La Comissió Europea identifica el compromís públic amb la recerca com un dels sis eixos clau per a una recerca i innovació responsable, i s’està plantejant com millorar l’avaluació de l’impacte de la recerca que finança. Us presentem aquest article que proposa un marc conceptual i metodològic que serveixi d’estàndard comú d’avaluació per a projectes de compromís públic amb la recerca aplicable a un ampli ventall de possibles contextos i propòsits.

A common standard for the evaluation of public engagement with research
 

A continuació reproduïm la introducció de l’article «A common standard for the evaluation of public engagement with research» publicat a la revista Research for All, 2 (1) 2018.

Autors: Mark S. Reed (Newcastle University, UK), Sophie Duncan (National Co-ordinating Centre for Public Engagement, UK), Paul Manners (National Co-ordinating Centre for Public Engagement, UK), Diana Pound (Dialogue Matters, UK), Lucy Armitage (Dialogue Matters, UK), Lynn Frewer (Newcastle University, UK), Charlotte Thorley (Queen Mary University of London, UK), Bryony Frost (Queen Mary University of London, UK)

Introduction

Interest in public engagement with research has never been higher. The Higher Education Funding Council for England (HEFCE) defined public engagement with research as:
 

Specialists in higher education listening to, developing their understanding of, and interacting with non-specialists. The ‘public’ includes individuals and groups who do not currently have a formal relationship with an HEI [higher education institution] through teaching, research or knowledge transfer.
(HEFCE, 2006: 1)

It has been proposed that public engagement is a means of ‘ensuring that science contributes to the common good’ (Wilsdon and Willis, 2004: 1) and restoring public trust in science (Wynne, 2006). A major driver for this in the UK is the Research Excellence Framework (REF), which evaluates the social and economic benefits of excellent research in the UK. Similar systems are being considered in other countries that have significant public investment in research (for example, Australia and Germany are currently considering introducing impact into their national research evaluation exercises, Excellence for Research in Australia and Forschungsrating; Reed, 2016). The European Commission (2015) identifies public engagement as one of the six ‘keys’ for responsible research and innovation, and is considering ways of better evaluating the impact of its research in the successor to Horizon 2020.
 

However, there are many challenges to the evaluation of public engagement. As many public engagement activities are unplanned, there is often limited budget, staffing or evaluation expertise available. Even when the resources are available to evaluate public engagement, it may be difficult to motivate researchers to evaluate their engagement practice (Rowe and Frewer, 2005; Burchell, 2015). Often, this is due to resource constraints and a lack of structured techniques for identifying relevant publics and other end users (Emery et al., 2015). Pathways from public engagement to impact can be complex, non-linear and indirect (ESRC, 2009; ESRC, 2011; Molas-Gallart et al., 2000). In addition, issues of time lags and attribution plague the evaluation of impacts A common standard for the evaluation of public engagement with research 145 Research for All 2 (1) 2018 arising from public engagement, given the complex range of factors that may delay or influence impacts (Morris et al., 2011; Fazey et al., 2013; Fazey et al., 2014).
 

We define the evaluation of public engagement with research as a process that collects, analyses and reports data (via quantitative or qualitative means) on the effectiveness of public engagement programmes and activities in terms of their design (in relation to their context and purpose), delivery and immediate outputs, and the beneficial impacts that arise for participants and wider society, and subsequently improves the effectiveness of future engagement and/or enables timely, reliable and credible judgements to be made about the effectiveness of engagement (after Stufflebeam, 1968; Stufflebeam, 2001; Patton, 1987). There is a normative assumption within this definition that public engagement should produce benefits for the economy or society, and that evaluation should therefore assess the subjective worth or value of engagement to different publics and stakeholders (Hart et al., 2009).

There are a large number of toolkits and resources available to guide the evaluation of engagement projects (NCCPE, 2017a). Useful work has also been done to develop indicators to allow institutions to evaluate and audit their engagement at a macrolevel (Hart et al., 2009; Neresini and Bucchi, 2011; Vargiu, 2014; European Commission, 2015). Despite this, there are claims that evaluation of public engagement tends to be done rather poorly (not just in higher education, but in most sectors) (Bultitude, 2014), and that evaluation findings are rarely shared widely or lead to demonstrable changes in engagement practice (Davies and Heath, 2013). As a result, there are now calls for the establishment of a common ‘evaluation standard’ to provide tools and guidance for evaluating public engagement in order to promote good practice and enable comparison between projects (Smithies, 2011; Neresini and Bucchi, 2011; Bultitude, 2014). This paper is a first step towards developing such a standard, which can subsequently be applied to compare the efficacy of different methodological approaches in different engagement contexts, and to monitor changes in the effectiveness of public engagement across time and space.

The aim of the paper is to propose a linked conceptual and methodological framework that can be used as a common evaluation standard for public engagement projects across a wide range of possible contexts and purposes. The conceptual framework summarizes the three main ways in which evaluation can provide judgements about, and enhance the effectiveness of, public engagement. A methodological framework is then proposed to operationalize the conceptual framework. The development of the standard is informed by literature review, an online survey and semistructured interviews in Queen Mary University of London (QMUL), who commissioned the development of a ‘public engagement evaluation toolkit’ to inform their work, which could be used across the higher education sector. The standard is then tested and refined in situ in a large public engagement event hosted by QMUL and post hoc to a range of public engagement impact case studies from the 2014 Research Excellence Framework.

Podeu llegir l’article sencer aquí.