TRANSFORM Monitoring and Evaluation Guide – Final Version
Creators: name: Strand Roger
name: Slaattelid Rasmus
name: Völker Thomas
name: Mazzonetto Marzia
Publisher: name: Zenodo
Year: 2023

description: TRANSFORM is a coordination and support action funded under the SwafS-14 call topic of Horizon 2020 to help implement the principles and practices of RRI – Responsible Research and Innovation – into institutions, policies and practices of innovation at the regional scale. To this effect, the project involves itself in processes of innovation, policy-making and practice in three European regions: Lombardy, Catalonia and the Brussels-Capital region, our so-called “clusters”. The project initiatives include efforts to introduce RRI approaches into S3 policies in the three regions. The TRANSFORM project has developed and refined its own strategy for monitoring and evaluating the progress of its activities and approaches, in close collaboration with other SwafS-14 funded actions and the methodologically oriented RIA on the topic, SuperMoRRI. In the first version of the monitoring and evaluation guide, D7.1, we reported the results from project task 7.1 “Mapping of TRANSFORM’s ecosystem of monitoring and evaluation practices” and the first results from project task 7.2 "Design of TRANSFORM’s own suite of monitoring and evaluation activities”. D7.1 presented and discussed to the many fundamental methodological challenges of monitoring and evaluating RRI initiatives, not the least within the time frame of short (3-year) projects. From the perspective of research evaluation studies, the focus on documentation of impacts and even benefits within the lifetime of a project, as was found in the underlying calls and work programmes of H2020/SwafS, was not always well founded. Still, the various SwafS-14 actions, together with the broader SwafS action ecosystem, have made great efforts to develop reasonable and sensible approaches to the challenge of monitoring and evaluation. Especially in 2021 and 2022, the novel approach of "evaluative inquiry" has gained traction within the SwafS/RRI community. This deliverable, D7.5, is a scientific publication that represents and presents the state-of-the-art outcome of this methodological development. The publication is our manuscript for an original research article, "Translating tools and indicators in territorial RRI", recently (September 2022) submitted to the journal Frontiers in Research Metrics and Analytics, which is preparing a special collection monitoring and evaluating RRI. The results were presented in a preliminary form at the annual -SPRI conference of the EU-SPRI Forum (the European Forum for Studies of Policies for Research and Innovation) in Utrecht, June 2022. The article presents our adaptation of the method of evaluative inquiry for the use in monitoring and evaluating territorial RRI. It has the following abstract: <em>By a series of calls within the Horizon 2020 framework programme, the EU funded projects intended to deploy Responsible Research and Innovation (RRI) at a territorial level, in regional research and innovation ecosystems. This paper presents efforts to document and evaluate the achievements in TRANSFORM, one of these projects. Noting the need for a general principle to be interpreted, adapted and translated in order to be rendered meaningful at a local level, we studied precisely these multiple territorial translations of RRI, the organizational and institutional orderings with which they co-emerge and in the challenges that come with these translations. An important shared feature is that RRI work does not start from zero, but rather builds on pre-existing relationships and repertoires of collaboration. The RRI project is hence a way to continue ongoing work and follow pre-set purposes, aims and objectives, as a form of “maintenance work”. In this very human sense, RRI is deployed with a logic of care in the regional context, while the Horizon 2020 calls and proposals above all are formulated in a logic of choice, to be assessed by indicators. We warn against undue standardization of RRI by toolification and use of quantitative indicators, and recommend that RRI performance is monitored by methods of evaluative inquiry. </em> The choice in the Grant Agreement of making TRANSFORM Deliverable D7.5 a scientific publication that presents original research, has the advantage of increasing the dissemination level and potentially the impact of the results. The potential disadvantage is that scientific publications have a genre that lends itself less to offering immediate practical guidance. For this reason, we have decided to use the results to prepare a voluntary update D7.1 into a Second Version, keeping the guidance document format and revising the content and incorporating the methodological developments of 2021-2022. The new version will be produced during project month M36.