VITALISE D2.2 Evaluation process and tools
Creators: name: David, Servais
name: Vervoort Koen
name: Petsani Despoina
Publisher: name: Zenodo
Year: 2022

description: Work Package 2 (WP2), “Common Standards, Procedures and Practices” is a core WP for the VITALISE project as it handles all the work and implementation for the Harmonization procedure that is being followed. Within this scope, WP2 attempts to define a methodology for managing the Harmonization activities performed in the project but also guide the Harmonization process of living labs in other domains beyond the VITALISE project. This document aims to present the evaluation process and tools of VITALISE from a twofold perspective: On one hand, present the Harmonization process that will be followed throughout VITALISE along with the events performed and tools used for the Harmonization methodology improvement. On the other hand, present the work of Task 2.2 towards developing an evaluation process and tools for supporting living labs in reaching higher maturity levels while providing them with harmonized tools to do so. The VITALISE Harmonization methodology was initially developed and presented in D2.1. This deliverable presents the updated version of the methodology which was tested (M6-M12) in order to understand and identify existing difficulties and improve accordingly. The updated version of the methodology is based on the Plan-Do-Check-Act (PDCA) and Plan-Do-Study-Act (PDSA), but is also underpinned on agile development principles. The idea is that every six months a whole Sprint cycle is performed. During each Sprint there are ongoing activities that serve the objectives of the Standard and aim to improve and deliver specific items of the Standard. The events described in the VITALISE Harmonization methodology are: Co-creation, to explore and identify new procedures or Standard items that have not been mapped before Consensus, to come to an agreement about the harmonization process among the members of the Harmonization Body Approval Process, to Approve and Publish the current version of the Standard Implement, for Living Labs to adopt and implement in their practice the current approved and public Standard Revise &amp; Adjust, to reflect on the feedback and KPIs collected during the implementation Phase for each of the services and procedures. The Harmonization process should be monitored in order to understand if the Standard Versions that are produced are serving their objective of facilitating and improving the access for external researchers to Health and Wellbeing living lab research infrastructures but also if the Standard improve the way that living labs collaborate and work together towards improved outcomes. The Harmonization monitoring is being done in two fronts: 1) monitoring the effectiveness of the process, 2) monitoring the effectiveness of the results. Within Τask 2.2 the VITALISE envisages to create a structured measurement for living labs that will be used so that each living lab can self-evaluate its strengths and weaknesses. In short, the following steps were integrated in the plan of action for the development of the evaluation framework: literature review, ENoLL certification &amp; evaluation framework, collection and evaluation existing self-assessment toolboxes/approaches for living labs, interactive workshop with VITALISE partners, scoping of criteria with group of living lab experts and validation workshop with a specially created pilot group. The concluded evaluation framework is composed by: The strategy of a LL: with the elements of governance, business model and culture of the living lab Operations of the LL: with the elements of operations, human resources and equipment &amp; infrastructure Openness of the LL: with the elements of innovation processes &amp; partnerships and ownership of results Users &amp; reality: with the elements of quality of the iterative living lab processes in real-life settings, user-centricity of the user &amp; stakeholder engagement approach and the quality of the participatory tools &amp; methods Impact &amp; value creation: with the elements of co-created values and impacts of the living lab Stability &amp; harmonization: with the elements of stability of the living lab and harmonization &amp; scale-up Finally, the consortium is striving throughout VITALISE to have an iterative approach for the evaluation of the procedures within the project (Harmonization methodology) but also beyond the project (the evaluation of all living labs). The approach of a continuous evaluation throughout the remaining duration of the project will allow us to have more concrete feedback, taking into account the possible difficulties and risks.