# Evaluation Methodology for Circular and Resilient Information Systems
## 2 Background Literature
### CRIS Evaluation Methodology Design  
A practical evaluation methodology design for an ICT (Information and Communication Technology) system will ensure its functionality, efficiency, and alignment with the overall organisational goals and justify the reason for its development. A robust evaluation framework can enable the identification of its strengths and weaknesses toward guidingimprovements and potential adaptations to meet the pilots' actual needs. Furthermore, a well-structured evaluation methodology fosters accountability and transparency, providing stakeholders with clear insights into the system's value and potential. Towards developing the evaluation methodology of Plooto (and thus the CRIS system), the process of 5W1H was followed to act as a structured approach to developing an evaluation methodology grounded in theory. The 5W1H methodology is a systematic tool that helps break down complex problems into manageable parts by asking key questions that cover all essential aspects of a situation. This ensures that no critical factor is overlooked during the evaluation process. In particular, and for evaluating the circular and resilient information system of Plooto, the 5W1H methodology will initially be used for information gathering and problem-solving relevant to the approach followed. The 5W1H methodological tool, an enhanced version of the 5W tool, answers six specific questions (Who, What, When, Where, Why, and How) during the initial information-gathering stages. Both 5W and 5W1H have been previously used by researchers relevant to manufacturing sector studies [35; 36; 37]. This approach also serves as a useful tool for cross-case analysis in the case of multiple pilots [38]. Additionally, this process helps us to adapt measures in the improvement phase and thoroughly understand the current situation in detail, following the principle of 5W1H [39]. Our followed methodology will support involved stakeholders and evaluators to thoroughly understand and then evaluate the system's requirements, functionalities, impacts, and areas for improvement by addressing issues such as (i) identifying the people involved or affected (Who), (ii) defining what the issue/s or situation is (What), (iii) specifying the time frame or when the issue occurs or occurred (the critical times when the system is most needed (When), (iv) determining the location or context of the event or problem (the physical and virtual environments in which the system operates) (Where), (v) seeking to understand the cause or reason behind the situation (problems it aims to solve, the processes it intends to improve, etc.) (Why), and (vi) describing the method, solution, or process involved in the situation or in problem-solving (How). The existing body of literature proposes a plethora of IS evaluation methodologies regarding different aspects, stages, and angles. Most of them include quantitative and qualitative methods, considering pre-existing KPIs, examination, and information gathering. In this case, a mixed-method approach with a systematic methodological design for each dimension and stage is contemplated to evaluate the information system efficiently. Thus, Table 1 presents the results of the WS11 methodological approach followed to delineate the theoretical grounding of the CRIS evaluation methodology. Having presented the overarching theoretical evaluation methodology framework, the focus shifts to selecting the most meaningful KPIs for consideration for the different value networks in order to be able to define a list of KPIs per value network that respond to a successful and effective pilot implementation. In that direction, all identified KPIs were examined for their validity, computability, and availability of data and past values, as well as computation methodology in parallel with the pilot partners. In particular, relevant to the KPI validity, each KPI was examined for its relevance to the specific value network and pilot context for gaining additional maturity.