# Evaluation Methodology for Circular and Resilient Information Systems
## 3 Results
### CRIS System evaluation of functionality performance

Table 2: Modules of CRIS evaluation metrics. the overall system and its evaluation from the end-user perspective, particularly the CRIS evaluation of its learnability, usability, and user experience. Relevant to the CRIS learnability, it refers to the ease with which new users can begin to effectively interact with the system and achieve a reasonable level of proficiency. To enable the learnability evaluation, the \"Software Evaluation: Criteria-based Assessment\" of the Software Sustainability Institute [43] offers a dedicated section with questions relevant to the existence of a getting started guide, the provision of instructions for basic (or all) use cases, the provision of reference guides, and API documentations, all of which are relevant for the CRIS evaluation. Furthermore, as the newly developed system will be fed with live data from the shopfloor, interoperability and integration issues might arise that need to be accounted for and evaluated. Another section of the criteria-based assessment can be utilised in this direction as it offers a dedicated section on sustainability and maintainability that includes evaluation instruments and processes on the analysability, changeability, evolvability, and interoperability categories [43]. Firstly, from the end-user perspective, its overall usability was found to be important for evaluation, as well as the user experience the users will have in the process of its application. For the CRIS usability, the evaluation methodology will employ the widely utilised System Usability Scale (SUS). The SUS was developed by [PERSON] [44] as a \"quick and dirty\" survey scale that quickly and easily assesses the usability of a given product. Its ease of use and reliable results have set the SUS as an industry standard in terms of usability evaluation, as it has been reported as the prime instrument for usability evaluation in over 2300 individual surveys in over 200 studies [45]. Lastly, for the user experience evaluation part of the CRIS system, the widely utilised and validated instrument of the User Experience Questionnaire (UEQ) by [PERSON] et al. [33] will be used to record the overall experience of the factory employees using the developed system, based on its attractiveness, perspicuity, efficiency, dependability, stimulation, and overall novelty. The aspects above of the CRIS evaluation relate to the end-users' (factory employees') point of view. They will be administered during the pilot rounds of evaluations and will serve as the data collection instruments for the qualitative part of the evaluation of the CRIS system. Finally, focusing only on real-time key performance indicators is not sufficient. To this end, a periodical assessment of the systems and performance under different conditions is necessary to evaluate the long-term impact of circular and resilient information systems (CRISs). In other words, evaluations at regular intervals can allow businesses to track an CRIS's ongoing and sustained system performance that may emerge over time, whether positive or negative, after the introduction of the CRIS in the day-to-day activities on the shop floor.