D2.4.1: Monitoring and reporting procedures
Creators: name: Merida-Campos, Carlos
name: Bela Dias, Ana
Publisher: name: Zenodo
Year: 2011
Subjects: Deliverable
HPC
Research infrastructure
Monitoring
Reporting
Impact Assessment

description: Monitoring and reporting is an invaluable tool for good management providing a useful base for the evaluation of the health status of the organization. It enables to determine whether the resources available are sufficient and are being well used, whether the capacity is sufficient and appropriate, whether the organization is doing what it is supposed to do, and whether the environment and contextual conditions are changing.<br> The practice is not meant only for business or commercial oriented organizations and it has become increasingly important also for entities like PRACE that have non-profit missions and objectives, but need to implement good management practices for their sustainability.<br> A preliminary analysis of what is implemented in worldwide institutions that like PRACE manage the allocation of resources showed that all of them operate some monitoring and reporting process model.<br> While it is already a common practice of computing centres to prepare and publish an annual report on the performance of the organization mainly concerning financial and service aspect, a pan-European organization such as PRACE, given its more ambitious nature, would benefit from widen its monitoring scope to better assess the efficiency of its performance, the effectiveness of its actions, and its impact. We have found examples of organizations that are applying different monitoring methodologies showing a good level of maturity. One is the Department of Energy, Office of Science (DoE SC) whose approach is based on an operational assessment (OA) of the performance of its HPC facilities. The second one is the UK Research Council that adopted specific methodologies for measuring the impact of science using quantitative (e.g. macroeconomic, microeconomic or survey data) and<br> qualitative approaches (e.g. interviews or expert testimonies).<br> The development of a monitoring and reporting process for PRACE has to take into account the structure of the organization as well as its objectives.<br> PRACE shows some peculiarities that should be considered carefully: complex processes that require intense collaboration with stakeholders and interaction with the users; infrastructure of geographically distributed computer centres managed on the basis of national laws and with managers with different traditions and mentalities. Those peculiarities influence the selection of the monitoring targets to measure efficiency, effectiveness and impact in the different perspectives of PRACE’s activities.<br> Based on the PRACE workflow model, the monitoring targets and the derived variables are classified accordingly to four groups: Input: what resources PRACE uses in order to create and run its services such as monetary funding and budget, personal resources and equipment. Delivery: the services provided by PRACE. Output: the concrete results obtained with allocated compute resources and with the support for application development. Environment: the market environment in which PRACE delivers its services. The set of variables defined is fairly large and covers as much as possible the complete work flow of PRACE. Each one is described taking into consideration the following characteristics: Periodicity Priority the entity responsible for it the recipient body of PRACE the procedure to be followed for gathering the data. The descriptions provide the basis for designing the process of reporting and the definition of the roles involved in the process. The latter are important for the implementation of the process tasks in PRACE AISBL organization and the mapping of roles to the actual staff profiles.