## 2 Testing and Evaluation Methodology
### Metrics and Product Quality Model for Evaluation of SLS  
#### 2.2.1 Metrics for Performance Evaluation  
The SLS deals with data recording, transfer, and storage in a centralized database through the integrated action of a smart box and a web-based information platform (see Figure 3). The functional features of the smart box tool and information platform were integrated using Cargolog PC software, Cargolog FAT90 V2 (Mobitron, Huskvarna, Sweden). In this report, as part of the evaluation of the smart system, metrics were selected and adapted from international standards for systems and software quality requirements and evaluation (ISO/IEC 25010:2011) as described in [9] and Table 3.  
\begin{table}
\begin{tabular}{c p{284.5 pt}} \hline \hline \multicolumn{2}{c}{**Basic**} \\ \multicolumn{2}{c}{**Characteristics**} \\ \hline System to evaluate & Used to evaluate the SLS. Usability testing is an evaluation approach that can be applied to evaluate almost any product or technology; for example, software for database management, network management tools, early-stage prototypes, and related help manuals. \\ \hline Focus & The focus was on usability of the SLS. The test was intended to validate the first release of the smart system (smart box and information platform) prototype, with less consideration of marketability of the product. \\ \hline Participants & The intended main participants were (potential) end users of the smart system who were actors of the pruning supply chain (farmers, pruning traders, transporters, power plants). Test participants and administrators interacted during tests. Training and instruction guides were provided before starting. During this testing, research centers such as Leibniz Institute for Agricultural Engineering Potsdam-Bornim (ATB) in Germany, Research Centre for Energy Resources and Consumption (CIRCE) in Spain, and the Swedish University of Agricultural Sciences (SLU) as well as Gruyser (transport dealer) participated actively. \\ \hline Tasks & The smart system has different functional features. Each participant considers the specific functionality of the system that best fits him/her. Test participants who act as farmers perform tasks that the pruning producers perform, while traders perform what processors and traders of biomass perform using the system. Similarly, transporters and consumers test features of the system that serve the transport company and end consumers of the pruning biomass. \\ \hline Data & Pruning biomass–related data were recorded, stored in a database, and analyzed. Both problems and positive aspects noticed during tests were analyzed. Data collected with three prototypes were used to evaluate the functionality of similar features of the system but with different smart box prototypes. Accordingly, a data triangulation test approach was used, especially to test data storage and display capability of the central information platform by analyzing data sourced from testing with three different prototypes. \\ \hline Results & Test results were used to improve the smart system and communicate the research outputs. These analysis results are documented, archived, and used to identify what problems surfaced and how to solve them. \\ \hline \hline \end{tabular}
\end{table}
Table 2: Usability testing characteristics.