# A Big Data and Time Series Analysis Technology-Based Multi-Agent System for Smart Tourism
## 6 Evaluation of System Presentation and Efficacy
### System Evaluation  
System evaluation is for the analysis of user satisfaction, the usability of the content provided by the webpage of the system, and the ease of use of the system. The analysis of the former is in the International Standard Organization (ISO) 9241. According to its definition, three dimensions --effectiveness, efficiency, and satisfaction--are used to evaluate usability. [PERSON] ([[http://www.wqusability.com/](http://www.wqusability.com/)]([http://www.wqusability.com/](http://www.wqusability.com/)), 2009) suggested using the \"5E\" (efficient, effective, engaging, error tolerant, and easy to learn) function, which could measure usability. Regarding the ease of using  
Figure 10: Bulk recommendation based on St. John’s University GPS. (**a**) Information about quality restaurant; (**b**) information about the corresponding parking lot; (**c**) information on the website of Big Mac Ice Cream; and (**d**) destination navigation and presentation. the system, [PERSON] ([[http://www.useit.com/jakob/](http://www.useit.com/jakob/)]([http://www.useit.com/jakob/](http://www.useit.com/jakob/)), 2005) strongly recommended 10 basic principles for user interface design, including visibility of system status, match between system and the real world, user control and freedom, consistency and standards, error prevention, recognition rather than recall, flexibility and efficiency of use, aesthetic and minimalist design, helps users recognize, diagnose, and recover from errors, and help and documentation. Relevant evaluation parameters, as well as the analysis results, are shown in Table 3. The overall satisfaction of the usability of the system and the ease of using the interface design is 80% (((4/5) + (8/10))/2), which satisfies basic principles (1)-(8) of the 10 principles proposed by [PERSON], which indicates that this system must be improved in two aspects: error tolerance and help documentation. The design preference to importance ratio (DIR) (Equation (6)), as proposed by [PERSON] [30], is taken as the design principle of the human-machine interface (HMI) of this system, as it is perfect for combining it with the balancing index (BI) (see Equation (7)) to define the interface. In other words, if BI is zero, all the HMI elements in this design are balanced and perfect. Its physical meaning is that the HMI design meets the principle of the design preference for importance, and that users' operation of the system is more consistent with user demand:  
\[DIR_{ijk}=\frac{\frac{DP_{ij}}{\frac{\sum_{i=1}^{n}DF_{ij}}{\sum_{i=1}^{n}I_{ik} }}}{\frac{I_{ik}}{\sum_{i=1}^{n}I_{ik}}} \tag{6}\]  
\[BI_{jk}=\frac{\left|\sum_{i=1}^{n}\log_{10}DIR_{ijk}\right|}{n} \tag{7}\]  
Specifically, \(DIR_{ijk}\) refers to the \(DIR\) of the design attribute \(j\) and importance attribute \(k\) of HMI interface element \(i\). \(DP_{ij}\) indicates the design preference of design attribute \(j\) of the HMI interface element \(i\). \(I_{ik}\) represents design importance \(k\) of importance attribute of the HMI interface element \(i\). \(BI_{jk}\) refers to the balance index of design attribute \(j\) and importance attribute \(k\). \(n\) indicates the total number of HMI interface elements. A five-scale evaluation scheme is utilized to evaluate each design preference (\(DP_{ij}\)). In other words, the design preferences include very good, good, moderate, weak, and very weak, and their corresponding values are 5, 4, 3, 2, and 1, respectively. Table 4 illustrates the evaluation of HMI elements in our design attributes, as well as their corresponding informational importance, as evaluated using the analytic hierarchy process [30], while Table 5 shows the evaluation results of \(DIR\) with \(BI\) = 0.003165