# Using Statistical Test Method to Establish a Decision Model of Performance Evaluation Matrix
## 5 Case Study
Based on the improvement ranking rules, since evaluation coordinates \((x_{5},y_{5})\in Q_{2}\) and \((x_{6},y_{6})\in Q_{2}\), service item 5 and service item 6 were ranked to be of the first priority of improvement. Similarly, evaluation coordinates \((x_{7},y_{7})\in Q_{3}\) and \((x_{8},y_{8})\in Q_{3}\), then service item 7 and service item 8 were ranked to be of the second priority of improvement. \begin{table}
\begin{tabular}{c c c c c c c} \hline \hline
**Dimensions** & **Items** & \(\hat{\theta}_{i}^{S}\) & \(\hat{\theta}_{i}^{I}\) & \(C_{X0i}\) & \(C_{Y0i}\) & **Quadrant** \\ \hline \multirow{3}{*}{Learner Interface} & 1. Ease of use & 0.65 & 0.77 & 0.49 & 0.69 & Q1 \\  & 2. User-friendliness & 0.70 & 0.77 & 0.50 & 0.69 & Q1 \\  & 3. Ease of understanding & 0.68 & 0.76 & 0.50 & 0.69 & Q1 \\  & 4. Operational stability & 0.59 & 0.75 & 0.49 & 0.69 & Q1 \\ \hline \multirow{3}{*}{Learning Community} & 5. Ease of discussion with other learners & 0.41 & 0.77 & 0.49 & 0.69 & Q2 ** \\  & 6. Ease of discussion with teachers & 0.39 & 0.77 & 0.49 & 0.69 & Q2 ** \\  & 7. Ease of accessing shared data & 0.37 & 0.63 & 0.49 & 0.68 & Q3 * \\  & 8. Ease of exchanging learning with the others & 0.34 & 0.57 & 0.49 & 0.68 & Q3 * \\ \hline \multirow{3}{*}{System Content} & 9. Up-to-date content & 0.52 & 0.67 & 0.49 & 0.68 & Q4 \\  & 10. Sufficient content & 0.52 & 0.67 & 0.49 & 0.68 & Q4 \\  & 11. Useful content & 0.53 & 0.66 & 0.49 & 0.68 & Q4 \\ \hline \multirow{3}{*}{Personalization} & 12. Capability of controlling learning progress & 0.51 & 0.65 & 0.49 & 0.68 & Q4 \\  & 13. Capability of recording learning performance & 0.51 & 0.67 & 0.49 & 0.68 & Q1 \\ \hline \hline \end{tabular} Note.* indicates the need for improvement; ** indicates the need for the top priority of improvement.  
\end{table} 
Table 1: Satisfaction and Importance survey for CALL System. Figure 2: PEM of CALL System. Next, managers would be able to refer to the recommendations of these rules to carry out improvement procedures. After the improvement concludes, a test of the improvement can be carried out. Based on Equations (38) and (39), the assumptions of its statistical test are as follows:  
\[\text{null hypothesis }H_{0}:\;\theta_{bi}^{S}=\theta_{ai}^{S}; \tag{44}\]  
\[\text{alternative hypothesis }H_{1}:\;\theta_{bi}^{S}\
eq\theta_{ai}^{S}. \tag{45}\]  
Based on Equations (36) and (37), the 95% confidence intervals with sample size \(n=258\) of the satisfaction indices of the four service items that need to implement improvements are calculated. 1. Service Item 5 before improvement : \[[L\theta_{55},U\theta_{55}]=\left[0.41-1.96\frac{0.28}{\sqrt{344}},0.41+1.96 \frac{0.28}{\sqrt{344}}\right]=[0.38,0.44];\] (46) after improvement : \[[L\theta_{55},U\theta_{55}]=\left[0.48-1.96\frac{0.27}{\sqrt{258}},0.48+1.96 \frac{0.27}{\sqrt{258}}\right]=[0.45,0.51].\] (47)
2. Service Item 6 before improvement : \[[L\beta_{bi},U\theta_{bi}]=\left[0.39-1.96\frac{0.29}{\sqrt{344}},0.39+1.96 \frac{0.29}{\sqrt{344}}\right]=[0.36,0.42];\] (48) after improvement : \[[L\theta_{bi},U\theta_{bi}]=\left[0.47-1.96\frac{0.28}{\sqrt{258}},0.47+1.96 \frac{0.28}{\sqrt{258}}\right]=[0.43,0.50].\] (49)
3. Service Item 7 before improvement : \[[L\beta_{77},U\theta_{77}]=\left[0.37-1.96\frac{0.29}{\sqrt{344}},0.37+1.96 \frac{0.29}{\sqrt{344}}\right]=[0.34,0.40];\] (50) after improvement : \[[L\beta_{77},U\theta_{77}]=\left[0.42-1.96\frac{0.27}{\sqrt{258}},0.42+1.96 \frac{0.27}{\sqrt{258}}\right]=[0.39,0.46].\] (51)
4. Service Item 8 before improvement : Figure 2:0 (52) after improvement : Figure 2:1 (53)  
In summary, the aforementioned test rules are listed as follows:  
1. For service items 5 and 6, their upper limits before improvement are smaller than their lower limits after improvement (\(U\theta_{55}=0.44<L\theta_{55}=0.45;\,U\theta_{bi}=0.42<L\theta_{a6}=0.43\)), indicating that the improvements of service items 5 and 6 are effective. 2.