## 4 Evaluation Results and Discussion
### Performance across Viewing Devices  
To investigate the performance of the QoE models across different viewing devices, we consider three separate device combinations corresponding to three database sets. Particularly, the first set (denoted _DBS1_) includes three sub-databases of _W-SQoE-IV_pH264_, _W-SQoE-IV_pH264_, and _W-SQoE-IV_uH264_, which corresponds to three devices of smart phones, full high definition (FHD) monitor, and ultra high definition (UHD) TV, respectively. With the focus on smart phone and PC, the second set (denoted _DBS2_) contains two databases of _TR04p_ and _TR04f_. Similarly, the third set (denoted _DBS3_) is composed of _TR06p_ and _TR06f_ databases. It is worth noting that the difference between the databases in a given set is the viewing device only. This allows us to reliably compare the performance of the same model across different devices.  
To analyze the impact of viewing devices on the performances of the models, we use two metrics of average performance (denoted _av_) and mean difference (denoted \(\Delta\)). The use of _av_ and \(\Delta\) is to respectively measure the main tendency and the variation of performances across viewing devices. The formulas of these metrics are given by (4) and (5). In particular, given a model, the first metric is measured as the average performance computed over all the seven databases in the three sets. For the second metric, the performance differences are firstly calculated between all the database pairs in the same set. Then, the mean of all the differences is considered as the mean difference value:  
\[av=\frac{\sum_{i=1}^{N}\sum_{j=1}^{K_{i}}M_{i,j}}{\sum_{i=1}^{N}K_{i}}, \tag{4}\]  
\[\Delta=\frac{\sum_{i=1}^{N}\sum_{j=1}^{K_{i}-1}\sum_{l=j+1}^{K_{i}}\mid M_{i,j }-M_{i,l}\mid}{\sum_{i=1}^{N}\binom{K_{i}}{2}}, \tag{5}\]  
Figure 2: Average performances of individual QoE models over _H.264-encod_ set and _HEVC-encoded_ set. (**a**) PCC; (**b**) SROCC; (**c**) RMSE.  
where \(N=3\) is the number of sets, \(K_{i}\) is the number of databases in set \(i\), and \(M_{i,j}\) is the performance metric value corresponding to database \(j\) in set \(i\).  
Figure 3 shows the results of individual QoE models. It can be seen that, among the models, the _[PERSON]'s_ model has the lowest average performance with avPCC = 0.27, avSROCC = 0.44, and avRMSE = 0.84. Meanwhile, the _[PERSON]'s_ model is found to have the strongest performance variation. Specifically, its mean differences of \(\Delta\)PCC, \(\Delta\)SROCC, and \(\Delta\)RMSE are respectively 0.11, 0.10, and 0.04 in the _ori_ case, and 0.11, 0.13, and 0.04 in the _mod_ case. By contrast, the performance of the _LSTM_, _SQI_, and _KSQI_ models is not only high (i.e., high _av_) but also quite consistent (i.e., low \(\Delta\)). Particularly, the avPCC, avSROCC, and avRMSE values are 0.87, 0.88, and 0.39 for the _LSTM_ model, 0.85, 0.85, and 0.39 for the _SQI_ model, and 0.87, 0.87, and 0.36 for the _KSQI_ model. In addition, the \(\Delta\)PCC, \(\Delta\)SROCC, and \(\Delta\)RMSE values are respectively 0.06, 0.03, and 0.04 for the _LSTM_ model, 0.03, 0.03, and 0.04 for the _SQI_ model, and 0.04, 0.03, and 0.04 for the _KSQI_ model. The reason might be that these models use either MOS or VMAF as a segment quality metric. Both of the metrics inherently take into account the impact of a viewing device on the user perceived quality. Thus, the _LSTM_, _SQI_, and _KSQI_ models still perform quite well across viewing devices. This implies that, to obtain the high and stable performance for different devices, segment quality metrics should be ones that consider the effect of viewing devices such as MOS and VMAF. In addition, it is suggested that the _LSTM_, _SQI_, and _KSQI_ models can be employed in the QoE prediction for different viewing devices.