# Video Quality Modelling--Comparison of the Classical and Machine Learning Techniques
## 3 Materials and Methods
### Research Methods Used  
The quality assessment of the analysed samples included three steps:  
* Step 1: subjective quality assessment using an appropriate group of testers;
* Step 2: objective quality assessment using a dedicated software tool;
* Step 3: quality assessment using a machine learning model. The detailed research procedure (flow diagram) is presented in Figure 2.  
Step 1 is based on the results of subjective measurements obtained in previous studies [29; 34]. The Double Stimulus Impairment Scale Method (DSISM) [24] was used to subjectively assess video quality. In this method, the viewer compares two video sequences and evaluates the degree of deterioration of the second signal (the evaluated sequence) in relation to the first signal (the reference sequence). The assessment used a five-point  
Figure 1: A frame from the reference video. Figure 2: Flow diagram of the work. MOS scale, where a value of 5 means imperceptible deterioration and 1 is very bothersome [35]. Subjective quality measurements were performed in a laboratory room that met the requirements of the recommendations of the International Telecommunication Union [24; 35]. Test signals with different transmission conditions (bitrate, resolution) were randomly presented to viewers on a 60-inch TV screen. The audience consisted of students from Wroclaw University of Science and Technology, aged 20-21, with normal visual acuity and correct colour discrimination. The test signals encoded with the H.264 standard were evaluated by a team of 45 people, and H.265 by 35 people. Each observer provided their assessment of the deterioration of the quality of the second video compared to the first one in a special questionnaire after watching the original and encoded sequences. After entering the spreadsheet, all evaluations were subjected to statistical analysis according to the procedure described in the ITU-R BT.500 recommendation [24]. This analysis allowed for the rejection of estimates beyond the assumed 95% confidence interval. The statistical analysis procedure according to the ITU-T P500 recommendation is presented below. The MOS value was calculated separately for each encoding technique, screen resolution, and bitrate, denoted as (c), (r), and (i), respectively:  
\[MOS_{c,r,i}=\frac{1}{N}{\sum_{k=1}^{N}O_{c,r,i,k}}\, \tag{1}\]  
where O is the assessment value of the deterioration of the quality of the k-th observer, N is the size of the observer group, c is the coding technique (c = 1--H.264, 2--H.265), r is the image resolution (r = 1--1280 \(\times\) 720, 2--1920 \(\times\) 1080), and i is the bitrate (i = 1,2,...18). The standard deviation S\({}_{c,r,i}\) was calculated according to the relationship as follows:  
\[S_{c,r,i}=\sqrt{\sum_{k=1}^{N}\frac{MOS_{c,r,i}-O_{c,r,i,k}}{(N-1)}}. \tag{2}\]  
According to BT.500, a 95% confidence interval is defined as:  
\[[MOS_{c,r,i}-\delta_{c,r,i},\ MOS_{c,r,i}+\delta_{c,r,i}], \tag{3}\]  
where the confidence interval coefficient \(\sigma_{c,r,i}\) is calculated according to the relationship:  
\[\delta_{c,r,i}=1.96\frac{S_{c,r,i}}{\sqrt{N}}. \tag{4}\]  
In Step 2, an objective assessment of the quality of individual video samples was carried out. The evaluation was performed using the full reference method, where each tested video sample was compared with the reference material (uncompressed) frame by frame, pixel by pixel. For each video sample tested, the average values of the following ten objective metrics were calculated: APSNR, Delta, MSAD, MSE, NQI, PSNR, SSIM, VII, VMAF, and VQM. It was made using a software tool called the Video Quality Estimator (VQE), version 4.3 [36]. The primary goal of the third part of the research (Step 3) was to prepare a model that, based on input data that objectively describes the quality of a video, would predict the subjective quality rating. The prediction of subjective ratings can be performed on the basis of a single objective quality metric.