# Assessment of the Quality of Video Sequences Performed by Viewers at Home and in the Laboratory
## 2 Materials and Methods
Each group of viewers evaluated the video signal subjected to one encoding technique for all resolutions and bit rates. In both cases, the team of observers consisted of second-year electronics students at the Faculty of Electronics, Photonics, and Microsystems of the Wroclaw University of Science and Technology, aged 20-21, with normal visual acuity and correct color discrimination. As recommended by the International Telecommunication Union BT. 500 [1], the minimum number of observers should be 15. In the presented studies, three groups were created for home measurements and three groups for laboratory measurements. Each group evaluated a different type of coding. The number of individual test groups examining individual codec was as follows:  
1. H.264--25 people under home conditions and 45 people under laboratory conditions;
2. H.265--35 people under home conditions and 35 people under laboratory conditions;
3. VP09--30 people under home conditions and 40 people under laboratory conditions. The different size of the groups was, among other reasons, the result of a different number of individuals willing to participate in a given measurement session, the effect of a statistical analysis of observer ratings, and the elimination of those observers who were characterized by low participation (regardless of the bit rate or resolution, they gave the same quality rating). Before beginning the measurements, the participants were familiarized with the assessment method and had one training session. During the training, the observers became acquainted with the technique of presenting the test material and how to assess the changes in video quality. After the training, the actual measurements began. After viewing the original and encoded sequences, each study participant recorded their assessment of quality deterioration in a special form. In the second part of the research, the authors performed video quality assessment using the objective double stimulus method, which relies on a comparison of the encoded video samples with the reference original video (see Figure 2). The original video footage was encoded using the FFmpeg [48] tool with implemented H.264, H.265, and VP9 video codecs. Four spatial resolutions and coding bit rates in the range from 300 to 6000 kbps, mentioned above, were taken into account. There were 216 video samples (3 codecs \(\times\) 4 spatial resolutions \(\times\) 18 coding bit rates) prepared in total. Figure 1: An example frame from the original video. Figure 2: Video quality assessment using double stimulus method. Each set of video samples for specific spatial resolution should be compared with source video footage of the same resolution. This way, the quality of each set of videos is objectively assessed independently of the other sets of videos. When it comes to subjective quality assessment, each set of videos should be presented on a display with proper resolution that should be fitted to the resolution of the assessed video. This may be difficult to achieve when the quality assessment is performed by many different users in their home environments, where a specific display resolution may be used by default. Thus, the authors assumed that the objective assessment should be conducted using one display resolution. The most popular spatial resolution of the displays used by end users was 1920 \(\times\) 1080 pixels (FHD). Therefore, the sample preparation process was a bit more complicated than just encoding. It also included upsizing all of the videos of smaller spatial resolution, i.e., 640 \(\times\) 360, 858 \(\times\) 480 and 1280 \(\times\) 720, to FHD (Figure 3). This way, the authors wanted to achieve the same effect observed on the end-user equipment, which usually resizes smaller resolution videos to the maximum display size, with FHD resolution set by default. After this preparation, the tested video samples were objectively assessed, by comparison with the reference video (denoted in Figure 3 as '1920 \(\times\) 1280/ref./'), using three metrics, i.e., PSNR, SSIM, and VMAF.