No-reference video quality assessment : high motion and high dynamic range videos
Creators: name: Ebenezer, Joshua Peter
name: 0000-0003-4936-9784
Publisher: name: The University of Texas at Austin
Year: 2023
Subjects: Video processing
Machine learning
Video quality assessment

description: Video content continues to proliferate, already accounting for more than 70% of internet traffic, and projected to exceed 82% of internet traffic by 2021. Distortions can affect videos as they are captured, transmitted, and received. It is of vital importance to providers of video content to be able to monitor and predict the perceptual quality of their videos, since this directly impacts customer satisfaction. Video quality tools can also help make well-informed design choices while creating systems for capturing, processing, transmitting, and displaying videos. Video quality assessment algorithms also have applications in video denoising, designing loss functions for deep learning, video compression, and many other high-impact areas. Measuring the quality of a video without a reference is referred to as No-Reference Video Quality Assessment (NR VQA). NR VQA has many modalities, one of which is its application to livestreamed videos, which can have defects at the source the that need to be identified quickly and accurately before they reach consumers. Sports content has a great deal of motion, due to which additional perceptual effects come into play such as motion blur and poorer compression due to rapidly changing motion estimates. In this dissertation, we propose an NR VQA algorithm called ChipQA that achieves remarkable performance on Sports VQA databases. ChipQA introduces the concept of space-time chips, which are small oriented slices of the video in space and time. Another modality of NR VQA is its application to High Dynamic Range (HDR) videos. Dynamic range is the ratio of the largest and smallest usable values of a signal. In the context of video, it is the luminance range over which a video system can capture, store, and display scenes without loss of details. HDR videos have a greater range of brightnesses and colors than traditional Standard Dynamic Range videos. Distortions can be more apparent in HDR due to its increased contrast and details. We designed an extension of ChipQA, called HDR ChipQA, that incorporates special nonlinearities that enhance the extreme ends of the luminance range in order to highlight distortions. We also conducted a large-scale study of human perceptual quality judgments of HDR and SDR videos subjected to scaling and compression levels and viewed on three different display devices. While conventional expectations are that HDR quality is better than SDR quality, we have found subject preference of HDR versus SDR depends heavily on the display device, as well as on resolution scaling and bitrate. To study this question, we collected more than 23,000 quality ratings from 67 volunteers who watched 356 videos on OLED, QLED, and LCD televisions. Since it is of interest to be able to measure the quality of videos under these scenarios, e.g. to inform decisions regarding scaling, compression, and SDR vs HDR, we tested several well-known full-reference and no-reference video quality models on the new database. Towards advancing progress on this problem, we also developed a novel no-reference model called HDRPatchMAX, that uses both classical and bit-depth sensitive distortion statistics more accurately than existing metrics.