On quality aware adaptation of Internet video (2004)

The main issue with the transmission of streamed video over the Internet is that of adaptation: due to the best-effort service model of the Internet, abundance of bandwidth to guarantee good quality is not always feasible. Instead, congestion control of the shared resources needs to be employed, resulting in variations in bandwidth availability. These variations of the transmission rate introduce a first level of quality degradation. In addition, the time-varying complexity of the underlying visual scenes requires a widely fluctuating transmission bandwidth in order to achieve good quality, otherwise quality oscillations occur. In order to tackle these problems, a multitude of rate-adaptation techniques have been proposed. However, these proposals either consider adaptation solely from the network point of view (e.g., TCP-friendliness, rate-adaptation, layering) and completely disregard the effect on quality, or employ simplistic metrics of quality (e.g., peak signal-to-noise ratio) that are not necessarily true representations of the quality as experienced by the user of the video service. This thesis advocates the integration of emerging objective video quality metrics within the adaptation cycle of Internet video. A result of recent research efforts, objective quality metrics are computational models that produce quality ratings which are highly correlated with human judgements of quality. By considering the time-changing relationship between properties of the video content, the available bit rate, and the effect of both on perceived quality, this dissertation studies quality-aware rate adaptation techniques that improve end quality perception in the context of two different application scenarios of video streaming. Firstly, this dissertation examines applications that involve the transmission of multiple concurrent media streams to a receiver (e.g., the transmission and display of several video streams, relevant to the application scenario). We encounter the problem of efficiently apportioning the bandwidth available to a multi-stream session among its constituent media flows, by allowing participating flows to jointly adapt their transmission rates in consideration of their respective time-varying quality. Suitable adaptation timescales that coincide with changes in the video content (scene cuts) and an inter-stream adaptation mechanism that considers the time-varying objective quality of the participating streams are proposed. Experimental results show the benefits of the proposed method, in terms of improved session quality and utilisation of the session bandwidth, in comparison to (i) a priority-based inter-stream adaptation and (ii) the case where the session flows are transmitted over independent congestion controlled connections. Secondly, the thesis deals with the problem of providing smooth quality rate adaptation for live, unicast, real-time video streaming. Since real-time performance is necessary, an objective quality metric cannot be applied in-line, as it is computationally intensive. For this reason, artificial neural networks are utilised to predict quality ratings in real-time. Predictions are sought based on descriptive features of the video content and the bandwidth that is available to the stream. The limitations of current approaches to provide stable or smooth quality are identified. A rate-quality controller, built on the principles of fuzzy logic, is then developed to alleviate annoying short-term quality variations that appear due to mismatches between the available bandwidth and the rate required for stable quality. Based on the neural network's quality predictions, the controller continuously monitors the recent quality values, the nominal transmission rate and the occupancy levels of the participating buffers, to calculate appropriate encoding rates that eliminate short-term quality fluctuations. Experimental results show that the proposed solution offers significant stability of short-term quality and 'smoothes out' annoying oscillations of quality to extreme low and high values, while at the same time respects transmission rate constraints and preserves buffer stability. By presenting numerous experimental results with a wide variety of video sequences, this dissertation shows that video streaming systems can utilise objective measures of perceived quality to deliver improved presentation quality, by applying quality-aware adaptation techniques which are tailored to the semantics of the specific streaming application.