Patent Description:
HTTP Adaptive Streaming (HAS), for example as implemented by MPEG Dynamic Adaptive Streaming over HTTP (DASH), is currently a widely used technology for over-the-top (OTT) video streaming. Videos accounted for <NUM>% of the consumer Internet traffic in <NUM>, and that share is predicted to increase to <NUM>% by <NUM> (Cisco. For Internet Service Providers (ISPs) and OTT providers, assessing the perceptual quality of the videos that are transmitted over these services is important to ensure a high degree of satisfaction among their customers. For OTTs in particular, the possibility of providing a video with the best quality for the lowest bandwidth possible can lead to energy and cost reductions and thus is an important goal.

In HAS, for a given video, several representations with varying bit rates are created from an original reference. These representations are segmented for transmission. The representations target different bandwidths; clients can request the respective representations for their current network conditions, and switch between them (adaptation). The adaptation algorithms in HAS mainly try to optimize the requests for encoded video segments in a way that the overall bandwidth consumed or the risk of depleting the client-side buffer is minimized. A more advanced adaptation algorithm may also optimize for delivering the best possible overall quality. Hence, there is a need for a prediction of the visual quality of a video segment, so that the adaptation algorithms can use this as a fundamental criterion to improve streaming quality. This prediction is performed by a model that takes as input the video stream and outputs the visual quality. The output should match as closely as possible the quality that a human subject would rate when watching the same video stream. Such a model is usually trained with the help of ground truth data, for example actual ratings of human viewers. The training is usually done by optimizing the values of the internal model coefficients so that the residuum (difference) between model output and the ground truth data is minimized, thereby improving overall prediction accuracy.

Depending on the input data that is used for quality assessment, video quality models can be classified into three main categories.

Media-layer models use the decoded frames to estimate video quality. They can be further categorized into three types namely ((A. Takahashi, <NUM>) (A. Raake, <NUM>)),.

Bit stream-layer models: Depending on the level of access to the bit stream, three main types have recently been distinguished in the context of ITU-T Rec. <NUM> (<NPL>) namely,.

Hybrid models use a combination of decoded frames and bit stream as input to estimate video quality.

Conventional approaches for estimating or predicting video quality using either parametric- or machine-learning-based computation are known in the art.

In general video quality can be predicted using several approaches, for example pixel based or bit stream-based (M. Shahid <NUM>). There have been several pixel-based and bit stream-based video quality models reported in literature. Several approaches can be used to develop bit stream-based video quality models. Some of the approaches are: parametric approach, machine learning approach etc. This section will mostly focus on the parametric and machine learning based video quality models.

Concerning the parametric-based models, in (ITU-T, <NUM>), a parametric video quality model for video quality estimation for IPTV applications is presented. This model takes into account distortions due to both compression and erroneous transmission.

Considering that most of today's popular video streaming providers used HAS for video transmission, effects other than compression artifacts need to be handled by video quality models. In <CIT>, bit stream-based video quality prediction models for HAS have been proposed. These models are applicable up to a resolution of 1080p and frame rate of 30fps. Moreover, only videos encoded with H. <NUM> video codec are considered in these models. All presented models are standardized as the ITU-T Recommendation P. <NUM> (<NPL>).

Beside bit stream-based video quality models other models exist, e.g. using pixel data of the decoded video frames to estimate video quality. A categorization of such pixel models according to their input data, for example full-reference, reduced-reference, no-reference was outlined above (M. Shahid <NUM>).

One example of a full-reference model is the perceptual video quality model designed using a Support Vector Machine (SVM)-based approach that was presented in <CIT>. This model referred to as "Video Multimethod Assessment Fusion (VMAF)" combines various objective metrics to produce a perceptual quality score for a target video.

On the other side, several pixel-based no-reference models have been presented in the literature. A Support Vector Regression (SVR)-based model to predict the level of quantization and PSNR is proposed in (J. Søgaard, <NUM>).

A pixel-based, no-reference video-quality model based on deep neural networks (DNN) is proposed in S. Göring, <NUM>. In the domain of gaming videos, a machine learning based no-reference video quality model is presented in S. Göring, <NUM>. In addition to proposing a new model for no-reference evaluation of gaming videos, the study described in S. Göring, <NUM> also presents new features such as staticness, blockmotion etc. that is used in the model proposed in that paper. Barman, <NUM>, an SVR-based hybrid no-reference video quality model for gaming videos has been proposed. This study uses VMAF as the ground truth for model development.

In contrast to the parametric bit stream models, machine learning based bit stream models have also been proposed in the literature. One main advantage of pure bit stream based models in contrast to pixel based models is their application scope, due to a faster processing and prediction of video quality scores.

A no-reference bit stream-based machine learning video quality model is proposed in M. Shahid, <NUM>. The model presented in this work is based on Least Squares (LS) SVM as the employed machine-learning method. This model was evaluated on its ability to predict metrics such as PSNR, PEVQ, SSIM (Z. Wang, <NUM>), MSSIM (Z. Wang <NUM>) and subjective MOS obtained from video-quality tests. The LS-SVM-based model (M. Shahid, <NUM>) is trained on CIF and QCIF videos at <NUM> and 30fps with videos encoded using H. This model is not designed to take into account the effects that are encountered in HAS and also does not consider resolution and video codecs that are typical used in current HAS applications.

In Mocanu, <NUM>, a machine-learning-based no-reference bit stream model is proposed. The main focus of that work was to explore the added value of machine learning in video quality estimation. To this end, the proposed model is designed to output two values, one of which is the predicted mean opinion score (MOS) and the other value provides the variability of subjective data. However, only one codec was considered. The model uses a random neural network as the machine-learning approach.

Demirbilek, <NUM>, various machine-learning-based video-quality models have been presented. Decision Tree-, Deep Learning- and Genetic Programming-based models have been presented in this work. These models use quantization parameter, frame rate, packet loss ratio, bit rate, resolution and codec as the main features for all model variants. These models are only tested on H. <NUM> and H. <NUM> encoded videos. Similar to the model by (M. Shahid, <NUM>), these models are not capable of handling the effects seen in HAS and for newer codecs.

Muhammad Shahid et at. disclose a reduced complexity no-reference artificial neural network based video quality predictor.

Wikipedia features an article on Ensemble Learning.

Robi Polikar discloses an article about ensemble learning on Scholarpedia.

Elwerghemmi Radhia et al. disclose an online QoE prediction model based on stacked multiclass incremental support vector machine.

According to the present invention, an information-adaptive mixed deterministic/machine-learning-based bit stream video-quality model is proposed.

Further aspects of the invention are specified in the dependent claims.

Disclosed herein is a method for predicting the quality Q of a video bit stream. The method comprises obtaining a video segment and a corresponding bit stream, parsing the bit stream by a bit stream parser, and obtaining bit stream parameters. The quality Q is predicted by using a deterministic model and a machine learning model with the obtained bit stream parameters, wherein the predicted video quality is a weighted sum of the prediction of both models. A residual of the deterministic model is predicted by the machine learning model. That is, during optimizing of the model coefficients, a residuum or prediction error of the deterministic model is estimated by the machine learning model. It thereby compensates the prediction error.

Particularly, the visually perceived video quality Q of a given video sequence based on the bit stream of said video sequence, and expressed on a video-quality rating scale may be predicted by the proposed method.

Various embodiments may preferably implement the following features:
Preferably, the predicted video quality Q is computed as <MAT> wherein Mparametric and Mmachine learning are the results of the deterministic model and the machine learning model, respectively, and w<NUM> and w<NUM> are weighting factors of the deterministic model and the machine learning model, respectively with w<NUM>+ w<NUM>=<NUM>. Preferably, the weighting factor values are dependent on the available bit stream parameters. The weighting factor of the machine learning model may increase with the number of available bit stream parameters.

In other words, as the machine learning model gets better - since more information or input is available -, the weight of the machine learning model may increase.

Preferably, the bit stream parameters comprise at least one of bit rate or resolution or frame rate or codec information or video codec information.

Preferably, the bit stream parameters further comprise at least one of frame sizes or ratio of I-frame to non-I frame size or a prediction of the deterministic model.

Preferably, the bit stream parameters comprise at least one of average motion per frame or standard deviation of motion in the x-direction (horizontal motion) per frame or maximum quantization parameter of non-I frames or minimum quantization parameter or average quantization parameter of non-I frames or frame sizes and/or quantization degradation,.

Preferably, a quantization parameter, QP, is predicted using said parameters. A quantization degradation, quant, may be computed as <MAT> wherein QPnon-I frames is the quantization parameter of non-I frames in the bit stream and QPmax is the maximum quantization parameter possible in the bit stream.

Preferably, quant is a normalized value of QP. The QP is a parameter directly related to the level of compression applied during encoding of a video segment and indirectly related to the quality of a video segment. QPnon-I frames is the quantization parameter of non-I frames in the bit stream. QPmax is the maximum quantization parameter possible for the codec used in the bit stream. Non-<NUM> frames are frames within the bit stream that are dependent on other frames in order to be decoded.

Preferably, the machine learning model is one of Random Forest, Support Vector Machine (SVM), or Support Vector Regression (SVR). Preferably, the machine learning model comprises an initial data set which is trained by the machine learning model. Other machine learning models may also be used.

Disclosed herein is further a data processing apparatus. The data processing apparatus comprises an obtaining unit for obtaining a video segment and a corresponding bit stream, a receiving unit configured to parse the bit stream by a bit stream parser, and a parameter unit for obtaining bit stream parameters. A prediction unit may be configured to predict the video quality by using a deterministic model and a machine learning model with the obtained bit stream parameters, wherein the prediction unit is configured to calculate the video quality as a weighted sum of both models.

The invention is described with reference to the following figures:.

An exemplary embodiment of the invention is shown in <FIG>. Therein, a video segment is parsed by a bit stream parser. Depending on the level of access to the bit stream provided to it, the bit stream parser generates the available bit stream parameters. Depending on the type of codec used, a different parser may be used. The obtained parameters are then aggregated. This can be done e.g. per segment, per group of pictures (GOP) or in any other suitable manner.

The obtained parameters are then fed into a deterministic model which calculates an initial parametric prediction of video quality. The residuum between parametric prediction and ground truth data is then calculated. In order to reduce the remaining prediction error of the deterministic model, a machine learning model is additionally used, which estimates video quality, additionally compensating this residuum. Said machine learning model is based on the same data as the parametric model and it is trained to determine not only the video quality, but also the residuum of the parametric prediction. The output of the deterministic model and the machine learning model are then added using weighting factors. Here, the weighting factors are dependent on the type, complexity and number of parameters available. In other words, the more detailed the bit stream may be analysed and the more bit stream parameters are available, the better the machine learning model gets. Thus, a higher and more detailed number of bit stream parameters may lead to a higher weighting factor of the machine learning model.

The present invention describes bit stream-based video-quality prediction models that determine the perceptual quality of a short-duration (e.g., <NUM> long) video sequence. These models predict the video quality as perceived by a user on a scale from <NUM>-<NUM>, with <NUM> representing the worst quality and <NUM> representing the best quality. The output can be provided on a per-segment or per-second basis. The main focus of the presented models is the (no-reference) bit stream-based video quality prediction, however the general concept is not limited to this use case. In other words, the visually perceived video quality of a given video sequence is predicted based on the bit stream of said video sequence. The perceived video quality may be expressed on a video-quality rating scale. The presented models can handle resolutions from 240p up to 3840p (<NUM> UHD), with frame rates from <NUM> up to <NUM> fps for three different codecs (H. <NUM> and VP9). It is understood by the skilled person that further codecs, higher frame rates, and higher resolutions might be processed with the method and apparatus according to the present disclosure by adapting the available and used parameters accordingly.

The provided models can be used as building blocks in various applications, e.g., as part of a video quality model that integrates the short-term predictions over a longer period (e.g., <NUM>, <NUM> or <NUM> minutes), which can use the calculated per-segment and per-second scores as input with other side-information to predict overall (visual or audiovisual) quality.

More particularly, the present disclosure refers to information-adaptive mixed deterministic / machine-learning-based video-quality models. These models are capable of predicting video quality for every <NUM> second period and for a segment of a length of multiple seconds (e.g., up to <NUM> or <NUM> seconds). The models that are described are combinations of deterministic parametric models and machine learning models. In the following, a Random Forest based machine learning model will be used as an example only. However, other machine learning models might also be suitable for the task. The video quality (Q) can be predicted as follows: <MAT> where Q is the overall quality prediction, Mparametric and MrandomForest refer to the parametric model and Random Forest model parts, respectively, and w<NUM> and w<NUM> are weights to emphasize one model part over the other, which may improve prediction accuracy. The weights may be selected freely according to the chosen models and other factors. Preferably, the weighting factors are in a range between <NUM> and <NUM>. More preferably, the weighting factors add up to <NUM>.

The "information-adaptive" concept associated with the model refers to the ability of the regression-based machine learning algorithm part of the model to scale in terms of prediction accuracy based on the available content-related features.

The proposed model comprises three possible layers based on the available input information for the parametric part on a per-segment basis. Herein, they are characterized and referred to as different "levels":.

With a higher level, i.e. more available input information for the models, a higher prediction accuracy can be achieved at the expense of higher computational complexity, and the requirement that the model actually has access to that information. It is to be noted that decoding of frames (such as done with pixel-based models) is not necessary in any of the levels.

The features for the machine-learning part of the model, which is exemplarily based on a Random Forest regression, are derived from the available input information for each level. Moreover, it is not limited to a Random Forest, as other models that can act as classifiers or regressors can be used.

In the following, the parametric part or deterministic model will be discussed in detail.

The parametric part of the model Mparametric is based on the principle of degradation-based modeling. In the proposed approach, three different degradations are identified which may affect the perceived quality of a given video. The general concept is that the higher the degradation, the lower the quality of the video.

The three degradations that affect that quality of a given video are as follows:.

As described above, an output scale of <NUM>-<NUM> is typically used for video quality model predictions, as it aligns with the <NUM>-point absolute category rating scale mostly used in subjective video-quality tests, typically referred to as the MOS-scale, mean opinion score scale. In the degradation-based modeling case, the computation of all the three degradations are performed on the so-called R-scale (<NUM>-scale), which was originally introduced in the so-called E-model, cf. This scale is used internally to the model to avoid saturation of the quality in the higher part of the <NUM>-point scale.

The RfromMOS and MOSfromR computations shown below involve information loss due to the fact that these two functions assume that the highest MOS that can be reached is <NUM>, thereby resulting in clipping on the MOS-scale for ratings higher than <NUM>. To avoid this information loss, all the subjective data used to train the model is compressed to the <NUM>-scale by a simple linear transformation, and the model is trained on this data. Therefore, the resulting coefficients predict the initial prediction on a <NUM>-scale. To obtain the prediction on the original <NUM>-point scale, the initial prediction is scaled back to the <NUM>-scale using the inverse transformation.

The parametric / deterministic part in all three models uses the same "core model" structure. The feature set for the Random Forest part of the model is adapted based on the mode.

Note that the Random Forest part of the model is used to take into account the effects which are not captured by the three types of degradation used in the parametric/ deterministic, "core-part" of the model.

The core model will now be presented in detail. The values of the coefficients may be found in the tables provided below, dependent on the Mode or Level and type of device used. Here, an <NUM>-frame (intra coded frame) denotes a picture that is coded independently of all other pictures. Video content can be characterized by its amount of spatial and temporal information. The higher the spatial and temporal information is, the higher is its complexity. A quantization parameter (QP) is chosen to encode a particular video at a given target bit rate depending on spatiotemporal complexity of the particular video, taking into account the content-related properties.

The determination of the quantization degradation quant is performed as follows: <MAT>.

Therein, QPmax is codec and bit depth dependent and quant is in the range <NUM> - <NUM>.

quant is a normalized value of the quantization parameter QP, wherein the QP is a parameter directly related to the level of compression applied during encoding of a video segment and indirectly related to the quality of a video segment. QPnon-I frames is the quantization parameter of non-I frames in the bit stream and QPmax is the maximum quantization parameter possible for the codec used in the bit stream. Non-I frames are frames within the bit stream that are dependent on other frames in order to be decoded.

Furthermore, the following equations are used <MAT> <MAT> <MAT>.

Determination of the upscaling degradation is calculated as follows: <MAT>.

Where display_res = (<NUM> * <NUM>) for PC and (<NUM> * <NUM>) for mobile and tablet, coding_res corresponds to the resolution at which the video is encoded and the scale_factor is in the range <NUM> - <NUM>.

The determination of the frame rate degradation can be computed as follows: <MAT>.

Where <MAT> and framerate_scale_factor is in the range <NUM> - <NUM>.

With <MAT> the parametric part related final MOS can be calculated. <MAT> <MAT> <MAT>.

Scaling is done as the coefficients are trained by compressing the subjective scores to a scale of <NUM> to avoid the information loss that can be introduced by the RfromMOS and MOSfromR calculations, as noted above.

The machine-learning-based part of the model will now be further described. In general, video content can be characterized by its amount of spatial and temporal information. The higher the spatial and temporal information, the higher its complexity. The quantization parameter (QP) chosen to encode a particular video at a given target bit rate depends on spatiotemporal complexity of the particular video, taking into account the content-related properties.

The parametric part of the model accommodates the content-related complexity by using the values of quantization parameter (QP) from the bit stream, which, however, is only available in level <NUM>. When the bit stream is not available (Level <NUM> and <NUM>), estimations of the QP are used.

In addition to QP, other parameters such as average motion or frame sizes can be used to more accurately estimate the complexity of a video. The general idea of the usage of the Random Forest component in the model is to take into account the effect of these different content-related features on user perception of quality and thereby enhance the accuracy of quality prediction.

The proposed Random Forest model estimates a "residuum prediction", that is, the difference between the ground truth data (e.g. a real video quality score obtained from subjective tests) during model training and the prediction of the parametric/deterministic part of the model, which uses only QP or estimated QP and the separate components addressing upscaling and temporal degradation due to the given frame rate. This difference can be explained by the contribution of features to the overall quality score, which are not available in the parametric/deterministic model part. In other words, the machine learning model calculates the prediction error, i.e. residuum, of the deterministic/parametric model. That is the part that may not be properly captured by the deterministic model.

The features available to the Random Forest depend on the chosen level. Different statistical aggregations of the features are then computed and used as the input to the Random Forest model. In addition to the content-related features, the Random Forest model may explicitly take into account the prediction from the parametric part of the model as further input. The final Random Forest-based prediction is the summation of the prediction of the parametric part and the predicted residuum.

The procedure and the parameters of a Level <NUM> quality prediction are now discussed in detail. The model corresponding to Level <NUM> has access to the entire bit stream as input. The deterministic part is computed as follows.

The coding degradation is codec- and bit-depth-dependent. This results in five sets of coefficients, one each for H. <NUM>-8bit, H. <NUM>-10bit, H. <NUM>-8bit, H. <NUM>-10bit and VP9 codecs.

QPmax is <NUM> for H. <NUM>-8bit, H. <NUM>-8bit; <NUM> for H. <NUM>-10bit, H. <NUM>-10bit and <NUM> for VP9.

Since the QP value is available directly in Level <NUM>, there is no need for any pre-computation related to QP. The model part based on parametric/deterministic prediction is realized using the core model as described above.

The machine learning part of the model will now be presented. Any number of features that can be extracted from an encoded bit stream may be used as input to the Random Forest for Level <NUM>, as the model corresponding to this mode has access to the entire bit stream. However, for reasons of computational complexity, such an approach is not reasonable in practice. Hence, an analysis of feature importance may be performed, by which the features with the maximum impact on the residuum quality prediction can be selected as final input features to the model.

Features with high impact on the prediction can be, but are not limited to, the following, representing the preferred embodiment of the disclosure:.

Various statistical aggregation of the features may be computed, including, but not limited to the following:.

The following features and the corresponding aggregations, considered alone or in combination, are preferred in the disclosure.

The depth and number of trees used in the Random Forest may depend on the chosen features and number samples. In this exemplary embodiment of the disclosure, the Random Forest model uses <NUM> trees with a depth of <NUM>. This hyper-parameter combination of the Random Forest is the preferred embodiment of the disclosure.

The more parameters are available to the machine learning model and the better said model is being trained, the better the residuum prediction gets. Thus, a Level <NUM> model having access to all relevant information is able to yield the best results.

The final prediction of quality is the weighted average of the prediction from the parametric part and the random forest part. For the case of Level <NUM>, for example, equal weights can be assigned to both parts in equation (<NUM>): <MAT>.

Here, w<NUM> = <NUM> and w<NUM> = <NUM>. Further, a high weight on the Random Forest part can be justified by the large number of features available compared to lower levels of the model, as indicated in the following.

In addition to the per-segment video quality score, the model also outputs the per-second score. The per-second video quality score according to a preferred embodiment of the disclosure is calculated as follows: <MAT>.

Wherein QPnon-I,per-seg is the average QP of all non-I frames in a segment, QPnon-I,per-sec is the average QP of all non-I frames per second and Q is the per-segment video quality score as described above.

In case a Level <NUM> model is applied, the deterministic model has access to frame sizes, bit rate, resolution, frame rate and codec as input.

The coding degradation is only codec-dependent, as there is no access to the bit depth as input information. This results in three sets of coefficients, one each for H. <NUM> and VP9. Also, the quantization parameter (QP) which is required to calculate the coding degradation is unavailable in this mode. To overcome the unavailability of QP, the value of QP is predicted using the available input parameters. The QP prediction is performed according to equation (<NUM>): <MAT> wherein mean_nonIframeSizes is the average over the sizes of all non-I frames, <MAT> and the coefficients a<NUM>, b<NUM>, c<NUM>, d<NUM> and e<NUM> are codec-dependent.

The maximum QP has <NUM> different values - <NUM> for H. <NUM> and H. <NUM>; <NUM> for VP9. There is no case for <NUM> or <NUM>-bit codec profiles in this mode as there is no access to the bit depth information.

Once the QP is predicted as described above, the coding degradation and other degradations are calculated as described in the core model to obtain the prediction for the parametric/deterministic part of the model.

The following coefficients refer to a preferred embodiment of the disclosure:.

The resolution and temporal upscaling coefficients are the same as for the Level <NUM> case.

The features available to the Random Forest part for this mode are the ones that are described in the parametric part. Instead of just using the mean value of different features as in the parametric part, the Random Forest may use other statistical aggregations to take into account the variation of the features based on the content.

The features used may comprise at least one of.

In addition to the above features, bits per pixel ( <MAT>) and resolution scale factor <MAT> are used as inputs to the random forest.

The above features and the corresponding aggregations is a preferred embodiment of the disclosure.

The QP prediction is performed according to equation (<NUM>) as follows: <MAT>.

As in the Level <NUM> model, the final prediction is the weighted sum of the parametric part output and the Random Forest part output according to equation (<NUM>), for example: <MAT>.

Where w<NUM> = <NUM> and w<NUM> = <NUM>.

In this disclosure, the Random Forest model uses <NUM> trees with a depth of <NUM>. This hyper-parameter combination of the Random Forest is a preferred embodiment of the disclosure. The per-second quality score in this level is the same as the per-segment quality score as described above.

In case of a Level <NUM> model, the computation is performed as described below.

The deterministic model related to Level <NUM> preferably has access to at least one of bit rate, resolution, frame rate, codec and codec profile as input on a per-segment basis.

With respect to coding degradation, like the Level <NUM> model, the model corresponding to Level <NUM> has no access to the QP value. As in the case of Level <NUM>, the value of QP is predicted using the available input parameters. The QP prediction for this mode is done as follows: <MAT>.

The coefficients a<NUM>, b1, c<NUM> and d<NUM> are codec dependent. The maximum QP has <NUM> different values - <NUM> for H. <NUM> and H. <NUM> and <NUM> for VP9. There is no case for <NUM> or <NUM>-bit codec in this mode as there is no access to the bitdepth information.

Once the QP is predicted as described above, the coding degradation and other degradations are calculated as described in the core model to obtain the parametric part related prediction. The following coefficients refer to the preferred embodiment of the disclosure:.

For the Level <NUM> Random Forest model, in addition to the features used in the parametric part of the model, bits per pixel ( <MAT>), resolution scale factor ( <MAT>) and the parametric part output are used as inputs, like in the Level <NUM> model.

In addition to the linear values, the logarithm of each of the features is used as input to the random forest model.

As in the Level <NUM> model, the final prediction is the weighted sum of the parametric part output and the Random Forest part output according to equation (<NUM>): <MAT>.

According to a preferred embodiment of the disclosure, w<NUM> = <NUM> and w<NUM> = <NUM> for Level <NUM> modeling.

In this disclosure, the Random Forest model uses <NUM> trees with a depth of <NUM>. This hyper-parameter combination of the Random Forest is the preferred embodiment of the disclosure. The per-second quality score in this level is the same as the per-segment quality score as described above.

In addition to the model embodiments described above, a variant of the models consists in including the video codec profile as an additional input, if it is available, for all <NUM> Levels of the presented models. This feature can be used as either an additional input to the parametric /deterministic part of the model or as an additional feature in the random forest / machine-learning part of the model.

<FIG> depicts a flowchart of a method of predicting the visually perceived video quality Q of a given video sequence based on the bit stream of said video sequence, and expressed on a video-quality rating scale, according to an embodiment of the disclosure. Therein, a video segment and a corresponding bit stream are obtained in step S101 and the bit stream is parsed in step S102 by a bit stream parser. Furthermore, in step S103 bit stream parameters are obtained. The quality Q is predicted in step S104 by using a deterministic model and a machine learning model with the obtained bit stream parameters, wherein the predicted video quality is a weighted sum of both models calculated in S105.

<FIG> shows a data processing apparatus <NUM> according to an embodiment. The data processing apparatus <NUM> comprises an obtaining unit <NUM> for obtaining a video segment and a corresponding bit stream and a receiving unit <NUM> configured to parse the bit stream by a bit stream parser. A parameter unit <NUM> is configured to obtain bit stream parameters. A prediction unit <NUM> is configured to predict the video quality by using a deterministic model and a machine learning model with the obtained bit stream parameters, wherein the prediction unit <NUM> is configured to calculate the video quality as a weighted sum of both models.

According to the present disclosure, an information adaptive mixed deterministic/machine-learning-based bit stream video-quality model is proposed. By integration of the parametric part of the model with the machine learning part and its specific implementation by addressing the residuum error between deterministic prediction and the ground-truth used during model training and using the machine learning component for predicting that residuum, a greatly enhanced video quality prediction is achieved.

Therein, the same model structure is re-used. Where it is depending on the amount and type of available input information, the deterministic and machine-learning parts are adapted, as well as the weights of the two components.

In Level <NUM> and Level <NUM> models, the quantization parameter is predicted, while for Level <NUM> that feature is available and used directly.

Furthermore, a prediction method for per-second video quality score for the model is presented.

Furthermore, in the claims the word "comprising" does not exclude other elements or steps, and the indefinite article "a" or "an" does not exclude a plurality. A single unit may fulfil the functions of several features recited in the claims. The terms "essentially", "about", "approximately" and the like in connection with an attribute or a value particularly also define exactly the attribute or exactly the value, respectively.

MOSfromR can be expressed as follows:
<IMG>.

Claim 1:
Method for predicting the quality Q of a video bit stream, the method comprising:
Obtaining (S101) a video segment and a corresponding bit stream,
Parsing (S102) the bit stream by a bit stream parser,
Obtaining (S103) bit stream parameters,
Predicting (S104) the quality Q by using a parametric model and a machine learning model with the obtained bit stream parameters, wherein the predicted video quality is a weighted sum of the prediction of both models,
wherein the machine learning model predicts a residuum of the parametric model, the machine learning model being trained on the difference between the prediction of the parametric model and ground truth data used for the training.