<figure><img src="image_1.png" /></figure>

HAL
open science

# Objective No-Reference Error Estimation at the Macroblock Level for Compressed Videos

Melan Vijayaratnam, Didier Nicholson

<figure><img src="image_2.png" /></figure>

## To cite this version:

Melan Vijayaratnam, Didier Nicholson. Objective No-Reference Error Estimation at the Macroblock Level for Compressed Videos. 2025. hal-05421509

HAL Id: hal-05421509

https://hal.science/hal-05421509v1

Preprint submitted on 22 Dec 2025

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

Public Domain
# Objective No-Reference Error Estimation at the Macroblock Level for Compressed Videos

Melan Vijayaratnam*, Didier Nicholson*

*EKTACOM

19 Avenue des Indes

Les Ulis, 91940 France

{mvijayaratnam,dnicholson}@ektacom.com

## Abstract

The accelerating pace of video capture across modern devices is producing an unprecedented volume of visual data every day. As a result, there is a growing need for methods that can objectively analyze errors introduced during compression or manipulation. While many recent approaches rely on learning-based models trained for subjective quality prediction, we revisit a signal-based strategy grounded in the statistical distribution of DCT coefficients. Such approaches remain crucial to provide objective and reproducible evidence of how the signal evolves under compression, enabling the analysis of degradation mechanisms at their origin that can serve forensic applications. Our method estimates error without requiring a reference video and allows fine-grained analysis down to the macroblock level in H.264 content, making it possible to identify localized regions within a frame where distortions occur. We demonstrate, both quantitatively and qualitatively, the reliability of the proposed approach as well as its limitations under different quantization parameters across diverse videos, highlighting its potential use in forensic scenarios.

## Introduction

The explosive growth of online video content has made it one of the most dominant forms of digital media, driving the need for advanced methods of efficient indexing, retrieval, and understanding. In many real-world scenarios such as streaming, social media uploads, or video surveillance, the original uncompressed video is not available [1]. These videos are typically compressed using codecs, notably H.264 [2, 3], which inevitably introduce distortions in the signal. As a result, methods that can directly analyze compressed-domain features without requiring a reference video are of particular interest. Examples include the use of macroblock motion vectors [4] or the statistical behavior of transform coefficients.

Early efforts approached this problem from a statistical standpoint by analyzing the distribution of compressed Discrete Cosine Transform (DCT) coefficients. However, with the rise of deep learning [5], statistical approaches have often been replaced by data-driven models [6]. Nevertheless, signal-based approaches remain highly relevant, since they provide a direct understanding of how compression artifacts arise in the compressed domain. Beyond their scientific interest, such methods offer interpretable and reproducible evidence of degradation mechanisms, which is particularly valuable in sensitive contexts such as forensic analysis or judicial investigations [7], where the ability to trace and explain errors in the signal is essential.
While learning-based approaches focus on predicting visual perception in the spatial domain of user-generated content (UGC), traditional signal-based methods address the frequency domain, at the very level where degradation originates from quantization. In this paper, we revisit no-reference objective error estimation for compressed UGC content, with a focus on forensic applications where understanding the error introduced by compression is critical. More precisely, we:

* refine error estimation by modeling the distribution of both zero and non-zero DCT coefficients,

* provide accurate error estimation at a fine-grained level, down to the macroblock level.

# 1 Related Work

Early works on no-reference error estimation mainly focus on studying the behavior of the DCT coefficients in the compressed domain. Turaga *et al.* [8] as well as Ichi-gaya *et al.* [9] introduce methods for estimating Peak-to-Signal Noise Ratio (PSNR) using quantized coefficients from MPEG-2 streams. For the later introduced H.264 codec, Altunbasak *et al.* [10] propose a Cauchy model distribution for the DCT coefficients of the Intra (I) frames and Brandao *et al.* [11] extend it by using a Laplace modeling of the DCT coefficients for the predictive (P and B) frames for blind error estimation. These approaches were only tested on QCIF and CIF resolution videos, that are videos of small resolutions and based on the estimation on the full sequence of the video, while we want a fine-grained and accurate error information on modern high-resolution videos.

In recent years, research on signal-domain error estimation has been largely replaced by learning-based approaches aimed at predicting subjective visual scores. Min *et al.* [12] describe the general framework of no-reference perceptual prediction. They train a deep neural network model and subsequently use feature fusion and regression to predict a quality score. These quality scores are learned from a dataset where users give a subjective score on a video. Barman *et al.* [13] and Sun *et al.* [14] both present a model to predict subjective quality scores from a given video. The drawback of such approaches is that they operate at the scale of a frame or a video, and reducing the scale to a macroblock is difficult to be assessed in a subjective manner.

By contrast, our objective is not to predict perceptual quality, but to analyze the error directly at the signal level without reference. This distinction is especially important in forensic contexts, where compression traces and degradation patterns have been used to detect manipulations, identify recompression, and authenticate digital evidence [15, 16, 7]. Our work builds on this line of research, refining error estimation models down to the macroblock level for applicability in forensic investigations.
## 2 Methodology

Starting from the model [10] of error estimation of compressed content of Intra frames from H.264 content, the DCT coefficients are modelized using a Cauchy distribution and the error is induced as the following:

$$ D^{\text{zero}}(Q, \rho) = 2 \sum_{i=1}^{M} \frac{\mu Q}{\pi} - \frac{i\mu Q}{\pi} \ln \left( \frac{\mu^2 + \left(i + \frac{1}{2}\right) Q^2}{\mu^2 + \left(i - \frac{1}{2}\right)^2 Q^2} \right) \\ - 2 \sum_{i=1}^{M} \frac{\mu^2 - i^2 Q^2}{\pi} \tan^{-1} \left( \frac{\mu Q}{\mu^2 + \left(i^2 - \frac{1}{4}\right)} \right) \\ + \left[ \frac{\mu Q}{\pi} - \frac{2\mu^2}{\pi} \tan^{-1} \left( \frac{Q}{2\mu} \right) \right], \quad (1) $$

where $\mu$ is the parameter of the Cauchy distribution and $Q$ the quantization step and $M = Q^2/4$.

$$ \mu = \frac{Q}{2 \tan \left( \frac{\pi \rho}{2} \right)}. \quad (2) $$

The parameter $\mu$ depends on $\rho$ which is essential and needs further explanation. We denote $N_{QP=4}^{\text{nonzero}}$ the number of non-zero transform coefficients after quantization with $QP = 4$, which corresponds to a quantization step $Q = 1$, with the following:

$$ Q = \left\lfloor 2^{\frac{QP-4}{6}} \right\rfloor, \quad (3) $$

and $N_{QP=4 \to QP=x}^{\text{nonzero}}$ represents the number of those same coefficients that remain non-zero after quantization with a higher QP value $x$. Then, we define the coefficient $\rho$ as such:

$$ \rho = \frac{N_{QP=4 \to QP=x}^{\text{nonzero}}}{N_{QP=4}^{\text{nonzero}}}. \quad (4) $$

Here, $N_{QP=4 \to QP=x}^{\text{nonzero}}$ does not imply a re-encoding of a $QP = 4$ bitstream. Instead, the reference set of non-zero coefficients is obtained by quantizing the original transform coefficients with $QP = 4$, and we then check how many of these same coefficients remain non-zero when quantized with a higher QP value $x$. In other words, both counts are derived directly from the original transform coefficients, with different quantization parameters, rather than by sequential re-encoding.

However, this definition requires access to the reference video to obtain the coefficients at $QP = 4$. In practical applications, such as forensic analysis, the original video is typically unavailable. To address this limitation, we propose the following proxy measurement $\tilde{\rho}$, which bypasses the need for the original video:

$$ \tilde{\rho} = \frac{N_{QP=x}^{\text{zero}}}{N_{QP=x}^{\text{total}}}, \quad (5) $$
which makes it applicable in a real case scenario where only the pre-encoded bitstream is available. These contributions emphasizes the zero coefficients in the DCT coefficients. To account for the non zero coefficients quantization error, we approximated it by the quantization error of a uniform distribution [17]:

$$D^{\text{non zero}}(Q) = \frac{Q^2}{12} \qquad (6)$$

that we regulate with a factor $(1 - \tilde{\rho})$ and thus the total error $MSE_{\text{Tot}}$ becomes:

$$MSE_{\text{Tot}}(Q, \tilde{\rho}) = \tilde{\rho} \cdot D^{\text{zero}}(Q, \tilde{\rho}) + (1 - \tilde{\rho}) \cdot D^{\text{non zero}}(Q) \qquad (7)$$

This error can be computed at the frame level, or at the macroblock level. We can compute the PSNR [18] from each macroblock (MB) and compare it to the PSNR at the frame (F):

$$\text{PSNR}_{\text{F}} = \phi \left( \frac{1}{N_{\text{MB}}} \sum_{j=1}^{N_{\text{MB}}} \text{MSE}_{\text{MB},j} \right) \qquad (8)$$

with $N_{\text{MB}}$ corresponding to the total number of macroblock in the frame and $\phi(\cdot)$ corresponds to the conventional PSNR definition: $\phi(\text{MSE}) = 20 \cdot \log_{10} \left( \frac{255}{\sqrt{\text{MSE}}} \right)$.

# 3 Experimental results

## 3.1 Test conditions

We use the Youtube UGC dataset intended for video compression research (YouTube-UGC) [19]. YouTube-UGC is an authentic database which includes test video sequences with diverse authentic distortions. It has a rich collection of contents such as gaming, lecture, sports. We use 10 video from 10 diverse categories at 1280 × 720 image size resulting in a total of 100 videos, explicit in Table 1 alongside the number of frames from each class. Each of the video are either 19 or 20 seconds clips at diverse framerate. The estimated error using the true value of $\rho$ is computed, according to previous research at the frame level. We include observation of estimated error when using the approximate $\tilde{\rho}$ at the frame level and at the macroblock level, then with the contribution of $D^{\text{non zero}}(Q)$. Since the videos are available in their raw format, it allows to have control on the compression to be applied. We use different quantization parameters $QP \in \{16, 22, 27, 32, 37, 42\}$ which correspond to quantization steps $Q \in \{4, 8, 14, 25, 45, 80\}$ with higher quantization implying higher artifacts of compression. All macroblocks within the image are forced to be of the same value to evaluate the prediction results per specific quantization parameter. All frames are intra coded and use the JM implementation of H.264 [20]. The modified JM version of VQEG [21] allows to extract the DCT coefficients directly per macroblock that are essential to compute the $\rho$ for the $\mu$ parameter of the Cauchy distribution.
<table><caption>Table 1: Youtube-UGC chosen test sequences.</caption><thead><tr><td>Class Sequences</td><td>Number of frames</td></tr></thead><tbody><tr><td>Animation</td><td>4857</td></tr><tr><td>CoverSong</td><td>5399</td></tr><tr><td>Gaming</td><td>7602</td></tr><tr><td>HowTo</td><td>6176</td></tr><tr><td>Lecture</td><td>5173</td></tr><tr><td>LiveMusic</td><td>6077</td></tr><tr><td>MusicVideo</td><td>6435</td></tr><tr><td>NewsClip</td><td>5396</td></tr><tr><td>Sports</td><td>6685</td></tr><tr><td>Vlog</td><td>5685</td></tr></tbody></table>

## 3.2 Local error estimation

We evaluate the quality of the videos specified in Table 1 and report the results in Table 2 averaged across all frames in the sequences of all videos. We report the true PSNR, meaning the true degradation when comparing the degraded frame to the original uncompressed version. $D^{zero}(Q, \rho)$ corresponds to the proposed model [8] at the frame level. Then we also display $D^{zero}(Q, \tilde{\rho})$ as the frame and macroblock level, with the approximation of $\rho$ when the raw video is not available and thus $\rho$ can not be computed. And finally we evaluate the contribution of the zero coefficients from $D^{zero}(Q, \tilde{\rho})$ and the non zero coefficients from $D^{non zero}(Q)$ resulting in MSE<sub>Tot</sub>(Q, $\tilde{\rho}$) at the frame and local, macroblock level. The errors displayed in Table 2 are displayed using the PSNR representation. Alongside this table we display the MSE of the values in Figure 1 of the average across the first 400 frames of each video in terms of mean squared error directly. We observe the predicted MSE to be close to the ground truth when looking at the PSNR, even at the macroblock level. This is an interesting finding as it allows to have a more fine-grained investigation of the error according to small area of the image, as opposed to having a single value for the full image, as a 720P video is constituted of 3600 16×16 macroblocks. The contribution of non zero coefficients from $D^{non zero}(Q)$ brings the results closer to the ground truth. Furthermore Figure 1 shows that the trend of the predicted curve of MSE<sub>Tot</sub>(Q, $\tilde{\rho}$) is closely related to the true MSE to be predicted. With the different quantization parameters, we see the offset between the prediction and the ground truth being higher as QPs grows, as the prediction is more complex to predict.

## 3.3 Qualitative analysis

With this approach, it is now possible to estimate the error at a local level. Figure 2 illustrates the error estimation using the signal approach and the colormap associated. We use two sequences from the same Youtube-UGC dataset and encode them using the x264 codec from ffmpeg. We set the group of picture size, the number of frames
<table><caption>Table 2: Quantitative results using different models.</caption><thead><tr><th>QP</th><th>Q</th><th>True PSNR</th><th>φ(D<sup>zero</sup>(Q, ρ))</th><th>φ(D<sup>zero</sup>(Q, ρ̂))<br>(frame level)</th><th>φ(MSE<sub>Tot</sub>(Q, ρ̂))<br>(frame level)</th><th>φ(D<sup>zero</sup>(Q, ρ̂))<br>(MB level)</th><th>φ(MSE<sub>Tot</sub>(Q, ρ̂))<br>(MB level)</th></tr></thead><tbody><tr><td>16</td><td>4</td><td>45.19</td><td>47.81</td><td>49.23</td><td>47.53</td><td>50.04</td><td>48.06</td></tr><tr><td>22</td><td>8</td><td>41.14</td><td>42.20</td><td>40.42</td><td>39.12</td><td>41.01</td><td>39.56</td></tr><tr><td>27</td><td>14</td><td>37.39</td><td>38.35</td><td>39.00</td><td>37.73</td><td>39.68</td><td>38.22</td></tr><tr><td>32</td><td>25</td><td>34.53</td><td>34.80</td><td>37.13</td><td>35.97</td><td>37.63</td><td>36.34</td></tr><tr><td>37</td><td>45</td><td>32.47</td><td>31.59</td><td>34.23</td><td>33.17</td><td>34.60</td><td>33.45</td></tr><tr><td>42</td><td>80</td><td>30.16</td><td>29.26</td><td>32.26</td><td>31.26</td><td>32.49</td><td>31.44</td></tr></tbody></table>

<figure><img src="image_3.png" /><figcaption>Figure 1: Framewise analysis on blind MSE estimation at multiple degradation level.</figcaption></figure>

between two successive Intra frames to be 10 and assign set the bitrate to 40 Mbit/s for the sequence Sports_720P-5ea4 that has a framerate of 50 frames per second and select frame number 100 and 1 Mbit/s for the sequence Sports_720P-5e39 that has a
framerate of 30 frames per second and select frame number 100 as well. We apply
on purpose high compression to highlight the different compression zone as in this
case the QP per macroblock is not uniform across the full image. The optimal QP
is found by the rate-distortion optimization algorithm. We can see in Figure 2 in
warmer colors areas where the distortion is higher and thus more degraded areas and
in colder colors (blue) areas where the distortion is lower and closer to the original
ground truth area.

<figure><img src="image_4.png" /><figcaption>Figure 2: Overview of the blind error estimation at the macroblock level using the x264 codec with the fast preset for (a) in Sports_720-5ea4 and (b) Sports_720P-5e39.</figcaption></figure>

3.4 Correctness of the approach

The results shown in Table 2 at the macroblock level are averaged across the full
image to be compared with the frame level approach. However, to see the correctness
of the approach, we want to observe how the values spread over each macroblock. We
display the standard deviation over the 400 first frames of the sequences in Figure 3 for
QP ∈ {22, 27, 32}. We observe that for low quantization parameter values, that the
macroblock level is more accurate than the frame level approach and the standard
deviation Dzero(Q, ̄ρ). Indeed, the frame level approaches use more parameter to
estimate $\rho$ and $\tilde{\rho}$ but the prediction for each macroblock will be the same across the frame, whereas the prediction of $D^{\text{zero}}(Q, \tilde{\rho})$ at the macroblock will use fewer DCT coefficients to estimate $\tilde{\rho}$ but the error prediction will account for the specific macroblock. Starting from QP=32, we observe the macroblock level estimation starts to be less accurate than the approach accounting only for zeros. This is due to the fact at higher compression, there are indeed more zero coefficients and at the macroblock level, there are mostly null coefficients and $\tilde{\rho}$ tends to be close to 1 and so the prediction in those particularly macroblocks will have a high error. We observe as well overall that the standard deviation increases at higher quantization parameter as the estimation becomes harder to predict.

<figure><img src="image_5.png" /><figcaption>Figure 3: Spread of the error prediction</figcaption></figure>

# 4 Conclusion

In this work, we addressed blind video objective quality estimation for compressed content using signal analysis of DCT coefficients, in contrast to the current trend of subjective, learning-based approaches. We focused exclusively on H.264 videos due to the targeted use-case of forensic analysis on real-world data that still use on the vast majority H.264 encoded videos.
Nevertheless, the approach can naturally be extended to newer standards such as H.265/HEVC and H.266/VVC as these more recent codecs are sharing many core coding tools, and can been seen as enhancement to H.264. In HEVC, the macroblock structure is replaced by larger coding tree units (CTUs), while VVC introduces further partitioning strategies with multi-type tree partitioning. However, in both cases the underlying statistical distribution of DCT coefficients remains the same, which makes the proposed methodology directly applicable.

The objective approach allows having an error that is true to its signal, and we believe it will be essential with the abundance of videos in the traffic. A local level, here at the macroblock level with H.264, is the closer level to where the errors were generated. It allows to understand the zones that can be used for analysis or even for post-processing algorithms or being used within video decoding algorithm. Most importantly, it allows to understand the degradation happening due to compression.

As future work, we plan to extend the method to predictive P/B frames and to explore hybrid approaches that combine this signal analysis with deep learning methods.

# 5 Acknowledgments

This work is supported by The French Research Funding Agency (ANR) under project IMPROVED ANR-22-CE39-0006.

# References

[1] J. Søgaard, S. Forchhammer, and J. Korhonen, "No-reference video quality assessment using codec analysis," *IEEE Transactions on circuits and Systems for Video Technology*, vol. 25, no. 10, pp. 1637-1650, 2015.

[2] R. Pereira and E. Pereira, "Video streaming: H. 264 and the internet of things," in *2015 IEEE 29th International Conference on Advanced Information Networking and Applications Workshops*. IEEE, 2015, pp. 711-714.

[3] I. E. Richardson, *The H. 264 advanced video compression standard*. John Wiley & Sons, 2011.

[4] M. Leny, D. Nicholson, and F. Preteux, "De l'estimation de mouvement pour l'analyse temps réel de vidéos dans le domaine compressé," *Motion estimation for the real-time analysis of videos in the compressed domain*, GRETSI, 2007.

[5] S. Dong, P. Wang, and K. Abbas, "A survey on deep learning and its applications," *Computer Science Review*, vol. 40, p. 100379, 2021.

[6] Q. Zheng, Y. Fan, L. Huang, T. Zhu, J. Liu, Z. Hao, S. Xing, C.-J. Chen, X. Min, A. C. Bovik et al., "Video quality assessment: A comprehensive survey," *arXiv preprint arXiv:2412.04508*, 2024.

[7] M. Hafner, A. Radovic, M. Langer, S. Findenig, and A. Uhl, "Forensic recognition of codec-specific image compression artefacts," in *Proceedings of the 2024 ACM Workshop on Information Hiding and Multimedia Security*, 2024, pp. 131-136.

[8] D. S. Turaga, Y. Chen, and J. Caviedes, "No reference psnr estimation for compressed pictures," *Signal Processing: Image Communication*, vol. 19, no. 2, pp. 173-184, 2004.

[9] A. Ichigaya, M. Kurozumi, N. Hara, Y. Nishida, and E. Nakasu, "A method of estimating coding psnr using quantized dct coefficients," *IEEE Transactions on circuits and systems for video technology*, vol. 16, no. 2, pp. 251-259, 2006.
[10] Y. Altunbasak and N. Kamaci, "An analysis of the dct coefficient distribution with the H.264 video coder," in *2004 IEEE International conference on acoustics, speech, and signal processing*, vol. 3. IEEE, 2004, pp. iii-177.

[11] T. Brandao and M. P. Queluz, "No-reference psnr estimation algorithm for H. 264 encoded video sequences," in *2008 16th European Signal Processing Conference*. IEEE, 2008, pp. 1-5.

[12] X. Min, H. Duan, W. Sun, Y. Zhu, and G. Zhai, "Perceptual video quality assessment: A survey," *Science China Information Sciences*, vol. 67, no. 11, p. 211301, 2024.

[13] N. Barman, E. Jammeh, S. A. Ghorashi, and M. G. Martini, "No-reference video quality estimation based on machine learning for passive gaming video streaming applications," *IEEE Access*, vol. 7, pp. 74511-74527, 2019.

[14] W. Sun, T. Wang, X. Min, F. Yi, and G. Zhai, "Deep learning based full-reference and no-reference quality assessment models for compressed ugc videos," in *2021 IEEE International Conference on Multimedia & Expo Workshops (ICMEW)*. IEEE, 2021, pp. 1-6.

[15] V. L. Thing, Y. Chen, and C. Cheh, "An improved double compression detection method for jpeg image forensics," in *2012 IEEE International Symposium on Multimedia*. IEEE, 2012, pp. 290-297.

[16] Y. Li, M. Gardella, Q. Bammey, T. Nikoukhah, J.-M. Morel, M. Colom, and R. G. Von Gioi, "A contrario detection of h. 264 video double compression," in *2023 IEEE International Conference on Image Processing (ICIP)*. IEEE, 2023, pp. 1765-1769.

[17] J. Rosenblatt and M. K. Roychowdhury, "Uniform distributions on curves and quantization," *arXiv preprint arXiv:1809.08364*, 2018.

[18] G. Sullivan and K. Minoo, "Objective quality metric and alternative methods for measuring coding efficiency," in *document JCTVC-H0012, ITU-T/ISO/IEC Joint Collaborative Team on Video Coding (JCT-VC), 8th Meeting: San Jose, CA, USA, 2012, pp. 1-10.*

[19] Y. Wang, S. Inguva, and B. Adsumilli, "Youtube ugc dataset for video compression research," in *2019 IEEE 21st international workshop on multimedia signal processing (MMSP)*. IEEE, 2019, pp. 1-5.

[20] J. V. T. J. of ISO/IEC MPEG and I.-T. VCEG, "JM Reference Software," https://iphome.hhi.de/suehring/tml/, version JM 19.0, accessed June 2025.

[21] K. Brunnstrom, D. Hands, F. Speranza, and A. Webster, "Vqeg validation and itu standardization of objective perceptual video quality metrics [standards in a nutshell]," *IEEE Signal processing magazine*, vol. 26, no. 3, pp. 96-101, 2009.

**MELAN VIJAYARATNAM** is a Research Engineer at EKTACOM, where he works on AI solutions for compressed video content. He earned his PhD from Telecom Paris, Institut Polytechnique de Paris, where his research focused on temporal extrapolation for zero-latency video transmission, including contributions to ICIP, VCIP, and EUVIP conferences.

**DIDIER NICHOLSON** is the Research and Technology Director of EKTA-COM. His work focuses on image and video coding, transmission and processing. He has supervised one PhD student, was a jury member of more than 20 PhD thesis, authored more than 30 publications, led multiple national and international projects as well as has been participated actively in standardisation (JPEG and MPEG) for more than 25 years.