# Joint multi-profile video coding for cost optimization of standard ABR streaming

Reda Kaafarani, Julien Le Tanou, Michael Ropert, Thomas Maugey, Aline Roumy

<figure><img src="image_1.png" /></figure>

## To cite this version:

Reda Kaafarani, Julien Le Tanou, Michael Ropert, Thomas Maugey, Aline Roumy. Joint multi-profile video coding for cost optimization of standard ABR streaming. MHV 2025 - 4th ACM Mile-High Video Conference, Feb 2025, Denver (CO), United States. pp.33 - 39, 10.1145/3715675.3715795. hal-05291651

HAL Id: hal-05291651

https://inria.hal.science/hal-05291651v1

Submitted on 1 Oct 2025

HAL is a multi-disciplinary open access archive for the deposit and dissemination of scientific research documents, whether they are published or not. The documents may come from teaching and research institutions in France or abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est destinée au dépôt et à la diffusion de documents scientifiques de niveau recherche, publiés ou non, émanant des établissements d'enseignement et de recherche français ou étrangers, des laboratoires publics ou privés.

<figure><img src="image_2.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
# Joint multi-profile video coding for cost optimization of standard ABR streaming

Reda Kaafarani, Julien Le Tanou, Michael Ropert
MediaKind
Cesson-Sévigné, France
{firtdname.lastname}@mediakind.com

Thomas Maugey, Aline Roumy
INRIA
Rennes, France
{firtdname.lastname}@inria.fr

## Abstract

This paper addresses the problematic of multi-profile encoding and delivery system optimization for the purpose of standard HTTP-based Adaptive Bitrate (ABR) video streaming. Such delivery system must process, encode, and usually store the same content in different (bitrate, resolution) pairs, which defines a set of encoding profiles or coded representations to serve and adapt the video content to various end-user bandwidth requirements and device capabilities. The presented research work specifically targets for such a system to improve the trade-off between the storage bit-cost of the different representations, the transcoding complexity and transmission efficiency (i.e. bitrate-quality trade-off at transmission) of the requested representation by the end-client while guaranteeing that the delivered output bitstream remains compliant with the standard decoding system available at the client. For that purpose, a joint multi-profile coding format with corresponding fast transcoding method is proposed and assessed against State-of-the Art methods. The coding format relies on the concepts of predictive residual coding across video representations for storage saving and of partial decoding for light standard stream regeneration on client-request which induce no loss in quality in comparison to independent coding of the different profiles.

## CCS Concepts

* Information systems → Multimedia streaming; Multimedia streaming; * Mathematics of computing → Coding theory.

## Keywords

ABR delivery, multi-profile video coding, predictive residual coding, transcoding complexity, storage cost optimization

## ACM Reference Format:

Reda Kaafarani, Julien Le Tanou, Michael Ropert and Thomas Maugey, Aline Roumy. 2025. Joint multi-profile video coding for cost optimization of standard ABR streaming. In *Proceedings of Mile-High Video Conference (MHV'25)*. ACM, New York, NY, USA, 7 pages. https://doi.org/10.1145/3715675.3715795

## 1 Introduction

Video streaming services heavily rely on HTTP-based Adaptive Bitrate (ABR) streaming technologies, such as Dynamic Adaptive Streaming over HTTP [1] or HTTP Live Streaming (HLS) [2], to serve video content to varying end-user device capabilities and network conditions. To adapt to the end-client request, an ABR delivery system must commonly process, encode, and store the same video content in different resolution and bit-rate pairs, which define a set of encoding profiles or coded representations (commonly referred to as bitrate ladder). As a first path of system optimization the bitrate ladder can be optimized per content, using traditional [3, 4, 5] or machine learning [6, 7, 8] approaches, and with knowledge of the video coding standard or codec efficiency in use [9]. Complementary, and by considering the signal redundancy between the representations, we investigate and introduce novel joint coding formats and transcoding methods to optimize the trade-off between storage cost, transcoding complexity, and transmission cost of those representations in a multi-profile video coding system for standard ABR streaming. The two most common approaches for multi-profile video delivery are Simulcast (SC) and Full Transcoding (FT) which set two extremes in terms of optimization criteria. Simulcast offers the lowest transcoding complexity (i.e. highest scalability to profile requests) and best transmission efficiency but requires the largest amount of storage, while FT has the lowest storage cost but the highest transcoding cost and transmission bitrate overhead. As a third, in-between approach, we investigate methods that seek to optimize a better trade-off between storage, transcoding, and transmission costs, with similar motivations than a past MPEG initiative formalized by a Call for Evidence (CfE) on Transcoding for Network Distributed Video Coding (NDVC) [10]. The CfE aimed to proposed methods that lower the storage cost of SC while improving transmission efficiency (i.e. reducing bitrate overhead for same quality) of the delivered streams against FT or its transcoding complexity. It includes the additional constraint to keep the final served bitstream compliant with any standard decoding system. In that context, we specifically focus on the best-in-class method from the State-of-the Art (SOTA): the so-called Guided Transcoding using Deflation and Inflation (GTDI) method [11]. The GTDI method is based on predictive residual coding across profiles (or equivalently layers) to reduce storage requirements of low-quality representations. The method preserves the best transmission efficiency of SC and reduces its storage cost but still induces transcoding complexity to serve a dependent representation which requires the full decoding of the representations involved in an inter-layer residual prediction process. As a new trade-off, we propose and discuss a novel predictive residual coding scheme based on partial decoding for standard stream re-generation which significantly lowers the

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from permissions@acm.org.

MHV'25, Denver, CO, USA

© 2025 Copyright held by the owner/author(s). Publication rights licensed to ACM.
ACM ISBN 979-8-4007-1488-7/25/02
https://doi.org/10.1145/3715675.3715795
<figure><img src="image_3.png" /><figcaption>**Figure 1:** Overview of an ABR video streaming service – main involved processes (green) and associated costs under consideration in the context of this paper (red)</figcaption></figure>

transcoding complexity of the GTDI approach by sacrificing on storage or transmission cost. In addition, we propose novel optimization techniques which improve storage saving of any method based on predictive residual coding. Typically, for the GTDI method, it can further improve the storage saving by -10% on average with a negligible transmission bitrate overhead. For sake of clarity, it is pointed out that scalable video coding techniques [12, 13] are not considered in this work since our motivations (as for the CfE [10]) are to deliver a standard non-scalable stream and to guarantee the lowest bitrate overhead at transmission for the highest quality representation (i.e. nominal or reference bitstream) – representation which is expected to be the most served profile and to impact the CDN costs the most.

## 2 Overview of an ABR video delivery system and associated costs for optimization

An ABR video streaming service, Live or On-Demand, is composed of 3 main parts: Encoding, Content management and Delivery, for which an overview is given Figure 1 with the involved processes (green) and the associated costs (dashed red) which are considered for optimization in the context of this paper.

**Encoding** – as introduced earlier the first step of the streaming system consists in encoding the input video in one or multiple representations with different (resolution, bitrate) pairs according to a pre-defined video bitrate ladder and coding strategy. Therefore, the cost associated with the Encoding directly depends on the processing complexity of the bitrate ladder and the codec that are in use – it defines the Encoding Complexity as a first possible system optimization criterion. Encoding Complexity reduction of multiple profiles has been extensively studied [14, 15, 16, 17] and is not of focus in this paper.

**Content management** – at this stage the compressed streams are either packaged for direct publishing through the origin server (e.g. for Live video delivery) or re-purposed (e.g. transcode in a proprietary format for storage optimization) and stored for later playback (e.g. for On-Demand delivery). For the latter case, on a client request the corresponding stored representation must be inverse repurposed to a standard format for delivery. This system part induces two costs or criteria to optimize: the Storage Bit-cost and the Transcoding Complexity.

**Delivery** – in the last stage of the system the encoded and packaged videos are received from the origin server and published on

the Content Delivery Network (CDN) [18] which optimizes the distribution of the contents to end users from different geographical regions. The CDN costs are directly dependent to the bitrate of the content to deliver which is itself correlated to the Rate-Distortion (R-D) performance achieved by the coding strategy used for delivery. The R-D performance criteria at transmission is defined as Transmission Efficiency.

In this paper we investigate and focused on joint-coding schemes (e.g. at an origin server) and associated lightweight transcoding approach (e.g. at the edge) for balancing the trade-off between the three criteria: storage bit-cost, transcoding complexity, and transmission efficiency. The joint optimization of storage, transcoding and transmission costs for ABR streaming have been modeled and formalized in several works [19, 20], resulting in various delivery policies for edge caching or computing which include transcoding strategies, such as Full or Guided transcoding, discussed in next section.

## 3 Comparative analysis of state-of-the-art methods

In this section, we review state of the art methods for multi-profile video coding and delivery and provide a qualitative comparison of their performance.

### 3.1 Simulcast

Simulcast (SC) [10] offers no transcoding complexity and optimal transmission efficiency in return of high storage bit-cost, simply by encoding independently all the bitrate ladder representations of a video upfront. These independent video streams are then stored directly with no re-purposing (which gets reduced to a pass-through as in Figure 1) such that no extra processing is needed once a representation is requested. SC sets the norm for the optimal video quality since there is no modification of the independent encoding process for each profile which leads to no bitrate overhead or quality degradation at transmission.

### 3.2 Full Transcoding

In contrast to SC, the Full Transcoding (FT) [10] technique offers maximal storage savings at the cost of very high transcoding complexity and lowest transmission efficiency. This strategy works by encoding and storing only the highest quality (HQ) representation of the video. By doing so, the storage requirements for this method are heavily reduced. However, once a user requests a representation of the video that is different from the HQ one, a full transcoding process (inverse re-purposing) must be performed in which the HQ representation is decoded (optionally downsampled), and then re-encoded at the requested bitrate. This process is complex and requires costly computing power. In addition, and since the requested representation is a re-encoding of an already degraded video signal, the transmission efficiency of FT is sub-optimal. Either the quality of the dependent profiles would be degraded, or it would require targeting higher bitrate at encoding for the HQ profile to compensate for the quality degradation resulting in significant transmission bitrate overhead. Several additional research works in Academic literature investigated alternative strategies to Simulcast and Full transcoding, as developed next sections.
<figure><img src="image_4.png" /><figcaption>Figure 2: GTDI or PRC-Full-PTQ method; showing deflated stream generation on the left and standard stream re-generation on the right. It uses reconstructed pixel samples from a reference high quality video to generate the residual predictor of the dependent lower quality videos.</figcaption></figure>

## 3.3 Guided Transcoding

A Guided Transcoding (GT) approach was first proposed in [21] which aims to reduce the transcoding complexity of FT while still maintaining storage savings in comparison to SC. Similar to FT, it encodes a HQ representation and stores it as is. For the lower quality (LQ) encodings, the HQ representation is decoded, downsized, and then encoded at the required resolution/rate. However, the LQ streams are fully stripped down from their transform coefficients before storage. Consequently, all the decisions of the encoder for the LQ streams are saved in what is called a Control Stream (CS). When a LQ representation is requested for delivery, the HQ representation is decoded, downsized, and then re-encoded by guiding the encoding process using the corresponding CS. Since the CS contains all the decisions needed for the encoder, complex R-D search operations for mode decisions can be skipped and the re-encoding process is reduced to the regeneration and entropy coding of the transformed coefficients. Another variant of GT was then proposed in [22] to further reduce its transcoding complexity. In this variant, not all the transform coefficients of the LQ streams are omitted but a fraction of them, which belong to pictures assigned to lower temporal layers in a dyadic hierarchical B picture prediction structure. The idea is that transform coefficients of pictures assigned to higher temporal layers usually have lower residual energy than those in lower layers and won't contribute much to the storage savings if omitted. Consequently, keeping them in the stream wouldn't require re-generation of these coefficients and thus, decreases the transcoding complexity for a small storage penalty. The method is flexible allowing the control of the number of layers for which the coefficients of pictures are removed. This offers a trade-off between storage savings and transcoding complexity.

The GT techniques achieve a good trade-off between storage savings and transcoding complexity. However, they achieve the same non-optimal transmission efficiency of FT resulting in significant bitrate overhead or quality degradation at transmission. Consequently, in this work, the GT schemes of [21, 22] aren't considered for a full objective comparison.

## 3.4 Guided Transcoding using Deflation and Inflation

Finally, a Guided Transcoding using Deflation and Inflation (GTDI) method has been proposed in [11]. The GTDI strategy specifically targets to reduce storage cost of SC with lower transcoding complexity than FT under the constraint of having the same transmission efficiency of SC. For that purpose, and despite not being formally defined as such, the scheme introduces the concept of Predictive Residual Coding (PRC) with Full decoding using spatial pixel domain reference samples (PRC-Full-PTQ). For the rest of the paper, and to better highlight the commonalities and differences with the new proposed method and optimizations in our work we renamed GTDI as PRC-Full-PTQ. The scheme is depicted in Figure 2 with new functionalities (in comparison to any standard hybrid coding scheme as specified in H.264/AVC, HEVC, VVC or AV1, etc.) to perform the prediction and differential coding of the residual shown in red. It shows the principles of deflation and inflation for an example of two layers (i.e. representations): a reference layer video $V_0$ and a dependent one $V_1$ where $V_0$ is the video representation of highest quality and resolution, and $V_1$ can be a representation of any quality and/or resolution lower than $V_0$.

The method starts by applying a deflation (re-purposing) process on LQ dependent videos before storage as follows. $V_0$ is normally encoded to generate a standard stream $S_0$. $V_1$ is normally encoded up to the point of entropy encoding that would have produced a standard stream $S_1$. The reconstructed images $\tilde{I}_0$ of $V_0$ are (optionally) downsized into $\tilde{I}_{01}$ to match the resolution of $V_1$. The prediction $p_1$ resulting from the encoding of $V_1$ is subtracted from $\tilde{I}_{01}$ to form an approximate residual $\epsilon_{01}$ of $\epsilon_1$. The approximate residual $\epsilon_{01}$ is then transformed and quantized using the quantization parameter of $V_1$ to get $q_{01}$. A difference between $q_{01}$ and the original residual $q_1$ is then calculated to get $\Delta q = q_1 - q_{01}$ which is the delta residual to be entropy coded. The delta residual and encoder decisions (modes, motion data etc.) of $V_1$ are entropy encoded to form a non-standard (i.e. deflated) stream that is called $\Delta S_1$.

When a user requests a LQ stream that is represented by a dependent stream $\Delta S_1$ in the scheme, the inverse of the repurposing process (inflation) must be invoked. Such, $S_0$ is fully decoded to get
back the images $\tilde{I}_0$ which are required to generate back the residual $\epsilon_{01}$ used for prediction. The residual $\epsilon_{01}$ is further transformed and quantized to $q_{01}$, which is added back to $\Delta q$ to get back the original residual $q_1$. A standard Context Adaptive Binary Arithmetic Coding (CABAC) (or any entropy coder as adopted in the considered codec) encoding process of $q_1$ along with modes and motion data is then carried out to form compliant stream $S_1$. Both deflation and inflation operations use the same configurations to ensure that the exact same LQ stream is generated as in the case of Simulcast. Consequently, this scheme achieves the same transmission efficiency as SC but with lower storage requirements. On the transcoding side, the inflation process to re-generate a LQ stream is coarsely equivalent to the cost of two full decoding loops, which results in this method being faster than Full Transcoding which requires to perform a full decoding followed by a complete encoding with complex RD optimization.

# 4 Proposal methods for system cost optimization

The proposed work is twofold by targeting to lower the transcoding complexity, and to improve the trade-offs between storage bit-cost and transmission efficiency of the best-in-class method from SOTA namely the GTDI approach (or equivalently PRC-Full-PTQ). For that purpose, we leverage the redundancy between the residuals of the various video representations by means of predictive residual coding techniques with the main contributions being as follows.

To lower the transcoding complexity of the GTDI approach, we propose the idea of Predictive Residual Coding (PRC) with Partial decoding using spatial residual domain reference samples (PRC-Part-TQ).

To further improve the coding efficiency of any method based on predictive residual coding, such as PRC-Full-PTQ or PRC-Part-TQ, we propose two optimizations. The first optimization conditions the delta residual coding and signaling to ensure lower residual energy, the second optimization modifies the Rate-Distortion optimization criteria commonly used for coding mode decisions to favor prediction and splitting modes that minimize the final coded delta residual – hence improving the prediction of the residual data.

# 4.1 Partial decoding using spatial residual domain reference samples (PRC-Part-TQ)

PRC-Part-TQ method relies on partial decoding using spatial residual domain reference samples to lower the transcoding complexity of the GTDI approach. The corresponding coding scheme is depicted in Figure 3. In this approach, a residual predictor based on the inverse transformed and inverse quantized residual image of the reference layer video is used (instead of spatial pixel domain reference samples in case of GTDI).

A reference layer video $V_0$ is normally encoded at the highest resolution/quality and saved as is.

For the dependent video $V_1$ the following re-purposing process is invoked. The residual image of $V_0$ is re-scaled to the resolution of $V_1$ and stored in a buffer. The encoding process of $V_1$ is carried out normally and the encoder is left to make its optimal decisions as for a SC stream. Before entropy encoding, and for each coding unit (CU), the corresponding position and area in the residual image of

$V_0$ is transformed and quantized to obtain $q_{01}$ (after re-scaling if necessary) before being subtracted from the original residual $q_1$ of $V_1$ which leads to $\Delta q = q_1 - q_{01}$. The delta residual $\Delta q$ along with the optimal encoder decisions are entropy encoded to generate $\Delta S_1$ dependent stream.

To re-generate the standard Simulcast version $S_1$ from the dependent stream $\Delta S_1$, upon user request, the following inverse re-purposing process must be invoked. $S_0$ is entropy decoded and the coefficients are inverse quantized, and inverse transformed to obtain the residual image which is then re-scaled to match the resolution of $S_1$. From $\Delta S_1$, the delta coefficients are entropy decoded to obtain $\Delta q$. Then, for each CU, the co-located area in the residual image of $S_0$ is transformed and quantized to get $q_{01}$ which is added to $\Delta q$ to get back the original residual $q_1$. Finally, the original residual $q_1$ is entropy encoded along with the encoder decisions to obtain the standard simulcast stream $S_1$.

This basis proposal method reduces the transcoding complexity, to generate back a standard stream equivalent to Simulcast, to only one partial decoding loop (with re-scaling if necessary) and an entropy encoding operation. It achieves the same optimal transmission efficiency as SC or GTDI while saving on storage bit-cost of dependent streams, by encoding a difference of residuals.

# 4.2 Coding Efficiency Optimizations

To further improve the coding efficiency of the base method PRC-Part-TQ, or any SOTA method based on predictive residual coding such as in GTDI, we propose two complementary optimizations.

## 4.2.1 Conditional Delta Residual (CDR) coding and signaling.
For the base method PRC-Part-TQ or GTDI, a delta residual is calculated and coded for every coding unit (CU) in a group of pictures. However, for some cases, if the residual predictor is not well correlated with the residual blocks to predict then coding the delta-residual can result in a significant bit-cost overhead. To address this issue, we introduce a novel condition for coding the delta-residual of the dependent layer. The idea is to code the inter-layer delta residual only if it lowers the residue energy. More precisely, the differential residual is coded if and only if it satisfies:

$$ \sum_{k=1}^{numComp} \sum_{j=1}^{h/s} \sum_{i=1}^{w/s} |\Delta q(i, j)^k| < \sum_{k=1}^{numComp} \sum_{j=1}^{h/s} \sum_{i=1}^{w/s} |q_{1,i,j}^k| \quad (1) $$

Where numComp is the number of color components, w and h are the width and height of the current coding block (or unit) respectively, $\Delta q$ and $q_1$ are the delta residual and original residual, s a scale factor according to the color component and chroma subsampling. If this condition is not satisfied, then the original residual $q_1$ of the block is coded. To control this condition and to be able to have a decodable stream, a flag called InterLayerResidualPrediction is added and coded for each CU which indicates if it is inter-layer predicted (true) or not (false). The flag can be entropy coded using CABAC (or any other entropy coder as per the considered codec for implementation) using either the bin probability initialization states of the root Coded Block Flag if available (root CBF as standardized in H264/AVC, HEVC or VVC) or any custom bin probability model that can be typically contextualized according to neighboring flag values (e.g. top or left coding block neighbors). CDR coding and
<figure><img src="image_5.png" /><figcaption>Figure 3: proposed PRC-Part-TQ method; showing deflated stream generation on the left and standard stream re-generation on the right. It uses spatial residual samples from a reference high quality video to generate the residual predictor of dependent lower quality videos</figcaption></figure>

signaling demonstrates to improve the storage savings on dependent streams with no impact on the quality at transmission and on the transcoding complexity.

### 4.2.2 Rate-Distortion Optimization based on Delta Residuals (RDODR)
The idea is to update the Rate-Distortion Optimization (RDO) process commonly used for coding mode search and decision by using delta residual bit-cost for the rate estimations, to favor prediction and splitting modes that will minimize the delta residual to code for the dependent streams. In a common RDO process, the encoder exhaustively decides which coding mode to use for a given block or unit based on the minimization of a rate-distortion cost function defined as $J(R, D) = D + \lambda.R$ where R is the bit-cost, D is the distortion and $\lambda$ is the Lagrange multiplier that balances the importance of bit-cost and distortion. For each coding unit, and candidate coding mode $\forall p \in P$, the distortion D is typically estimated by performing the prediction, transform, quantization and inverse processes plus optional in-loop filtering and measured the distance (e.g. L2 based on MSE) of the reconstructed samples with the source samples. The rate or bit-cost is usually estimated by invoking the pseudo-coding of the prediction mode and transformed quantized residuals (i.e. $q_1$) using a CABAC (or any other entropy coder as per the considered codec) estimation process, as formalized in (2).

$$p^* = \operatorname*{argmin}_p (D(p) + \lambda.R(q_1|p)) \quad (2)$$

In the context of any predictive residual coding scheme, we propose to update the bitrate estimations in the RDO process, such that the delta residual bit-cost (i.e. $\Delta q$) is calculated for each block instead of the default residuals $q_1$, as formalized in (3). Such optimization can be combined with the prior proposal of CDR coding and signaling such that the appropriate bit-cost of delta-residuals or residuals is estimated according to the condition defined in (1).

$$p^* = \operatorname*{argmin}_p (D(p) + \lambda.R(\Delta q | p)) \quad (3)$$

Such optimization enables further storage bit-cost saving for no impact on the transcoding complexity. However, it can slightly lower the transmission efficiency (e.g. vs SC) but still being negligible with respect to storage saving benefits.

## 5 Performance assessment

The different proposals, as well as methods from SOTA namely Simulcast, Full Transcoding, GTDI, have been implemented and compared in the context of VVC codec [23]. They were all implemented on top of VTM version 19.0 [24]. For techniques based on Predictive Residual Coding, including the different proposal variants and GDTI, the VVC multi-layer coding structure was leveraged on using VTM Multi-Layer Main 10 profile. With the Layer0 set as the reference video layer and Layer1 set as the dependent video layer. For the re-scaling of the reconstructed and residual reference samples between layers, the Reference Picture Resampling (RPR) filter (as specified in VVC standard) was used (but any resampling filter can be used).

The performance of the different predictive residual coding schemes, including the proposed method, optimizations, and GDTI, are assessed and compared to SC and FT. The performances at different stages of the video delivery scheme are considered: storage bit-cost, transmission efficiency and transcoding complexity for two Multi-Rate and Multi-Resolution video delivery scenarios defined as follows.

A Multi-Rate scenario: in this scenario, we consider a fixed resolution bitrate ladder where the representations vary in bitrate only according to the chosen $QP$ value. All the streams are encoded using the native resolution of the test sequence. The reference stream is encoded with a $QP$ value $QP0 \in \{22, 27\}$. The dependent streams are then encoded using the following $QP$ values: $QP1 = QP0 + offset$ where $offset \in \{2, 4, 6, 8\}$.

A Multi-Resolution scenario: in this scenario, we consider a bi-lateral ladder where the dependent streams can be of resolutions different from the native one with varying bitrates for the same resolution. The reference layer is fixed at the native resolution $L0$ of the test sequence which is 2160p for classA and 1080p for classB sequences. The $QP$ value of the reference layer is $QP0 \in \{22, 27\}$. As for the dependent streams, the resolution is defined as $L1 \in \{1440p, 1080p, 720p, 540p, 360p\}$ for classA sequences, and $L1 \in \{720p, 540p, 360p\}$ for classB sequences, as per the Common
<caption>Table 1: Performance of the different approaches and their variants</caption>

(a) Multi-Rate Scenario

<table><thead><tr><th rowspan="3">Approach</th><th rowspan="3">Variant</th><th colspan="4">Storage Bitcost</th><th colspan="2">Transmission efficiency</th><th colspan="2">Transcoding complexity</th></tr><tr><th colspan="2">All</th><th colspan="2">Dependent</th><th colspan="2">vs SC</th><th colspan="2">vs FT</th></tr><tr><th>QP<sub>1</sub> 22</th><th>QP<sub>1</sub> 27</th><th>QP<sub>1</sub> 22</th><th>QP<sub>1</sub> 27</th><th>QP<sub>1</sub> 22</th><th>QP<sub>1</sub> 27</th><th>QP<sub>1</sub> 22, 27</th><th>QP<sub>1</sub> 22, 27</th></tr></thead><tbody><tr><td rowspan="4">Full Transcoding* [10]</td><td>NA</td><td>-60.5%</td><td>-66.1%</td><td>-100%</td><td>-100%</td><td>14.2%</td><td>17.5%</td><td>0%</td><td>2079.8%</td></tr><tr><td rowspan="3">PRC-Full-PTQ</td><td>Base* [11]</td><td>-25.3%</td><td>-25.5%</td><td>-41.4%</td><td>-38.5%</td><td>0%</td><td>0%</td><td>-95.2%</td><td>0%</td></tr><tr><td>CDR</td><td>-24.8%</td><td>-24.4%</td><td>-40.5%</td><td>-36.8%</td><td>0%</td><td>0%</td><td>-95.2%</td><td>0%</td></tr><tr><td>RDODR</td><td>-28.8%</td><td>-30.3%</td><td>-47.1%</td><td>-45.6%</td><td>0.8%</td><td>-0.2%</td><td>-95.2%</td><td>0%</td></tr><tr><td>CDR+RDODR</td><td>-29.0%</td><td>-30.3%</td><td>-47.3%</td><td>-45.7%</td><td>0.4%</td><td>-0.9%</td><td>-95.2%</td><td>0%</td></tr><tr><td rowspan="4">PRC-Part-TQ</td><td>Base</td><td>-6.4%</td><td>-6.9%</td><td>-10.5%</td><td>-10.5%</td><td>0%</td><td>0%</td><td>-98.5%</td><td>-67.5%</td></tr><tr><td>CDR</td><td>-7.9%</td><td>-7.8%</td><td>-13.0%</td><td>-11.8%</td><td>0%</td><td>0%</td><td>-98.5%</td><td>-67.5%</td></tr><tr><td>RDODR</td><td>-10.1%</td><td>-10.5%</td><td>-16.5%</td><td>-15.9%</td><td>3.4%</td><td>1.7%</td><td>-98.5%</td><td>-67.5%</td></tr><tr><td>CDR+RDODR</td><td>-10.7%</td><td>-10.8%</td><td>-17.6%</td><td>-16.4%</td><td>1.8%</td><td>0.8%</td><td>-98.5%</td><td>-67.5%</td></tr></tbody></table>

(b) Multi-Resolution Scenario

<table><thead><tr><th rowspan="3">Approach</th><th rowspan="3">Variant</th><th colspan="4">Storage Bitcost</th><th colspan="2">Transmission efficiency</th><th colspan="2">Transcoding complexity</th></tr><tr><th colspan="2">All</th><th colspan="2">Dependent</th><th colspan="2">vs SC</th><th colspan="2">vs FT</th></tr><tr><th>QP<sub>1</sub> 22</th><th>QP<sub>1</sub> 27</th><th>QP<sub>1</sub> 22</th><th>QP<sub>1</sub> 27</th><th>QP<sub>1</sub> 22</th><th>QP<sub>1</sub> 27</th><th>QP<sub>1</sub> 22, 27</th><th>QP<sub>1</sub> 22, 27</th></tr></thead><tbody><tr><td rowspan="4">Full Transcoding* [10]</td><td>NA</td><td>-76.8%</td><td>-82.1%</td><td>-100%</td><td>-100%</td><td>6.1%</td><td>6.4%</td><td>0%</td><td>910.3%</td></tr><tr><td rowspan="3">PRC-Full-PTQ</td><td>Base* [11]</td><td>-22.5%</td><td>-25.7%</td><td>-28.9%</td><td>-31.4%</td><td>0%</td><td>0%</td><td>-79.9%</td><td>0%</td></tr><tr><td>CDR</td><td>-23.6%</td><td>-25.7%</td><td>-33.2%</td><td>-31.5%</td><td>0%</td><td>0%</td><td>-79.9%</td><td>0%</td></tr><tr><td>RDODR</td><td>-29.3%</td><td>-31.6%</td><td>-38.0%</td><td>-38.7%</td><td>7.2%</td><td>3.0%</td><td>-79.9%</td><td>0%</td></tr><tr><td>CDR+RDODR</td><td>-30.6%</td><td>-32.5%</td><td>-39.6%</td><td>-39.7%</td><td>0.7%</td><td>-0.9%</td><td>-79.9%</td><td>0%</td></tr><tr><td rowspan="4">PRC-Part-TQ</td><td>Base</td><td>-2.5%</td><td>-3.3%</td><td>-3.5%</td><td>-4.2%</td><td>0%</td><td>0%</td><td>-89.9%</td><td>-48.2%</td></tr><tr><td>CDR</td><td>-5.7%</td><td>-5.4%</td><td>-7.6%</td><td>-6.6%</td><td>0%</td><td>0%</td><td>-89.9%</td><td>-48.2%</td></tr><tr><td>RDODR</td><td>-5.9%</td><td>-6.5%</td><td>-7.9%</td><td>-8.0%</td><td>1.9%</td><td>1.2%</td><td>-89.9%</td><td>-48.2%</td></tr><tr><td>CDR+RDODR</td><td>-8.3%</td><td>-8.2%</td><td>-11.0%</td><td>-10.1%</td><td>0.7%</td><td>0.4%</td><td>-89.9%</td><td>-48.2%</td></tr></tbody></table>

Test Conditions (CTC) of the MPEG CFE on NDVC [10]. The down-scaled versions of each of the sequences are generated with FFmpeg using its bi-cubic filter. In addition, for each sequence and each resolution, dependent streams were encoded using same *QP* values *QP1* than in the previous *Multi-Rate* scenario.

The Table 1a and Table 1b summarize the performance results for the different SOTA methods and proposals. The results for storage bit-cost savings are shown for two cases: when considering all streams ("All" column) and when only considering dependent streams ("Dependent" column). For transmission efficiency and transcoding complexity, the results can only be shown for dependent streams and are averaged over the different sequences and *QP* values *QP1*. The transmission efficiency results were compared to those of the SC encodings on a similar quality basis. For that purpose, 3rd order R-D polynomial functions were estimated using bitrates and PSNRs of each of the SC sequences. Then, for each PSNR of a sequence in the tested methods, the corresponding SC bitrate is interpolated using the polynomial function. Hence, the resulting bitrate is the SC bitrate at the same quality of the tested approach. The methodology to calculate the different savings at the different stages were taken from the CfE on NDVC [10].

The proposed PRC-Part-TQ method based on the concept of partial decoding using spatial residual domain reference samples enables significant reduction of the GTDI transcoding complexity, with about -68% and -48% transcoding run-time acceleration in *Multi-Rate* and *Multi-Resolution* scenarios, respectively. The transcoding complexity reduction comes at the expense of loss in storage savings which can be mitigated by the proposed optimizations CDR+RDODR - resulting in about -17% and -11% storage bit-costs on dependent streams in comparison to SC for the *Multi-Rate* and *Multi-Resolution* scenarios, respectively, and negligible impacts on transmission efficiency (i.e. < 1%) - which can set an

interesting trade-off if transcoding complexity is of concern for the targeted application case. If transcoding complexity is less of concern - the GTDI approach combined with the two proposed optimizations CDR+RDODR can significantly improve the base method in terms of storage savings - with -9.5% additional savings on average across the tested scenarios - while keeping the same initial method benefits in comparison to FT: i.e. a much lower transcoding complexity (about -95% and -80% across the tested scenarios) than FT and near equivalent transmission efficiency than SC; while 11% bitrate overhead is observed on average for FT at transmission for the tested conditions.

## 6 Conclusion

In this paper, we studied the problematic of multi-profile video coding and delivery system optimization for ABR video streaming. We looked at optimizing the system globally for the three criteria: storage bit-cost, transmission efficiency, and transcoding complexity. We first proposed the idea of partial decoding in a differential predictive coding scheme, such as the Guided Transcoding Using Deflation and Inflation (GTDI) method. This allows transcoding complexity reduction at the expense of loss in storage savings. Across the different test conditions and scenarios, the best proposed variant achieves -48.2% transcoding complexity reduction over GTDI and 89.9% over Full Transcoding (FT) for about -9.5% additional storage savings on average and negligible transmission bitrate increase (<1%) in comparison to Simulcast (SC); while 11% bitrate overhead can be observed on average for FT. Moreover, the two R-D optimizations introduced show to further improve the storage savings of GTDI by -11% on average for near equivalent transmission efficiency than SC and same transcoding complexity reduction benefit (i.e. 85+% faster than FT). This later proposal shows an excellent trade-off for common ABR delivery applications.
References

[1] ISO/IEC. Information technology — dynamic adaptive streaming over http (dash). Standard.

[2] Apple. Http live streaming (hls) authoring specification for apple devices. Standard. https://developer.apple.com/documentation/http-live-streaming/hls-authoring-specification-for-apple-devices.

[3] Jan De Cock, Zhi Li, Megha Manohara, and Anne Aaron. Complexity-based consistent-quality encoding in the cloud. In *2016 IEEE International Conference on Image Processing (ICIP)*. IEEE, September 2016.

[4] Yuriy A Reznik, Xiangbo Li, Karl O Lillevold, Abhijith Jagannath, and Justin Greer. Optimal Multi-Codec adaptive bitrate streaming. In *2019 IEEE International Conference on Multimedia & Expo Workshops (ICMEW)*. IEEE, July 2019.

[5] Yuriy A Reznik, Karl O Lillevold, Abhijith Jagannath, Justin Greer, and Jon Corley. Optimal design of encoding profiles for ABR streaming. In *Proceedings of the 23rd Packet Video Workshop*, New York, NY, USA, June 2018. ACM.

[6] Angeliki V Katsenou, Joel Sole, and David Bull. Efficient bitrate ladder construction for content-optimized adaptive video streaming. *IEEE Open J. Signal Process.*, 2:496–511, 2021.

[7] Hadi Amirpour, Klaus Schoeffmann, Mohammad Ghanbari, and Christian Timmerer. DeepVCA: Deep video complexity analyzer. *IEEE Trans. Circuits Syst. Video Technol.*, 34(9):8836–8847, September 2024.

[8] Jinhai Yang, Mengxi Guo, Shijie Zhao, Junlin Li, and Li Zhang. Optimal transcoding resolution prediction for efficient per-title bitrate ladder estimation. In *2024 Data Compression Conference (DCC)*. IEEE, March 2024.

[9] Reda Kaafarani, Mederic Blestel, Thomas Maugey, Michael Ropert, and Aline Roumy. Evaluation of bitrate ladders for versatile video coder. In *2021 International Conference on Visual Communications and Image Processing (VCIP)*. IEEE, December 2021.

[10] MPEG. Call for evidence on transcoding for network distributed video coding. Technical report, MPEG, 2017.

[11] Christopher Hollmann and Rickard Sjöberg. Guided transcoding using deflation and inflation. In *Proceedings of the 23rd Packet Video Workshop*, New York, NY, USA, June 2018. ACM.

[12] H Schwarz, D Marpe, and T Wiegand. Overview of the scalable video coding extension of the H.264/AVC standard. *IEEE Trans. Circuits Syst. Video Technol.*, 17(9):1103–1120, September 2007.

[13] Jill M Boyce, Yan Ye, Jianle Chen, and Adarsh K Ramasubramanian. Overview of SHVC: Scalable extensions of the high efficiency video coding standard. *IEEE Trans. Circuits Syst. Video Technol.*, 26(1):20–34, January 2016.

[14] Damien Schroeder, Adithyan Ilangovan, Martin Reisslein, and Eckehard Steinbach. Efficient multi-rate video encoding for HEVC-based adaptive HTTP streaming. *IEEE Trans. Circuits Syst. Video Technol.*, 28(1):143–157, January 2018.

[15] Bichuan Guo, Yuxing Han, and Jiangtao Wen. Fast block structure determination in av1-based multiple resolutions video encoding. In *2018 IEEE International Conference on Multimedia and Expo (ICME)*. IEEE, July 2018.

[16] Johan De Praeter, Antonio Jesus Diaz-Honrubia, Niels Van Kets, Glenn Van Wallendael, Jan De Cock, Peter Lambert, and Rik Van de Walle. Fast simultaneous video encoder for adaptive streaming. In *2015 IEEE 17th International Workshop on Multimedia Signal Processing (MMSP)*. IEEE, October 2015.

[17] Ekrem Cetinkaya, Hadi Amirpour, Christian Timmerer, and Mohammad Ghanbari. FaME-ML: Fast multirate encoding for HTTP adaptive streaming using machine learning. In *2020 IEEE International Conference on Visual Communications and Image Processing (VCIP)*. IEEE, December 2020.

[18] A Vakali and G Pallis. Content delivery networks: Status and trends. *IEEE Internet Comput.*, 7(6):68–74, November 2003.

[19] Alireza Erfanian, Farzad Tashtarian, Anatoliy Zabrovskiy, Christian Timmerer, and Hermann Hellwagner. OSCAR: On optimizing resource utilization in live video streaming. *IEEE Trans. Netw. Serv. Manag.*, 18(1):552–569, March 2021.

[20] Alireza Erfanian, Hadi Amirpour, Farzad Tashtarian, Christian Timmerer, and Hermann Hellwagner. CD-LwTE: Cost- and delay-aware light-weight transcoding at the edge. *IEEE Trans. Netw. Serv. Manag.*, 20(3):3104–3118, September 2023.

[21] Glenn Van Wallendael, Jan De Cock, and Rik Van de Walle. Fast transcoding for video delivery by means of a control stream. In *2012 19th IEEE International Conference on Image Processing*. IEEE, September 2012.

[22] Thomas Rusert, Kenneth Andersson, Ruoyang Yu, and Harald Nordgren. Guided just-in-time transcoding for cloud-based video platforms. In *2016 IEEE International Conference on Image Processing (ICIP)*. IEEE, September 2016.

[23] Benjamin Bross, Ye-Kui Wang, Yan Ye, Shan Liu, Jianle Chen, Gary J Sullivan, and Jens-Rainer Ohm. Overview of the versatile video coding (VVC) standard and its applications. *IEEE Trans. Circuits Syst. Video Technol.*, 31(10):3736–3764, October 2021.

[24] MPEG ITU-T. Test model of versatile video coding (vtm). https://vcgit.hhi.fraunhofer.de/jvet/VVCSoftwareVTM.