Paper: Effortless Cross-Platform Video Codec: A Codebook-Based Method (ICLR-2024)

Reviewer comment (Experiments):
Why were tests conducted only on 1080p videos? To validate the method's generality, the authors should test it on videos of different resolutions. Typically, in video compression tasks, testing is done on classes like B, C, D, and E, but the authors have provided results for only class B. To ensure the method's effectiveness across various resolutions, the authors should supplement the results on these different video classes. Additionally, what issues arise from center-cropping input images to multiples of 128? Why not consider the original image dimensions?

Author response:
To further demonstrate the effectiveness of our method, we conduct experiments on additional datasets with different resolutions, including HEVC Class C (480P), D (240P), and E (720P). Beyond PSNR and MS-SSIM, we introduced the perceptual evaluation metric LPIPS[1] as a supplementary experiment. In addition, we compare the performance of different GOP sizes (e.g., 12 and 32), which are commonly used in present methods[2,3,4,5,6]. Furthermore, for H.264 and H.265, we analyze the impact brought by quality control parameters (e.g., *qp* and *crf*). The detailed FFMPEG settings are shown in [Table 7](https://drive.google.com/file/d/1Ymp0lZ-a7s0JGbZy5LHyWFF8WVcxd4jK/view?usp=sharing) in the appendix of our paper.  

Finally, we conduct experiments on data of various resolutions and obtain rate-distortion curves for PSNR, MS-SSIM, and LPIPS, as shown in [Figure 12](https://drive.google.com/file/d/12wqNVLGpkB6RnFg0COiZJ6cclCTmjeil/view?usp=sharing), [Figure 13](https://drive.google.com/file/d/1baGqDeCr9PhDX-B71xQ2taZvbdvBIVLD/view?usp=sharing), and [Figure 14](https://drive.google.com/file/d/1O8G7a4CONyq9F6juuiQPWoovwMGqXHy8/view?usp=sharing) in the appendix of paper, respectively.  

As shown in Figures 13 and 14, our method significantly outperforms the best H.265 results in terms of both MS-SSIM and LPIPS. As shown in Figure 12, our method can partially outperform the H.265 results in terms of PSNR. It is worth noting that for the MCL-JCV dataset, our method presents different degrees of degradation for different metrics. The main reason for this is the presence of slight moving objects in the MCL-JCV dataset, which is discussed in [Figure 8](https://drive.google.com/file/d/1W_aAIVrQRZDiWAFHUYkCqsB_6I_xSbSH/view?usp=sharing) of our paper. And our method uses a fixed number of indices for video compression, which inevitably wastes indices in videos with high redundancy, resulting in higher bitrates for the same reconstruction quality, as shown in [Table 4](https://drive.google.com/file/d/1fNuULtK_7an9yg-PnXVaY7UI9fhy12Vw/view?usp=sharing) of our paper. This is also a direction that we need to explore in the future.  

More than that, we have deployed our model on mobile devices(e.g., iPhone 15 Pro) for practical validation, and the [demo video](https://drive.google.com/file/d/1F_6DOe8Rz_tIxRvROxjzbyJA9YppY1jX/view?usp=drive_link) fully demonstrate the advantages of our approach in terms of compression performance and decoding efficiency.  

***Re:*** As shown in the model architecture in [Figure 6](https://drive.google.com/file/d/1-V4UjzXD2ih_z1BVbB8BRi-IQrIf2CW9/view?usp=sharing) of the appendix, we obtain $\mathbf{y}$ by 8× downsampling. As shown in [Figure 4](https://drive.google.com/file/d/1e-hnkIWQoXTPLMmglUX0A1XD3WJ8magw/view?usp=sharing) of our paper, in the process of multi-stage multi-codebook VQ on $\mathbf{y}$, an additional 8× downsampling is required. For $\mathbf{y}^{''}_d$, it also needs to undergo a multi-codebook quantization, which includes a 2× downsampling. Therefore, our smallest scale feature needs to undergo 128× downsampling, which is why we crop to ensure that the input image shape is divisible by 128. Certainly, there may be an optimal choice, which is to perform padding before cropping. However, for alignment with performance calculation and efficiency evaluation, we chose the cropping approach.

Reviewer comment (Experiments):
The proposed method is trained on V100 FP32 and tested on both V100 FP32 and P40 FP16. I agree that different precision platforms can impact results. However, when testing with the lower precision platform, it's customary to convert FP32 to FP16 for comparison. Have the authors provided such comparisons, particularly with traditional methods like H.265 and H.264?

Author response:
To our best knowledge, existing high-performance AI **video codecs** [2,3,4,5,6,9] are all based on **entropy model**, the core principle of which is to transmit data at a lower bitrate by more accurate estimation of the probability distribution, i.e., the output of the entropy model. To accurately transmit data using arithmetic coding, it is necessary to ensure that the probability distributions used during compression and decompression are identical. However, in practice, the entropy model runs on **different platforms**. At this point, when running the entropy model in floating point operations (either half-precision or single-precision, i.e., FP16 or FP32), there is usually a slight deviation, caused by unavoidable system errors, which leads to decoding errors.  

In summary, the cross-platform problem is not caused by low precision such as FP16, but by the fact that the entropy model needs to be identical on the encoding and decoding side. This is the motivation for our proposal of a codebook-based approach to alternative entropy model-based approaches.  

We focus on how to enable video codecs to be cross-platform without the need for quantization. Therefore, we validate the cross-platform capability of our model under varying platforms with different precisions. Existing entropy model-based methods face serious cross-platform issues even when using single precision floating points (i.e., FP32) on different machines. The cross-platform problem analysis for this phenomenon is presented in [Table 2](https://drive.google.com/file/d/1kZKcof-vRrr9FXsaOnjA6eBkaeRIjIZ8/view?usp=sharing) of the paper[5].  

H.264 and H.265 are representative of traditional codecs, and their specification and synchronization operations are mainly performed on **integer precision**[8]. This helps ensure consistency and reproducibility of the encoding and decoding process across different hardware and software platforms. This is also why we chose H.264 and H.265, which are completely cross-platform codecs, for comparison.

Reviewer comment (Reproducibility):
How was the codebook generated? It is mentioned that Zhu et al.'s method is used. Could they provide specific details on the training process, or did they directly employ a pre-trained codebook? If that, how can we guarantee that this codebook can adapt to the testing dataset?

Author response:
All of our codebooks are trained from scratch. Zhu et al. [7] described in the discussion section, where they trained their model end-to-end and used the Gumbel reparameterization method to train the codebook. We adopt this strategy and train our video codec, including all codebooks trained from scratch, on the training set. Ultimately, we evaluate the performance of the model on the test datasets.

Reviewer comment (Experiments):
Could the authors provide an explanation for the design of codebook sizes as mentioned here? “While for predicted frames, we use three different groups of codebook sizes to achieve different video compression ratios as {8192, 2048, 512}, {64, 2048, 512} and {8, 2048, 512}.”

Author response:
In paper [7], different compression ratios for images are achieved by adjusting the value of parameter $\mathbf{m}$. For video compression, due to the spatial and temporal redundancy in predicted frames, the bits per pixel (bpp) of predicted frames are typically smaller than that of keyframes. Based on the keyframe configuration in our proposed method, there are two options for adjusting the bpp of predicted frames: reducing $\mathbf{m}$ or reducing the size of the codebook. Since $\mathbf{m}$ is already set to 2, there is limited margin for adjustment. Therefore, we choose to reduce the size of the codebook to adjust the bitrate of the predicted frame. Certainly, the codebook size and $\mathbf{m}$ are hyperparameters, and we can have many different adjustment strategies, which is also one of our future research areas.

Reviewer comment (Experiments):
I believe that using codebooks for compression is not only cross-platform but also an efficient approach. Why haven't the authors compared their method with more recent approaches like VVC and DCVC-DC?

Author response:
Yes, you are absolutely correct. The codebook-based method not only solves the cross-platform issue but is also relatively efficient. We also validate VVC and SOTA AI video codecs on the same machine, and the experimental results in Table S1 demonstrate the high efficiency of our method.  

The essence of this article is to propose a novel codebook-based framework as a replacement for the existing entropy model-based framework, which can address cross-platform issues. Due to the absence of autoregressive modeling and optical flow alignment, we can design an extremely minimalist framework that can greatly benefit computational efficiency. Our previous experiments focus on demonstrating the feasibility of the new framework and its ability to address cross-platform issues, but lack efficiency validations. We hope that the supplementary information provided can help address any questions or concerns you may have.  

More than that, we have deployed our model on mobile devices(e.g., iPhone 15 Pro) for practical validation, and the [demo video](https://drive.google.com/file/d/1F_6DOe8Rz_tIxRvROxjzbyJA9YppY1jX/view?usp=drive_link) fully demonstrate the advantages of our approach in terms of decoding efficiency.  

**Table S1** : Efficiency comparison of different methods for the same machine and same input shape.  
| Method | Encoding time | Decoding time |  
| :-----:| ----: | ----: |  
| VVC(VTM-22.2) | 647300 ms | 161 ms |  
| DCVC(autoregressive) [2] | 6442 ms | 33865 ms |  
| DCVC-TCM [10] | 683 ms | 428 ms |  
| DCVC-HEM [3] | 680 ms | 333 ms |  
| DCVC-DC [4] | 438 ms | 225 ms |  
| WCA-based-4 (Ours) | 122 ms | 229 ms |  
| light-decoder (Ours) | 122 ms | 35 ms |  
| light-decoder (on iPad Pro) | -- | 25 ms |  
| light-decoder (on iPhone 15 Pro) | -- | 26 ms |  
| light-decoder (on iPhone 13 Pro) | -- | 31 ms |

Reviewer comment (Experiments):
The authors used GOP=12 for H.264 and H.265 configurations, but the model in the paper used GOP=32. Setting H.264/H.265 configurations to GOP=32 would ensure a fairer comparison.

Author response:
To further demonstrate the effectiveness of our method, we conduct experiments on additional datasets with different resolutions, including HEVC Class C (480P), D (240P), and E (720P). Beyond PSNR and MS-SSIM, we introduced the perceptual evaluation metric LPIPS[1] as a supplementary experiment. In addition, we compare the performance of different GOP sizes (e.g., 12 and 32), which are commonly used in present methods[2,3,4,5,6]. Furthermore, for H.264 and H.265, we analyze the impact brought by quality control parameters (e.g., *qp* and *crf*). The detailed FFMPEG settings are shown in [Table 7](https://drive.google.com/file/d/1Ymp0lZ-a7s0JGbZy5LHyWFF8WVcxd4jK/view?usp=sharing) in the appendix of our paper.  

Finally, we conduct experiments on data of various resolutions and obtain rate-distortion curves for PSNR, MS-SSIM, and LPIPS, as shown in [Figure 12](https://drive.google.com/file/d/12wqNVLGpkB6RnFg0COiZJ6cclCTmjeil/view?usp=sharing), [Figure 13](https://drive.google.com/file/d/1baGqDeCr9PhDX-B71xQ2taZvbdvBIVLD/view?usp=sharing), and [Figure 14](https://drive.google.com/file/d/1O8G7a4CONyq9F6juuiQPWoovwMGqXHy8/view?usp=sharing) in the appendix of paper, respectively.  

As shown in Figures 13 and 14, our method significantly outperforms the best H.265 results in terms of both MS-SSIM and LPIPS. As shown in Figure 12, our method can partially outperform the H.265 results in terms of PSNR. It is worth noting that for the MCL-JCV dataset, our method presents different degrees of degradation for different metrics. The main reason for this is the presence of slight moving objects in the MCL-JCV dataset, which is discussed in [Figure 8](https://drive.google.com/file/d/1W_aAIVrQRZDiWAFHUYkCqsB_6I_xSbSH/view?usp=sharing) of our paper. And our method uses a fixed number of indices for video compression, which inevitably wastes indices in videos with high redundancy, resulting in higher bitrates for the same reconstruction quality, as shown in [Table 4](https://drive.google.com/file/d/1fNuULtK_7an9yg-PnXVaY7UI9fhy12Vw/view?usp=sharing) of our paper. This is also a direction that we need to explore in the future.  

More than that, we have deployed our model on mobile devices(e.g., iPhone 15 Pro) for practical validation, and the [demo video](https://drive.google.com/file/d/1F_6DOe8Rz_tIxRvROxjzbyJA9YppY1jX/view?usp=drive_link) fully demonstrate the advantages of our approach in terms of compression performance and decoding efficiency.

Reviewer comment (Reproducibility):
H.264 and H.265 compression configurations seem a little different from the common test configuration. Could you provide specific instructions for H.264 and H.265 compression configurations?

Author response:
In our experiments, the conventional codecs H.264 and H.265 in FFmpeg are used as the anchor with *medium* preset and Constant Rate Factor (CRF) mode. In current methods [2, 3, 9], *qp* and *crf* are commonly used for compressing video with quality control, while it is known that *crf* can often achieve the maximum compression efficiency(https://trac.ffmpeg.org/wiki/Encode/H.264). Therefore, we provide separate configurations for experiments using *qp* (i.e., for quick encoding) and *crf* (for achieving the best possible quality), which are shown in Table 1.

Reviewer comment (Writing):
In Sec 2.1, “Consequently, To avoid….” here 'T' should be lowercase.

Author response:
Thank you for the meticulous review. We have corrected the [typo](https://drive.google.com/file/d/15S7VB6gj0O5-VvN8zOK9Vm_4RmkBPWl_/view?usp=sharing) and successfully uploaded the updated paper.

Reviewer comment (Novelty):
The novelty of this paper may be limited as the overall framework bears resemblance to Mentzer et al.'s (2022) approach, which avoids explicit motion estimation by employing a transform-like architecture. Additionally, the use of vector quantization for compression is not new and may have been inspired by Zhu's work in image compression. Furthermore, the windows-based cross attention appears similar to SwinTransformer. Consequently, the major architecture and designs lack sufficient novelty.

Author response:
***Re:*** Video compression and image compression are two related but different issues. Existing codebook-based approaches [1] can only address **image compression**, whereas we focus more on **video compression**, which has more complex scenarios. 

In video compression, the mainstream framework is based on the **entropy model**, whose critical prediction frame model mainly consists of a motion compensation module and a context compensation module. Therefore, within this framework, Mentzer et al. proposed a novel transformer-based entropy model. This model is capable of estimating the probability distribution of the current frame without the need for motion compensation [2], while still operating within the framework of **entropy model**-based methods. 

**In contrast**, we propose a new **codebook-based** framework rather than an entropy model-based framework for video compression. First, our proposed method eliminates the need for entropy modeling and achieves cross-platform compatibility through a codebook strategy. Furthermore, we propose a **WCA-based** context model that **avoids** the need for motion compensation and autoregressive components. Due to the absence of autoregressive modeling and optical flow alignment, we can design an extremely minimalist framework that can greatly benefit computational efficiency. 

Additionally, SwinTransformer [3] is a **self-attentive** framework, but our goal is to achieve more adequate information fusion between successive frames. Therefore, we propose a context model based on **cross-attention**. To improve the efficiency of our model, we further introduce a WCA(window-based cross-attention)-based context model.

Reviewer comment (Experiments):
The experimental results are unconvincing as they only provide data on 1080p videos without including other resolutions such as Class C/D/E. Moreover, the authors solely compare their approach with traditional video codecs like H.265; however, it would be beneficial to include comparisons with learned video codecs that can also be utilized on CPU platforms to reduce decoding errors. These methods should be incorporated into the study.

Author response:
***Re:*** To further demonstrate the effectiveness of our method, we conduct experiments on additional datasets with different resolutions, including HEVC Class C (480P), D (240P), and E (720P). Beyond PSNR and MS-SSIM, we introduced the perceptual evaluation metric LPIPS[4] as a supplementary experiment. In addition, we compare the performance of different GOP sizes (e.g., 12 and 32), which are commonly used in present methods[5,6,7,8,9]. Furthermore, for H.264 and H.265, we analyze the impact brought by quality control parameters (e.g., *qp* and *crf*). The detailed FFMPEG settings are shown in [Table 7](https://drive.google.com/file/d/1Ymp0lZ-a7s0JGbZy5LHyWFF8WVcxd4jK/view?usp=sharing) in the appendix of our paper.

Finally, we conduct experiments on data of various resolutions and obtain rate-distortion curves for PSNR, MS-SSIM, and LPIPS, as shown in [Figure 12](https://drive.google.com/file/d/12wqNVLGpkB6RnFg0COiZJ6cclCTmjeil/view?usp=sharing), [Figure 13](https://drive.google.com/file/d/1baGqDeCr9PhDX-B71xQ2taZvbdvBIVLD/view?usp=sharing), and [Figure 14](https://drive.google.com/file/d/1O8G7a4CONyq9F6juuiQPWoovwMGqXHy8/view?usp=sharing) in the appendix of paper, respectively. 

As shown in Figures 13 and 14, our method significantly outperforms the best H.265 results in terms of both MS-SSIM and LPIPS. As shown in Figure 12, our method can partially outperform the H.265 results in terms of PSNR. It is worth noting that for the MCL-JCV dataset, our method presents different degrees of degradation for different metrics. The main reason for this is the presence of slight moving objects in the MCL-JCV dataset, which is discussed in [Figure 8](https://drive.google.com/file/d/1W_aAIVrQRZDiWAFHUYkCqsB_6I_xSbSH/view?usp=sharing) of our paper. And our method uses a fixed number of indices for video compression, which inevitably wastes indices in videos with high redundancy, resulting in higher bitrates for the same reconstruction quality, as shown in [Table 4](https://drive.google.com/file/d/1fNuULtK_7an9yg-PnXVaY7UI9fhy12Vw/view?usp=sharing) of our paper. This is also a direction that we need to explore in the future.

Reviewer comment (Experiments):
In the experiment section, it is important to set an equal Group of Pictures (GoP) size when comparing the author's model with traditional codecs. Therefore, both H.265 and H.264 GoP sizes should be set at 32 for a fair comparison. Additionally, regarding the proposed codebook method, there might be concerns about error propagation if a larger GoP size is used; thus, it would be valuable for the authors to address this issue in their discussion.

Author response:
***Re:*** First, existing entropy-based methods require consistent probability distribution estimation, which is difficult to guarantee in cross-platform scenarios. Therefore, entropy model-based methods **cannot** achieve cross-platform compatibility. In this paper, we propose a codebook-based video codec without entropy modeling, which **can** achieve cross-platform compatibility. 

Of course, like all neural networks, different platforms can lead to error propagation. This can be optimized by choosing operators with smaller errors or other engineering strategies. According to our experiments, setting the GOP size to 32 (the same as the existing SOTA method[6,7]) does not cause significant cumulative errors in our model in all test environments (e.g., iPad Pro ANE, iPad Pro GPU, iPhone 15 Pro ANE, iPhone 15 Pro GPU, etc).

Reviewer comment (Reproducibility):
How do you calculate the bitrate? And how do you compress the index in the proposed framework? The authors don't provide many details.

Author response:
***Re:*** I'm sorry for any inconvenience I may have caused you. The paper discusses the upper bound of bitrate required for transmitting indices of multi-stage multi-codebook[1]. For example, if the codebook size is **8192**, transmitting an index requires **13** bits, since $log_{2}8192=13$. If we set a multi-codebook size of 2, transmitting a code requires **26** bits, which is 2 times the number of bits needed for a single codebook index (i.e., $2*13=26$ bits). The latent shape is $H/16 \times W/16$, where $H$ and $W$ are the shape of the input image. Therefore, transmitting one frame requires $H/16 \times W/16 \times 26 / (H \times W) = 0.1015$ bits per pixel (bpp). 

Certainly, we can calculate the probability of different indices appearing in the training set, and then take advantage of arithmetic coding to compress the indexes with higher probabilities into smaller bits, which reduces the final bpp required for transmission. **For all experiments in our paper**, the bitstream that can be transmitted is obtained, and the bpp is calculated based on it.

Reviewer comment (Novelty):
Related work should be updated.

Author response:
***Re:*** Yes, there is a new interesting direction using the generative model for video or image compression [1,2]. There are also methods focusing on compression efficiency [3,4]. We have updated the related works. If there is still interesting work that we have neglected, we will be happy that the reviewer can point it out for us.

Reviewer comment (Novelty):
Include the recent papers in references.

Author response:
***Re:*** Yes, there is a new interesting direction using the generative model for video or image compression [1,2]. There are also methods focusing on compression efficiency [3,4]. We have updated the related works. If there is still interesting work that we have neglected, we will be happy that the reviewer can point it out for us.

Reviewer comment (Presentation):
Mention the future scope.

Author response:
***Re:*** As mentioned in the conclusions, we will consider designing entropy constraints for the indices of different videos to achieve content-based adaptive compression in future work. In addition, several reviewers have mentioned the potential efficiency benefits of the codebook-based approach, which we are currently testing on mobile devices. As shown in Table S1, the experimental results up to this point indicate significant efficiency advantages of our approach.

**Table S1** : Efficiency comparison of different methods for the same machine and same input shape.
| Method | Encoding time | Decoding time |
| :-----:| ----: | ----: |
| VVC(VTM-22.2) | 647300 ms | 161 ms |
| DCVC(autoregressive) [6] | 6442 ms | 33865 ms |
| DCVC-TCM [11] | 683 ms | 428 ms |
| DCVC-HEM [7] | 680 ms | 333 ms |
| DCVC-DC [8] | 438 ms | 225 ms |
| WCA-based-4 (Ours) | 122 ms | 229 ms |
| light-decoder (Ours) | 122 ms | 35 ms |
| light-decoder (on iPad Pro) | -- | 25 ms |
| light-decoder (on iPhone 15 Pro) | -- | 26 ms |
| light-decoder (on iPhone 13 Pro) | -- | 31 ms |

Reviewer comment (Evaluation):
Detailed analysis of figure 5 results is required.

Author response:
***Re:*** To further demonstrate the effectiveness of our method, we conduct experiments on additional datasets with different resolutions, including HEVC Class C (480P), D (240P), and E (720P). Beyond PSNR and MS-SSIM, we introduced the perceptual evaluation metric LPIPS[5] as a supplementary experiment. In addition, we compare the performance of different GOP sizes (e.g., 12 and 32), which are commonly used in present methods[6,7,8,9,10,11]. Furthermore, for H.264 and H.265, we analyze the impact brought by quality control parameters (e.g., *qp* and *crf*). The detailed FFMPEG settings are shown in [Table 7](https://drive.google.com/file/d/1Ymp0lZ-a7s0JGbZy5LHyWFF8WVcxd4jK/view?usp=sharing) in the appendix of our paper.

Finally, we conduct experiments on data of various resolutions and obtain rate-distortion curves for PSNR, MS-SSIM, and LPIPS, as shown in [Figure 12](https://drive.google.com/file/d/12wqNVLGpkB6RnFg0COiZJ6cclCTmjeil/view?usp=sharing), [Figure 13](https://drive.google.com/file/d/1baGqDeCr9PhDX-B71xQ2taZvbdvBIVLD/view?usp=sharing), and [Figure 14](https://drive.google.com/file/d/1O8G7a4CONyq9F6juuiQPWoovwMGqXHy8/view?usp=sharing) in the appendix of paper, respectively. 

As shown in Figures 13 and 14, our method significantly outperforms the best H.265 results in terms of both MS-SSIM and LPIPS. As shown in Figure 12, our method can partially outperform the H.265 results in terms of PSNR. It is worth noting that for the MCL-JCV dataset, our method presents different degrees of degradation for different metrics. The main reason for this is the presence of slight moving objects in the MCL-JCV dataset, which is discussed in [Figure 8](https://drive.google.com/file/d/1W_aAIVrQRZDiWAFHUYkCqsB_6I_xSbSH/view?usp=sharing) of our paper. And our method uses a fixed number of indices for video compression, which inevitably wastes indices in videos with high redundancy, resulting in higher bitrates for the same reconstruction quality, as shown in [Table 4](https://drive.google.com/file/d/1fNuULtK_7an9yg-PnXVaY7UI9fhy12Vw/view?usp=sharing) of our paper. This is also a direction that we need to explore in the future. 

More than that, we have deployed our model on mobile devices(e.g., iPhone 15 Pro) for practical validation, and the [demo video](https://drive.google.com/file/d/1F_6DOe8Rz_tIxRvROxjzbyJA9YppY1jX/view?usp=drive_link) fully demonstrate the advantages of our approach in terms of compression performance and decoding efficiency.

Reviewer comment (Experiments):
**Decoding efficiency**: The paper should also report on decoding efficiency relative to existing neural video compression methods, extending beyond the ablation studies of the proposed modules.

Author response:
> Decoding efficiency: The paper should also report on decoding efficiency relative to existing neural video compression methods, extending beyond the ablation studies of the proposed modules.

***Re:*** We thank the reviewer for pointing this out. We have compared the efficiency of our method with traditional VVC(VTM) and SOTA AI video codecs. The specific encoding and decoding times are shown in Table S1. Although our performance only exceeds that of H.265, our approach has significant advantages in both encoding and decoding efficiency than the AI video codecs. This will play a crucial role in future practical applications. 

**Table S1** : Efficiency comparison of different methods for the same machine and same input shape.
| Method | Encoding time | Decoding time |
| :-----:| ----: | ----: |
| VVC(VTM-22.2) | 647300 ms | 161 ms |
| DCVC(autoregressive) [4] | 6442 ms | 33865 ms |
| DCVC-TCM [10] | 683 ms | 428 ms |
| DCVC-HEM [5] | 680 ms | 333 ms |
| DCVC-DC [6] | 438 ms | 225 ms |
| WCA-based-4 (Ours) | 122 ms | 229 ms |
| light-decoder (Ours) | 122 ms | 35 ms |
| light-decoder (on iPad Pro) | -- | 25 ms |
| light-decoder (on iPhone 15 Pro) | -- | 26 ms |
| light-decoder (on iPhone 13 Pro) | -- | 31 ms |

Reviewer comment (Experiments):
**More consideration of cross-platform scenario when encoding/decoding**: The paper presents the scenarios where encoding is done using an NVIDIA Tesla V100 and decoding with a Tesla P40, as detailed in Table 5 of the Appendix. However, this scenario is limited. More examination of diverse scenarios including a broader range of encoding/decoding machines should be included to verify the robustness of the proposed method regardless of the systematic errors across different platforms.

Author response:
> More consideration of cross-platform scenario when encoding/decoding: The paper presents the scenarios where encoding is done using an NVIDIA Tesla V100 and decoding with a Tesla P40, as detailed in Table 5 of the Appendix. However, this scenario is limited. More examination of diverse scenarios including a broader range of encoding/decoding machines should be included to verify the robustness of the proposed method regardless of the systematic errors across different platforms.

***Re:*** First, Since our approach eliminates the dependence on the entropy model, there is inherently no need for the computations at the encoding and decoding ends to be identical. Then our codebook-based video codec is able to run cross-platform. In addition, we have conducted cross-platform experiments on more heterogeneous platforms, and the results, as shown in Table S2, are sufficient to demonstrate the advantages of our approach in cross-platform scenarios. 

More than that, we have deployed our model on mobile devices(e.g., iPhone 15 Pro) for practical validation, and the [demo video](https://drive.google.com/file/d/1F_6DOe8Rz_tIxRvROxjzbyJA9YppY1jX/view?usp=drive_link) fully demonstrate the advantages of our approach in terms of compression performance and decoding efficiency.

**Table S2** : BD-rate calculated on different platforms by SSIM on UVG dataset.
| Decoder <br> precision | V100 <br> FP32 | V100 <br> FP16 |  P40 <br> FP32 | P40 <br> FP16 |  iPad GPU <br> FP16 | iPad ANE <br> FP16 | 
| :-----:| :----: | :----: | :----: | :----: | :----: | :----: |
| UVG | 0% | +0.9% | 0% | +0.9% | +0.9% | +1.0% |

Reviewer comment (Experiments):
**More comparison with existing calibration methods**: The paper should incorporate a detailed comparison with existing calibration methods. This is crucial for contextualizing the results presented in Table 2 and Table 5. The inclusion of performance comparison with existing calibration methods will significantly enhance the paper's comprehensiveness and the validity of its conclusions.

Author response:
> More comparison with existing calibration methods: The paper should incorporate a detailed comparison with existing calibration methods. This is crucial for contextualizing the results presented in Table 2 and Table 5. The inclusion of performance comparison with existing calibration methods will significantly enhance the paper's comprehensiveness and the validity of its conclusions.

***Re:*** To our knowledge, the only existing cross-platform AI video codec is the calibration method proposed by Tian et al. [1] They developed a plugin for entropy model-based video codec, which obtains cross-platform capabilities by transmitting additional bitstream. In contrast, we propose a new codebook-based method to address the cross-platform issue in AI video codecs. Therefore, we compare our method with the calibration method in terms of cross-platform capability, and the results are shown in Table S3. It can be seen that our method has more universal cross-platform capabilities due to the differences in the framework.

Reviewer comment (Novelty):
The proposed WCA-based context model is intriguing. However, the presented modules, including the codebook approach and the keyframe and prediction frame categorization, appear as already well-established techniques in the field of neural data compression. While the focus on cross-platform scenarios is an interesting aspect that underscores the effectiveness of your proposed method, it also requires a deeper investigation to distinguish this paper from existing methods. Therefore, to strengthen the claim of the paper, a more comprehensive comparison as mentioned in the weakness section seems crucial. Without them, my evaluation of the paper may be lower.

Author response:
> The proposed WCA-based context model is intriguing. However, the presented modules, including the codebook approach and the keyframe and prediction frame categorization, appear as already well-established techniques in the field of neural data compression. While the focus on cross-platform scenarios is an interesting aspect that underscores the effectiveness of your proposed method, it also requires a deeper investigation to distinguish this paper from existing methods. Therefore, to strengthen the claim of the paper, a more comprehensive comparison as mentioned in the weakness section seems crucial. Without them, my evaluation of the paper may be lower.

***Re:*** Video compression and image compression are two related but different issues. Zhu et al. propose a codebook-based approach that can only address image compression [3], whereas we focus more on video compression, which has more complex scenarios. In video compression, the mainstream framework is based on entropy modeling, whose critical prediction frame model mainly consists of a motion compensation module and a context compensation module. However, the SOTA approach also requires a parallel-unfriendly autoregressive component. **In contrast**, we propose a new codebook-based video codec. First, our framework eliminates the need for entropy modeling and achieves cross-platform compatibility through a codebook strategy. Furthermore, we propose a WCA-based context model that avoids the need for motion compensation and autoregressive components. Due to the absence of autoregressive modeling and optical flow alignment, we can design an extremely minimalist framework that can greatly benefit computational efficiency.