# OSLO-IC: On-the-Sphere Learned Omnidirectional Image Compression with Attention Modules and Spatial Context

Paul Wawerek-López, Navid Mahmoudian Bidgoli, Pascal Frossard, Andre Kaup, Thomas Maugey

<figure><img src="image_1.png" /></figure>

## To cite this version:

Paul Wawerek-López, Navid Mahmoudian Bidgoli, Pascal Frossard, Andre Kaup, Thomas Maugey. OSLO-IC: On-the-Sphere Learned Omnidirectional Image Compression with Attention Modules and Spatial Context. ICASSP 2025 - IEEE International Conference on Acoustics, Speech, and Signal Processing, Apr 2025, Hyderabad, India. pp.1-4. hal-05442204

HAL Id: hal-05442204

https://inria.hal.science/hal-05442204v1

Submitted on 5 Jan 2026

HAL is a multi-disciplinary open access archive for the deposit and dissemination of scientific research documents, whether they are published or not. The documents may come from teaching and research institutions in France or abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est destinée au dépôt et à la diffusion de documents scientifiques de niveau recherche, publiés ou non, émanant des établissements d'enseignement et de recherche français ou étrangers, des laboratoires publics ou privés.

<figure><img src="image_2.png" /></figure>

Distributed under a Creative Commons CC BY 4.0 - Attribution - International License
# OSLO-IC: On-the-Sphere Learned Omnidirectional Image Compression with Attention Modules and Spatial Context

Paul Wawerek-López*, Navid Mahmoudian Bidgoli, Pascal Frossard†, André Kaup*, and Thomas Maugey‡

*Multimedia Communications and Signal Processing, Friedrich-Alexander-Universität Erlangen-Nürnberg, Germany

†École Polytechnique Fédérale de Lausanne (EPFL), Lausanne, Switzerland

‡Institut National de Recherche en Informatique et en Automatique (INRIA), Rennes, France

***Abstract***—Developing effective 360-degree (spherical) image compression techniques is crucial for technologies like virtual reality and automated driving. This paper advances the state-of-the-art in on-the-sphere learning (OSLO) for omnidirectional image compression framework by proposing spherical attention modules, residual blocks, and a spatial autoregressive context model. These improvements achieve a 23.1% bit rate reduction in terms of WS-PSNR BD rate. Additionally, we introduce a spherical transposed convolution operator for upsampling, which reduces trainable parameters by a factor of four compared to the pixel shuffling used in the OSLO framework, while maintaining similar compression performance. Therefore, in total, our proposed method offers significant rate savings with a smaller architecture and can be applied to any spherical convolutional application.

***Index Terms***—360-degree, spherical transposed convolution, end-to-end compression, image compression

## I. INTRODUCTION

The ongoing progress of technologies including Virtual Reality (VR) and autonomous driving results in an increasing amount of omnidirectional content. Especially for consumer-oriented devices like VR glasses, high resolution images are required to obtain a good visual impression at each viewing angle. Thus, developing efficient compression techniques for omnidirectional images is necessary to store and transmit high-resolution omnidirectional images. In the field of 2D image and video compression, recent end-to-end learned methods based on autoencoders [1]–[6] have led to promising compression performance.

However, a direct extension of Convolutional Neural Networks (CNNs) to omnidirectional images is not straightforward since various representations exist. One method consists in mapping the sphere to one or multiple planes as a preprocessing step. The most popular among these mappings is the Equirectangular Projection (ERP). Due to the inevitable geometrical distortions introduced by such mappings, the interpixel correlations become position-dependent. To deal with the distortions of the ERP, the authors of [7]

introduced a latitude-dependent loss function as well as an additional latitude adaptive scale network. For the same reason, using multiple motion models for video compression was investigated in [8]. In contrast, the authors of [9] defined convolution operations directly on a spherical representation based on HEALPix sampling [10]. This framework is called On-the-Sphere Learning for Omnidirectional Images (OSLO), which enhances the performance of CNNs for omnidirectional image compression and denoising compared to training the equivalent 2D models on ERP images [9], [11]. However, recent advancements in 2D image compression models, including attention modules, residual blocks as nonlinearities, and a spatial context model for on-the-sphere learned image compression, have not yet been integrated into the OSLO framework. Additionally, OSLO only supports periodic pixel shuffling as an unpooling operation, resulting in four times the number of parameters compared to other well-known unpooling methods. Defining computationally efficient on-the-sphere operations is challenging since the HEALPix images are stored in vectors, containing only information about the direct neighborhoods of each pixel.

In this paper, we build on OSLO and propose an updated spherical end-to-end learned image compression model containing attention modules [4], the replacement of Generalized Divisive Normalization (GDN) layers [12] by residual blocks [5], and a spatial autoregressive context model [3]. The context model is realized by defining a spherical masked convolution. Compared to the spherical scale hyperprior model, the proposed model saves 23% of the bit rate in terms of BD rate [13], [14] calculated on WS-PSNR [15]. In addition, we define a spherical transposed convolution, reducing the number of parameters by a factor of four while maintaining its expressiveness. Since transposed convolutions are often used for other tasks including image generation or denoising, our proposed spherical transposed convolution reduces the complexity of many other spherical models.<fnref n="1" />

The authors gratefully acknowledge that this work has been supported by the Deutsche Forschungsgemeinschaft (DFG, German Research Foundation) under project number 418866191.

Navid M. Bidgoli is with Trimble Inc. This work was developed while he was with INRIA.

<footnote n="1">¹The source code is available at www.gitlab.inria.fr/on-the-sphere-learning-for-omnidirectional-images-oslo/oslo-360-degree-image-compression</footnote>
## II. ON-THE-SPHERE LEARNED IMAGE COMPRESSION

### A. HEALPix Sampling

Using the Hierarchical Equal Area isoLatitude Pixelization (HEALPix) proposed in [10], each pixel covers the same area on the sphere. Initially, the sphere is divided into 12 equal-area base tiles (red lines in Fig. 1). Higher resolutions are achieved by recursively splitting each pixel into 4 sub-pixels of equal area. The resolution parameter $N_{side}$ represents the number of pixels per base tile, resulting in a total of $N_{pix} = 12 \cdot N_{side}^2$ pixels. Pixel borders for $N_{side} \le 8$ are shown in Fig. 1. Each pixel has 8 neighbors, except for 24 pixels with 7 neighbors located at the intersections of polar and equatorial tiles. This number of exceptional pixels is constant, making their effect negligible at high resolutions. Pixel indices follow a nested tree structure, where a parent pixel index $[...p_2p_1p_0]_2$ (in binary notation) generates child pixel indices by appending two bits $b_1b_0$, resulting in $[...p_2p_1p_0b_1b_0]_2$.

### B. Spherical Convolution

The OSLO framework proposed in [9] defines spherical convolution as follows: Let $L_{in}$ and $L_{out}$ be the number of input and output features. The input features at pixel index $i = 1, ..., N_{pix}$ are $x_i \in \mathbb{R}^{L_{in}}$, and the weights of the $l$-th kernel for the $k$-th neighbor are $\Theta_k^l \in \mathbb{R}^{L_{in}}$, with $k = 1, ..., 8$ and $l = 1, ..., L_{out}$. The kernel shape is shown in Fig. 1, with each color representing a different kernel entry. The output feature $l$ of the spherical convolution is [9]:

$$x_i^l = \langle \Theta_0^l, x_i \rangle + \sum_{k=1}^{8} \langle \Theta_k^l, x_{N_i(k)} \rangle \cdot w_{N_i(k),i}, \quad (1)$$

where $N_i(k)$ is the index of the $k$-th neighbor of pixel $i$ and $w_{N_i(k),i}$ is 0 if $N_i(k)$ is missing, otherwise 1. Only 1-hop convolutions are supported due to computational complexity and storage demands, but larger receptive fields can be achieved by performing $n$ subsequent 1-hop convolutions and summing the outputs.

This spherical convolution meets three key requirements [9]: 1) Consistency: The filter orientation towards the north pole, interpixel correlations, and pixel areas remain nearly constant over the sphere in HEALPix sampling. 2) Expressiveness: The anisotropic filter makes distinction between neighboring pixels, similar to standard 2D convolution. 3) Efficiency: The computation scales linearly with the number of pixels.

### C. End-to-End Learned Image Compression

Most recent end-to-end learned image compression models are based on the scale hyperprior model introduced in [2]. This model consists of two autoencoders: the image encoder and decoder $(e, d)$, and the statistics encoder and decoder $(e_s, d_s)$, known as the hyperprior network.

The image encoder $e$ generates a latent space $y$ from the input $x$. The latent space is quantized through rounding, but since rounding is not differentiable, it is emulated by additive uniform noise during training [16]. The hyperprior network estimates the likelihood of $y$ using a Gaussian entropy

<figure><img src="image_3.png" /><figcaption>Fig. 1. HEALPix sampled sphere with resolution $N_{side} = 8$. Red, orange and blue lines show the pixel boundaries for $N_{side} = 1, 2, 4$, respectively. The gray line indicates the equator. Additionally, a spherical convolution kernel is shown at two positions on the sphere.</figcaption></figure>

model to improve the performance of the lossless arithmetic encoder and decoder. More precisely, the latent space $y$ is fed into $e_s$ to produce a representation $\nu$ of the distribution of standard deviations $\sigma$. This representation $\nu$ is quantized to $\hat{\nu}$ and losslessly compressed by an arithmetic encoder, forming the hyperprior bitstream transmitted as side information. The scale estimations $\hat{\sigma}$ are obtained by decoding the hyperprior bitstream with an arithmetic decoder and $d_s$. The arithmetic encoder and decoder use a Gaussian entropy model based on $\hat{\sigma}$ to generate the image bitstream and reconstruct the quantized latent space $\hat{y}$, respectively. The reconstructed image $\hat{x}$ is obtained by feeding $\hat{y}$ into $d$.

During training, the model optimizes the trade-off between the bit rate $R$ and the distortion $D$ by minimizing the loss function:

$$\mathcal{L} = R(\hat{y}, \hat{\nu}) + \lambda D(x, \hat{x}), \quad (2)$$

where $\lambda$ denotes the quality parameter. A higher $\lambda$ places more importance on reconstruction quality. The distortion $D$ is calculated from the mean squared error between the input $x$ and its reconstruction $\hat{x}$.

Various modifications of the scale hyperprior model have been proposed to improve compression performance. Min- nen et al. introduced a spatial autoregressive context model in [3], estimating mean values in addition to standard devia- tions. This spatial context model uses already decoded parts of the quantized latent space $\hat{y}$ to predict future mean and standard deviation values, implemented with a masked convo- lution kernel and a parameter aggregation network. Sequential decoding is required due to causality constraints in the spatial context model. Channel-wise context models for faster parallel decoding were investigated in [5], [17], [18], but are not the focus here. In [4], Cheng et al. included attention modules in the image autoencoder, reducing bits for simple regions and spending more bits on challenging parts [4]. He et al. proposed replacing (inverse) GDN layers [12] with three consecutive residual blocks, which are more effective for Gaussian density modeling due to larger receptive fields [5].
<figure><img src="image_4.png" /><figcaption>Fig. 2. Extended spherical model with Parameter Aggregation (PA) and Residual Blocks (RBs). Attention modules (Attn) follow the simplified structure in [4]. The description "Conv $hn \downarrow 4 \times N$" represents a spherical n-hop convolution downsampled by 4 with N filters. "TConv $hn \uparrow 4 \times N$" represents a spherical n-hop transposed convolution upsampled by 4 with N filters.</figcaption></figure>

### III. EXTENDED MODEL

In the following, we build our extended spherical compression model as shown in Fig. 2, where 0-, 1-, and 2-hop convolutions are the equivalents of 1 × 1, 3 × 3, and 5 × 5 convolutions for 2D images.

The spherical attention module and residual blocks are only built from 0- and 1-hop convolutions as well as element-wise operations. For reference, we use the simplified attention module from [4]. Let $x$ denote the input of the spherical attention layer, then the output can be calculated as $x + f_{\text{RBs}}^1(x) \odot \sigma(f_{\text{h0}}(f_{\text{RBs}}^2(x)))$, where $f_{\text{RBs}}^1(\cdot)$ and $f_{\text{RBs}}^2(\cdot)$ denote two different stacks of three residual blocks as indicated in the top-right (RBs) of Fig. 2, $f_{\text{h0}}(\cdot)$ denotes a 0-hop convolution, and $\odot$ denotes an element-wise multiplication. The calculation of 0-hop convolutions is a special case of (1) when only the center of the kernel $\Theta_0^l$ is considered. The computation of 0-hop convolutions thus includes only a single matrix multiplication. Therefore, all necessary operations are defined inside the previous OSLO framework. The resulting structure of the image autoencoder is adopted from [5].

For the spatial context model, we define the spherical masked convolution by replacing the mask $w_{N_i(k),i}$ in (1) by $w'_{i,k} = w_{N_i(k),i} \cdot w_{N_i(k),i}^c$ with the position-dependent causal mask:

$$w_{N_i(k),i}^c = \begin{cases} 0, & N_i(k) \ge i \\ 1, & i < N_i(k) \end{cases}, \quad (3)$$

which is illustrated for a masked 1-hop convolution at two different positions on the sphere in Fig. 3. We also change the spherical scale hyperprior to also estimate the distribution of mean values by adapting the output features of the convolutions inside the decoder of statistics $d_s$. The parameter aggregation module, which combines the output of $d_s$ with the context model for the prediction of mean and standard

<figure><img src="image_5.png" /><figcaption>Fig. 3. Proposed causal 1-hop convolution mask $w_{N_i(k),i}^c$ for the spatial context model at two different pixel positions $i$. The pixel indices follow the nested HEALPix indexing scheme. For example, when computing the convolution for pixel $i=6$, the neighboring pixels $i=1, 3, 4, 5$ have already been processed and are included in the masked convolution.</figcaption></figure>

deviation distributions, consists only of 0-hop convolutions and can thus be implemented with the previous OSLO framework.

Additionally, we modify the unpooling operation by replacing the pixel shuffling operation with a spherical transposed convolution, illustrated in Fig. 4. The idea is to multiply the weights $\Theta$ with the input values $x_i$ for each pixel $i$. Each multiplied kernel is then added to the high-resolution output with the kernel center placed at the child pixels with index $i' = [...]i_2i_1i_000]_2$ in binary notation. To make this operation computationally efficient, we calculate matrix multiplications of the input resolution signals as $x\Theta_0$ for the kernel center and $x\Theta_k$ for each neighbor with $x \in \mathbb{R}^{N_{\text{pix}} \times L_{\text{in}}}$ and $\Theta_0, \Theta_k \in \mathbb{R}^{L_{\text{in}} \times L_{\text{out}}}$. The resulting $N_{\text{pix}} \times L_{\text{out}}$ values are added to the high-resolution output image at the positions $i'$ for the center and $N_i(k)$ for each neighbor $k$.

The proposed transposed convolution kernel thus consists of $(9L_{\text{in}} + 1)L_{\text{out}}$ parameters with additional bias. In contrast, using pixel shuffling as unpooling operation requires $4 \cdot L_{\text{out}}$ output channels to provide the same output resolution, and therefore four times the number of parameters, due to the rearrangement of the output channels after the convolution.
<figure><img src="image_6.png" /><figcaption>Fig. 4. Proposed spherical transposed convolution operation with input **x** and filter kernel **Θ**. The kernel gets multiplied by each input pixel value and added to the high-resolution output at the corresponding pixel position.</figcaption></figure>

## IV. PERFORMANCE EVALUATION

To evaluate our extended spherical model, we use the same train and test setup as the authors of [9]. We use 2170 ERP images of resolution 9104×4552 from the SUN360 dataset [19] and split them into 1737 train, 10 validation, and 423 test images with the identical split used in [9]. We resample the ERP images to HEALPix with resolution $N_{side} = 2^{10}$. During training, random patches equivalent to a size of 256×256 are fed into the model in batches of size 10. We train our models for 1000 epochs with a fixed learning rate of $10^{-4}$ during the first 800 epochs and reduce the learning rate based on the validation loss during the last 200 epochs.

As a quantitative measurement of the reconstruction quality, we use the Weighted-to-Spherically-Uniform Peak Signal-to-Noise Ratio (WS-PSNR) [15]. The WS-PSNR places more importance to pixels with large area and, therefore, is stronger correlated with subjective quality than PSNR. Similar results were obtained for Spherical PSNR (S-PSNR) [20], which is not discussed here due to space constraints. To evaluate the compression performance, we plot the Rate-Distortion (RD) curves of each model in Fig. 5, where the WS-PSNR values for different quality parameters $\lambda$ are averaged over the test images and plotted over the rate, given in bits per pixel (bpp).<fnref n="2" />

Fig. 5 also contains the Bjøntegaard-Delta (BD) rates [13], [14] in % and model sizes in number of trainable parameters, which are given for the three highest quality parameters $\lambda$ for each model.<fnref n="3" /> We can see in the red curve that our model including all extensions outperforms the spherical scale hyperprior model from [9] at all quality levels and saves 23.1% of the bit rate in terms of BD rate. However, the number of trainable parameters is considerably higher, which is mainly caused by the pixel shuffling operation. Using our transposed convolution instead (dotted red curve), reduces the model size by more than a factor of 3 while providing almost the same rate saving of 23.0% in terms of BD rate. We can also see from the green curves that the spherical attention modules and residual blocks reduce the bit rate by almost 10% while

<figure><img src="image_7.png" /><figcaption>Fig. 5. RD curves of different spherical models. The first number in parentheses is the BD-rate compared to the spherical Scale Hyperprior (SH) model (negative values show an improvement). Marker sizes scale with the number of trainable parameters (second number in parentheses).</figcaption></figure>

only leading to a moderate increase of the trainable parameters of roughly 25%. Similarly to the full model, our transposed convolution (dotted green curve) reduces the model size by a factor of 2.6 while only leading to a minor decrease in BD rate. Using transposed convolution for the original spherical scale hyperprior model (dotted blue curve) reduces the model size by a factor of around 4.1 with 1.3% more bits needed. Despite the transposed convolution increasing the bit rate by a small portion, all models using spherical transposed convolution achieve better reconstruction quality at higher bit rates compared to the equivalent models using pixel shuffling.

## V. CONCLUSION

In this paper, we showed that recent 2D image compression models significantly improve on-the-sphere learned compression models based on the OSLO framework. By incorporating attention modules, residual blocks as nonlinearities, and a spatial context model into the previous spherical scale hyperprior model, we save 23.1% of the bit rate based on WS-PSNR. For the unpooling operation, the proposed transposed convolution reduces the required number of parameters by a factor of 4 while maintaining almost the same filter expressiveness. In the future, we aim to extend the OSLO framework by a computationally efficient channel-wise context model as well as different model architectures including transformer-based networks.

<footnote n="2">²$\lambda \in \{0.0005, 0.0018, 0.0067, 0.0130, 0.025, 0.0483, 0.0932, 0.18\}$ and optionally $\lambda \in \{0.0009, 0.0035\}$.</footnote>

<footnote n="3">³$(N, M) = (142, 270)$ for $\lambda \ge 0.0483$, and otherwise (128, 192).</footnote>
REFERENCES

[1] J. Ballé, V. Laparra, and E. P. Simoncelli, "End-to-end optimized image compression," in *Proc. International Conference on Learning Representations (ICLR)*, 2017.

[2] J. Ballé, D. Minnen, S. Singh, S. J. Hwang, and N. Johnston, "Variational image compression with a scale hyperprior," in *Proc. International Conference on Learning Representations (ICLR)*, 2018.

[3] D. Minnen, J. Ballé, and G. D. Toderici, "Joint autoregressive and hierarchical priors for learned image compression," in *Proc. Advances in Neural Information Processing Systems (NeurIPS)*, vol. 31, 2018, pp. 10771-10780.

[4] Z. Cheng, H. Sun, M. Takeuchi, and J. Katto, "Learned image compression with discretized gaussian mixture likelihoods and attention modules," in *Proc. IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)*, 2020.

[5] D. He, Z. Yang, W. Peng, R. Ma, H. Qin, and Y. Wang, "Elic: Efficient learned image compression with unevenly grouped space-channel contextual adaptive coding," in *Proc. IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)*, 2022.

[6] F. Brand, J. Seiler, and A. Kaup, "Conditional residual coding: A remedy for bottleneck problems in conditional inter frame coding," *IEEE Transactions on Circuits and Systems for Video Technology (CVVT)*, vol. 34, no. 7, pp. 6445-6459, 2024.

[7] M. Li, J. Li, S. Gu, F. Wu, and D. Zhang, "End-to-end optimized 360° image compression," *IEEE Transactions on Image Processing*, vol. 31, pp. 6267-6281, 2022.

[8] A. Regensky, C. Herglotz, and A. Kaup, "Multi-model motion prediction for 360-degree video compression," *IEEE Access*, vol. 11, pp. 117004-117017, 2023.

[9] N. Mahmoudian Bidgoli, R. G. de A. Azevedo, T. Maugey, A. Roumy, and P. Frossard, "Oslo: On-the-sphere learning for omnidirectional images and its application to 360-degree image compression," *IEEE Transactions on Image Processing*, vol. 31, pp. 5813-5827, 2022.

[10] K. M. Gorski, E. Hivon, A. J. Banday, B. D. Wandelt, F. K. Hansen, M. Reinecke, and M. Bartelmann, "Healpix: A framework for high-resolution discretization and fast analysis of data distributed on the sphere," *The Astrophysical Journal*, vol. 622, no. 2, p. 759, 2005.

[11] R. Fermanian, T. Maugey, and C. Guillemot, "Spheredrunet: A spherical denoiser for omnidirectional images," in *Proc. IEEE International Symposium on Mixed and Augmented Reality Adjunct (ISMAR-Adjunct)*, 2023, pp. 417-422.

[12] J. Ballé, V. Laparra, and E. P. Simoncelli, "Density modeling of images using a generalized normalization transformation," in *Proc. International Conference on Learning Representations (ICLR)*, 2016.

[13] G. Bjøntegaard, "Calculation of average psnr differences betweenrd-curves, VCEG-M33," in *Proc. 13th Meet. Video Coding Experts Group*, Mar. 2001, pp. 1-5.

[14] C. Herglotz, H. Och, A. Meyer, G. Ramasubbu, L. Eichermüller, M. Kränzler, F. Brand, K. Fischer, D. T. Nguyen, A. Regensky, and A. Kaup, "The bjøntegaard bible why your way of comparing video codecs may be wrong," *IEEE Transactions on Image Processing*, vol. 33, pp. 987-1001, 2024.

[15] Y. Sun, A. Lu, and L. Yu, "Weighted-to-spherically-uniform quality evaluation for omnidirectional video," *IEEE Signal Processing Letters*, vol. 24, no. 9, pp. 1408-1412, 2017.

[16] J. Ballé, V. Laparra, and E. P. Simoncelli, "End-to-end optimization of nonlinear transform codes for perceptual quality," in *Proc. Picture Coding Symposium (PCS)*, 2016, pp. 1-5.

[17] D. Minnen and S. Singh, "Channel-wise autoregressive entropy models for learned image compression," in *Proc. IEEE International Conference on Image Processing (ICIP)*, 2020, pp. 3339-3343.

[18] L. Yuan, J. Luo, S. Li, W. Dai, C. Li, J. Zou, and H. Xiong, "Learned image compression with channel-wise grouped context modeling," in *Proc. IEEE International Conference on Image Processing (ICIP)*, 2021, pp. 2099-2103.

[19] J. Xiao, K. A. Ehinger, A. Oliva, and A. Torralba, "Recognizing scene viewpoint using panoramic place representation," in *Proc. IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)*, 2012, pp. 2695-2702.

[20] M. Yu, H. Lakshman, and B. Girod, "A framework to evaluate omnidirectional video coding schemes," in *Proc. IEEE International Symposium on Mixed and Augmented Reality (ISMAR)*, 2015, pp. 31-36.