# Enhancing Perceptual Quality Assessment for 360-Degree Images Based on Adaptive Patch Labeling and Multi-Label Learning

Abderrezzaq Sendjasni, Mohamed-Chaker Larabi

► To cite this version:

Abderrezzaq Sendjasni, Mohamed-Chaker Larabi. Enhancing Perceptual Quality Assessment for 360-Degree Images Based on Adaptive Patch Labeling and Multi-Label Learning. 2024 IEEE International Conference on Image Processing (ICIP), IEEE Signal Processing Society, Oct 2024, Abu Dhabi, France. pp.1267-1273, 10.1109/ICIP51287.2024.10647256. hal-05120335

HAL Id: hal-05120335

https://hal.science/hal-05120335v1

Submitted on 19 Jun 2025

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
ENHANCING PERCEPTUAL QUALITY ASSESSMENT FOR 360-DEGREE IMAGES BASED
ON ADAPTIVE PATCH LABELING AND MULTI-LABEL LEARNING

Abderrezzaq Sendjasni and Mohamed-Chaker Larabi

CNRS, Univ. Poitiers, XLIM, UMR 7252, France

ABSTRACT

This paper delves into the intricate field of perceptual
quality assessment specifically tailored for 360-degree im-
ages, aiming to advance the precision of quality models. In
contrast to conventional methodologies that associate differ-
ent regions within images to mean opinion scores (MOS),
our study introduces a paradigm shift. We propose a novel
approach where the model is trained to predict multi-labels
derived from subjective and objective measures, leveraging
a sophisticated quality labeling framework designed to cap-
ture nuanced perceptual distinctions across diverse regions
in panoramic content. This allows for a flexible and stable
training process. In addition, a loss function taking into ac-
count the magnitude and direction of quality labels under
a multi-label learning scheme is designed, using absolute
and directional distances as loss functions. Experimental re-
sults underscore the limitations of relying solely on MOS as
unique labels. The efficiency of our approach becomes evi-
dent through improved performance, showcasing its potential
for advancing the precision of perceptual quality assessment
models.

***Index Terms***— Image quality assessment, 360-degree, Quality score distribution, Multi-label learning, Convolutional neural networks.

# 1. INTRODUCTION

In the evolving landscape of immersive technologies, 360-
degree images have become integral components of virtual
experiences, offering viewers panoramic and interactive ex-
periences. As the demand for high-quality visual content con-
tinues to surge, the need for robust perceptual quality assess-
ment models becomes paramount. In recent years, several
models have been designed, mimicking the visual experience
of these types of media, as well as addressing the associated
challenges in quantifying their perceptual quality accurately.
Still, various intricacies persist in capturing the nuanced as-
pects of perceptual quality, prompting a need for innovative
approaches to refine existing conventional methodologies.

The complexity inherent to immersive media underscores
the requirement for sophisticated perceptual quality assess-
ment models that surpass conventional methodologies. Tradi-

tional perceptual quality assessment is typically tied to mean
opinion scores (MOS) as the exclusive target for model train-
ing [1–11]. These models adopt a viewport or patch-based
strategy [12, 13]. Both strategies involve sampling regions
from 360-degree images, and quality assessment is then per-
formed on these localized regions. A labeling step is required
for the latter strategy, where the MOS is used to represent
the global quality of each patch. However, relying solely on
MOS may not adequately represent local quality, especially
given the complex spatial content inherent to 360-degree im-
ages. The intricate spatial characteristics of these images ne-
cessitate a more appropriate approach to accurately capture
local variations. In addition, the use of MOS as the sole la-
bel implies the use of simpler distance measures as objec-
tive functions, such as mean squared error (MSE) as in [6, 7,
14], between ground truth and predicted quality scores. This
constraint may overlook the intricate perceptual relationships
within and between patches, potentially limiting the model’s
ability to discern subtle perceptual differences in complex
spatial environments and accurately capture the local quality.
Besides, it limits the various possibilities that can be used to
refine and improve image quality assessment (IQA) in general
and 360-degree assessment in particular.

Subjective quality assessment datasets [14–18], the an-
chor to date for IQA designs, often exhibit a limitation in
that they provide solely a MOS per 360-degree image. While
this score is a valuable indicator of overall quality, its sin-
gular nature tends to oversimplify the perceptual evaluation,
potentially overlooking localized variations and nuanced as-
pects within the panoramic content. Understanding how the
MOS is aggregated becomes a challenging aspect, given the
complexity of the scene and the intricate spatial resolutions
of 360-degree images, 4k+, coupled with their diverse visual
content. This necessitates a more granular and comprehensive
approach to quality assessment. Besides, the restriction of a
single MOS per image hampers the capture of fine-grained
perceptual differences within various regions or patches of the
360-degree scene. Addressing this limitation becomes crucial
for the advancement of perceptual quality assessment models,
as it enables a more detailed understanding of how viewers
perceive specific elements within immersive media. It also
opens up more possibilities for addressing the quality evalua-
tion of 360-degree images.
Relying on a single MOS value, often not locally representative of patches and viewports, poses potential challenges. Local perceptual quality, specific to the content within patches, may not consistently align with the global quality of the entire image. This inconsistency arises due to the high non-stationarity of image content across spatial locations and the intricate interactions between content and distortions, as highlighted in previous studies [19]. The complexities associated with these variations are particularly emphasized in the context of 360-degree images, where exploration is viewported-based, yet the images encompass a high diversity of content within the entire 360 degrees. The interplay between local and global perceptual quality becomes crucial to consider, as it influences the effectiveness of IQA models in various ways. How, then, can we efficiently represent the local quality within patches and viewports to ensure a more comprehensive understanding of perceptual quality in 360-degree images?

In this paper, we investigate possible solutions to solve the abovementioned limitations, aiming to enhance the precision of quality models. Our primary contribution lies in the shift from associating different parts of 360-degree images with MOS to training a model capable of predicting quality labels. This paradigm shift is made possible by a novel quality labeling framework that enables the model to capture nuanced perceptual differences across diverse regions within the panoramic content. In addition, we introduce a carefully designed loss function that combines absolute difference and directional distance, allowing the model to learn both the magnitude and direction of quality labels under a multi-label learning scheme. By using a more granular and comprehensive approach to perceptual quality assessment, our methodology provides a richer representation of localized variations and nuances, ultimately contributing to the refinement of immersive media quality evaluation.

## 2. THE PROPOSED STUDY

The goal of this study is to leverage well-known 2D IQA models to accurately label patches sampled from 360-degree images, subsequently solving the limitations of solely labeling with MOS. Besides, such a configuration enhances the model training by using a more suitable loss function that takes into account multi-label regression. Fig. 1 illustrates the fundamental components involved in the proposed study, including patch sampling and generation, objective and subjective labeling, and the model's training.

### 2.1. Patches generation

In this step, non-overlapping patches of size 128 × 128 pixels are systematically sampled from the 360-degree image I using the equirectangular representation. The equirectangular projection maps the spherical 360-degree image onto a 2D plane, allowing for systematic sampling.

Let $P_I$ represent the set of patches sampled from the 360-degree image I (see Fig. 1 (a)), where each patch $p_i$ is defined by its position in the equirectangular projection. The sampling process can be formally expressed as:

$$P_I = \{(u, v) | u = u_{\min} + \Delta u \cdot i \cdot s, v = v_{\min} + \Delta v \cdot j \cdot s\}, (1)$$

here, $u_{\min}$ and $v_{\min}$ represent the minimum coordinates in the equirectangular projection, $\Delta u$ and $\Delta v$ are the intervals between sampled patches, and $s$ is the size of each non-overlapping patch (128 pixels). The indices $i$ and $j$ determine the position of the patch in the sampled grid. The non-overlapping nature of these patches ensures systematic coverage of the entire 360-degree image.

### 2.2. Subjective and objective measures for quality labeling

In this step, each patch $p_i \in P_I$ is labeled using a variety of 2D IQA models denoted as $M_1, M_2, ..., M_m$. These models include traditional approaches such as NIQE [20], PIQUE [21], BRISQUE [22], BIQAA [23], BLIINDS [24], CPBD [25], FADE [26], and deep learning ones such as DB-CNN [19], MUSIQ [27], TRES [28], NIMA [29], PaQ-2-PiQ [30]. The labeling process results in a quality vector $Q_i$ for each patch, where $Q_i$ is a multi-label quality vector representing the perceptual assessment of the patch $p_i$ by different 2D IQA models. Each element of $Q_i$ corresponds to the quality score assigned by a specific IQA model. This process ensures a comprehensive evaluation of perceptual quality, considering various aspects captured by different IQA methodologies. In addition, the multi-label per patch emphasizes the multidimensional nature of the quality vector, capturing the diverse perspectives provided by different 2D IQA models. It facilitates a thorough characterization of perceptual quality in local regions.

At this stage, we enhance $Q_i$ by incorporating MOS as an additional quality label, representing the subjective quality of patch $p_i$ inherited from the 360-degree image I:

$$Q_i = \begin{bmatrix} M_1(p_i) \\ M_2(p_i) \\ \vdots \\ M_m(p_i) \\ \text{MOS}_I \end{bmatrix} \quad (2)$$

$Q_i$ encapsulates both the objective quality derived from the quality vector obtained through various 2D IQA models ($M_1, M_2, ..., M_m$), and the subjective quality expressed by the MOS associated with the patch, as illustrated in Fig. 1 (b). This representation signifies the comprehensive representation of perceptual quality, combining the insights from objective IQA models and subjective human evaluations.
<figure><img src="image_2.png" /><figcaption>Fig. 1: Illustration of the fundamental component involved in the proposed study. "n" represents the number of patches and "m" the number of objective quality labels. GAP stands for global average pooling.</figcaption></figure>

## 2.3. Model training

In contrast to training a quality model to associate different parts from 360-degree images to the MOS, as explained in Sec.1, we can train a model to predict a quality vector. This is possible thanks to the quality labeling framework described in the previous section. Therefore, the model is no longer limited to a unique label, i.e. MOS, but is trained to predict a vector of quality scores. This approach allows for a more granular understanding of perceptual quality, capturing localized variations and nuances across the panoramic content. The shift towards predicting a quality vector aligns with the advancements in quality labeling methodologies and provides a richer representation of perceptual quality for immersive media. At this stage, a multi-layer perception (MLP) is used for regressing the obtained feature vectors VFs to quality vectors $\hat{Q}$s as depicted in Fig. 1 (c). The feature vectors are generated using the ResNet-50 [31] with its pre-trained weights.

### 2.3.1. Directional distance as a loss function

We designed a loss function using the absolute difference and directional distance to comprehensively capture localized perceptual quality. This loss function combines $\mathcal{L}_{L_1}$, to measure the absolute differences between predicted and ground truth quality scores, with directional distance, captured by cosine similarity. By measuring the cosine of the angle between the vectors, the loss function captures the directional relationship between predicted and ground truth vectors, providing a measure of alignment between them [32, 33].

Building upon the perceptual quality assessment model, the training process introduces a multi-label loss function, denoted as $L(\hat{Q}_i, Q_i)$, where $\hat{Q}_i$ represents the predicted quality vectors, and $Q_i$ corresponds to the ground truth labels:

$$ \mathcal{L}(\hat{Q}_i, Q_i) = \lambda \cdot \mathcal{L}_{L_1} + (1 - \lambda) \cdot \mathcal{L}_{CS}, \quad (3) $$

with:

$$ \mathcal{L}_{L_1} = \frac{1}{N} \sum_{k=1}^{N} |\hat{Q}_{ik} - Q_{ik}|, \quad (4) $$

and:

$$ \mathcal{L}_{CS} = 1 - \frac{1}{N} \sum_{k=1}^{N} \alpha \cdot \cos(\hat{Q}_{ik} - Q_{ik}), \quad (5) $$

where $\lambda$ controls the influence of $\mathcal{L}_{L_1}$ and $\mathcal{L}_{CS}$ losses, respectively. This strategic weighting enables a tailored emphasis on different aspects of perceptual quality during model training. $N$ represents the batch size and $\alpha$ is a learnable weight to adjust the importance of the quality labels prior to measuring the cosine distance between $\hat{Q}_i$ and $Q_i$. As a result, the overall objective during training is to minimize the defined loss function $\mathcal{L}(\hat{Q}_i, Q_i)$ with respect to both the model parameters $\Theta$ and the learnable weights:

$$ \min_{\Theta, \alpha} \mathcal{L}(\hat{Q}_i, Q_i) \quad (6) $$

# 3. RESULTS AND DISCUSSION

## 3.1. Implementation details and evaluation protocols

The proposed study is conducted using two widely recognized 360-IQA datasets, namely OIQA [15] and CVIQ [14]. The CVIQ dataset comprises 528 distorted 360-degree images, all with a 4K resolution. These images are generated from 16 distortion-free images by three types of compression (JPEG, AVC, and HEVC) at 11 different levels. The OIQA dataset is a collection of 320 distorted 360-degree images characterized by high resolutions (approximately 11K). These images are derived from 16 reference images, each subject to distortion
through four different types of distortions (JPEG, JPEG 2000, Gaussian noise, and Gaussian blur) at five distinct levels.

The validation process for the proposed study is carried out on a computer equipped with 32GB of RAM and an Nvidia Quadro T2000 MAX-Q 4GB GPU. A robust 4-fold cross-validation approach is employed within each of the selected datasets. Each fold undergoes extensive training for 100 epochs. The datasets are partitioned into an 80%/20% split for training and testing, respectively. To ensure a complete separation between the training and testing sets, distorted samples associated with the same pristine image are exclusively allocated to the same set. This meticulous partitioning strategy enhances the reliability of the model evaluation by preventing any potential data leakage between the training and testing phases. The performances are reported using the Pearson linear correlation coefficient (PLCC) to evaluate accuracy, the Spearman rank-order correlation coefficient (SRCC) for monotonicity, and the root mean squared error (RMSE) to evaluate prediction errors.

## 3.2. Performance analysis

With the objective of assessing the efficacy of locally labeling patches through a combination of subjective and objective labels, our analysis compares the performances under different loss functions, specifically $\mathcal{L}_{L_1}$, $\mathcal{L}_{L_2}$, and $\mathcal{L}_{L_1} + \mathcal{L}_{CS}$. The performances in terms of correlation and prediction errors are summarized in Table 1. The incorporation of subjective opinions alongside objective labels is a key aspect in evaluating the models. The use of $\mathcal{L}_{L_1}$ demonstrates robust performance, and the addition of quality labels $Q$ further enhances the results. $\mathcal{L}_{L_2}$, although competitive, shows slightly lower performance. Additionally, the $\mathcal{L}_{L_1}$ combined with cosine similarity ($\mathcal{L}_{L_1} + \mathcal{L}_{CS}$) presents an interesting approach, leveraging both local labels and similarity measures to compute the absolute and directional distances between the predicted labels and ground truth ones. This behavior remains consistent across various datasets, underscoring a reproducible pattern. Specifically, there is a noteworthy improvement of approximately 3.14% in PLCC and 3.58% in SRCC within OIQA when training the model using $\mathcal{L}_{L_1} + \mathcal{L}_{CS}$ compared to $\mathcal{L}_{L_1}$ (MOS). A similar trend is discernible within the CVIQ dataset. This observation indicates that relying exclusively on the MOS for training not only constrains the range of available loss functions but also potentially limits the model's capability to adeptly correlate patches with distinct characteristics to the same label. This comprehensive comparison aims to shed light on the most effective strategy for locally labeling patches, considering the interplay between subjective and objective metrics in the assessment process.

Moreover, delving into the learning curve analysis during training can offer valuable insights into the behavior of the model. Fig. 2 illustrates the average training loss over K-fold cross-validation, providing insights into the central tendency

<figure><img src="image_3.png" /><figcaption>**Fig. 2:** Mean loss curve with standard deviation over K-fold cross-validation. The curves represent the loss function (training labels).</figcaption></figure>

of the model's performance. The shaded region represents the standard deviation, offering a measure of the variability and stability of the model's learning across different folds. As it can be seen, the plot showcases the impact of different loss functions on model training. The fluctuating curve using only $\mathcal{L}_{L_1}$ with MOS labels contrasts with the smoother trend observed when using $\mathcal{L}_{L_1}$ with quality labels $Q$. The fluctuating curve implies higher variability in the model's performance during training, potentially stemming from noise in the training data due to the inherent redundancy introduced by relying solely on the MOS labels. This redundancy may contribute to the model struggling to converge consistently. Conversely, the smoother curve observed with the use of quality labels $Q$ suggests a more stable and consistent learning process. The loss curve indicates that the model is adapting well to the training data, leading to reliable convergence. This enhanced stability is attributed to the representativeness of quality labels $Q$, which contributes to a more robust and effective learning process. Notably, the combination of the magnitude and direction computations of the quality labels, represented by $\mathcal{L}_{L_1} + \mathcal{L}_{CS}$, yields a stable and minimized loss, indicating superior convergence and performance. These behaviors align with the earlier findings related to correlation performances.

### 3.2.1. Cross-dataset evaluation

To demonstrate the robust generalization capabilities of the loss function with the multi-label regression across diverse content generated under distinct conditions, we conducted a comprehensive cross-dataset evaluation. In this evaluation, we first trained the model on the OIQA dataset and assessed its performance on the CVIQ dataset, and vice versa. The outcomes of this evaluation, summarized in Table 2, are reported with PLCC, SRCC and RMSE. These performances are reported for the overall datasets and specifically highlight the model's efficiency in handling JPEG compression, a common distortion present in both datasets.

Based on the reported performances, when training on OIQA and evaluating on CVIQ, the $\mathcal{L}_{L_1}$ with MOS achieves notable performance in both overall and JPEG-specific evalu-
<table><caption>**Table 1:** Performance evaluation in terms of PLCC, SRCC, and RMSE for various loss functions, exclusively using the predicted MOS for overall quality computation (The median over 4-fold is provided and the best performances are highlighted in **bold**).</caption><thead><tr><th rowspan="2">Loss function</th><th rowspan="2">Training</th><th colspan="3">OIQA</th><th colspan="3">CVIQ</th></tr><tr><th>PLCC</th><th>SRCC</th><th>RMSE</th><th>PLCC</th><th>SRCC</th><th>RMSE</th></tr></thead><tbody><tr><td rowspan="2">L<sub>L1</sub></td><td>MOS</td><td>0.9368</td><td>0.9279</td><td>0.0498</td><td>0.9380</td><td>0.9233</td><td>0.0483</td></tr><tr><td>Q</td><td>0.9448</td><td>0.9320</td><td>0.0476</td><td>0.9525</td><td>0.9281</td><td>0.0410</td></tr><tr><td rowspan="2">L<sub>L2</sub></td><td>MOS</td><td>0.9313</td><td>0.9230</td><td>0.0516</td><td>0.9355</td><td>0.9100</td><td>0.0476</td></tr><tr><td>Q</td><td>0.9342</td><td>0.9243</td><td>0.0520</td><td>0.9496</td><td>0.9283</td><td>0.0416</td></tr><tr><td>L<sub>L1</sub> + L<sub>CS</sub></td><td>Q</td><td><strong>0.9667</strong></td><td><strong>0.9618</strong></td><td><strong>0.0366</strong></td><td><strong>0.9606</strong></td><td><strong>0.9498</strong></td><td><strong>0.0384</strong></td></tr></tbody></table>

<table><caption>**Table 2:** Cross-dataset performances (the best performances are highlighted in **bold**).</caption><thead><tr><th rowspan="3">Loss function</th><th colspan="3">Train on OIQA / Test on CVIQ</th><th colspan="9">Train on CVIQ / Test on OIQA</th></tr><tr><th colspan="3">Overall</th><th colspan="3">JPEG</th><th colspan="3">Overall</th><th colspan="3">JPEG</th></tr><tr><th>PLCC</th><th>SRCC</th><th>RMSE</th><th>PLCC</th><th>SRCC</th><th>RMSE</th><th>PLCC</th><th>SRCC</th><th>RMSE</th><th>PLCC</th><th>SRCC</th><th>RMSE</th></tr></thead><tbody><tr><td>L<sub>L1</sub> (MOS)</td><td>0.8223</td><td>0.7584</td><td>0.0813</td><td>0.9226</td><td>0.8360</td><td>0.0572</td><td>0.7026</td><td>0.6986</td><td>0.1024</td><td>0.8771</td><td>0.8608</td><td>0.0753</td></tr><tr><td>L<sub>L1</sub> (Q)</td><td>0.8640</td><td>0.8112</td><td>0.0720</td><td>0.9401</td><td>0.8421</td><td>0.0525</td><td>0.7519</td><td>0.7514</td><td>0.0948</td><td><strong>0.9324</strong></td><td><strong>0.9198</strong></td><td><strong>0.0567</strong></td></tr><tr><td>L<sub>L1</sub> + L<sub>CS</sub> (Q)</td><td><strong>0.8737</strong></td><td><strong>0.828</strong></td><td><strong>0.0695</strong></td><td><strong>0.9562</strong></td><td><strong>0.8669</strong></td><td><strong>0.0499</strong></td><td><strong>0.7735</strong></td><td><strong>0.7743</strong></td><td><strong>0.0912</strong></td><td>0.9292</td><td>0.9162</td><td>0.0580</td></tr></tbody></table>

Nevertheless, the version of the $L_{L1}$ employing multi-label (Q) surpasses its MOS version, underscoring its efficacy in capturing nuanced quality aspects. This underscores the limitations associated with relying solely on MOS as labels. Notably, the combined $L_{L1} + L_{CS}$ loss function stands out as the top-performing configuration, emphasizing the collaborative efficiency of absolute and directional differences in enhancing the model's predictive capabilities.

When training on CVIQ and evaluating on OIQA, a similar pattern emerges, mirroring the observations from the previous scenario. Notably, the exception lies in $L_{L1}$ with multi-label (Q), which exhibits the best performances, slightly outperforming $L_{L1} + L_{CS}$. This difference underscores the intricate dynamics involved in cross-dataset evaluations and emphasizes the effectiveness of the proposed multi-label approach in capturing perceptual quality across diverse datasets. Besides, the observed performance difference between $L_{L1}$ with multi-label (Q) and $L_{L1} + L_{CS}$ when training on CVIQ and evaluating on OIQA could be attributed to the distinct characteristics of the two datasets. As OIQA contains a broader variety of distortions compared to CVIQ, the multi-label approach might better adapt to the diverse set of distortions present in OIQA. This flexibility allows the model to capture and generalize across different types of distortions, contributing to its superior performance in a cross-dataset evaluation. However, as CVIQ predominantly focuses on compression-related distortions, the model trained on CVIQ might be optimized to excel in scenarios that involve similar compression distortions. The use of co-

sine similarity, which measures the directional relationships between vectors, may perform slightly less effectively when dealing with a diverse set of distortions, as highlighted by the provided performances.

### 3.2.2. Impact of the regularization parameter $\lambda$

<table><caption>**Table 3:** Performance evaluation with varied regularization parameter $\lambda$ (The median over 4-fold is provided and the best performances are highlighted in **bold**).</caption><thead><tr><th></th><th>λ Value</th><th>λ=0.25</th><th>λ=0.5</th><th>λ=0.75</th></tr></thead><tbody><tr><td rowspan="3">OIQA</td><td>PLCC</td><td>0.9440</td><td>0.9531</td><td><strong>0.9667</strong></td></tr><tr><td>SRCC</td><td>0.9336</td><td>0.9442</td><td><strong>0.9618</strong></td></tr><tr><td>RMSE</td><td>0.0464</td><td>0.0441</td><td><strong>0.0366</strong></td></tr><tr><td rowspan="3">CVIQ</td><td>PLCC</td><td>0.9490</td><td>0.9582</td><td><strong>0.9606</strong></td></tr><tr><td>SRCC</td><td>0.9269</td><td>0.9421</td><td><strong>0.9498</strong></td></tr><tr><td>RMSE</td><td>0.0422</td><td>0.0385</td><td><strong>0.0384</strong></td></tr></tbody></table>

Table 3 presents results in terms of PLCC, SRCC, and RMSE, illustrating the impact of the regularization parameter $\lambda$ on the loss function's performance and the trade-off between the two loss components, i.e. $L_{L1}$ and $L_{CS}$. This parameter plays a crucial role in determining the relative importance of each loss term in the overall objective function. A smaller $\lambda$ assigns more weight to $L_{CS}$, emphasizing the importance of the cosine similarity loss. This may encourage
the model to focus more on capturing relational information and structural similarities between data points. While a larger value prioritizes $\mathcal{L}_{CS}$. The analysis of different values of $\lambda$ allows for the fine-tuning of the model's behavior, providing insights into the optimal balance between the two loss terms for achieving the desired training objectives for the selected datasets. The observed trend in performance suggests that the best results are attained with $\lambda = 0.75$, emphasizing cosine similarity and demonstrating robust correlations, potentially at the expense of accentuating directional relationships over absolute differences. However, the performances obtained on CVIQ indicate that a balanced approach with $\lambda = 0.5$ is still effective, showcasing the importance of achieving equilibrium between $\mathcal{L}_{L_1}$ and $\mathcal{L}_{CS}$ for optimal model performance.

<figure><img src="image_4.png" /><figcaption>**Fig. 3:** Mean loss curve with standard deviation over K-fold cross-validation. The curves represent the loss associated with each $\lambda$ value.</figcaption></figure>

In addition to the performance analysis, we provide in Fig. 3 the mean loss curves with standard deviation over K-fold cross-validation, each curve corresponding to a different $\lambda$ value. The curves appear smooth and stable. This observation suggests that the model's performance, as measured by the loss, is consistent across the K-fold cross-validation, and the variation in the loss values is relatively small. The smooth and stable nature of the curves implies that the used loss function, i.e. $\mathcal{L}_{L_1} + \mathcal{L}_{CS}$ with a regularization parameter $\lambda$ is contributing to a stable training process. Particularly, the loss associated with $\lambda = 0.75$, shows promising results in terms of minimizing the loss. This supports the obtained performances reported in Table 3.

### 3.2.3. Impact of the learnable parameter $\alpha$

It is widely recognized that the incorporation of learnable parameters often leads to automatic enhancements in performance. In various machine learning contexts, the adaptability and optimization capabilities of learnable parameters contribute to improved model outcomes, allowing the system to refine its representations and adapt to intricate patterns within the data. This inherent ability to learn and adjust enables models to achieve higher levels of accuracy and effectiveness in diverse tasks [34]. Therefore, it is essential to assess their effectiveness from a performance point of view. Table 4

presents an evaluation of the impact of the trainable parameter $\alpha$ on both OIQA and CVIQ performances. The results highlight significant improvements in terms of PLCC and SRCC with the inclusion of $\alpha$ compared to the configuration without it. The percentage gain, depicted in the final row, underscores the noteworthy positive impact of $\alpha$ across both quality assessment metrics, with an increase of 1.113% and 1.794% in OIQA, as well as 1.311% and 2.884% in CVIQ for PLCC and SRCC, respectively. These findings emphasize the beneficial contribution of the learnable parameter $\alpha$ to overall model performance.

<table><caption>**Table 4:** Assessment of the impact of the trainable parameter $\alpha$ on the performances. The final row illustrates the percentage improvement achieved between the two configurations.</caption><thead><tr><th rowspan="2"></th><th colspan="2">OIQA</th><th colspan="2">CVIQ</th></tr><tr><th>PLCC</th><th>SRCC</th><th>PLCC</th><th>SRCC</th></tr></thead><tbody><tr><th>With α</th><td><strong>0.9667</strong></td><td><strong>0.9618</strong></td><td><strong>0.9606</strong></td><td><strong>0.9498</strong></td></tr><tr><th>W/O α</th><td>0.9560</td><td>0.9447</td><td>0.9481</td><td>0.9228</td></tr><tr><th>Perc. gain</th><td><strong>+1.113</strong></td><td><strong>+1.794</strong></td><td><strong>+1.311</strong></td><td><strong>+2.884</strong></td></tr></tbody></table>

## 4. CONCLUSION

In conclusion, this study represents a significant value in the field of perceptual quality assessment for 360-degree images. By contrasting from conventional approaches that rely solely on MOS, we introduced a novel paradigm where our model learns from both subjective and objective measures, capturing nuanced differences across diverse regions in panoramic content. The emphasis on multi-label prediction and the strategic use of a sophisticated quality labeling framework contribute to a more refined understanding of perceptual quality. Our exploration of customized loss functions and the varied regularization parameters further highlights the importance of a balanced approach in considering both absolute differences and directional relationships. The proposed methodology showcases promising results in cross-dataset evaluations, demonstrating the potential for generalization to new content and conditions. As the field of immersive media processing continues to evolve, our study paves the way for enhanced perceptual quality assessment pipelines, contributing valuable insights to the landscape of 360-degree image processing and evaluation.

## 5. ACKNOWLEDGEMENT

This work is partially funded by the Nouvelle-Aquitaine research council under projects SIMOREVA360 2018-1R50112 and PERCEIVE AAPR2020-2019-8498010.
## 6. REFERENCES

[1] Y. Liu, X. Yin, Y. Wang, Z. Yin, and Z. Zheng, "HVS-based perception-driven no-reference omnidirectional image quality assessment," *IEEE TIM*, vol. 72, pp. 1-11, 2022.

[2] Y. Wang, H. Li, and Q. Jiang, "Dynamically attentive view-port sequence for no-reference quality assessment of omnidirectional images," *Frontiers in Neuroscience*, vol. 16, pp. 1022041, 2022.

[3] A. Sendjasni and MC. Larabi, "Attention-aware patch-based CNN for blind 360-degree image quality assessment," *Sensors*, vol. 23, no. 21, 2023.

[4] C. Tian, X. Chai, G. Chen, F. Shao, Q. Jiang, X. Meng, L. Xu, and YS. Ho, "VSOIQE: A novel viewport-based stitched 360 omnidirectional image quality evaluator," *IEEE TCSVT*, vol. 32, no. 10, pp. 6557-6572, 2022.

[5] Y. Liu, X. Yin, G. Yue, Z. Zheng, J. Jiang, Q. He, and X. Li, "Blind omnidirectional image quality assessment with representative features and viewport oriented statistical features," *JVCIR*, vol. 91, pp. 103770, 2023.

[6] T. Wu, S. Shi, H. Cai, M. Cao, J. Xiao, Y. Zheng, and Y. Yang, "Assessor360: Multi-sequence network for blind omnidirectional image quality assessment," *arXiv preprint arXiv:2305.10983*, 2023.

[7] M. Zhou, L. Chen, X. Wei, X. Liao, Q. Mao, H. Wang, H. Pu, J. Luo, T. Xiang, and B. Fang, "Perception-oriented u-shaped transformer network for 360-degree no-reference image quality assessment," *IEEE TB*, vol. 69, no. 2, pp. 396-405, 2023.

[8] C. Zhang and S. Liu, "No-reference omnidirectional image quality assessment based on joint network," in *30th ACM ICM*, 2022, pp. 943-951.

[9] J. Li, Y. Zhao, W. Ye, K. Yu, and S. Ge, "Attentive deep stitching and quality assessment for 360° omnidirectional images," *IEEE JSTSP*, vol. 14, no. 1, pp. 209-221, 2020.

[10] C. Tian, F. Shao, X. Chai, Q. Jiang, Long Xu, and Yo-Sung Ho, "Viewport-sphere-branch network for blind quality assessment of stitched 360° omnidirectional images," *IEEE TCSVT*, vol. 33, no. 6, pp. 2546-2560, 2023.

[11] L. Yang, M. Xu, T. Liu, L. Huo, and X. Gao, "TVFormer: Trajectory-guided visual quality assessment on 360° images with transformers," in *30th ACM ICM*, 2022, pp. 799-808.

[12] A. Sendjasni, MC. Larabi, and FA. Cheikh, "Patch-based CNN model for 360 image quality assessment with adaptive pooling strategies," *Electronic Imaging*, vol. 34, no. 9, pp. 395-1, 2022.

[13] A. Sendjasni, MC. Larabi, and FA. Cheikh, "Convolutional neural networks for omnidirectional image quality assessment: A benchmark," *IEEE TCSVT*, vol. 32, no. 11, pp. 7301-7316, 2022.

[14] W. Sun, X. Min, G. Zhai, K. Gu, H. Duan, and S. Ma, "MC360IQA: A multi-channel CNN for blind 360-degree image quality assessment," in *IEEE JSTSP*, 2020, vol. 14, pp. 64-77.

[15] H. Duan, G. Zhai, X. Min, Y. Zhu, Y. Fang, and X. Yang, "Perceptual Quality Assessment of Omnidirectional Images," in *IEEE ISCAS*, Florence, Italy, 2018, pp. 1-5.

[16] X. Zheng, G. Jiang, M. Yu, and H. Jiang, "Segmented spherical projection-based blind omnidirectional image quality assessment," *IEEE Access*, vol. 8, pp. 31647-31659, 2020.

[17] L. Yang, M. Xu, X. Deng, and B. Feng, "Spatial attention-based non-reference perceptual quality prediction network for omnidirectional images," in *IEEE ICME*, Shenzhen, China, 2021, pp. 1-6.

[18] X. Sui, K. Ma, Y. Yao, and Y. Fang, "Perceptual quality assessment of omnidirectional images as moving camera videos," *IEEE TVCG*, vol. 28, no. 8, pp. 3022-3034, 2022.

[19] W. Zhang, K. Ma, J. Yan, D. Deng, and Z. Wang, "Blind image quality assessment using a deep bilinear convolutional neural network," *IEEE TCSVT*, vol. 30, no. 1, pp. 36-47, 2020.

[20] A. Mittal, R. Soundararajan, and AC. Bovik, "Making a "completely blind" image quality analyzer," *IEEE SPL*, vol. 20, no. 3, pp. 209-212, 2013.

[21] N. Venkatanath and D. Praneeth et al., "Blind image quality evaluation using perception based features," in *IEEE NCC*, 2015, pp. 1-6.

[22] A. Mittal, A. K. Moorthy, and A. C. Bovik, "No-reference image quality assessment in the spatial domain," *IEEE TIP*, vol. 21, no. 12, pp. 4695-4708, 2012.

[23] G. Salvador and C. Gabriel, "Blind image quality assessment through anisotropy," *J. Opt. Soc. Am. A*, vol. 24, no. 12, pp. B42-B51, 2007.

[24] MA. Saad, AC. Bovik, and C. Charrier, "Blind image quality assessment: A natural scene statistics approach in the dct domain," *IEEE TIP*, vol. 21, no. 8, pp. 3339-3352, 2012.

[25] N. Narvekar and L. Karam, "A no-reference perceptual image sharpness metric based on a cumulative probability of blur detection," in *IEEE QoMEX*, San Diego, USA, 2009, pp. 87-91.

[26] LK. Choi, J. You, and AC. Bovik, "Referenceless prediction of perceptual fog density and perceptual image defogging," *IEEE TIP*, vol. 24, no. 11, pp. 3888-3901, 2015.

[27] J. Ke, Q. Wang, Y. Wang, P. Milanfar, and F. Yang, "MUSIQ: Multi-scale image quality transformer," in *IEEE ICCV*, Montreal, Canada, 2021, pp. 5128-5137.

[28] A. Golestaneh, S. Dadsetan, and K. Kitani, "No-reference image quality assessment via transformers, relative ranking, and self-consistency," in *IEEE/CVF WCACV*, 2022, pp. 3209-3218.

[29] H. Talebi and P. Milanfar, "Nima: Neural image assessment," *IEEE TIP*, vol. 27, no. 8, pp. 3998-4011, 2018.

[30] Z. Ying and H. Niu et al., "From patches to pictures (PaQ-2-PiQ): Mapping the perceptual space of picture quality," in *IEEE CVPR*, Seattle, USA, 2020, pp. 3572-3582.

[31] K. He, X. Zhang, S. Ren, and J. Sun, "Deep residual learning for image recognition," in *IEEE CVPR*, Las Vegas, NV, USA, 2016, pp. 770-778.

[32] B.W. Keelan, *Handbook of image quality: characterization and prediction*, CRC Press, 2002.

[33] KR. Castleman, *Digital image processing*, Prentice Hall Press, 1996.

[34] I. Goodfellow, Y. Bengio, and A. Courville, *Deep learning*, MIT press, 2016.