# TransformAR: A light-weight transformer-based metric for Augmented Reality quality assessment

Aymen Sekhri, Mohamed-Chaker Larabi, Seyed Ali Amirshahi

<figure><img src="image_1.png" /></figure>

## To cite this version:

Aymen Sekhri, Mohamed-Chaker Larabi, Seyed Ali Amirshahi. TransformAR: A light-weight transformer-based metric for Augmented Reality quality assessment. Signal Processing: Image Communication, 2026, 140, pp.117437. 10.1016/j.image.2025.117437. hal-05369692

HAL Id: hal-05369692

https://hal.science/hal-05369692v1

Submitted on 17 Nov 2025

HAL is a multi-disciplinary open access archive for the deposit and dissemination of scientific research documents, whether they are published or not. The documents may come from teaching and research institutions in France or abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est destinée au dépôt et à la diffusion de documents scientifiques de niveau recherche, publiés ou non, émanant des établissements d'enseignement et de recherche français ou étrangers, des laboratoires publics ou privés.

<figure><img src="image_2.png" /></figure>
<figure><img src="image_3.png" /></figure>

<figure><img src="image_4.png" /></figure>

# TransformAR: A light-weight transformer-based metric for Augmented Reality quality assessment

Aymen Sekhri<sup>a,b ID,*</sup>, Mohamed-Chaker Larabi<sup>a ID</sup>, Seyed Ali Amirshahi<sup>b ID</sup>

## ARTICLE INFO

**Keywords:**

Augmented Reality

VIT-based framework

Image processing

Image quality assessment

## ABSTRACT

As Augmented Reality (AR) technology continues to gain traction in various sectors, ensuring a superior user experience has become an essential challenge for both academic researchers and industry professionals. However, the task of automatically predicting the quality of AR images remains difficult due to several inherent challenges, particularly the issue of visual confusion arising from the overlap of virtual and real-world elements. This paper introduces transformAR, a novel and efficient transformer-based framework designed to objectively assess the quality of AR images. The proposed model uses pre-trained vision transformers to capture content features from AR images, calculates distance vectors to measure the impact of distortions, and employs cross-attention-based decoders to effectively model the perceptual qualities of the AR images. Additionally, the training framework uses regularization techniques and label smoothing-like method to reduce the risk of overfitting. Through comprehensive experiments, we demonstrate that transformAR outperforms existing state-of-the-art approaches, offering a more reliable and scalable solution for AR image quality assessment.

## 1. Introduction

Augmented Reality (AR) technologies are reshaping the way users engage with digital content by overlaying virtual elements onto the real world. Achieving consistently high visual fidelity in these environments remains a significant challenge, one that critically influences the user experience across fields such as navigation, healthcare, education, and entertainment [1]. To improve the Quality of Experience (QoE) in AR applications, it is crucial to implement objective quality assessment metrics that account for multiple perceptual factors [2,3]. Image quality, being a vital determinant of QoE [4], has traditionally been evaluated through extensive psychophysical studies. However, these methods are labor-intensive and result in the limited availability of subjective datasets.

Unlike conventional 2D images, AR scenes pose unique challenges due to the superposition of virtual and real content. A key artifact in this context is *visual confusion*, referring to the perceptual interference caused by overlapping foreground and background elements. Visual confusion in AR can be broadly divided into two types: *binocular visual confusion*, which arises when conflicting information is presented to both eyes in stereoscopic displays, and *monocular visual confusion*, which occurs when virtual content is superimposed on real scenes within a single eye's field of view. Both forms of interference can

degrade visual clarity, reduce task performance, and even induce discomfort or disorientation [5-8]. Monocular visual confusion is particularly problematic in single-eye AR presentations, where virtual overlays directly compete with real-world content in the same visual space, producing interference distinct from binocular rivalry [9,10]. Under such conditions, the visual system must resolve competing streams of information, often leading to segmentation difficulties that suppress critical augmented content from conscious awareness. This results in unstable perception and figure-ground ambiguity, where virtual content may fade, flicker, or become masked by the background. The severity of monocular confusion further depends on the properties of the overlays: high-contrast content tends to create stronger interference, while low-contrast overlays risk becoming imperceptible against complex backgrounds [11,12].

Objective Image Quality Assessment (IQA) metrics play an important role in many applications, such as dynamically monitoring image and video compression methods [13]. They provide standardized and repeatable ways to assess and optimize algorithms without relying on subjective human judgments, and they are generally categorized into three types: Full-Reference IQA (FR-IQA), Reduced-Reference IQA (RR-IQA), and No-Reference IQA (NR-IQA). FR-IQA methods rely on both pristine and degraded images, RR-IQA methods use partial reference

<footnote><sup>a</sup> CNRS, Université de Poitiers, XLIM, Poitiers, France</footnote>

<footnote><sup>b</sup> Norwegian University of Science and Technology, Gjøvik, Norway</footnote>

<footnote>* Corresponding author at: CNRS, Université de Poitiers, XLIM, Poitiers, France.</footnote>

<footnote>E-mail addresses: aymen.sekhri@univ-poitiers.fr (A. Sekhri), chaker.larabi@univ-poitiers.fr (M.-C. Larabi), s.ali.amirshahi@ntnu.no (S.A. Amirshahi).</footnote>
information, while NR-IQA methods require no reference at all, making them particularly attractive for real-world deployments. Classical FR-IQA metrics such as Mean Squared Error (MSE) and Peak Signal-to-Noise Ratio (PSNR) quantify pixel-level distortions but often fail to align with human perception. Perceptual metrics like the Structural SIMilarity index (SSIM) [14] addressed this issue by considering luminance, contrast, and structural information. Variants such as MS-SSIM [15], IW-SSIM [16], and FSIM [17] further improved robustness, while other methods including GMSD [18] and VSI [19] have also been proposed. More recently, deep learning-based approaches have emerged. Kang et al. [20] introduced CNN-based architectures for simultaneous quality prediction and distortion identification, while other studies explored convolutional sparse coding [21] and learned feature-based IQA [22]. LPIPS [23], which measures distances between deep features extracted from networks such as AlexNet [24] and VGG [25], has become one of the most widely used methods. With the rise of transformer architectures, CNN-transformer hybrids have been proposed to capture both local and global dependencies, addressing the limited receptive field of CNNs [26–28]. Despite their success for natural images, these methods have not been fully adapted to the AR setting, where the quality depends not only on classical distortions, but also on the impact of visual confusion and the trade-off between the influences of virtual objects and the real background. To address AR-specific challenges, Duan et al. [9] developed CNN-based AR-IQA models such as CFIQA, ARIQA, and ARIQA+. These approaches leverage architectures such as VGG and ResNet [29], combined with spatial and channel attention mechanisms and edge detection to better predict perceptual quality. Nevertheless, they remain constrained by the local receptive fields of CNNs, which limit their ability to capture global contextual information in AR scenes. In contrast, transformer architectures are well-suited to modeling long-range dependencies [30], enabling them to capture both local details and global semantic context, which are crucial in the presence of complex background-foreground interactions and visual confusion.

Progress in AR-IQA also depends on the availability of high-quality datasets. Early works focused mainly on distortions in 3D meshes and point clouds rendered on uniform backgrounds [31,32]. To bridge this gap, Duan et al. [9] introduced ARIQA, the first publicly available dataset for AR image quality was collected in a Virtual Reality (VR) environment designed to simulate AR scenarios. In this setting, Virtual content including webpages, natural images, and cartoons, was overlaid on omnidirectional real-world backgrounds. The dataset incorporates a variety of distortions, such as JPEG compression, scaling, contrast changes, and monocular visual confusion, where the same view is rendered to both eyes. More recently, ARIQA-PICO [33] was proposed to better reflect real AR conditions. Unlike ARIQA, which relied on VR-based simulation, ARIQA-PICO was captured using the PICO4 AR headset in real mixed-reality environments. It includes 15 virtual images rendered at three transparency levels ($\lambda \in \{0.25, 0.5, 0.75\}$) combined with 10 real backgrounds, yielding 450 AR scenes. Subjective ratings from 20 participants revealed three distinct levels of visual confusion—unilateral suppression, acceptable confusion, and intense confusion—with medium transparency ($\lambda = 0.5$) generating the strongest interference. Both datasets are discussed in detail in Section 3.1.

Based on these observations, there is a clear need for new and efficient AR-IQA methods that can account for visual confusion and background-foreground interactions. In this paper, we propose a novel transformer-based AR-IQA metric that leverages the strength of transformers in capturing global dependencies. Compared to the preliminary version of this work presented at MMSP, the contributions of this extended study are fourfold:

* A lightweight encoder-decoder transformer architecture designed for efficient global quality feature extraction, even with limited training data;

* An adaptation of a label smoothing-like strategy to mitigate over-confidence in predictions and enhance model robustness;

* Perceptual confusion modeling that jointly considers real background and virtual foreground images along with superimposed AR image, providing a more comprehensive quality assessment;

* Extensive experiments and benchmarking on both ARIQA and ARIQA-PICO datasets, going beyond the MMSP paper to validate generalization and robustness.

The remainder of this paper details the proposed framework (Section 2), presents experimental results (Section 3), and concludes with insight and future research directions (Section 4).

## 2. Proposed method

This work is an extension of our proposed approach, called TransformAR, described in [34]. It is composed of three stages (Fig. 1). First, content-aware encoders, based on the Vision Transformer (ViT) architecture [35,36], which are responsible for capturing the global features of both reference and distorted images. The second stage involves the quality-aware decoder, designed to extract quality-related attributes from the processed content. Lastly, the third stage consists of the regressors, mapping the high-dimensional feature embeddings to continuous values, ultimately producing the predicted Mean Opinion Score (pMOS). Throughout the remainder of this paper, we use the term AR image or superimposed image to refer to the distorted image, defined as the fusion of the virtual foreground image and the real background image. In addition, we denote by TransformAR-s the simplified variant of our framework, which relies on a single reference (either the virtual foreground or the real background image) rather than both simultaneously. Architecturally, TransformAR-s is identical to TransformAR, except that one of the reference branches has been removed (Fig. 1).

### 2.1. Content-aware encoders

As highlighted in Section 1, although CNNs have made significant contributions to the field of IQA, their inability to capture long-range dependencies and global context [38] due to the limited receptive field prevents them from effectively mimicking the human visual system in its observation of a given scene. To address this limitation, we employ three frozen ViT encoders, pre-trained using a self-supervised algorithm called DINO [37]. These encoders are designed to capture global content information from both reference and distorted images. For instance, if the reference and distorted images are identical — that is, the distorted image is actually perfect — the frozen encoders will produce identical representations in a deterministic manner. However, any alteration in the distorted image will lead to differences at the feature level, providing a signal about the applied noise or distortions that cause this deviation from the perfect representation. The ViT architecture [36] divides the input image into non-overlapping patches of size $P \times P$. Let the input image have dimensions $H \times W \times 3$, where $H$ is the height, $W$ is the width. The image is first reshaped into $N = \frac{H \times W}{p^2}$ patches, each of size $P \times P \times 3$. These patches are then flattened into a sequence of 1D vectors, also known as tokens, where each token $x_i$ represents a flattened patch. Next, these tokens are projected through a linear layer $W_{embed}$ to obtain a sequence of embedding tokens $z_i$ of dimension $C$. The resulting sequence of tokens is then passed through a series of transformer layers. Each transformer layer consists of layer normalization, attention mechanism, skip-connection, and feed-forward network. The self-attention computes pairwise attention scores between tokens to capture dependencies between different regions of the image. The self-attention operation can be written as:

$$ \text{Attention}(Q, K, V) = \text{softmax}\left(\frac{QK^T}{\sqrt{d}}\right) \cdot V \qquad (1) $$

where $Q$, $K$, and $V$ are the queries, keys, and values, respectively, and $d$ is the dimension of the embedding space. This mechanism allows ViT to focus on different parts of the image, improving the extraction
<figure><img src="image_5.png" /><figcaption>**Fig. 1.** Illustration of the proposed TransformAR, which consists of, (i) Content-aware encoders $F_{\theta}(\cdot)$: frozen ViT encoders [37] that map an input image $I_{*} \in \mathbb{R}^{H \times W \times 3}$ to patch embeddings $f_{*} \in \mathbb{R}^{N \times C}$, where $H \times W$ is the image resolution, $N$ the number of patches, and $C$ the embedding dimension. (ii) Quality-aware decoders $G_{SA}(\cdot)$ and $G_{SB}(\cdot)$: trainable cross-attention based transformer block that receives $d_{SA} = |f_{S} - f_{A}|$ and $d_{SB} = |f_{S} - f_{B}|$ and the reference features $f_{A}$ and $f_{B}$ producing intermediate quality-aware representations $h_{SA}$ and $h_{SB}$. (iii) Regressors $R_{SA}(\cdot)$ and $R_{SB}(\cdot)$: map $h_{SA}$ and $h_{SB}$ into quality scores $S_{SA}$ and $S_{SB}$. These are combined by a weighting factor $\zeta$ into the pMOS.</figcaption></figure>

<figure><img src="image_6.png" /><figcaption>**Fig. 2.** Illustration of reference and superimposed (distorted) images from the ARIQA dataset [9].</figcaption></figure>

<figure><img src="image_7.png" /><figcaption>**Fig. 3.** Illustration of reference and superimposed (distorted) images from the ARIQA-PICO dataset [33].</figcaption></figure>

of relevant features. Since ViTs are data-hungry models [39], and given the scarcity of AR-IQA data, using the full transformer blocks resulted in rapid overfitting. Since our task does not involve solving high-level semantic tasks such as segmentation or classification, we use only the first two blocks of the ViT to map the input images into meaningful embeddings for the quality assessment task. For a given superimposed image $I_S$, regarded as the distorted image whose quality is to be predicted, two associated reference images are provided: the real background image $I_B$ and the virtual foreground image $I_A$ (Fig. 2). The quality is then estimated as $pMOS = FR - IQA(I_S, I_A, I_B)$. The superimposed image $I_S$ is generated as follows:

$$I_S = \lambda \circ D(I_A) + (1 - \lambda) \circ I_B, \qquad (2)$$

where $D(\cdot)$ denotes the applied distortions, and $\lambda$ is the mixing parameter that determines the visual confusion level $\lambda \in [0.26, 0.42, 0.58, 0.74]$.

Each encoder $F(\cdot)$ generates feature vectors $f_{*} = \{f_{*_{i}}\}_{i=0}^{N}$ for each image, where $* \in \{S, A, B\}$. Since the content encoders are frozen, they generate identical representations for the same images, and any alteration to one of the inputs will be reflected at the feature level as the impact of distortion and visual confusion. To capture this, we compute the absolute differences between the distorted features $f_S$ and their corresponding references $f_A$ and $f_B$, producing the following sequence of tokens:

$$\begin{aligned} d_{SA_i} &= |f_{S_i} - f_{A_i}| \\ d_{SB_i} &= |f_{S_i} - f_{B_i}| \end{aligned} \qquad (3)$$

Here, $d_{SA}$ and $d_{SB}$ represent the distances between feature representations of $I_S$ and $I_A$, and $I_S$ and $I_B$, respectively. These distance vectors are then further refined to represent the quality features by being fed as inputs to the decoders, along with the reference features, described in the next section.
## 2.2. Quality-aware decoders

This stage is crucial for the framework, as its role is to produce the quality features by refining the distortion features computed earlier. To achieve this, we adapt a transformer block by replacing the self-attention mechanism with a cross-attention mechanism. This modification allows the model to compute quality features more effectively making use of both the reference information — produced by DINO — and the distortion-induced distance information, computed in Eq. (3). In this configuration, queries are derived from the reference embeddings, while keys and values are obtained from the distance embeddings. The decoders $G_{SA}(\cdot)$ and $G_{SB}(\cdot)$ use cross-attention to model the relationship between the reference image and the distortions applied:

$$CA(Q_{f_A}, K_{d_{SA}}, V_{d_{SA}}) = \text{softmax} \left( \frac{Q_{f_A} K_{d_{SA}}^T}{\sqrt{d_k}} \right) V_{d_{SA}} \quad (4)$$

$$CA(Q_{f_B}, K_{d_{SB}}, V_{d_{SB}}) = \text{softmax} \left( \frac{Q_{f_B} K_{d_{SB}}^T}{\sqrt{d_k}} \right) V_{d_{SB}}$$

where $Q_{f_A}$ and $Q_{f_B}$ represent the queries from $f_A$ and $f_B$, respectively, while $K_{d_{SA}}$, $V_{d_{SA}}$, $K_{d_{SB}}$, and $V_{d_{SB}}$ correspond to the keys and values derived from the distance vectors $d_{SA}$ and $d_{SB}$, respectively. The parameter $d_k$ refers to the dimension of the key vectors. The output embeddings are then normalized, followed by a skip connection and projection layers. This process generates the final output vectors $g_{SA} = G_{SA}(f_A, d_{SA})$ and $g_{SB} = G_{SB}(f_B, d_{SB})$. The latters capture long-range quality information, considered as crucial for assessing the impact of distortions and visual confusion encoded in the distance vectors.

## 2.3. Regressors

Since predicting the mean opinion score is a regression problem, we use two regression modules, $R_{SA}(\cdot)$ and $R_{SB}(\cdot)$, to generate quality scores for each patch $x_i$ from the superimposed image $I_S$. With the outputs of the regression modules, we obtain $h_{SA} = R_{SA}(g_{SA})$ and $h_{SB} = R_{SB}(g_{SB})$, where $g_{SA}$ and $g_{SB}$ are the embeddings generated by the decoders. Patch-level scores are then aggregated using learnable linear layers with parameters $W_{SA}$ and $W_{SB}$, respectively. This yields the final scores $S_{SA} = W_{SA} \cdot h_{SA}^\top$, which reflects the quality between the foreground and the superimposed image, and $S_{SB} = W_{SB} \cdot h_{SB}^\top$, which reflects the quality between the background and the superimposed image. The predicted MOS (pMOS) is finally computed as a weighted sum of these scores:

$$pMOS = \zeta S_{SA} + (1 - \zeta) S_{SB}, \quad (5)$$

where $\zeta$ is empirically set to 0.51. This approach allows for an effective combination of both quality scores in the final MOS prediction.

## 2.4. Training procedure

Our training procedure includes key aspects such as selecting the loss function and addressing overfitting with regularization techniques. We selected the Huber loss [40] as our main objective function. The benefits of using this type of loss lie in the fact that it combines the characteristics of the Mean Squared Error (MSE) and Mean Absolute Error (MAE), providing a balance between robustness to outliers and sensitivity to small errors. The Huber loss is given by:

$$HL_{\delta}(y, \hat{y}) = \begin{cases} \frac{1}{2}(y - \hat{y})^2 & \text{for } |y - \hat{y}| \le \delta, \\ \delta(|y - \hat{y}| - \frac{1}{2}\delta) & \text{otherwise.} \end{cases} \quad (6)$$

where $y$ represents the target value, $\hat{y}$ represents the predicted value, and $\delta$ is a hyperparameter. The latter determines the limit where the loss transitions from a quadratic behavior for small residuals to a linear one for larger residuals [40]. Due to data scarcity limited to the novelty of the field, our model is prone to overfitting. To address this issue,

we use elastic net regularization [41]. This combines Lasso and Ridge regularization methods, effectively preventing overfitting and serving as a feature selection mechanism in regression models. In our case, given that we employed the Huber objective function, the overall loss function is defined as follows:

$$L(y, \hat{y}, \beta) = HL_{\delta}(y, \hat{y}) + \gamma(\alpha \|\beta\|_1 + (1 - \alpha) \|\beta\|_2^2), \quad (7)$$

where $\beta$ represents the learnable weights, $\gamma$ controls the penalty terms, and $\alpha$ balances $l_1$ and $l_2$ penalties. Since MOS depend on various subjective factors such as the number of participants, evaluation environments, and individual biases, they inherently contain noise and can thus be viewed as noisy labels. To address issues related to limited data and mitigate the model's tendency toward overconfidence, we adapt the concept of label smoothing — a regularization technique traditionally employed in classification — to the regression context of MOS prediction. Specifically, we introduce small random perturbations to each MOS label during training:

$$y_e = y + \lambda_n e, \quad (8)$$

where $\lambda_n$ is uniformly sampled from the interval [-1, 1], and $e$ is drawn from a normal distribution ($\epsilon \sim N(0, \sigma^2)$). This approach effectively reduces overfitting, encourages model generalization, and slightly improves predictive performance.

For implementation, we trained the model using the AdamW optimizer [42] with a learning rate of $1e^{-4}$, a batch size of 32 images, and for 150 epochs. The learning rate was automatically lowered if there was no improvement after 10 epochs. The implementation in PyTorch was run on an NVIDIA Tesla V100S-PCIE-32 GB GPU.

# 3. Experimental results

## 3.1. Datasets

To evaluate the effectiveness of the proposed method, we conducted experiments on both the ARIQA dataset [9] and the ARIQA-PICO dataset [33]. The ARIQA dataset comprises 560 superimposed AR images, each associated with a single MOS. It contains 20 omnidirectional background images evenly distributed between indoor and outdoor scenes, as well as 20 reference virtual foreground images classified into three categories: webpages, natural images, and graphics. All images have a resolution of 1440 × 900. Each AR reference image was subjected to one of six different degradations — including JPEG compression, scaling, and contrast adjustments — along with visual confusion as a quality factor [9]. From these combinations, 560 stimuli were selected for subjective evaluation involving 23 participants using HTC VIVE Pro Eye VR headsets. Participants rated the perceived image quality on a 10-point scale using a single-stimulus evaluation approach. The purpose of this experiment was to study the effects of various image degradations and visual confusion on perceived quality in AR contexts. Each evaluated viewport, consisting of a foreground AR layer overlaid on a background image, was recorded along with its corresponding quality score (Fig. 2).

The ARIQA-PICO dataset, on the other hand, was introduced to better represent real AR viewing conditions and to study visual confusion as a perceptual distortion. Unlike ARIQA, which simulated AR using VR with omnidirectional backgrounds, ARIQA-PICO was captured with the PICO4 AR headset, recording real mixed-reality scenes. It includes 15 virtual images (nature, websites, cartoons) displayed at three transparency levels $\lambda \in \{0.25, 0.5, 0.75\}$, combined with 10 real-world backgrounds to form 450 AR scenes of resolution 1920 × 1080 (Fig. 3). Twenty participants rated the visual quality of both the virtual and real components using a 10-point single-stimulus protocol, producing 900 MOS values. This dataset highlights how transparency influences visual confusion — low $\lambda$ suppresses virtual content, high $\lambda$ suppresses reality, and medium $\lambda$ produces the strongest mutual interference — and serves as a challenging benchmark for AR image quality assessment.
For evaluation, ARIQA-PICO is divided into six folds for training and testing. For the ARIQA dataset, we introduced a 50-fold random partition, denoted by $\mathcal{X}$, to enable a more robust evaluation. Each fold $j$ was randomly split into a training set $\mathcal{T}_j$ with 280 samples and a test set $S_j$ with 280 samples. This split ensures no scene overlap, as formally defined in Eqs. (9) and (10), where $\mathcal{D}$ denotes the complete ARIQA dataset.

$$D = \{[I_{A_i}, I_{B_i}, I_{S_i}, \text{MOS}_i]\}_{i=1}^{560} \qquad (9)$$

$$\mathcal{X} = \{(\mathcal{T}_j, S_j) | \mathcal{T}_j \cap S_j = \emptyset\}_{j=1}^{50}, \quad \text{where} \quad \bigcup_{j=1}^{50} S_j = \mathcal{D} \qquad (10)$$

## 3.2. Performance metrics

To assess the performance of the evaluated models, we employ three widely recognized statistical correlation metrics, the Spearman's Rank-Order Correlation Coefficient (SROCC), the Kendall's Rank-Order Correlation Coefficient (KROCC), and the Pearson Linear Correlation Coefficient (PLCC). These metrics serve to quantify the degree of agreement between the predicted quality scores and human subjective judgments. The SROCC ($\rho$) measures the strength and direction of a monotonic relationship between two ranked variables. It is computed as:

$$\rho = 1 - \frac{6 \sum_{i=1}^{N} d_i^2}{N(N^2 - 1)} \qquad (11)$$

where $d_i$ denotes the difference between the ranks of the predicted and ground-truth quality scores for the $i$th sample, and $N$ is the total number of samples. The KROCC ($\tau_b$) evaluates the ordinal association between two variables by comparing the number of concordant (C) and discordant (D) pairs:

$$\tau_b = \frac{C - D}{\sqrt{(C + D + T)(C + D + U)}} \qquad (12)$$

where $T$ is the number of pairs tied only in the first variable, and $U$ is the number of pairs tied only in the second variable [43]. The PLCC, denoted by $r$, quantifies the strength and direction of the linear relationship between two continuous variables. It is mathematically expressed as:

$$r = \frac{\sum_{i=1}^{N} (P_i - \bar{P})(S_i - \bar{S})}{\sqrt{\sum_{i=1}^{N} (P_i - \bar{P})^2 \cdot \sum_{i=1}^{N} (S_i - \bar{S})^2}} \qquad (13)$$

where $P_i$ and $S_i$ are respectively the values in the vectors $P$ and $S$ for the image $i$, $\bar{P}$ and $\bar{S}$ are respectively the mean values of vectors $P$ and $S$, while $N$ is the number of images in the database. To accurately compute the PLCC, a non-linear regression function is applied to transform the objective quality scores into a scale that aligns better with human perception. Hence, the following five-parameter logistic function is used:

$$Q' = \beta_1 \left( \frac{1}{2} - \frac{1}{1 + e^{\beta_2(Q - \beta_3)}} \right) + \beta_4 Q + \beta_5 \qquad (14)$$

where $Q$ represents the objective quality score, $Q'$ denotes the mapped (fitted) score, and $\beta_i$ ($i$ = 1, 2, 3, 4, 5) are the parameters estimated during curve fitting. Overall, higher values across these three metrics reflect stronger perceptual consistency, indicating that the model's predictions align more closely with human quality judgments.

## 3.3. Comparison with state-of-the-art methods

In this section, we present a comprehensive comparison between our proposed model, TransformAR, and a range of state-of-the-art IQA metrics for AR content. Table 1 reports the performance on the ARIQA dataset, while Table 2 summarizes the results on the ARIQA-PICO benchmark. Both tables include classical full-reference (FR) metrics (e.g., PSNR, SSIM [14], FSIM [17]), deep learning-based metrics such as LPIPS [23], and AR-specific approaches such as CFIQA [9], ARIQA [9],

<table><caption>**Table 1**
Comparison with state-of-the-art performance. Higher values indicate better correlation with human perceptual judgments.</caption><thead><tr><th>Model</th><th>SROCC↑</th><th>KROCC↑</th><th>PLCC↑</th><th># Params</th></tr></thead><tbody><tr><td>PSNR</td><td>0.2900</td><td>0.1977</td><td>0.3091</td><td>-</td></tr><tr><td>SSIM [14]</td><td>0.2521</td><td>0.1698</td><td>0.2850</td><td>-</td></tr><tr><td>MS-SSIM [15]</td><td>0.1019</td><td>0.0687</td><td>0.1491</td><td>-</td></tr><tr><td>GMSD [18]</td><td>0.2680</td><td>0.1842</td><td>0.3261</td><td>-</td></tr><tr><td>MDSI [44]</td><td>0.2931</td><td>0.1978</td><td>0.3023</td><td>-</td></tr><tr><td>IW-SSIM [16]</td><td>0.3499</td><td>0.2410</td><td>0.3635</td><td>-</td></tr><tr><td>DSS [45]</td><td>0.3587</td><td>0.2448</td><td>0.3906</td><td>-</td></tr><tr><td>HaarPSI [46]</td><td>0.3883</td><td>0.2677</td><td>0.4243</td><td>-</td></tr><tr><td>VIF [47]</td><td>0.4848</td><td>0.3417</td><td>0.5175</td><td>-</td></tr><tr><td>VSI [19]</td><td>0.5632</td><td>0.4021</td><td>0.5802</td><td>-</td></tr><tr><td>SR-SIM [48]</td><td>0.6093</td><td>0.4282</td><td>0.6212</td><td>-</td></tr><tr><td>FSIM [17]</td><td>0.6124</td><td>0.4419</td><td>0.6320</td><td>-</td></tr><tr><td>LPIPS [23]</td><td>0.7624</td><td>0.5756</td><td>0.7591</td><td>20.02M</td></tr><tr><td>CFIQA [9]</td><td>0.7787</td><td>0.5863</td><td>0.7695</td><td>20.12M</td></tr><tr><td>ARIQA [9]</td><td>0.7902</td><td>0.5967</td><td>0.7824</td><td>20.12M</td></tr><tr><td>ARIQA+ [9]</td><td>0.8124</td><td>0.6184</td><td>0.8136</td><td>35.02M</td></tr><tr><td>TransformAR-s (ours)</td><td><strong>0.7938</strong></td><td><strong>0.6011</strong></td><td><strong>0.7945</strong></td><td><strong>11.39M</strong></td></tr><tr><td>TransformAR (ours)</td><td><strong>0.8267</strong></td><td><strong>0.6359</strong></td><td><strong>0.8251</strong></td><td><strong>15.32M</strong></td></tr></tbody></table>

and its improved variant ARIQA+. On the ARIQA benchmark (Table 1), our full model achieves SROCC = 0.8267, KROCC = 0.6359, and PLCC = 0.8251, outperforming ARIQA+ (SROCC = 0.8124, KROCC = 0.6184, PLCC = 0.8136) while requiring less than half the parameters (15.32M vs. 35.02M). Moreover, our model processes three images (two references and one distorted) in approximately 65.3 ms. These results demonstrate that the transformer-based design captures quality-relevant features more efficiently than previous AR-specific deep models, even in the presence of challenging degradations and perceptual phenomena such as visual confusion. We also introduce a lightweight variant, TransformAR-s, which relies only on one reference content and does not require both pristine references. Despite the reduced input, it achieves competitive results on ARIQA (SROCC = 0.7938, KROCC = 0.6011, PLCC = 0.7945) using only the foreground.

On the ARIQA-PICO benchmark (Table 2), since each superimposed image has separate scores for the FG and BG, we used TransformAR-s, which shows strong robustness across different evaluation settings; both references (AR/BG), FG only (AR), or BG only (BG). The model achieves the best overall performance when both AR/BG are available (SROCC = 0.8251, KROCC = 0.6475, PLCC = 0.8304), while maintaining state-of-the-art results in single-reference scenarios. These findings highlight three key advantages; high correlation with human perception compared to classical and deep AR-specific metrics, model efficiency with fewer parameters yet higher accuracy, and robustness to limited reference information, making TransformAR-s practical for real-world AR applications.

## 3.4. Ablation study

To evaluate the contributions of individual components in our approach, we performed an ablation study using randomly chosen five folds from the folds set $\mathcal{X}$. Specifically, we systematically removed or altered components such as the Quality-Aware decoder, $l_1$-distance calculation, label smoothing, elastic net regularization, and Huber loss. The results are presented in Table 3, which indicates the importance of each component in improving the performance of the model. In particular, omitting the quality-aware decoder substantially reduced the performance of the quality metric (SROCC: 0.6161, KROCC: 0.4408, PLCC: 0.6242). This shows the crucial role of the quality-aware decoder in refining distance features to obtain more representative quality descriptors. Similarly, removing the $l_1$-distance computation even more severely impacted the performance of the model (SROCC: 0.4365, PLCC: 0.2960). This emphasizes how essential the $l_1$-distance is in capturing the impact of visual degradations and visual confusion. These
<caption>**Table 2**</caption>

Comparison with state-of-the-art performance on the ARIQA-PICO dataset [33], using both foreground and background reference images (AR/BG), only foreground (AR), and only background (BG). Best results are highlighted in **bold**.

<table><thead><tr><th rowspan="2">Model</th><th colspan="3">AR/BG</th><th colspan="3">AR</th><th colspan="3">BG</th></tr><tr><th>SROCC↑</th><th>KROCC↑</th><th>PLCC↑</th><th>SROCC↑</th><th>KROCC↑</th><th>PLCC↑</th><th>SROCC↑</th><th>KROCC↑</th><th>PLCC↑</th></tr></thead><tbody><tr><td colspan="10"><strong>Classical FR Metrics</strong></td></tr><tr><td>PSNR</td><td>0.4243</td><td>0.2969</td><td>0.4993</td><td>0.2725</td><td>0.1827</td><td>0.2989</td><td>0.6095</td><td>0.4420</td><td>0.6763</td></tr><tr><td>SSIM [14]</td><td>0.4079</td><td>0.2859</td><td>0.3346</td><td>0.3084</td><td>0.2205</td><td>0.4571</td><td>0.5002</td><td>0.3601</td><td>0.6327</td></tr><tr><td>MS-SSIM [15]</td><td>0.4452</td><td>0.3108</td><td>0.5007</td><td>0.4168</td><td>0.2863</td><td>0.4553</td><td>0.5214</td><td>0.3762</td><td>0.6564</td></tr><tr><td>GMSD [18]</td><td>0.3148</td><td>0.2196</td><td>0.4222</td><td>0.2044</td><td>0.1400</td><td>0.2783</td><td>0.4949</td><td>0.3567</td><td>0.5437</td></tr><tr><td>IW-SSIM [16]</td><td>0.4319</td><td>0.3018</td><td>0.5592</td><td>0.4947</td><td>0.3432</td><td>0.5104</td><td>0.4375</td><td>0.3127</td><td>0.5692</td></tr><tr><td>VIF [47]</td><td>0.1474</td><td>0.0967</td><td>0.3033</td><td>0.2086</td><td>0.1462</td><td>0.3741</td><td>0.1138</td><td>0.0773</td><td>0.2365</td></tr><tr><td>VSI [19]</td><td>0.4548</td><td>0.3199</td><td>0.4948</td><td>0.3562</td><td>0.2381</td><td>0.3847</td><td>0.6236</td><td>0.4633</td><td>0.7095</td></tr><tr><td>FSIM [17]</td><td>0.3571</td><td>0.2497</td><td>0.4538</td><td>0.1428</td><td>0.0966</td><td>0.1221</td><td>0.5322</td><td>0.3865</td><td>0.6246</td></tr><tr><td colspan="10"><strong>Deep FR Models</strong></td></tr><tr><td>LPIPS(VGG) [23]</td><td>0.5090</td><td>0.3621</td><td>0.6913</td><td>0.5540</td><td>0.3831</td><td>0.5693</td><td>0.6649</td><td>0.4997</td><td>0.8040</td></tr><tr><td>Baseline (SqueezeNet) [49]</td><td>0.5124</td><td>0.3666</td><td>0.5459</td><td>0.5324</td><td>0.3639</td><td>0.5745</td><td>0.6903</td><td>0.5296</td><td>0.5132</td></tr><tr><td>Baseline (VGG-16) [25]</td><td>0.5000</td><td>0.3548</td><td>0.5220</td><td>0.5203</td><td>0.3594</td><td>0.5669</td><td>0.6446</td><td>0.4783</td><td>0.4634</td></tr><tr><td>Baseline+ (VGG-16) [25]</td><td>0.5322</td><td>0.3803</td><td>0.5614</td><td>0.6036</td><td>0.4171</td><td>0.6176</td><td>0.6789</td><td>0.5151</td><td><strong>0.8061</strong></td></tr><tr><td>Baseline (ResNet-18) [29]</td><td>0.5205</td><td>0.3740</td><td>0.5208</td><td>0.5747</td><td>0.3963</td><td>0.5953</td><td>0.6920</td><td>0.5337</td><td>0.5273</td></tr><tr><td>Baseline (ResNet-34) [29]</td><td>0.5347</td><td>0.3868</td><td>0.3615</td><td>0.6214</td><td>0.4363</td><td>0.6427</td><td>0.6958</td><td>0.5372</td><td>0.5368</td></tr><tr><td colspan="10"><strong>CFIQA Models</strong></td></tr><tr><td>CFIQA(SqueezeNet) [9]</td><td>0.5454</td><td>0.3918</td><td>0.5409</td><td>0.5644</td><td>0.4059</td><td>0.6022</td><td>0.6250</td><td>0.4546</td><td>0.4707</td></tr><tr><td>CFIQA(AlexNet) [9]</td><td>0.5452</td><td>0.3898</td><td>0.5681</td><td>0.6282</td><td>0.4438</td><td>0.6632</td><td>0.6854</td><td>0.5213</td><td>0.7986</td></tr><tr><td>CFIQA(VGG-16) [9]</td><td>0.5250</td><td>0.3779</td><td>0.6051</td><td>0.6503</td><td>0.4852</td><td>0.6826</td><td>0.6153</td><td>0.4498</td><td>0.7512</td></tr><tr><td>CFIQA(VGG-19) [9]</td><td>0.5499</td><td>0.4000</td><td>0.5652</td><td>0.6913</td><td>0.5159</td><td>0.7252</td><td>0.6714</td><td>0.5058</td><td>0.4640</td></tr><tr><td colspan="10"><strong>Ours</strong></td></tr><tr><td>TransformAR-s</td><td><strong>0.8251</strong></td><td><strong>0.6475</strong></td><td><strong>0.8304</strong></td><td><strong>0.8400</strong></td><td><strong>0.6525</strong></td><td><strong>0.8580</strong></td><td><strong>0.7129</strong></td><td><strong>0.5702</strong></td><td><strong>0.7194</strong></td></tr></tbody></table>

<figure><img src="image_8.png" /><figcaption>**Fig. 4.** Progression of SROCC performance over epochs in a single fold.</figcaption></figure>

effects are measured at the feature level, which is extracted by a deterministic encoder as described in Section 2.1.

To statistically assess the impact of label smoothing, we performed a t-test between the predicted scores from the model trained with and without label smoothing. The test revealed a highly significant difference ($p \approx 5.014 \times 10^{-7}$), which is far below the threshold of 0.05. This indicates a significant difference in prediction accuracy when label smoothing is used. Furthermore, we observed a significant improvement during testing, as illustrated in Fig. 4, as well as in the ablation study (Table 3). Before epoch 20, the model achieved a high score during training, but a clear decline was observed during testing, indicating overfitting. However, after the 20th epoch, when label smoothing was applied, we noticed a modest decline in training performance, yet a significant improvement during testing. Fig. 5(a) displays the difference between the original and smoothed scores while Fig. 5(b) provides a better visual comparison of the data, showing the original labels (blue) versus the smoothed ones (red). As the model began to overfit on the original scores, we introduced label smoothing to reduce the model’s overconfidence, helping to generalize better during testing.

<caption>**Table 3**</caption>

Ablation study results across 5-fold validation.

<table><thead><tr><th>Model Variation</th><th>SROCC↑</th><th>KROCC↑</th><th>PLCC↑</th></tr></thead><tbody><tr><td>Without Decoder</td><td>0.6161</td><td>0.4408</td><td>0.6242</td></tr><tr><td>Without l₁-distance</td><td>0.4365</td><td>0.4443</td><td>0.2960</td></tr><tr><td>Without Label Smoothing</td><td>0.8269</td><td>0.6361</td><td>0.8221</td></tr><tr><td>Without Elastic Net</td><td>0.8427</td><td>0.6567</td><td>0.8471</td></tr><tr><td>Without Huber Loss</td><td>0.8374</td><td>0.6525</td><td>0.8443</td></tr><tr><td>All Components</td><td><strong>0.8461</strong></td><td><strong>0.6582</strong></td><td><strong>0.8481</strong></td></tr></tbody></table>

The exclusion of elastic net regularization exhibited only minimal changes, slightly decreasing the SROCC by 0.0034. This implies a limited role in influencing the overall predictive performance, despite its theoretical benefits in preventing overfitting. Replacing the Huber loss with MSE led to a noticeable reduction in performance, with the SROCC declining from 0.8461 to 0.8374. This clearly highlights the robustness advantage of the Huber loss against outliers in the training data. Ultimately, integrating all components achieved optimal results, demonstrating the highest scores in all evaluated metrics.

### 3.5. Impact of transformer depth and patch size

We investigate how varying transformer depth and patch size across DINO encoders affects TransformAR’s performance. Table 4 summarizes these impacts on SROCC and PLCC metrics. The results show that moderate transformer depths (with one and two blocks) consistently yield strong performance, while deeper configurations tend to degrade the results due to the increased model complexity compared to the small amount of data, leading to overfitting. Similarly, using smaller patch sizes (eight) does not provide significant improvements and often increases computational complexity without consistent gains in accuracy. This increase in complexity can be understood by examining the computational cost of the self-attention mechanism in transformers, which scales quadratically with the number of input tokens [35]. For an image of resolution $H \times W$ and patch size $P$, the number of tokens $n$ is $\frac{HW}{P^2}$. Thus, the complexity per layer would be:

$$ \mathcal{O}(n^2 \cdot d) = \mathcal{O}\left(\left(\frac{HW}{P^2}\right)^2 \cdot d\right), \qquad (15) $$
<figure><img src="image_9.png" /><figcaption>Fig. 5. Impact of label smoothing on the MOS.</figcaption></figure>

<figure><img src="image_10.png" /><figcaption>Fig. 6. t-SNE projection of the global pooled features from the quality-aware decoders. Colors indicate the quality score assigned to each superimposed image (see scale on the right).</figcaption></figure>

<table><caption>**Table 4**

Impact of transformer depth and patch size on TransformAR's performance (SROCC/PLCC).</caption><thead><tr><th>Num. Blocks</th><th>dino_vits16</th><th>dino_vitb16</th><th>dino_vits8</th><th>dino_vitb8</th></tr></thead><tbody><tr><td>1</td><td>0.818/0.822</td><td>0.805/0.815</td><td>0.798/0.806</td><td>0.811/0.816</td></tr><tr><td>2</td><td>0.803/0.814</td><td>0.814/0.815</td><td>0.807/0.814</td><td>0.810/0.819</td></tr><tr><td>4</td><td>0.797/0.805</td><td>0.807/0.812</td><td>0.812/0.817</td><td>0.787/0.796</td></tr><tr><td>6</td><td>0.797/0.801</td><td>0.800/0.797</td><td>0.801/0.806</td><td>0.783/0.788</td></tr><tr><td>8</td><td>0.805/0.810</td><td>0.809/0.818</td><td>0.801/0.806</td><td>0.808/0.809</td></tr><tr><td>10</td><td>0.806/0.815</td><td>0.794/0.802</td><td>0.796/0.799</td><td>0.806/0.809</td></tr><tr><td>12</td><td>0.812/0.815</td><td>0.783/0.789</td><td>0.801/0.803</td><td>0.806/0.810</td></tr></tbody></table>

where *d* is the embedding dimension. As the patch size *P* decreases, the number of tokens increases, leading to significantly higher computational demands.

These findings suggest that neither deeper transformers nor smaller patch sizes are necessary to achieve optimal performance. Instead, models with moderate depth and standard patch sizes strike a better balance between complexity and accuracy, making them more efficient and generalizable for AR quality assessment.

## 3.6. Qualitative analysis

To evaluate the effectiveness of the quality-aware decoder in capturing quality features, we propose to visualize the projected embeddings using t-SNE, an unsupervised non-linear dimensionality reduction technique for data exploration and visualizing high-dimensional data [50]. We begin by applying global average pooling to the decoder output, which consists of tokens representing patch-level information. The

pooled result serves as the overall quality embedding. Fig. 6(a) shows the two-dimensional t-SNE projection of the quality-aware decoder embeddings, colored by their mean opinion scores. We can see that samples with high MOS values (above 6) cluster mainly on the right side of the plot (yellow). Samples with lower MOS values lie to the left, shifting through orange and purple toward dark blue. Instead of forming rigid, isolated clusters, the points create a smooth gradient. This shows that the learned representation encodes quality as a continuous manifold. In other words, items judged to have similar perceptual quality are mapped to nearby regions. The transition from high to low quality follows a coherent path through the embedding space. This spatial organization confirms that the quality-aware decoder successfully preserves fine-grained distinctions in subjective quality judgments.

In contrast, Fig. 6(b) shows the t-SNE projection [50] of the output of the quality-aware decoder $G_{SB}(\cdot)$. The embeddings here appear to be more compact and exhibit relatively linear transitions in quality scores, indicating the difficulty of differentiating between background content. The background quality is primarily affected by the noise and distortions applied to the virtual image, resulting in tighter feature embeddings. Additionally, a perceptible gradual transition is becoming obvious on the diagonal from brighter samples at the bottom right (higher quality) to darker samples at the top left (lower quality). This gradual change demonstrates that, despite tighter grouping, the decoder effectively preserves a perceptually meaningful separation across quality levels. Although distinct clustering is not necessarily expected in regression-oriented tasks, this qualitative analysis confirms the decoder's ability to produce embeddings that closely align with their corresponding MOS.
<figure><img src="image_11.png" /><figcaption>**Fig. 7.** Cross-attention scores: The first row represents the AR image under JPEG compression artifacts, the second row corresponds to contrast adjustment, and the third row depicts image scaling. Each column corresponds to a different distortion level (two levels). Each attention map illustrates how reference tokens interact with the distance tokens for each distortion type, where brighter regions indicate higher attention scores (closer to one), and darker regions indicate lower attention scores (closer to zero).</figcaption></figure>

### 3.7. Cross-attention pattern analysis

In order to evaluate how the quality-aware decoder $G_{SA}(\cdot)$ responds to various distortions, we examine its cross-attention maps for a single AR image under the six degradations outlined in Section 3.1. For

each degradation, the content-aware encoder $F_0(\cdot)$ produces a fixed set of reference tokens (queries), while the superimposed image yields the distance tokens (keys), thus isolating the effect of each type of distortion as described in Section 2.1. Fig. 7 presents examples of cross-attention maps for each distortion (rows) at two severity levels
<caption>**Table 5**</caption>

Frobenius norms (Eq. (16)) of the differences between cross-attention maps and the reference map (without distortion). These values quantify how different distortion types and levels affect the overall attention scores.

<table><thead><tr><th>Distortion Type</th><th>Level 1</th><th>Level 2</th></tr></thead><tbody><tr><td>JPEG Compression</td><td>F<sub>01</sub> = 0.56</td><td>F<sub>04</sub> = 0.73</td></tr><tr><td>Contrast Adjustment</td><td>F<sub>02</sub> = 0.64</td><td>F<sub>05</sub> = 1.21</td></tr><tr><td>Image Scaling</td><td>F<sub>03</sub> = 0.95</td><td>F<sub>06</sub> = 1.43</td></tr></tbody></table>

(columns), showing the attention scores of the reference to distance tokens within a single-layer single-head decoder (Section 2.2). These heat maps illustrate how $G_{SA}$ adaptively reallocates its focus based on the severity of the applied distortion. To provide a comprehensive quantitative analysis for all types of distortion, Table 5 summarizes the Frobenius norms for all types and levels of distortion examined in this study.

As shown, the attention patterns remain broadly consistent across different distortions, indicating that the global attention structure is largely preserved. In particular, the presence of similar high-activation bands suggests that the decoder exhibits a degree of invariance to various degradations. Nonetheless, subtle differences are noticeable, with some regions appearing brighter than others, likely reflecting the type and level of the applied distortion. These observations highlight the need for a more quantitative measure to rigorously assess such variations. To this end, we compute the Frobenius norm (Eq. (16)) of the difference between each cross-attention map and the original one (without any distortion applied). This helps us understand how changes in the type and level of degradation affect the overall attention scores.

$$ \|A - B\|_F = \sqrt{\sum_{i=1}^{m} \sum_{j=1}^{n} (A_{ij} - B_{ij})^2}, \quad (16) $$

which provides a single scalar value summarizing the overall structural deviation. A low Frobenius norm indicates that the attention scores closely resemble those of the original map (without any distortion applied), whereas a high value implies that the decoder is focusing on different regions as a result of the applied distortion. This approach, therefore, allows us to quantitatively relate the structural differences across attention maps to the corresponding noise conditions.

Specifically, from Table 5 and Fig. 7, the image with JPEG compression shows the closest match to the original attention map, with the lowest Frobenius norm ($F_{01} = 0.56$). Contrast adjustment follows ($F_{02} = 0.64$), while image scaling leads to the largest discrepancy ($F_{03} = 0.95$), likely due to its stronger impact on image quality [9]. These patterns remain consistent as the distortion level increases (Table 5), confirming that the quality-aware decoder's cross-attention maps progressively adapt to higher noise levels.

## 4. Conclusion and future work

In this paper, we introduced TransformAR, a novel AR-IQA metric built on the transformer architecture. Our framework uses pre-trained ViT encoders alongside quality-aware decoders with cross-attention mechanisms to effectively extract quality features from AR images. Extensive evaluations on the ARIQA and the ARIQA-PICO datasets demonstrate that TransformAR achieves superior correlations with subjective human scores (SROCC, KROCC, and PLCC) compared to current state-of-the-art methods, while significantly reducing the complexity of the model. Our ablation studies further underscore the importance of each component, and the framework exhibits notable performance even when relying solely on the foreground content, highlighting its practical applicability for real-world AR applications. We also carried out a detailed qualitative study to show how quality-aware decoder attention maps are key to creating good quality representations. In addition,

we analyzed how these maps change when different distortions are applied.

Future work will focus on optimizing the model for real-time deployment and exploring additional modalities to further enrich the quality assessment process. Expanding the diversity and size of AR datasets will also be essential to validate and generalize our approach across a wider range of scenarios. Since access to reference images is not always feasible in real-world scenarios, we will also focus on developing a diverse AR-in-the-wild dataset.

## CRediT authorship contribution statement

**Aymen Sekhri:** Writing – original draft, Visualization, Validation, Software, Methodology, Investigation, Formal analysis, Data curation, Conceptualization. **Mohamed-Chaker Larabi:** Writing – review & editing, Supervision, Project administration, Methodology, Investigation, Funding acquisition, Conceptualization. **Seyed Ali Amirshahi:** Writing – review & editing, Supervision, Methodology, Funding acquisition, Conceptualization.

## Declaration of competing interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

## Acknowledgment

This work is partially funded by the Nouvelle-Aquitaine Research Council under project REALISME AAPR2022-2021-17027310.

## Data availability

The authors do not have permission to share data.

## References

[1] R. Vertucci, S. D'Onofrio, S. Ricciardi, M. De Nino, History of augmented reality, in: Springer Handbook of Augmented Reality, Springer, 2023, pp. 35–50.

[2] International Telecommunication Union, ITU-T Recommendation G.1036, 2022, https://www.itu.int/rec/T-REC-G.1036-202207-I. (Accessed 8 April 2025).

[3] A. Perkis, C. Timmerer, S. Baraković, et al., QUALINET white paper on definitions of immersive media experience (IMEx), 2020, arXiv preprint arXiv:2007.07032.

[4] J. Xu, C. Lin, W. Zhou, Z. Chen, Subjective quality assessment of stereoscopic omnidirectional image, in: 19th Pacific-Rim Conference on Multimedia, Hefei, China, Springer, 2018, pp. 589–599.

[5] J.R. Economides, D.L. Adams, J.C. Horton, Perception via the deviated eye in strabismus, J. Neurosci. 32 (30) (2012) 10286–10295.

[6] R.L. Woods, R.G. Giorgi, E.L. Berson, E. Peli, Extended wearing trial of trifield lens device for ‘tunnel vision’, Ophthalmic Physiol. Opt. 30 (3) (2010) 240–252.

[7] E. Peli, J.-H. Jung, Multiplexing prisms for field expansion, Optom. Vis. Sci.: Off. Publ. Am. Acad. Optom. 94 (8) (2017) 817.

[8] H. Apfelbaum, E. Peli, Tunnel vision prismatic field expansion: challenges and requirements, Transl. Vis. Sci. Technol. 4 (6) (2015) 8–8.

[9] H. Duan, X. Min, Y. Zhu, et al., Confusing image quality assessment: Toward better augmented reality experience, IEEE Trans. Image Process. 31 (2022) 7206–7221.

[10] A. Kitamura, Y. Kinosada, K. Shinohara, Monocular presentation attenuates change blindness during the use of augmented reality, Front. Psychol. 10 (2019) 1688, http://dx.doi.org/10.3389/fpsyg.2019.01688, URL https://www.frontiersin.org/articles/10.3389/fpsyg.2019.01688.

[11] M. Wang, J. Ding, D.M. Levi, E.A. Cooper, The effect of interocular contrast differences on the appearance of augmented reality imagery, ACM Trans. Appl. Percept. 21 (1) (2023) http://dx.doi.org/10.1145/3617684.

[12] M. Wang, E.A. Cooper, Perceptual guidelines for optimizing field of view in stereoscopic augmented reality displays, ACM Trans. Appl. Percept. 19 (4) (2022) http://dx.doi.org/10.1145/3554921.

[13] S. Wang, A. Rehman, Z. Wang, S. Ma, W. Gao, Perceptual video coding based on SSIM-inspired divisive normalization, IEEE Trans. Image Process. 22 (4) (2012) 1418–1429.
[14] Z. Wang, A. Bovik, H. Sheikh, E. Simoncelli, Image quality assessment: from error visibility to structural similarity, IEEE Trans. Image Process. (TIP) 13 (4) (2004) 600–612.

[15] Z. Wang, E. Simoncelli, A. Bovik, Multiscale structural similarity for image quality assessment, in: Asilomar Conference on Signals, Systems & Computers, vol. 2, 2003, pp. 1398–1402.

[16] Z. Wang, Q. Li, Information content weighting for perceptual image quality assessment, IEEE Trans. Image Process. 20 (5) (2010) 1185–1198.

[17] L. Zhang, L. Zhang, X. Mou, D. Zhang, FSIM: A feature similarity index for image quality assessment, IEEE Trans. Image Process. 20 (8) (2011) 2378–2386.

[18] W. Xue, L. Zhang, X. Mou, A. Bovik, Gradient magnitude similarity deviation: A highly efficient perceptual image quality index, IEEE Trans. Image Process. 23 (2) (2013) 684–695.

[19] L. Zhang, Y. Shen, H. Li, VSI: A visual saliency-induced index for perceptual image quality assessment, IEEE Trans. Image Process. 23 (10) (2014) 4270–4281.

[20] L. Kang, P. Ye, Y. Li, D. Doermann, Simultaneous estimation of image quality and distortion via multi-task convolutional neural networks, in: Int. Conf. on Image Processing, 2015, pp. 2791–2795.

[21] Y. Yuan, Q. Guo, X. Lu, Image quality assessment: a sparse learning way, Neurocomputing 159 (2015) 227–241.

[22] S. Bosse, D. Maniry, K. Müller, et al., Deep neural networks for no-reference and full-reference image quality assessment, IEEE Trans. Image Process. 27 (1) (2017) 206–219.

[23] R. Zhang, P. Isola, A. Efros, et al., The unreasonable effectiveness of deep features as a perceptual metric, in: IEEE Conf. on Computer Vision and Pattern Recognition, Salt Lake City, UT, 2018, pp. 586–595.

[24] A. Krizhevsky, I. Sutskever, G. Hinton, Imagenet classification with deep convolutional neural networks, in: Advances in Neural Information Processing Systems, Lake Tahoe, NV, USA, 2012, pp. 1097–1105.

[25] K. Simonyan, A. Zisserman, Very deep convolutional networks for large-scale image recognition, 2014, arXiv preprint arXiv:1409.1556.

[26] M. Cheon, S. Yoon, B. Kang, J. Lee, Perceptual image quality assessment with transformers, in: IEEE/CVF Conference on Computer Vision and Pattern Recognition, Virtual Event, 2021, pp. 433–442.

[27] S.A. Golestaneh, S. Dadsetan, K. Kitani, No-reference image quality assessment via transformers, relative ranking, and self-consistency, in: IEEE/CVF Winter Conference on Applications of Computer Vision, Waikoloa Village, HI, USA, 2022, pp. 1220–1230.

[28] W. Jiang, L. Li, Y. Ma, Y. Zhai, et al., Image quality assessment with transformers and multi-metric fusion modules, in: IEEE/CVF Conference on Computer Vision and Pattern Recognition, New Orleans, LA, USA, 2022, pp. 1805–1809.

[29] K. He, X. Zhang, S. Ren, J. Sun, Deep residual learning for image recognition, in: IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA, 2016, pp. 770–778.

[30] M. Raghu, T. Unterthiner, S. Kornblith, C. Zhang, A. Dosovitskiy, Do vision transformers see like convolutional neural networks? in: Advances in Neural Information Processing Systems, vol. 34, Curran Associates, Inc., 2021, pp. 12116–12128.

[31] E. Alexiou, E. Upenik, T. Ebrahimi, Towards subjective quality assessment of point cloud imaging in augmented reality, in: IEEE 19th Int. Workshop on Multimedia Signal Processing, MMSP, 2017, pp. 1–6.

[32] J. Gutiérrez, T. Vigier, P. Le Callet, Quality evaluation of 3D objects in mixed reality for different lighting conditions, Electron. Imaging 32 (2020) 1–7.

[33] P. Wang, H. Duan, Z. Xie, X. Min, G. Zhai, Subjective and objective quality assessment for augmented reality images, IEEE Open J. Immersive Displays 1 (2024) 135–145, http://dx.doi.org/10.1109/OJID.2024.3441511.

[34] A. Sekhri, S.A. Amirshahi, M.-C. Larabi, Towards light-weight transformer-based quality assessment metric for augmented reality, in: IEEE Int. Workshop on Multimedia Signal Processing, 2024, pp. 1–6, http://dx.doi.org/10.1109/MMSP61759.2024.10743448.

[35] A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A.N. Gomez, Ł. Kaiser, I. Polosukhin, Attention is all you need, Adv. Neural Inf. Process. Syst. 30 (2017).

[36] A. Dosovitskiy, L. Beyer, A. Kolesnikov, D. Weissenborn, et al., An image is worth 16x16 words: Transformers for image recognition at scale, 2020, arXiv preprint arXiv:2010.11929.

[37] M. Caron, H. Touvron, I. Misra, H. Jégou, J. Mairal, P. Bojanowski, A. Joulin, Emerging properties in self-supervised vision transformers, in: IEEE/CVF Int. Conference on Computer Vision, 2021, pp. 9650–9660.

[38] Z. Peng, Z. Guo, W. Huang, Y. Wang, L. Xie, J. Jiao, Q. Tian, Q. Ye, Conformer: Local features coupling global representations for recognition and detection, IEEE Trans. Pattern Anal. Mach. Intell. 45 (8) (2023) 9454–9468, http://dx.doi.org/10.1109/TPAMI.2023.3243048.

[39] S. Khan, M. Naseer, M. Hayat, S.W. Zamir, F.S. Khan, M. Shah, Transformers in vision: A survey, ACM Comput. Surv. 54 (10s) (2022) http://dx.doi.org/10.1145/3505244.

[40] P. Huber, Robust estimation of a location parameter, in: Breakthroughs in Statistics: Methodology and Distribution, Springer, 1992, pp. 492–518.

[41] H. Zou, T. Hastie, Regularization and variable selection via the elastic net, J. R. Stat. Soc. Ser. B Stat. Methodol. 67 (2) (2005) 301–320.

[42] I. Loshchilov, F. Hutter, Decoupled weight decay regularization, 2017, arXiv preprint arXiv:1711.05101.

[43] SciPy Developers, scipy.stats.kendalltau — SciPy v11.0.1 Manual, 2023, URL https://docs.scipy.org/doc/scipy/reference/generated/scipy.stats.kendalltau.html. (Accessed 02 April 2025).

[44] H.Z. Nafchi, A. Shahkolaei, R. Hedjam, M. Cheriet, Mean deviation similarity index: Efficient and reliable full-reference image quality evaluator, IEEE Access 4 (2016) 5579–5590.

[45] A. Balanov, A. Schwartz, Y. Moshe, N. Peleg, Image quality assessment based on DCT subband similarity, in: International Conference on Image Processing, IEEE, 2015, pp. 2105–2109.

[46] R. Reisenhofer, S. Bosse, G. Kutyniok, T. Wiegand, A haar wavelet-based perceptual similarity index for image quality assessment, Signal Process., Image Commun. 61 (2018) 33–43.

[47] H. Sheikh, A. Bovik, Image information and visual quality, IEEE Trans. Image Process. 15 (2) (2006) 430–444.

[48] L. Zhang, H. Li, SR-SIM: A fast and high performance IQA index based on spectral residual, in: 19th IEEE International Conference on Image Processing, IEEE, 2012, pp. 1473–1476.

[49] F.N. Iandola, S. Han, M. Moskewicz, K. Ashraf, W. Dally, K. Keutzer, SqueezeNet: AlexNet-level accuracy with 50x fewer parameters and < 0.5 MB model size, in: IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA, 2017.

[50] L. van der Maaten, G. Hinton, Visualizing high-dimensional data using t-SNE, J. Mach. Learn. Res. 9 (nov) (2008) 2579–2605, Punctuation: 27.