SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Tianhe Wu $^{1*}$ , Shuwei Shi $^{1,2,*}$ , Haoming Cai $^{3}$ , Mingdeng Cao $^{2}$ , Jing Xiao $^{4}$ , Yinqiang Zheng $^{2}$ , Yujiu Yang $^{1\dagger}$
<sup>1</sup> Shenzhen International Graduate School, Tsinghua University  
<sup>2</sup> The University of Tokyo <sup>3</sup> University of Maryland, College Park <sup>4</sup> Pingan Group {wth22, ssw20}@mails.tsinghua.edu.cn, cmd@g.ecc.u-tokyo.ac.jp  
hmcai@umd.edu, xiaojing661@pingan.com.cn, yqzheng@ai.u-tokyo.ac.jp  
yang.yujiu@sz.tsinghua.edu.cn
Blind Omnidirectional Image Quality Assessment (BOIQA) aims to objectively assess the human perceptual quality of omnidirectional images (ODIs) without relying on pristine-quality image information. It is becoming more significant with the increasing advancement of virtual reality (VR) technology. However, the quality assessment of ODIs is severely hampered by the fact that the existing BOIQA pipeline lacks the modeling of the observer's browsing process. To tackle this issue, we propose a novel multi-sequence network for BOIQA called Assessor360, which is derived from the realistic multi-assessor ODI quality assessment procedure. Specifically, we propose a generalized Recursive Probability Sampling (RPS) method for the BOIQA task, combining content and details information to generate multiple pseudo-viewport sequences from a given starting point. Additionally, we design a Multi-scale Feature Aggregation (MFA) module with a Distortion-aware Block (DAB) to fuse distorted and semantic features of each viewpoint. We also devise Temporal Modeling Module (TMM) to learn the viewpoint transition in the temporal domain. Extensive experimental results demonstrate that Assessor360 outperforms state-of-the-art methods on multiple OIQA datasets. The code and models are available at https://github.com/TianheWu/Assessor360.
With the development of VR-related techniques, viewers can enjoy a realistic and immersive experience with head-mounted displays (HMDs) [3, 20] by perceiving 360-degree omnidirectional information. However, the acquired omnidirectional image, also named panorama, is not always of high quality [12, 60]. Degradation may be introduced in any image processing [44, 58, 42], leading to low-quality content that may be visually unpleasant for users. Consequently, developing suitable quality metrics for panoramas holds considerable importance, as they can be utilized to guide research in omnidirectional image processing and maintain high-quality visual content.
Generally, most ODIs are stored in the equirectangular projection (ERP) format, which exhibits considerable geometric deformation at different latitudes. This distortion can have a negative impact on quality assessment. Therefore, as shown in Figure 1 (a), many researchers [36, 17, 43, 58, 14, 60] explore viewpoint-based methods by projecting the original ERP image into many undeformed viewpoints (actual content observed by users) and aggregate their features with 2D IQA method as the ODI quality score. However, this conventional viewpoint-based pipeline lacks the modeling of the
* Tianhe Wu and Shuwei Shi contribute equally to this work.
Figure 1: Illustration of the existing ordinary viewpoint-based pipeline (a) and the realistic omnidirectional image quality assessment procedure (b). Plenty of existing viewpoint-based methods follow pipeline (a) which is inconsistent with authentic assessment procedure (b), causing the predicted quality score to be far from the human perceptual quality score.
observer's browsing process, causing the predicted quality to be far from the human perceptual quality, especially when the ODI containing non-uniform distortion (such as stitching) [33, 12, 36, 61, 37]. In fact, during the process of observing ODIs, viewpoints are sequentially presented to viewers based on their browsing paths, forming a viewpoint sequence.
While the viewpoint sampling techniques proposed by Sui et al.[33] and Zhang et al.[53] can generate viewpoints with a fixed sequential order for ODIs, their methods lack the ability to generate versatile sequences. Specifically, their methods will produce identical sequences for a given starting point, and the sequential order of viewpoints will remain constant across different ODIs. This behavior is inconsistent with the observations of multiple evaluators in realistic scenarios, leading to an inability to provide subjectively consistent evaluation results. To address this issue, a logical approach is to employ the scanpath prediction model for generating pseudo viewpoint sequences on the ODI without an authentic scanpath. However, current scanpath prediction methods [47, 24, 32, 1, 34] are developed for undistorted ODIs and mainly focus on high-level regions. As evaluators are required to provide an accurate quality score for an ODI, their scanpaths are distributed across both low-quality and high-detail regions. This makes it potentially unsuitable for direct application in BOIQA task where all ODIs are distorted.
To move beyond these limitations, inspired by the realistic multi-assessor ODI quality assessment procedure, shown in Figure 1 (b), we first propose a multi-sequence network called Assessor360 for BOIQA, which can simulate the authentic data scoring process to generate multiple viewpoint sequences (corresponding to multiple scanpaths). Specifically, we propose Recursive Probability Sampling (RPS) to generate multiple pseudo viewpoint sequences, combining semantic scene and local distortion characteristics. In particular, based on Equator-guided Sampled Probability (ESP) and Details-guided Sampled Probability (DSP), RPS will generate different viewpoint sequences for the same starting point (details in Section 3.2). Furthermore, we develop Multi-scale Feature Aggregation (MFA) with Distortion-aware Block (DAB) to effectively fuse viewpoint semantic and distorted features for accurate quality perception. The Temporal Modeling Module (TMM) is devised by applying GRU [5] module and MLP layers to learn the viewpoints temporal transition information in a sequence and regress the aggregated features to the final score. Extensive experiments demonstrate the superiority and effectiveness of proposed Assessor360 on multiple OIQA datasets (MVAQD [18], OIQA [11], CVIQD [35], IQA-ODI [46], JUFE [12], JXUFE [33]). We summarize our contributions into four points:
- We propose Assessor360 for BOIQA, which can leverage multiple different viewpoint sequences to assess the ODI quality. To our knowledge, Assessor360 is the first pipeline to simulate the authentic data scoring process in ODI quality assessment.
- We propose an unlearnable method, Recursive Probability Sampling (RPS) that can combine semantic scene and local distortion characteristics to generate different viewpoint sequences for a given starting point.
- We design Multi-scale Feature Aggregation (MFA) and Distortion-aware Block (DAB) to characterize the integrated features of viewports and devise a Temporal Modeling Module (TMM) to model the temporal correlation between viewports.
- We apply our Assessor360 to two types of OIAA task datasets: one with real observed scanpath data and the other without. Extensive experiments show that our Assessor360 largely outperforms state-of-the-art methods.
Omnidirectional Image Quality Assessment. Similarly to traditional 2D IQA, according to the reference information availability, OIQA can also be divided into three categories: full-reference (FR), reduced-reference (RR), and no-reference (NR) OIQA, also known as blind OIQA [2]. Due to the structural characteristics of the panorama and the complicated assessment process, OIQA has not matured as much as 2D-IQA [48, 22, 13, 19, 15, 29]. Some researchers extend the 2D image quality assessment metrics to the panorama, such as S-PSNR [51], CPP-PSNR [52], and WS-PSNR [37]. However, these methods are not consistent with the Human Visual System (HVS) and they are poorly consistent with perceived quality [60, 43]. Although WS-SSIM [61] and S-SSIM [4] consider some impacts of HVS, the availability of non-distortion reference ODIs severely hinders their applications in authentic scenarios.
Therefore, some deep learning-based BOIQA methods [60, 43, 36, 16, 53, 46] are devised to achieve better capabilities. Due to the geometric deformation present in ODIs in ERP format, many existing BOIQA methods [59, 46, 60, 17] follow a similar pipeline: sampling viewpoints in a particular way and simply regressing their features to the quality score. MC360IQA [36] first maps the sphere into a cubemap, then employs CNN to aggregate each cubemap plane feature and regress them to a score. ST360IQ [16] samples tangent viewpoints from the salient parts and uses ViT [10] to estimate the quality of each viewpoint. VGCN [43] migrates the graph convolution network and pre-trained DBCNN [56] to establish connections between different viewpoints. However, these methods ignore the vital effect of the viewpoint sequences generated in multiple observers' browsing process, which have been demonstrated by Sui et al.[33] and Fang et al.[12] on the perception of the ODI quality.
Viewport Sampling Strategies in OIQA. Existing viewpoint sampling strategies can be categorized into three modes: 1) Uniformly sampling without sequential order. Zhou et al.[59] and Fang et al.[12] uniformly extract viewpoints over the sphere. Jiang et al.[17] and Sun et al.[36] use cube map projection (CMP) and rotate the longitude to obtain several viewpoint groups. This sampling pattern will cover the full areas, whether they are significant or non-significant. 2) Crucial region sampling without sequential order. Xu et al.[43] leverage 2D Gaussian Filter [45] to acquire a heatmap and generate viewpoints with corresponding locations. Tofighi et al.[16] apply ATSal [6] to predict salient regions of the panorama, which gives help to sampling viewpoints. This mode incorporates HVS, where most of the sampled viewpoint might be observed in the browsing process. Nevertheless, both of the above methods only focus on the image content and do not concern the effect of the sequential order between viewpoints in the quality assessment process of panoramas. 3) Sampling viewpoints along the fixed direction. Sui et al.[33] first considers the effect of sequential order between viewpoints in the browsing process. They default observers rotate their perspective in a specific direction along the equator. Zhang et al.[53] further introduce ORB detection to capture key viewpoints and follow Sui et al.[33] default sampling direction to extract viewpoints. In fact, different evaluators will produce various scanpaths when viewing a panorama. Even if the same evaluator observes the same ODI twice, the scanpath will be different. However, their sampling methods cause the sequential order of the viewpoint to be fixed, and even for the different image contents, the sequential order of sampled viewpoints is the same. This creates a significant gap with people's actual browsing process, leading to unreasonable modeling of the viewpoint sequence.
3.1 Overall Framework
As shown in Figure 2 (a), the proposed Assessor360 framework consists of Recursive Probability Sampling (RPS) scheme, Multi-scale Feature Aggregation (MFA) strategy, and Temporal Modeling Module (TMM). Given a degraded ODI  $\mathcal{I}$ , to be consistent with the authentic multi-assessor assessment, we initialize  $N$  starting points  $\mathcal{X} = \{(u_i,v_i)\}_{i=1}^N$ , where  $u_i \in [-90^\circ, 90^\circ]$  and  $v_i \in [-180^\circ, 180^\circ]$  are the corresponding latitude and longitude coordinates. Then, we apply the proposed RPS strategy  $\mathcal{G}$  with parameters  $\Theta_g$  to generate multiple viewpoint sequences  $\mathcal{S} = \{\{\mathcal{V}_j^i\}_{j=1}^M\}_{i=1}^N$ , where  $\mathcal{V}$  denotes
Figure 2: Architecture of proposed Assessor360 for BOIQA (a). Our Assessor360 consists of Recursive Probability Sampling (RPS) scheme (b), Multi-scale Feature Aggregation (MFA) strategy, and Temporal Modeling Module (TMM).
each viewpoint,  $M$  denotes the length of each sequence. Next, to perceive the semantic scene and distortion in each viewpoint, multi-scale features are extracted from multi-stage layers of our method for quality assessment. We use MFA  $\mathcal{F}$  with parameters  $\Theta_{f}$  to represent this process. The features are aggregated from  $\mathbb{R}^{H\times W\times C}$  to  $\mathbb{R}^{1\times 1\times D}$ . In this case,  $C$  is the viewpoint's original dimensions,  $H$  and  $W$  are the height and width of the viewpoint, and  $D$  denotes the embedding dimension. Then, we use TMM  $\mathcal{H}$  with parameters  $\Theta_{h}$  to model each sequence viewpoint temporal transition information and predict the final viewpoint sequence quality score. Finally, we average all predicted scores of each sequence as the ODI's quality  $\mathcal{Q}_{\mathcal{I}}$ . Overall, the whole process can be described as follows:
$$
\mathcal {Q} _ {\mathcal {I}} = \frac {1}{N} \sum_ {i = 1} ^ {N} \mathcal {H} \left(\mathcal {F} \left(\mathcal {G} \left(\mathcal {I}, \mathcal {X}; \Theta_ {g}\right); \Theta_ {f}\right); \Theta_ {h}\right) \tag {1}
$$
3.2 Recursive Probability Sampling
Recursive Probability Sampling (RPS) strategy can adaptively generate the probability of scene transition direction based on prior knowledge of semantic context and degraded features in ODI. It mainly consists of Equator-guided Sampled Probability (ESP) and Details-guided Sampled Probability (DSP). The viewpoint sequence is generated by selecting a certain starting point and sampling viewpoints based on generated probabilities.
Preprocessing for generating probabilities. As illustrated in [47], transition direction and distance are two important factors for locating the next viewpoint position. We first follow the theory of Moore neighborhood [28] and define  $K$  neighbor  $(K = 8)$  transition directions from the center coordinate of the viewpoint. Following [31, 33], the transition distance  $(\Delta u,\Delta v)$  is set to  $(24^{\circ},24^{\circ})$ , avoiding sampling overly overlapped viewpoints. In details, given a current position  $x_{c}^{t} = (u_{c}^{t},v_{c}^{t})$ , we first generate the corresponding viewpoint  $\mathcal{V}_t$  from  $\mathcal{I}$  by rectilinear projection [49]  $\mathcal{R}$ . Then, we uniformly split  $\mathcal{V}_t$  into  $K + 1$  overlapped patches, including  $K$  neighbor patches  $\mathcal{P}^t = \{\mathcal{P}_i^t\}_{i = 1}^K$  and one central patch  $\mathcal{P}_c^t$  with height  $\frac{H}{2}$  and width  $\frac{W}{2}$ . As shown in Figure 2 (b), the  $K$  direction sampled coordinates  $\mathcal{X}_{neighbor}^{t} = \{x_{i}^{t}\}_{i = 1}^{K}$  can be calculated by the central patch  $\mathcal{P}_c^t$  coordinate  $x_{c}^{t}$  and  $(\Delta u,\Delta v)$ . During the browsing process, assessors are not only drawn to the high-level scenario but also focus on low-level texture and details regions [35, 11] to give a reasonable quality score. Therefore, according to the generalized prior content information and pixel-level details metric, we present ESP and DSP for each direction sampled coordinate in  $\mathcal{X}_{neighbor}^{t}$ . Then, we choose the next sampled viewpoint  $\mathcal{V}_{t + 1}$  position  $x_{c}^{t + 1}$  based on them.
Equator-guided Sampled Probability (ESP). Since the equator entails more scene information [7, 31], we introduce the prior equator bias [9]  $\mathcal{M}$  to constrain the sampled viewpoint near the equator. Concretely, the prior equator bias obeys a Gaussian distribution with a mean of 0 and a standard deviation of 0.2 in latitude. Regions near the equator have higher sampled weights and regions close
Algorithm 1 Viewport Sequence Generation (RPS Algorithm)
Input:  $N$  starting points  $\{x_{i}\}_{i = 1}^{N}$ ; an ODI  $\mathcal{I}$ ; rectilinear projection  $\mathcal{R}$ ; ESP calculation function  $\mathcal{F}_{esp}(\mathcal{X})$ ; DSP calculation function  $\mathcal{F}_{dsp}(\mathcal{P})$ ; selecting function  $\Gamma (\mathcal{X}|\tilde{p})$
Output: A set of  $N$  length  $M$  viewpoint sequences  $\{s_i = \{\mathcal{V}_t\}_{t = 1}^M\}_{i = 1}^N$ ;
1: for  $i = 1 \rightarrow N$  do
2: Initialize the current coordinate  $x \gets x_{i}$
3: for  $t = 1\to M$  do
4: Generate viewpoint by the current coordinate  $\mathcal{V}\gets \mathcal{R}(x,\mathcal{I})$
5: Split  $\mathcal{V}$  to obtain overlapped neighbor patches  $\mathcal{P}$  and calculate sampled coordinate  $\mathcal{X}$
6: Calculate ESP and DSP  $p_{esp} \gets \mathcal{F}_{esp}(\mathcal{X}), p_{disp} \gets \mathcal{F}_{disp}(\mathcal{P})$
7: Generate next viewpoint coordinate  $x \gets \Gamma(\mathcal{X}|\text{Aggregate}(p_{esp}, p_{dsp}))$
8: Sequentially gather generated  $M$  viewports  $\{\mathcal{V}_t\}_{t=1}^M$  as a viewpoint sequence  $s_i$
9: Gather generated  $N$  viewpoint sequences  $\{s_i\}_{i = 1}^N$  as the output
to the poles have relatively low sampled weights. We apply the softmax function to  $\mathcal{M}$  to obtain the coordinate probability map  $\mathcal{M}_p$  where each coordinate corresponds to a sampled probability. Then we take  $\mathcal{X}_{\text{neighbor}}^t$  sampled probabilities  $\{p_x^i\}_{i=1}^K$  from  $\mathcal{M}_p$ . Meanwhile, due to the inhibition of return (IOR) [27] where regions that have been fixated by the eyes have a lower probability of being fixated again in the near future, we multiply  $p_x^k$  with a decreasing factor  $\gamma$  where  $k$  is the index of the viewpoint coordinate has been generated. Finally, we apply softmax function to calculate  $p_{esp} = \{p_{esp}^i\}_{i=1}^K$ . Overall, the ESP calculation function  $\mathcal{F}_{\text{esp}}(\mathcal{X})$  can be defined as:
$$
\mathcal {F} _ {e s p} (\mathcal {X}) = \operatorname {S o f t m a x} \left(Z \cdot \mathcal {M} _ {p} \left(\mathcal {X} _ {\text {n e i g h b o r}} ^ {t}\right)\right), Z = \{1, \dots , \gamma , \dots , 1 \} ^ {K} \tag {2}
$$
Details-guided Sampled Probability (DSP). To efficiently measure the texture complexity of each patch, we use pixel-level information entropy  $\mathcal{E}$ . The mathematical expression is:
$$
\mathcal {E} \left(\mathcal {P} _ {i} ^ {t}\right) = \sum_ {m = 1} ^ {H ^ {\prime}} \sum_ {n = 1} ^ {W ^ {\prime}} - p \left(\mathcal {P} _ {i} ^ {t} [ m, n ]\right) \log_ {2} p \left(\mathcal {P} _ {i} ^ {t} [ m, n ]\right) \tag {3}
$$
where  $\mathcal{P}_i^t[m,n]$  is the gray value of each pixel at position  $(m,n)$  in the patch  $\mathcal{P}_i^t$ , and  $p(\cdot)$  is the probability of occurrence of each gray value. After calculating the entropy of each neighbor patch, we normalize it with the softmax function to obtain  $p_{disp} = \{p_{disp}^i\}_{i=1}^K$  for  $\mathcal{X}_{neighbor}^t$ . The DSP calculation function  $\mathcal{F}_{disp}(\mathcal{P})$  can be formulated as:
$$
\mathcal {F} _ {d s p} (\mathcal {P}) = \operatorname {S o f t m a x} \left(\mathcal {E} \left(\mathcal {P} ^ {t}\right)\right) \tag {4}
$$
Generating viewpoint sequences. Ultimately, we multiply the ESP and DSP with a scale factor  $\beta$  to get the integrated probability. We use the softmax function to obtain the final probability distribution and select the next viewpoint position  $x_c^{t + 1}$  according to it, which can be written as:
$$
x _ {c} ^ {t + 1} = \Gamma \left(\mathcal {X} ^ {t} \mid \operatorname {S o f t m a x} \left(p _ {d s p} \cdot p _ {e s p} \cdot \beta\right)\right) \tag {5}
$$
where  $\Gamma (\mathcal{X}|\tilde{p})$  is the selecting function based on the set of probability  $\tilde{p}$ . We can recursively generate a viewport  $\mathcal{V}_{t + 1}$  with  $x_{c}^{t + 1}$  and keep performing the RPS strategy until the number of viewpoints in the sequence reaches the desired length. The whole generation algorithm  $\mathcal{G}$  is shown in Algorithm 1.
3.3 Multi-scale Feature Aggregation
To represent the semantic information and distortion pattern of each viewpoint, which are assumed as two key factors of quality assessment, we aggregate multi-scale features of the viewpoint. We first extract features from each stage in pre-trained Swin Transformer [23, 30]. The outputs of the first two stages  $\mathbf{F}_1$ ,  $\mathbf{F}_2$  are more sensitive to the distortion pattern, while the outputs of the last two stages  $\mathbf{F}_3$ ,  $\mathbf{F}_4$  tend to capture the abstract features (details in Supplementary Materials). Before fusing multi-scale features, we first employ four  $1 \times 1$  convolution layers to reduce the feature dimension of the output to  $D$ . Then, to further emphasize the local degradation, we devise and apply the Distortion-aware Block (DAB) which includes a  $3 \times 3$  convolution layer following  $n$  Channel Attention (CA) operations to the reduced shallow features  $\hat{\mathbf{F}}_1$ ,  $\hat{\mathbf{F}}_2$ . This operation can help the model to better perceive the distortion pattern in the channel dimension, achieving distortion-aware capacity. Finally, we concatenate and integrate these features with Global Average Pooling (GAP) and multiple  $1 \times 1$  convolution layers to get the quality-related representation. After that, each aggregated feature  $\mathbf{F}_{\mathrm{fuse}}$  will be sent to TMM for viewpoint sequence quality assessment.
Table 1: Quantitative comparison of the state-of-the-art methods and proposed Assessor360. The best are shown in **bold**, and the second best (except ours) are **underlined*. Two baselines w/ ERP and w/ CMP mean that we replace input viewpoint sequences generated by RPS with ERP and CMP.
Type	Method	MVAQD	OIQA	IQA-ODI	CVIQD
SRCC	PLCC	SRCC	PLCC	SRCC	PLCC	SRCC	PLCC
FR-IQA methods	PSNR	0.8150	0.7591	0.3929	0.3893	0.4018	0.4890	0.8015	0.8425
SSIM [39]	0.8272	0.7202	0.3402	0.2307	0.5014	0.5686	0.6737	0.7273
MS-SSIM [40]	0.8032	0.7136	0.5750	0.5084	0.7434	0.8389	0.9218	0.9272
WS-PSNR [37]	0.8152	0.7638	0.3829	0.3678	0.3780	0.4708	0.8039	0.8410
WS-SSIM [61]	0.8236	0.5328	0.6020	0.3537	0.5325	0.7098	0.8632	0.7672
VIF [54]	0.8687	0.8436	0.4284	0.4158	0.7109	0.7696	0.9502	0.9370
DISTS [8]	0.7911	0.7440	0.5740	0.5809	0.8513	0.8723	0.8771	0.8613
LPIPS [55]	0.8048	0.7336	0.5844	0.4292	0.7355	0.7411	0.8236	0.8242
NR-IQA methods	NIQE [26]	0.6785	0.6880	0.8539	0.7850	0.6645	0.5637	0.9337	0.8392
BRISQUE [25]	0.8408	0.8345	0.8213	0.8206	0.8171	0.8651	0.8269	0.8199
PaQ-2-PiQ [50]	0.3251	0.3643	0.1667	0.2102	0.0201	0.0419	0.7376	0.6500
MANIQA [48]	0.5531	0.5718	0.4555	0.4171	0.2642	0.2776	0.6013	0.6142
MUSIQ [19]	0.5436	0.6117	0.3216	0.3087	0.0565	0.0983	0.3483	0.3678
CLIP-IQA [38]	0.5862	0.4941	0.2330	0.2531	0.0927	0.1929	0.4884	0.4347
LIQE [57]	0.6837	0.7539	0.7634	0.7419	0.8551	0.9020	0.8594	0.8086
SSP-BOIQA [58]	0.7838	0.8406	0.8650	0.8600	-	-	0.8614	0.9077
MP-BOIQA [17]	0.8420	0.8543	0.9066	0.9206	-	-	0.9235	0.9390
MC360IQA [36]	0.6605	0.6977	0.9071	0.8925	0.8248	0.8629	0.8271	0.8240
SAP-net [46]	-	-	-	-	0.9036	0.9258	-	-
VGCN [43]	0.8422	0.9112	0.9515	0.9584	0.8117	0.8823	0.9639	0.9651
AHGCN [14]	-	-	0.9647	0.9682	-	-	0.9617	0.9658
baseline w/ ERP	0.9076	0.9240	0.8961	0.8857	0.9098	0.9196	0.9330	0.9485
baseline w/ CMP	0.8966	0.9324	0.9216	0.9170	0.9105	0.9122	0.9390	0.9412
Assessor360	0.9607	0.9720	0.9802	0.9747	0.9573	0.9626	0.9644	0.9769
3.4 Temporal Modeling Module
The browsing process of ODI naturally leads to the temporal correlation. The recency effect indicates that users are more likely to evaluate the overall image quality affected by the viewpoints they have recently viewed, especially during prolonged exploration periods. To model this relation, we introduce the GRU module to learn the viewpoint transition. Due to the fact that the last token encodes the most recent information and the representation at the last time step involves the whole temporal relationships of a sequence, we use MLP layers to regress the last feature output by the GRU module to the sequence quality score.