# Review of Visual Saliency Prediction: Development Process from Neurobiological Basis to Deep Models
## 6 Evaluation Measures for Visual Saliency Prediction  
The metrics of visual saliency prediction mostly use similarities and differences between estimated predicted values and the Ground Truth (GT) and then outputs an evaluation score to judge the similarity or difference degree between them. Given a set of true values used to define the scoring function, the saliency prediction map can be used as the input, and the result of evaluating the accuracy of the prediction is returned. The evaluation measures are as follows:  
AUC variant: The Area Under Curve (AUC) is used as a measurement standard for the two-class pattern recognition problem. Different from the AUC in tasks, such as target detection and image segmentation, given the particularity of the saliency prediction task, the following AUC variants are often used in the saliency prediction tasks:  
* AUC-Judd: [PERSON] et al. [102] proposed a variant of the AUC called AUC-Judd. For a given threshold, the true-positive probability is the ratio of the pixels predicted as significant on all true-valued salient points, whereas the false-positive probability is the ratio of pixels predicted as significant on non-salient points. * AUC-Borji: [PERSON] et al. [103] proposed another variant of the AUC called AUC-Borji. This variant uses the uniform random sampling of non-focus points to calculate the false positive rate and defines the saliency mapping value above the threshold of these pixels as false positive. The false positive calculation in AUC-Borji is a discrete approximation of the calculation in AUC-Judd. Due to the use of random sampling, the same model may be evaluated with different results. * Shuffled AUC: Shuffled AUC (sAUC) [97] is also a commonly used AUC variant. It reduces the sensitivity of the AUC to the center shift by sampling the salient point distribution of other images. AUC-Judd, AUC-Borji, and sAUC, as variants of AUC, are widely used in the evaluation of saliency models. Their values are positively correlated with model performance. Although AUC is an important evaluation measure, it cannot distinguish the relative importance of different regions. Therefore, other distribution-based similarity evaluation measures are needed:  
* Normalized Scanpath Saliency (NSS): NSS is a unique evaluation measure of saliency prediction. It is used to calculate the average normalized significance value at the point of interest [104]. The calculation formula of NSS is \[\text{NSS}=\frac{1}{\text{N}}\sum_{i=1}^{\text{N}}\overline{\text{P}}(i)\times \text{Q}(i)\] (1) where \(\overline{\text{P}}\) is the average value at the gaze point Q of the human eye, N is the total number of human eye gazes, i represents the i-th pixel, and N is the total number of pixels at the gaze point. A positive NSS indicates consistency between mappings, whereas a negative NSS is the opposite. The NSS value is negatively correlated with model performance. * Linear Correlation Coefficient (CC):The CC is the statistic used to measure the linear correlation between two random variables. For the significance prediction evaluation, the prediction significance map (P) and the true value view (G) can be regarded as the two random variables. The calculation formula of CC is \[\text{CC}=\frac{\text{cov}(\text{P},\text{G})}{\sigma(\text{P})\times\sigma( \text{G})}\] (2) where cov is the covariance, \(\sigma\) is the standard deviation. The CC can equally distinguish false positives and false negatives at the value range of (\(-1\),\(1\)). A value close to the two ends indicates a better model performance. * Earth Movers Distance (EMD): EMD [105] represents the distance between the two 2D maps denoted by G and S, and it calculates the minimum cost of converting the estimated probability distribution of the saliency map S into the probability distribution of the GT map denoted by G. Therefore, a low EMD corresponds to a high-quality saliency map.