# Review of Visual Saliency Prediction: Development Process from Neurobiological Basis to Deep Models
## 8 Commonalities and Limitations of the Deep Saliency Models  
Although the structures of the various deep saliency models differ from one another, they have many commonalities. Compared with the classic model, the deep saliency model automatically captures features. Although the classic models can manually encode features, deep networks with multi-layer structures can automatically capture more features. The CNN-based saliency model is trained in an end-to-end manner, and combined with feature extraction and saliency prediction, it can greatly improve the performance compared with that of the classic model. The success of these saliency prediction models indicates the importance of automatically capturing features based on the deep learning framework. Aiming at improving model performance, saliency models often perform similar optimization. First, in view of reducing the loss of image features in a series of convolution and pooling layers, some models use the multi-scale network or skip layers to preserve the loss information. Second, using transfer-learning methods or adding some pre-trained classification networks or LSTMs to the model can play a role in adding prior knowledge, and this scheme has a significant impact on the model results. Finally, as evaluation measures have a great influence on model performance, some models often select multiple evaluation measures to train the model (i.e., ML-Net). Dynamic models also include multi-stream, multi-modal, and 3D CNNs and other forms. However, the overall framework type is less than the static models in terms of multi-tasking, action recognition, and other frameworks and thus need to be developed. Although the deep saliency model can sufficiently capture features, a wide gap exists between the result and the GT. The problem can be resolved by studying how to imitate human analysis scenes and understand the mechanism of the human gaze. Aimed at achieving these aspects on the model, a higher level of visual understanding is required. In particular, besides using the conventional optimization model and finding a better loss function, saliency prediction can be explored and improved on the basis of the following:  
1. New Datasets: Datasets are extremely important to model performance [120]. The GT and measurement prediction errors obtained from the data have a significant impact on the model performance. In earlier years, the collection of saliency datasets relied on eye tracking data, and the datasets had fewer images. Although the emergence \begin{table}
\begin{tabular}{c c c c c c c} \hline \hline  & **Model Name** &
\begin{tabular}{c} **AUC-** \\ **Judd** \\ \end{tabular} & **sAUC** & **CC** & **NSS** & **SIM** \\ \hline Static & DVA [78] & 0.86 & 0.595 & 0.358 & 2.013 & 0.262 \\ Models & SALICON [69] & 0.857 & 0.59 & 0.327 & 1.901 & 0.232 \\  & JuntingNet [71] & 0.855 & 0.592 & 0.331 & 1.775 & 0.201 \\  & Shallow-Net [71] & 0.833 & 0.529 & 0.295 & 1.509 & 0.182 \\  & GBVS [28] & 0.828 & 0.554 & 0.283 & 1.474 & 0.186 \\  & ITTI [4] & 0.774 & 0.553 & 0.233 & 1.207 & 0.162 \\ \hline Dynamic & ACLNet [92] & 0.89 & 0.601 & 0.434 & 2.354 & 0.315 \\ Models & OM-CMM [90] & 0.856 & 0.583 & 0.344 & 1.911 & 0.256 \\  & Two-stream [85] & 0.834 & 0.581 & 0.325 & 1.632 & 0.197 \\  & FANG [141] & 0.819 & 0.537 & 0.273 & 1.539 & 0.198 \\  & RUDOY [115] & 0.769 & 0.501 & 0.285 & 1.498 & 0.214 \\  & STRA [88] & **0.895** & **0.663** & **0.458** & **2.588** & **0.355** \\  & AWS-D [116] & 0.703 & 0.513 & 0.174 & 0.94 & 0.157 \\  & PQFT [117] & 0.699 & 0.562 & 0.137 & 0.749 & 0.139 \\  & OBDL [118] & 0.638 & 0.5 & 0.117 & 0.495 & 0.171 \\  & SEO [119] & 0.635 & 0.499 & 0.07 & 0.334 & 0.142 \\ \hline \hline \end{tabular}
\end{table} 
Table 4: Performance of the dynamic models over the DHF1K dataset. of SALICON improved the result, the gap remains to be an order of magnitude with respect to datasets in related fields (e.g., ImageNet). The JFT-300M dataset recently collected by [PERSON] et al.