# Review of Visual Saliency Prediction: Development Process from Neurobiological Basis to Deep Models
## 4 Deep Visual Saliency Models
### Dynamic Models  
Unlike the settings of the static models, the observation time in dynamic models is reduced from approximately 4 s to 0.05 s. In addition, due to the obvious motion information in videos, predicting the saliency of the dynamic video is more difficult. As a result, much fewer dynamic models exist. Nevertheless, as the demand for applications continues to grow, the research on dynamic models has also been continuously developing.  
Dynamic models usually add temporal information to CNNs or use LSTMs for modeling. Early dynamic models mainly combined static saliency features with temporal information based on the bottom-up model. [PERSON] et al. [83] integrated additional motion information, and [PERSON] et al. [84] used a local regression kernel to calculate the similarity between the pixels in the video and its surrounding area. However, the performance of these models were restricted by their handcrafted features. The emergence of deep learning frameworks has improved this situation. [PERSON] et al. [85] proposed the dynamic model and added motion features based on the two-stream network. Due to the final fusion of the information of the two streams, the network was limited in learning spatiotemporal features. [PERSON] et al. [86] added residual motion and RGB color planes of two consecutive frames to CNN based on transfer learning. The model of [PERSON] et al. [87] merged the RGB color planes, dense optical flow map, and saliency map into a seven-layer CNN network. [PERSON] et al. [88] proposed a spatiotemporal residual attentive network (STRA-Net), which learned a stack of local attentions as well as global attention priors to filter out unrelated information. The model has advantages in precisely locating dynamic human fixations as well as capturing the temporal attention transitions.  
LSTMs are also widely used in dynamic models. [PERSON] et al. [89] used 3D CNNs to connect with the LSTMs and projected the output of the LSTMs to a Gaussian mixture model. The Object-to-Motion (OM)-CNN model proposed by [PERSON] et al. [90] analyzed intra-frame saliency based on the salient object networks and the motion information networks. On this basis, [PERSON] [91] proposed a multi-stream convolutional LSTM (ConvLSTM) network with three networks (gaze following, rapid scene change, and attention feedback) based on the static model. The ACLNet proposed by [PERSON] et al. [92] used an enhanced CNN-LSTMs to encode static saliency information. However, the ability of the network to capture motion information was limited. In this manner, LSTMs can focus on learning temporal saliency representations across consecutive frames and avoid overfitting. [PERSON] et al. [93] designed an attention-aware ConvLSTM to obtain spatial features from static networks and temporal features from dynamic networks, subsequently integrating them. The features extracted from consecutive frames were used to predict the salient regions, and a final salient map is generated for each video frame.  
In the past two years, the dynamic saliency field has gradually developed in the direction of omnidirectional images (ODIs) and 3D ODIs. [PERSON] et al. [94] used adversarial networks to predict the saliency of ODIs by imitating the head trajectory of the object and applied generative adversarial simulation models to train deep models. The development process of saliency prediction models is shown in Figure 3.