## 6 Video Saliency
### Methods for Video Saliency  
Table 9 enumerates recent solutions for saliency estimation in video sequences, along with additional pieces of information. Particular attention should be paid in differentiating methods that target salient object region annotations, and those who target fixations-related saliency maps [65; 66]. Specifically for the former category, some of the described solutions are tested against datasets that were originally annotated for video segmentation [122; 123; 124], and in some cases the method itself is described as addressing \"saliency-based video segmentation\" [121; 125; 126], showing once again the correlation between such tasks. An inherent characteristic of video-based processing is the temporal window, i.e., the number of frames that are jointly analyzed in order to exploit the time dimension. Methods indicated with \(\infty\) are not constrained with an explicit limit in the temporal window, although the influence of other \begin{table}
\begin{tabular}{l c c c c c c c} \hline \hline \multirow{2}{*}{**Method**} & **Saliency** & \multirow{2}{*}{**Temporal Window**} & **Optical** & \multirow{2}{*}{**Backbone CNN**} & **Deep** & **Hand-** & \multirow{2}{*}{**Target**} & **Output** \\  & **Shift** & & & Flow & & & **Learning** & **Crafted** & \\ \hline Fan 2019 [49] & ✓ & \(\infty\) (LSTM) & & ResNet-50 & ✓ & & OR & SM \\ [PERSON] 2019 [127] & 2 (OF) & ✓ & ResNet-34/101 & ✓ & & OR & SM \\ [PERSON] 2019 [128] & 4 (CNN) + \(\infty\)(GRU) & ✓ & ResNet-50 & ✓ & & OR & SM, BM \\ [PERSON] 2019-III [129] & 2 (OF) + \(\infty\) (energy) & ✓ & (none) & & ✓ & OR & SM \\ [PERSON] 2018 [125] & 2 (OF) + \(\infty\) (diffusion) & ✓ & (none) & & ✓ & OR & SM, BM \\ [PERSON] 2018 [130] & 3 & ✓ & (none) & & ✓ & OR & SM \\ [PERSON] 2018 [121] & 2 (OF) & ✓ & (none) & & ✓ & OR & SM, BM \\ [PERSON] 2017-II [126] & 5 (OF) & ✓ & (none) & & ✓ & OR & SM, BM \\ [PERSON] 2017 [131] & 4 \(\div\) 20 (diffusion) & ✓ & (none) & & ✓ & OR & SM \\ \hline Min 2019 [65] & 32 (CNN) & & & S3D (inception) & ✓ & & FM & FM \\ [PERSON] 2018 [66] & ✓ & \(\infty\) (LSTM) & & VGG-16 & ✓ & & FM & FM \\ \hline \hline \end{tabular}
\end{table} 
Table 9: Methods for video saliency. The temporal window is indicated in relation to the underlying technique: OF (Optical Flow), LSTM (Long-Short Term Memory), CNN (Convolutional Neural Network), GRU (Gated Recurrent Unit). The “Target” column indicates the nature of the ground truth used to train or develop the methods, while “Output” describes what data they explicitly generate (OR = Object Regions, SM = Sharp Maps, BM = Binary Masks, FM = Fixation Maps). frames to the current one typically decreases with their distance. Other criteria useful in discriminating among different methods include the type of representation involved (such as optical flow), and the type of model involved. In this case, for deep learning methods, the backbone CNN is also reported. In computer vision, optical flow can be defined as a displacement vector field that describes, for each pixel in each frame, the direction and intensity movement from the previous frame (or frames). Solutions for video saliency estimation based on optical flow [18; 21; 66; 121; 125; 127; 128; 130] demonstrate that explicitly and compactly representing the time-wise variations provide a valuable piece of information for accurate detection of salient objects in video sequences. [PERSON] et al. (2019-III) [129] designed a single-frame saliency model based on sparsity-based reconstruction, and an inter-frame saliency map based on progressive sparsity-based propagation. The two maps are then incorporated in a global consistency energy formulation to achieve spatio-temporal smoothness. [PERSON] et al. [125] framed the problem at hand as an \"unsupervised video segmentation\" task. They exploited edge-aware features and the optical flow representation to develop a novel diffusion technique based on a neighborhood graph.