# Deep Pixel-Level Matching via Attention for Video Co-Segmentation
## 4 Experiments
### Comparisons with the State-of-the-Art Methods  
We also compared the performance of our framework with those of the state-of-the-art methods on the DAVIS 2016 dataset [23] to prove that our framework can achieve higher accuracy in video co-segmentation.  
Since the objects in an input video-pair should belong to the same category or at least have similar features so that the common information between two objects can be captured, we selected 32 videos including 2238 frames from the training data and testing data in the DAVIS 2016 dataset to form 16 video-pairs that satisfy the requirements in our evaluation experiment. Because the lengths of the videos in the DAVIS 2016 dataset are so long and the properties such as sizes and viewpoints of the objects usually change, we randomly chose six frames, which include the same object in different sizes and viewpoints in each video sequence to generate proposals containing the target as the training data. In the training step, we still used the weight pre-trained by ImageNet as the initial weight for  
\begin{table}
\begin{tabular}{c c c} \hline \hline
**Method** & **Pixel Acc. (\%)** & **mIoU (\%)** \\ \hline First frame w/o attention & 95.99 & 61.20 \\ Second frame w/o attention & 97.11 & \\ \hline First frame with attention & **97.09** & **70.99** \\ Second frame with attention & **97.92** & **70.99** \\ \hline \hline \end{tabular}
\end{table}
Table 1: Comparison of the accuracies from our framework without (w/o) and with the attention module on the ObMiC validation dataset in pixel-wise accuracy (Pixel Acc.) and on the ObMiC testing dataset in mIoU accuracy (mIoU). Numbers in bold are the best performance.  
our baseline. In the testing step, we input the whole video sequence into our framework and the state-of-the-art methods to generate results.  
In Table 2, we compare the results of our framework with those from unsupervised learning methods and supervised learning methods. The unsupervised learning methods we used for comparison include VOS [25], FOS [26], BVS [27], and DVCS [28], they can efficiently utilize the common information in the video segmentation problem; the supervised learning methods include the ResNet (the baseline in our framework), and it was also trained by the same training data that were used to train our framework. In Table 2, we can observe that the accuracies of our framework are higher than those of the unsupervised learning methods except for the testing on the videos of _Blackswan_, _Dance-Jump_, _Horsejump-High_, _Kite-Surf_, _Kite-Walk_, _Libby_, _Paragliding-Launch_, and _Stroller_. As it is shown in Figure 4, some background information in the segmentation results of the frames in the videos of _Blackswan_, _Dance-Jump_, _Horsejump-High_, and _Stroller_ was recognized as the foreground object since our network is still weak in extracting the semantic information to distinguish the foreground and background. Moreover, our network is still not remarkable in capturing features of the small parts, for example, the lines in the videos of _Kite-Surf_ and _Kite-Walk_, the lines and the head of the human in the video of _Paragliding-Launch_, and the legs of the dog in the video of _Libby_ are not segmented in the segmentation results in Figure 4. On the contrary, compared with the ResNet (the baseline), the accuracies of our framework are better than those of the ResNet in all videos. The examples of comparing the segmentation results from our baseline (ResNet) with those from our network are shown in Figure 5, we can observe that the edges in our network's results are smoother and more details, such as the leg of the camel in the video frame, can be segmented by our network with the attention module. Overall, the mIoU score (Avg.) of our framework is higher than those of the other methods, proving that our framework can improve the accuracy of video co-segmentation.  
Figure 4: The cases of failure from our network.  
Figure 5: The examples of the segmentation results from our baseline and network.