# Complementary Segmentation of Primary Video Objects with Reversible Flows
## 5 Experiments
### Detailed Performance Analysis
#### 5.3.1 Performance of Complementary CNNs  
In this section, some detail analysis will be given to further verify the effectiveness of the proposed complementary CNN branches and complementary loss. **Impact of two complementary branches**. To explore the impact of two complementary network branches, we evaluated the foreground maps and background maps initialized by the two complementary branches, as well as their fusion maps. As shown in Table 3, in the first group the evaluation scores of case V-Init. FG and V-Init. BG are equally matched for their same branch structure, while the ones of their fusion maps increased to different degrees, which suggests that the complementary characteristics of initialized foreground maps and background maps can contribute to and constrain each other to generate more accurate prediction. Then what will happen if we abandon the background branch? To this end, we conducted two additional experiments in our previous work [39]. First, if we cut down the background branch and retrain only the foreground branch, the final performance decreases by about 0.9%. Second, if we re-train a network with two foreground branches, the final \(F_{\beta}\) and mIoU scores decrease from 0.806 to 0.800 and 0.710 to 0.700, respectively. These experiments indicate that, beyond learning more weights, the background branch does learn some useful cues that are ignored by the foreground branch, which are expected to be high-level visual patterns of typical background distractors. These results also validate the idea of training deep networks by simultaneously handling two complementary tasks. \begin{table}
\begin{tabular}{l c c c c c c c c c c} \hline \hline \multirow{2}{*}{**Test Cases**} & \multicolumn{2}{c}{**Backbone**} & \multicolumn{2}{c}{**Objective**} & \multicolumn{5}{c}{**Evaluation**} \\ \cline{2-11}  & **VGG16/ResNet50** & **CE** & **Comple.** & **NRF** & **Multi-Test** & **mAP** & **mAR** & \(F_{\beta}\) & **mIoU** & **mT** \\ \hline
**V-Init. FG** & VGG16 & ✓ & & & & 0.750 & 0.879 & 0.776 & 0.684 & 0.117 \\
**V-Init. BG** & VGG16 & ✓ & & & & 0.743 & 0.884 & 0.771 & 0.680 & 0.117 \\
**V-Init. (FG + BG)** & VGG16 & ✓ & & & & 0.791 & 0.834 & 0.800 & 0.689 & 0.121 \\
**V-Init. + NRF** & VGG16 & ✓ & & ✓ & & 0.789 & 0.870 & 0.806 & 0.710 & 0.109 \\ \hline
**R-Init. FG** & ResNet50 & ✓ & & & & 0.763 & 0.901 & 0.791 & 0.710 & 0.128 \\
**R-Init. BG** & ResNet50 & ✓ & & & & 0.764 & 0.899 & 0.791 & 0.711 & 0.128 \\
**R-Init. (FG + BG)** & ResNet50 & ✓ & & & & 0.808 & 0.863 & 0.820 & 0.724 & 0.127 \\
**R-Init. FGp** & ResNet50 & ✓ & ✓ & & & 0.768 & 0.925 & 0.800 & 0.726 & 0.124 \\
**R-Init. BGp** & ResNet50 & ✓ & ✓ & & & 0.763 & **0.927** & 0.796 & 0.723 & 0.124 \\
**R-Init. (FGp + BGp)** & ResNet50 & ✓ & ✓ & & & **0.814** & 0.883 & **0.829** & 0.739 & 0.122 \\
**R-Init. + NRF** & ResNet50 & ✓ & ✓ & ✓ & & 0.803 & 0.901 & 0.824 & 0.739 & 0.108 \\
**R-Init. + NRFp** & ResNet50 & ✓ & ✓ & ✓ & ✓ & 0.805 & 0.910 & 0.827 & **0.747** & **0.097** \\ \hline \hline \end{tabular}
\end{table} 
Table 3: Detailed performances of our approaches. The first test group is our previous work in [39], and the second group is our current work. V-Init/R-Init. : corresponding results initialized by previous/current network. FG (FGp)/BG (BGp): foreground/background estimation with (without) the constraint of complementary loss. NRF (NRFp): neighborhood reversible flow (with multi-test). CE: cross-entropy. Comple. : complementary loss. mT: mean temporal stability metric, the smaller the better. Bold and underline indicate the 1 st and 2 nd performance in each column. Therefore, network structures with two similar branches are still adopted in this extension work. What is different is that the new network structure is assisted with more designs based on the deeper and more effective ResNet50 instead of simple VGG16. From Table 3, we can find that the initialized results are distinctly improved when the backbone VGG16 is replaced by ResNet50.