# Complementary Segmentation of Primary Video Objects with Reversible Flows
## 3 Initialization with Complementary CNNs
### Problem Formulation  
Typically, a frame \(\mathcal{I}\) consists of the foreground area \(\mathcal{F}\) and the background area \(\mathcal{B}\) with \(\mathcal{F}\cap\mathcal{B}=\emptyset\) and \(\mathcal{F}\cup\mathcal{B}=\mathcal{I}\), i.e., the foreground and background should be complementary in image space. Considering that foreground objects and background distractors usually have different visual characteristics (e.g., clear versus fuzzy edges, large versus small sizes, high versus low objectness), we can attack the problem of primary object segmentation at the frame \(\mathcal{I}\) from a complementary perspective, estimating foreground and background maps, respectively. In this manner, the intrinsic characteristics of foreground and background regions can be better captured by two models with different focuses. Keeping this in mind, we propose the following formulation to explicitly consider foreground and background cues  
\[ \begin{split}\min_{\mathbb{W}_{F},\mathbb{W}_{B}}& \mathcal{L}(\mathbf{F},\mathbf{B},\mathbf{G})+\Omega(\mathbf{F}, \mathbf{B}),\\ \text{s.t.}&\phi_{F}(\mathcal{I};\mathbb{W}_{F})= \mathbf{F},\ \mathbf{F}(p)\in\{0,1\},\forall\ p\in\mathcal{I}\\ &\phi_{B}(\mathcal{I};\mathbb{W}_{B})=\mathbf{B},\ \mathbf{B}(p)\in\{0,1\},\forall\ p\in\mathcal{I},\end{split}  \tag{1}\]  
where \(\mathbf{F}\) and \(\mathbf{B}\) are two binary matrices representing \(\mathcal{F}\) and \(\mathcal{B}\). \(\mathbf{G}\) is the ground-truth map that equals 1 for a foreground pixel and 0 for a background pixel. \(\mathbb{W}_{F}\) and \(\mathbb{W}_{B}\) are two sets of parameters for the foreground and background prediction models \(\phi_{F}\) and \(\phi_{B}\). For the sake of simplifications, the values of \(\mathbf{F}\) and \(\mathbf{B}\) are assumed to be in the range \([0,1]\). The first term \(\mathcal{L}(\mathbf{F},\mathbf{B},\mathbf{G})\) is the empirical loss defined as  
\[\mathcal{L}(\mathbf{F},\mathbf{B},\mathbf{G})=E(\mathbf{F},\mathbf{G})+E( \mathbf{B},1-\mathbf{G}), \tag{2}\]  
where \(E(\cdot)\) is the cross-entropy loss that enforces \(\mathbf{F}\rightarrow\mathbf{G}\) and \(\mathbf{B}\to 1-\mathbf{G}\). Ideally, salient objects and background regions can be perfectly detected by minimizing these two losses. However, errors always exist even when two extremely complex models are used. In this case, conflicts and unlabeled areas may arise in the predicted maps (e.g., both \(\mathbf{F}\) and \(\mathbf{G}\) equal 1 or 0 at the same location). To reduce such errors, we refer to the constraint relationship \(\mathcal{F}\cap\mathcal{B}=\emptyset\) and \(\mathcal{F}\cup\mathcal{B}=\mathcal{I}\) and incorporate the complementary loss \(\Omega(\mathbf{F},\mathbf{B})\):  
\[\Omega(\mathbf{F},\mathbf{B})=\lambda_{\cap}\Omega_{\cap}(\mathbf{F},\mathbf{ B})+\lambda_{\cup}\Omega_{\cup}(\mathbf{F},\mathbf{B}), \tag{3}\]  
where \(\Omega_{\cap}(\cdot)\) and \(\Omega_{\cup}(\cdot)\) are two losses with non-negative weights \(\lambda_{\cap}\) and \(\lambda_{\cup}\) to encode the constraint \(\mathcal{F}\cap\mathcal{B}=\emptyset\) and \(\mathcal{F}\cup\mathcal{B}=\mathcal{I}\), respectively. Here, \(\lambda_{\cap}\) and \(\lambda_{\cup}\) are both set to 0.4. The intersection loss term \(\Omega_{\cap}(\cdot)\) tries to minimize the conflicts between \(\mathbf{F}\) and \(\mathbf{B}\):  
\[\Omega_{\cap}(\mathbf{F},\mathbf{B})=\frac{1}{\|\mathcal{I}\|}\sum_{p\in \mathcal{I}}(\mathbf{F}(p)\cdot\mathbf{B}(p))^{\sigma_{\cap}}, \tag{4}\]  
where \(\|\mathcal{I}\|\) indicates the number of pixels in the image \(\mathcal{I}\), and \(p\) is a pixel with predicted foregroundness \(\mathbf{F}(p)\) and backgroundness \(\mathbf{B}(p)\). \(\sigma_{\cap}\) is a positive weight to control the penalty of conflicts. The minimum value of (4) will be reached when \(\mathbf{F}(p)\cdot\mathbf{B}(p)=0\), implying that at least one map has zero prediction at every location.