# Saliency Detection with Bilateral Absorbing Markov Chain Guided by Depth Information
## 2 Methodology
### Mid-Level Saliency Maps Generation by Bilateral Absorbing Markov Chain
#### 2.3.3 Cross-Modal Multi-Graph Learning Model  
The two-layer sparse graph constructs the connections among the local regions, which will restrict the range of random walk to the local regions. Therefore, the absorption time may be inaccurate, especially when the long-range smooth background distributes near the center of image. To overcome it, we have improved the graph model from the connection relationship in the above section. However, in the absorbing Markov chain model, another key influencing factor is the weight of the edges between nodes. Similar to Formula (3), most of the existing graph models directly weight depth and color cues to measure the similarity between nodes. However, the models do not consider the effect of color and depth information on saliency detection in different scenarios. For example, in some scenes, color is more reliable than depth, so a larger weight of color is needed. Conversely, if depth is more reliable, we need to strengthen the weight of depth. Therefore, we propose a cross-modal multi-graph learning model (CMLM), which fully explores the complementary relationship between color and depth in different scenarios. The learning model constructs a more accurate affinity matrix and captures the optimal fusion state of color and depth information. Some algorithms [44; 48] have constructed the affinity matrix by the learning. In [48], the learning model based on the single graph is proposed, which construct an approximate full affinity matrix by using the following equation,  
\[\min_{\mathbf{Y}}\sum_{i,j=1}^{N}w_{ij}\|\mathbf{y}_{i}-\mathbf{y}_{j}\|^{2}+ \mu\sum_{i=1}^{N}\|\mathbf{y}_{i}-\mathbf{i}_{i}\|^{2} \tag{18}\]  
Figure 5: Visual comparisons of different graph-based results. (**a**) Original RGB images. (**b**) Original depth images. (**c**) Saliency maps produced by two-graph neighborhood graph. (**d**) Saliency maps produced by two-layer sparse graph. (**e**) Saliency maps produced by improved two-layer sparse graph based on background seed screening mechanism (BSSM). (**f**) Saliency maps produced by improved two-layer sparse graph without the non-local connections. (**g**) Saliency maps produced by improved two-layer sparse graph with the non-local connections. (**h**) Ground truth. where \(\mathbf{Y}=[\mathbf{y}_{i},\mathbf{y}_{2},\ldots,\mathbf{y}_{n}]\in\mathbb{R}^{N\times N}\) is an affinity matrix optimized by unsupervised learning based on the original sparse affinity matrix. \(\mathbf{y}_{i}=[y_{i1},y_{22},\ldots,y_{iN}]^{\top}\) is a column vector indicating the degree of affinity between the node \(i\) and all other nodes, \(\mathbf{i}_{i}^{\top}\) is the \(i\)-th column of an identity matrix \(\mathbf{I}\) which indicates the similarity with itself. In Equation (18), the first item is a smoothing constraint item, which indicates the difference between \(\mathbf{y}_{i}\) and \(\mathbf{y}_{j}\). The two nodes are more similar, the value of first item will be lower. The second item is a self-restraint item, which emphasizes that no matter how we update the value of \(\mathbf{y}_{i}\) of node \(i\), it should not be too different from its initial value. \(\mu\) is a parameter that balances the relationship between the two items, \(\mu>0\). Formula (18) is the learning process under the single-layer graph. To make full use of the complementarity of color and depth information, we explore feature spaces of multiple modes and develop a cross-modal multi-graph model to learn an affinity matrix. We use \(\beta=[\beta_{c},\beta_{d^{\prime}},\ldots]^{\top}\) to represent the set of multi-modal vectors, and its values indicate the importance of the corresponding affinity graph. In this work, we only adopt the modes of color and depth. \(\beta_{c}=[\beta_{c}^{(1)},\beta_{c}^{(2)},\ldots,\beta_{c}^{(m)}]\) is a sub-vector of the color mode and \(m\) is the number of feature maps in color space. \(\beta_{d}=[\beta_{d}^{(1)},\beta_{d}^{(2)},\ldots,\beta_{d}^{(n)}]\) is a sub-vector of the depth mode and \(n\) is the number of feature maps in depth space.