# Salient Region Detection Using Diffusion Process with Nonlocal Connections
## 2 Proposed Approach
### Compactness-Based Saliency Map
#### 2.2.3 Diffusion Process with Nonlocal Connections  
As shown in Figure 2, the variance of feature in the foregrounds may lead to the variance of the saliency value, which may result in the inconsistency of salient regions. To address this issue, many diffusion-based models have been presented [9; 16; 17; 18; 19; 20; 21; 22; 23; 24; 25; 26; 27; 28; 29; 30; 31; 32; 33; 34; 35; 36]. But most of them merely exploit the local intrinsic relevance on the graph. However, the weights between the local relevance may be inaccuracy, especially when the features of the salient regions have a complex distribution, after the diffusion process which is constructed only with local connections, the incorrect weights on the graph will inevitably lead to the suppression of the partial salient regions, which can be illustrated in Figure 2. Inspired by [36], we present a new diffusion model with nonlocal connections to address this issue. Figure 5: The jet color maps of different Gaussian refinement models (The values of the Gaussian models have been converted to 0–255). (**a**) Original image; (**b**) ground truth; (**c**) general Gaussian model (superpixel-level); (**d**) compactness-biased Gaussian model(superpixel-level); (**e**) uniqueness-biased Gaussian model (superpixel-level); (**f**) pixel-wise Gaussian model; (**g**) pixel-wise saliency-biased Gaussian model. In Section 2.2.1, we have roughly divided the superpixels into the foreground seeds and the background seeds. To find the more accurate foreground seeds and background seeds, we generate the \"true-foreground\" and \"true-background\" seed sets which are respectively referred as \(\textbf{{T}}_{\textbf{{f}}}\), \(\textbf{{T}}_{\textbf{{b}}}\) using:  
\[\left\{ \begin{array}{l}\textbf{{T}}_{\textbf{{f}}}=\left\{{{v}_{i}}|ComSal(i) \geq 0.9,i=1,2,\cdots,N\right\}\\ \textbf{{T}}_{\textbf{{b}}}=\left\{{{v}_{i}}|ComSal(i)\leq 0.1,i=1,2,\cdots,N \right\}\end{array} \right. \tag{14}\]  
As Figure 4 shows, the nonlocal connections linking each node and \(\textbf{{T}}_{\textbf{{f}}}\), \(\textbf{{T}}_{\textbf{{b}}}\) sets to each other are appended into the 2-layer sparse graph. For each node in graph, the nonlocal weighted connections measure the similarity with the foreground and background seed sets. After the manifold ranking through the proposed graph, the nodes will lean towards the region type (foreground or background) they belong to. This tendency will help to highlight the salient regions and compress the background regions. In addition, the value of the salient regions can have a fine consistency, which is difficultly acquired only with the local connections. With the compactness-based saliency information, we define the nonlocal weighted connections as:  
\[w_{ij}^{nonlocal}=\left\{ \begin{array}{cc}e^{-\left(ComSal(i)-ComSal(j) \right)^{2}}&\text{if $i$ or $j\in\textbf{{T}}_{\textbf{{f}}}$ or $\textbf{{T}}_{\textbf{{b}}}$}\\ 0&\text{others}\end{array} \right. \tag{15}\]  
Therefore, the weight of edge \(e_{ij}\) in 2-layer sparse graph with the nonlocal connections are defined as:  
\[w_{ij}^{com}=w_{ij}+w_{ij}^{nonlocal} \tag{16}\]  
where \(w_{ij}\) has been defined in Equation (1), similar with the Section 2.1, we construct the affinity matrix of the graph as \(W_{Com}=\left[w_{ij}^{com}\right]_{N\times N}\), and a degree matrix \(D_{com}{=\text{diag}\{d_{11},d_{22},\ldots,d_{NN}\}}\) is generated, where \(d_{ii}=\sum_{j}w_{ij}^{com}\). We propagate the \(S_{fg\_com}\) using the manifold ranking [9] with the following formula:  
\[S_{com}=\left(D_{com}-\alpha W_{com}\right)^{-1}S_{fg\_com} \tag{17}\]  
As shown in Figure 6, compare to the initial compactness-based saliency maps and the saliency maps generated by the 2-layer sparse graph, the salient regions produced after the proposed diffusion process are more uniform and highlighted. Figure 6: Main phases of compactness-based saliency calculation. (**a**) Original images. (**b**) Initial compactness-based saliency maps. (**c**) Saliency maps after compactness-biased Gaussian model.