# An Improved Image Semantic Segmentation Method Based on Superpixels and Conditional Random Fields
## 3 Key Techniques
### Accurate Boundary Recovery  
After the above boundary optimization, it is still necessary to improve the segmentation accuracy of the thin structure, weak edge, and complex superposition. Therefore, we employ the CRF model to recover the boundaries more accurately, i.e., to further optimization. To clearly show the effect of the CRF model, an example is given in Figure 5.  
Consider the pixel-wise labels as random variables and the relationship between pixels as edges, and correspondingly constitute a conditional random field. These labels can be modelled after we obtain global observations, which are usually the input images. In detail, a global observation \(I\) is represented as the input image of N pixels in our method. Then, given a graph \(G=(V,E)\), \(V\) and \(E\) denotes the vertices and the edges of the graph, respectively. Let \(X\) be the vector formed by the random variables \(X_{1},X_{2},\ldots,X_{N}\), in which \(X_{i}\) is the random variable representing the label assigned to the pixel \(i\). A conditional random field conforms to Gibbs distribution, and the pair \((I,X)\) can be modelled as  
\[P(X=x|I)=\frac{1}{Z(I)}.exp(-E(x|I)), \tag{1}\]  
in which \(E(x)\) is the Gibbs energy of a labeling \(\mathrm{x}\in L^{N}\) and \(Z(I)\) is the partition function [31]. The fully connected CRF model [32] employs the energy function  
\[E(x)=\sum_{i}\phi_{i}(x_{i})+\sum_{i,j}\psi_{i,j}\big{(}x_{i},x_{j}\big{)}, \tag{2}\]  
in which \(\phi_{i}(x_{i})\) is the unary potentials that represent the probability of the pixel \(i\) taking the label \(x_{i}\), and \(\psi_{i,j}\big{(}x_{i},x_{j}\big{)}\) is the pairwise potentials that represent the cost of assigning labels \(x_{i}\), \(x_{j}\) to pixels \(i\), \(j\) at the same time. In our method, the unary potentials can be treated as the boundary optimized feature map that can help improve the performance of CRF model. The pairwise potentials usually model  
Figure 4: Boundary optimization and partial enlarged details.  
Figure 5: An example of before/after CRF.  
the relationship among neighboring pixels and are weighted by color similarity. The expression is employed for pairwise potentials [32] as shown below:  
\[\psi_{i,j}\big{(}x_{i},x_{j}\big{)}=\mu\big{(}x_{i},x_{j}\big{)}\left[\omega_{1} exp\Bigg{(}-\frac{\big{|}\big{|}p_{i}-p_{j}\big{|}\big{|}^{2}}{2\sigma_{\alpha}^{2}}- \frac{\big{|}\big{|}I_{i}-I_{j}\big{|}\big{|}^{2}}{2\sigma_{\beta}^{2}}\Bigg{)} +\omega_{2}exp(-\frac{\big{|}\big{|}p_{i}-p_{j}\big{|}\big{|}^{2}}{2\sigma_{ \gamma}^{2}})\Bigg{]}, \tag{3}\]  
in which the first term depends on both pixel positions and pixel color intensities, and the second term only depends on pixel positions. \(I_{i}\), \(I_{j}\) are the color vectors, and \(p_{i}\), \(p_{j}\) are the pixel positions. The other parameters are described in the previous work [32]. As shown in the Potts model [33], \(\mu\big{(}x_{i},x_{j}\big{)}\) is equal to 1 if \(x_{i}\
eq x_{j}\), and 0 otherwise. It means that nearby similar pixels assigned different labels should be penalized. In other words, similar pixels are encouraged to be assigned the same label, whereas pixels that differ greatly in \"distance\" are assigned different labels. The definition of \"distance\" is related to the color and the actual distance; thus, the CRF can segment images at the boundary as much as possible. Partial enlarged details shown in Figure 6 are used to explain the analysis of the accurate boundary recovery.