## 2 Full-Body Pose Recovery
### GrabCut Segmentation  
In [4], the authors proposed an approach to find a binary segmentation(background and foreground) of an image by formulating an energy minimization scheme as the one presented in [9; 10; 11], extended using color instead of just gray-scale information. Given a color image \(I\), let us consider the array \(z=(z_{1},...,z_{n},...,z_{N})\) of \(N\) pixels where \(z_{i}=(R_{i},G_{i},B_{i})\), \(i\in[1,...,N]\) in RGB space. The segmentation is defined as array \(\boldsymbol{\alpha}=(\alpha_{1},...\alpha_{N})\), \(\alpha_{i}\in\{0,1\}\), assigning a label to each pixel of the image indicating if it belongs to background or foreground. A trimap \(T\) is defined by the user--in a semi-automatic way--consisting of three regions: \(T_{B}\), \(T_{F}\) and \(T_{U}\), each one containing initial background, foreground, and uncertain pixels, respectively. Pixels belonging to \(T_{B}\) and \(T_{F}\) are clamped as background and foreground respectively--which means GrabCut will not be able to modify these labels, whereas those belonging to \(T_{U}\) are actually the ones the algorithm will be able to label. Color information is introduced by GMMs. A full covariance GMM of \(K\) components is defined for background pixels (\(\alpha_{i}=0\)), and another one for foreground pixels (\(\alpha_{j}=1\)), parametrized as follows  
\[\boldsymbol{\theta}=\{\pi(\alpha,k),\mu(\alpha,k),\Sigma(\alpha,k),\alpha\in \{0,1\},k=1..K\}, \tag{1}\]  
being \(\pi\) the weights, \(\mu\) the means and \(\Sigma\) the covariance matrices of the model. We also consider the array \(\mathbf{k}=\{k_{1},...,k_{i},...k_{N}\}\), \(k_{i}\in\{1,...K\}\), \(i\in[1,...,N]\) indicating the component of the background or foreground GMM (according to \(\alpha_{i}\)) the pixel \(z_{i}\) belongs to. The energy function for segmentation is then  
\[\mathbf{E}(\boldsymbol{\alpha},\mathbf{k},\boldsymbol{\theta},\mathbf{z})= \mathbf{U}(\boldsymbol{\alpha},\mathbf{k},\boldsymbol{\theta},\mathbf{z})+ \mathbf{V}(\boldsymbol{\alpha},\mathbf{z}), \tag{2}\]  
where \(\mathbf{U}\) is the likelihood potential, based on the probability distributions \(p(\cdot)\) of the GMM:  
\[\mathbf{U}(\boldsymbol{\alpha},\mathbf{k},\boldsymbol{\theta},\mathbf{z})= \sum_{i}\text{--log }p(z_{i}|\alpha_{i},k_{i},\boldsymbol{\theta})-\text{log }\pi(\alpha_{i},k_{i}) \tag{3}\]  
and \(V\) is a regularizing prior assuming that segmented regions should be coherent in terms of color, taking into account a neighborhood \(C\) around each pixel  
\[\mathbf{V}(\boldsymbol{\alpha},\mathbf{z})=\gamma\sum_{\{m,n\}\in C}[\alpha_ {n}\
eq\alpha_{m}]\text{ exp }(-\beta\|z_{m}-z_{n}\|^{2}) \tag{4}\]  
Figure 1: Overall block diagram of the methodology.  
With this energy minimization scheme and given the initial trimap \(T\), the final segmentation is performed using a minimum cut algorithm [9; 10; 12]. The classical semi-automatic GrabCut algorithm is summarized in Algorithm 1.  
```
1: Trimap \(T\) initialization with manual annotation.
2: Initialize \(\alpha_{i}=0\) for \(i\in T_{B}\) and \(\alpha_{i}=1\) for \(i\in T_{U}\cup T_{F}\).
3: Initialize Background and Foreground GMMs from sets \(\alpha_{i}=0\) and \(\alpha_{i}=1\) respectively, with \(k\)-means.
4: Assign GMM components to pixels.
5: Learn GMM parameters from data z.
6: Estimate segmentation: Graph-cuts.
7: Repeat from step 4, until convergence.
```  
**Algorithm 1****Original GrabCut algorithm.**