# Multi-Layer Progressive Face Alignment by Integrating Global Match and Local Refinement
## 3 The Multi-Layer Progressive Face Alignment Method
### The Global Alignment Based on Shape Parameters Regression
#### 3.2.1 Multi-Mode LBP Images  
The original LBP only considers pixel-wise difference information. However, the higher-order variation tendency along different directions contains more monotonic properties. The more descriptive information in different pixel responses along a particular direction should be investigated. The second-order variation of gradient magnitude and gradient orientation is extracted to combine as the modified LBP, including M_LBP and O_LBP. The 3 \(\times\) 3 neighborhood around the current pixel contains four directions: horizontal, vertical, ascending and descending. The consistency and comparability between the three pixels along a certain direction is investigated. If the magnitude value of the center pixel is the median of three values, this direction satisfies tendency consistency, set to 1 or 0. The pixels in the neighborhood are labeled counterclockwise. M_LBP at a given position is defined as follows:  
\[M\_LBP(x_{0})=\sum_{i=1}^{4}s(x_{i})2^{i-1} \tag{3}\]  
\[s(x_{i})=\left\{ \begin{array}{ll}1&if\ (x_{i}-x_{0})\times(x_{0}-x_{i+4})>0 \\ 0&otherwise\end{array} \right. \tag{4}\]  
where \(x_{0}\) and \(x_{i}\) respectively indicates the magnitude value of the middle pixel and the two adjacent ones (as shown in Figure 4a). The errors of the orientation value between the middle one and the other two along a certain direction are calculated respectively. If the two errors are simultaneously less than a threshold, like pi/8, this direction satisfies tendency comparability, set to 1 or 0. The definition of O_LBP at a given position is similar to that of M_LBP:  
\[O\_LBP(x_{0})=\sum_{i=1}^{4}s(x_{i})2^{i-1} \tag{5}\]  
\[s(x_{i})=\left\{ \begin{array}{ll}1&if\ (|V_{i}-V_{0}|\leq\frac{\pi}{8}) \&(|V_{i+4}-V_{0}|\leq\frac{\pi}{8})\\ 0&otherwise\end{array} \right. \tag{6}\]  
where \(V_{0}\) and \(V_{i}\) respectively indicates the orientation value of the middle pixel and the adjacent one (as shown in Figure 4b). The orientation threshold is selected based on the prior knowledge of face gradient distribution. There are different gradient variations along different directions in different regions. Most face landmarks are mainly gathered in the component region with strong and rich variations, like eyes, eyebrows, mouth. In contrast, there are very few landmarks distributed in smooth regions with weak and little variations, like the cheeks and forehead. Because the specific application in this paper is face alignment, the feature representation on the component regions should be highlighted. We hope to keep the above two regions with different gradient distributions as separate as possible. In the other words, the pixels labeled 1 and 0 after the binary process should be distributed in the two regions. A binary threshold based on gradient orientation is selected to maximize the distance between the two classes. We suppose that the pixel gradient orientations in the two regions satisfy the different gauss distributions. The mean and variance of gradient orientations in two regions are calculated. The junction position of two gauss curves is the threshold. The threshold cannot be set to 0 because the gradient orientations in the local neighborhood are not strictly equal due to illumination and noise, even in the smooth region with a similar texture. After multiple experiments, pi/8 was selected as the threshold, as it has the best classification effect and can tolerate local variations. To combine multi-scale representation information, the three computing modes are further designed, including the original mode, the central mode and the average mode. The pixels in the different neighborhoods are selected for each mode. The adjacent pixels of the current pixel are selected for the original mode. The central pixels in the 3 \(\times\) 3 adjacent blocks are selected for the central mode. The averages in the adjacent blocks are selected for the average mode. The grey regions in Figure 5 indicate the selected pixel set of different modes.