## 6 Multi-Layer Fusion with Weighted AU for Facial Expression Recognition  
### Multi-Layer AUs' Fusion with Stacking Framework  
Stacking is a technique to fuse multiple classifiers applied to a specific classification problem [34]. It aims to improve the classification results over the individual classifier. It outperforms other fusion methods by simply voting or linear combination, which weights the function of individual classifiers through the sample training 29. This paper proposes a novel stacking-based ensemble structure combined with AUs' prediction in a facial expression recognition system. For each AU prediction, our proposed ensemble structure also takes full advantage of feature and classifier fusion to obtain better understanding of emotion action.  
Figure 4: Kappa-error diagram of six pairs of feature descriptors. The points obtained by experimenting on the Cohn-Kanade (CK) dataset—the IBK, SMO, MLP classification results—are shown in red, green and blue. Feature pairs Gab-Geo/Gab-LBP/Gab-WLD/Geo-LBP/Geo-WLD/LBP-WLD are marked by \({}^{*}\), \(+\), “diamond”, “angle”, “square”, “star”.  
The stacking framework has two parts: the base level, which processes the input respectively with several base classifiers, and the meta-level, which clearens and integrates the results of base classifiers. The detailed procedure of stacking is illustrated as follows. Supposing there are \(n\) base classifiers marked as \(F_{1}\), \(F_{2}\),..., \(F_{n}\), one meta-classifier marked as M, and \(m\) classes marked as C\({}_{1}\), C\({}_{2}\),..., \(C_{m}\), each sample S will be processed by the following procedure.  
\begin{tabular}{c l} \hline \hline \multicolumn{2}{c}{**Stacking Procedure**} \\ \hline Step 1 & \(i=1\). \\ \hline Step 2 & If \(i\leq n\), go to Step 3. Otherwise, go to Step 4. \\ \hline Step 3 & Classifying with the base classifier \(F_{i}\). For each sample S, the probability vector \(P_{i}=\{P_{i1},\)..., \(P_{ii}\),..., \(P_{in}\}\) under the base classifier \(F_{i}\) is derived, where \(p_{ij}\) indicates the probability that sample S is assigned to class \(C_{j}\) (\(j=1,\) 2,..., \(m\)). \\  & Then \(i++\) go to Step 2. \\ \hline Step 4 & The classification results of all base classifiers are obtained, here marked as \(P=\{P_{1},\)\(P_{2},\)..., \(P_{n}\}\). \\ \hline Step 5 & The meta-classifier M processes the input data, viz. the matrix \(P\) from base classifiers, and outputs the ultimate recognition result. \\ \hline \hline \end{tabular}  
Based on the stacking principle, we propose multiple channels for AUs' prediction at base level. As illustrated before, we pick eight affective related AUs, which include mouth, eyes, intercilium and middle of facial regions. At base level, we employ four alternate feature descriptors (Gabor, geometry, LBP, WLD) to represent AU blocks and use typical classifiers (SMO, MLP, IBK) to calculate the probability of whether the specific action happens or not. At the meta-level, we integrate the prediction probabilities of AUs as the input to the meta-classifier for further expression recognition. Figure 5 gives our proposed multi-layer facial expression recognition based on the stacking ensemble algorithm.  
Figure 5: Multi-layer facial expression recognition based on the stacking ensemblestrategy.