## 6 Multi-Layer Fusion with Weighted AU for Facial Expression Recognition
### AUs' Weight Learning by Association Rules  
To optimize the contribution and cooperation of eight AU channels for facial expression recognition, this paper provides a novel strategy, mining the strong relationship between AUs and emotions based on an a priori algorithm. Based on the learned rules, the weights of AUs are adjusted accordingly.  
Association rule learning is a method for discovering relations between variables in large datasets. It is intended to identify strong rules in datasets using some measures of interestingness [35]. In order to select interesting rules from the set of all possible rules, constraints on various measures of significance and interest are used. The best-known constraints are minimum thresholds on support and confidence [36]. Support is the fraction of transactions that contain both X and Y items; confidence is the fraction of transactions containing items X that also contain items Y. Intuitively, support measures the significance of the rule, so we are interested in rules with relatively high support. Confidence measures the strength of the correlation, so rules with low confidence are not meaningful, even if their support is high. We employed the basic a priori algorithm for finding the sets of items with high support, then used confidence as a threshold to filter the strong relationship. Therefore, a rule in this context is there relationship among transaction items with enough support and confidence.  
Motivated by this principle, we propose a strategy based on an a priori algorithm to help us find a strong relationship between AU and expressions. Counting the statistic rules from the data, we found the contribution of AU in expression prediction. The details of the process are described as follows.  
First of all, this paper selected a geometric feature to extract the AU feature, which is identified in a more stable manner by different classifiers on expression recognition performance and outperforms any single feature descriptors among all the recognition results of AUs' prediction, and used classifiers from base level to distinguish the prediction expression label of eight AU regions. Note that the output in this learning process is the expression prediction label instead of the prediction probability of each AU. Then, the a priori algorithm was used to carry out AU strong association mining for each expression, which treated the eight AU prediction labels of one sample and its real expression label as an item set for finding a strong relationship between AUs' prediction expressions and terminal expression. Table 4 shows part of one emotion sample: for one image, there are eight AU prediction labels, which are in the set of angry, disgust, fear, happy, sad and surprise and the real expression label is in the last column (italic). Thus, one item-set consists of eight expression predicting labels and one expression real label, and all the item-sets form a whole dataset for one expression. We made them the input of an a priori algorithm.  
We picked out strong relationships between AUs and expressions for which min-support was greater than 0.85 and min-confidence was greater than 1. Parts of the results are shown in Table 5. One rule noted was that \(P_{AUI}{\rightarrow}R_{Ej}\), where the AU prediction label was in the left set and the real label was in the right set.  
\begin{table}
\begin{tabular}{c c c c c c c c c c} \hline \hline \multirow{2}{*}{**Num**} & \multicolumn{8}{c}{**Expression Prediction Label**} & \multicolumn{2}{c}{_Real Label_} \\ \cline{2-10}  & **AU2** & **AU4** & **AU10** & **AU13** & **AU15** & **AU16** & **AU24** & **AU27** \\ \hline
**1** & angry & **disgust** & angry & angry & angry & angry & angry & angry & sad & _angry_ \\
**2** & angry & **happy** & angry & angry & **disgust** & angry & angry & angry & _angry_ \\
**3** & angry & angry & angry & angry & **sad** & angry & angry & angry & _angry_ \\ \hline... & & & &... & & & & &... \\ \hline \hline \end{tabular}
\end{table}
Table 4: Part of E1 (angry) item-set.