# An Approach for Selecting the Most Explanatory Features for Facial Expression Recognition
## 3 Methodology
### Expression Classification  
Consider a set of training signals \(D=[D_{1},D_{2},\ldots,D_{k}]\in R^{m\times p}\) from \(k\) different classes, where the columns of each sub-matrix \(D_{j}=\left[d_{1}^{j},d_{2}^{j},\ldots,d_{n_{j}}^{j}\right]\in R^{m\times n_{ j}}\) are signals from the class \(w_{j}\). Ideally, we have sufficient training samples of class \(w_{j}\), such that a test signal  
\begin{table}
\begin{tabular}{c c c} \hline \hline
**Measurement** & **Name** & **Distances Involved** \\ \hline \(M_{1}\) & Inner eyebrow displacement & \(d_{40,22},d_{43,23}\) \\ \(M_{2}\) & Outer eyebrow displacement & \(d_{18\beta_{i}},d_{28\beta_{i}}\) \\ \(M_{3}\) & Inner eyebrow corners dist. & \(d_{22,23}\) \\ \(M_{4}\) & Eyebrow from nose root dist. & \(d_{22,28},d_{23,28}\) \\ \(M_{5}\) & Eye opening & \(d_{39,41},d_{44,48}\) \\ \(M_{6}\) & Eye shape & \(d_{39,41}\)/ \(d_{37,40}\)/ \(d_{44,48}\)/ \(d_{29,46}\) \\ \(M_{7}\) & Nose length & \(d_{29,31}\) \\ \(M_{8}\) & Nose width & \(d_{32,36}\) \\ \(M_{9}\) & Lower lip boundary length & \(l_{55,56,57,58,59,60,49}\) \\ \(M_{10}\) & Mouth corners dist. & \(d_{49,55}\) \\ \(M_{11}\) & Mouth opening & \(d_{63,67}\) \\ \(M_{12}\) & Mouth shape & \(d_{63,67}\)/ \(d_{49,55}\) \\ \(M_{13}\) & Nose-mouth corners angle & \(\alpha(\xi_{32,49},\xi_{36,55})\) \\ \(M_{14}\) & Mouth corners to eye dist. & \(d_{49,49},d_{55,54}\) \\ \(M_{15}\) & Mouth corners to nose dist. & \(d_{49,34},d_{55,34}\) \\ \(M_{16}\) & Upper lip to nose dist. & \(d_{52,34}\) \\ \(M_{17}\) & Lower lip to nose dist. & \(d_{67,34}\) \\ \hline \hline \end{tabular}
\end{table}
Table 1: Geometric facial measurements. \(d_{ij}\): the Euclidean distance between landmarks \(i\) and \(j\); \(\alpha\): the angle between two lines; \(\xi_{ij}\): the line defined by \(i\) and \(j\); \(l_{ijk}\) the length of the curve defined by \(i\), \(j\), and \(k\); \(o_{1}\) and \(o_{2}\): the centers of the right and left eye, respectively.  
\begin{table}
\begin{tabular}{c c c} \hline \hline
**Type** & **Feature** & **Vector Size** \\ \hline Local & GW & 2176 \\ Local & LBP & \(256\times 256\) \\ Local & HOG & \(256\times 256\times 2\) \\ Local & LPQ & \(256\times 256\) \\ Local & RAW & \(256\times 256\) \\ Geometric & GEO & \(68\times 2\) \\ \hline \hline \end{tabular}
\end{table}
Table 2: Feature summary. For each feature, we provide its type and size.  
```
1:Calculate sparse representation: For \(i=1,\dots,L\) \[\alpha_{i}{}^{\star}=arg\min_{a\in R^{\prime}}\|\alpha\|_{0}\text{s.t.}g_{i}(x)=D_ {i}\alpha\] (5)
2: Calculate the vote of each representation to each class: For \(i=1,\dots,L\) and \(j=1,\dots,C\) \[r_{ij}=\|g_{i}(x)-D_{i}\beta_{j}(\mathbf{a}_{i}^{\star})\|_{2}^{2},\] (6)

where \(\delta_{j}\) selects the entries of \(\mathbf{a}^{\star}\) corresponding to the class \(j\), and \(r_{j}\) represents the residual test sample \(g_{i}(x)\) with the linear combination \(D_{i}\delta_{j}(\mathbf{a}^{\star})\). To obtain the vote for each class, the softmax function is applied to the inverse of the normalization of \(r_{ij}\):

\[d_{i,j}=\sigma_{softmax}\Big{(}1-\frac{r_{ij}}{|r_{-}|_{1}}\Big{)}, \tag{7}\]

where \(r_{-j}\) refers to the column \(j\) as a vector and \(d_{i,j}\) represents the decision profile.
3: Trained combination rules: \[\mu_{j}(x)=\sum\limits_{i=1}^{L}\theta_{ij}d_{i,j}(x).\] (8)

The weights \(\theta_{j}\) are estimated for the decision profile for class \(w_{j}\).
4: Classification: \[\hat{j}=arg\max_{j\in 1,\dots,C}\mu_{j}.\] (9)
5: Return  The estimated class \(\hat{j}\) for the signal \(x\)
```  
**Algorithm 1** Sparse representation fusion classification (SRFC). \(D_{1},\dots D_{L}\), with