# Facial Expression Recognition Based on Discrete Separable Shearlet Transform and Feature Selection
## 3 The Proposed Algorithm
### Feature Selection  
[PERSON] [17] defined the feature selection problem as the process of selecting the most relevant \(k,k<M\) features subset \(\textbf{W}=\{t_{1},t_{2},\cdots t_{w},\cdots,t_{k}\}\) from an initial set of \(\textbf{T}=\{t_{i}|i=1,2,\cdots,M\}\), and proposed a greedy selection method to solve it. Ideally, the problem can be solved by maximizing \(\text{M}\,I(\textbf{C};\textbf{W})\), the joint mutual information between the class variable **C** and the subset of selected features **W**. [PERSON] et al. [18] used an incremental search pattern to solve the problem of feature selection. They redefined a criterion function \(G\) as:  
\[G=\text{NM}\,I\left(c_{j};t_{i}\right)-\frac{1}{|\textbf{W}|}\underset{t_{s} \in W}{\sum}\,NM\,I(t_{i};t_{s}), \tag{12}\]  
where \(\text{NM}\,I(c_{j};t_{i})\) represents the normalized mutual information of \(c_{i}\) and \(t_{i}\), \(\text{NM}\,I(t_{i};t_{w})\) represents the normalized mutual information of \(t_{i}\) and \(t_{w}\), \(\text{NM}\,I(t_{i};t_{w})\) can be calculated as:  
\[\text{NM}\,I(t_{i};t_{w})=\frac{\text{M}\,I(t_{i};t_{w})}{\text{min}[H(t_{i}), H(t_{w})]}, \tag{13}\]  
where \(H(t_{i})\) is the entropy of information for \(t_{i}\) and \(H(t_{w})\) is the entropy of information for \(t_{w}\). However, the above method has the drawback of unequal weight in normalization, since the weight \(\text{min}[H(t_{i}),H(t_{w})]\) depends on the variables \(t_{i}\) and \(t_{w}\). To eliminate this drawback, an improved algorithm is proposed in this paper. Here, every feature \(t_{i}\) is quantized by employing the same number of levels (N) that has been decided to achieve the expected quantization error. Algorithm 1 illustrates the quantization algorithm, where \(t_{i}\) represents the original feature; \(U_{\text{Pper}}\) and \(L_{\text{ower}}\) represent the maximum and minimum values of the original feature respectively; \(S_{\text{step}}\) stands for quantization step; \(P_{\text{ partition}}\) is the segmentation vector, which represents the level of quantization range segmentation; \(y_{i}\) represents the quantized value of the original feature; \(Q_{\text{untiz}}\) is a quantization function defined in MATLAB; and \(C_{\text{odebook}}\) represents the set of quantized values. The number of quantization levels progressively increases until the quantization error is smaller than a predefined constant \(\xi\), which is the expected quantization error. Here, we used \(\xi=0.05\). Clearly, \(|\Omega_{\textbf{T}}|=N\), where \(\Omega_{\textbf{T}}\) is the alphabet of the variable **T**, and the entropy function \(H(\textbf{T})\) of **T** satisfies [PERSON]'s inequality [19]:  
\[H(\textbf{T})\leq\text{log}_{2}\Bigg{[}\underset{t_{i}\in\Omega_{\textbf{T}}}{ \sum}\,p(t_{i})\frac{1}{p(t_{i})}\Bigg{  
To eliminate the drawback of unequal normalization weights, we propose to use (17) to normalize the mutual information instead of (12) in [18].  
Hence, the improved criterion function \(G\) of the feature selection problem based on normalized mutual information is:  
\[G=\text{NM}\,I(\mathbf{C};t_{i})-\frac{1}{|\mathbf{W}|}\underset{t_{w}\in W}{ \sum}\,NM\,I(t_{i};t_{w})=\frac{\text{M}\,I(\mathbf{C};t_{i})}{\log_{2}(| \Omega_{\mathbf{C}}|)}-\frac{1}{|\mathbf{W}|}\underset{t_{w}\in W}{\sum}\, \frac{\text{M}\,I(t_{i};t_{w})}{\log_{2}(N)}. \tag{18}\]  
The improved feature selection algorithm based on normalized mutual information can be summarized as follows:  
1. Initialization: Assume \(\mathbf{T}=\{t_{i}|i=1,2,\cdots,M\}\) as the original set of features; initialize \(\mathbf{W}\) as an empty set. 2. Calculate the joint mutual information of each feature and class: \(\text{M}\,I(\mathbf{C};t_{i})\). 3. Find the first selected feature: find the feature \(t_{i}\) that maximizes \(\text{M}\,I(\mathbf{C};t_{i})\); delete \(t_{i}\) from set \(\mathbf{T}\); and then add \(t_{i}\) to set \(\mathbf{W}\), i.e., \(\mathbf{T}=\mathbf{T}\backslash\{t_{i}\}\), \(\mathbf{W}=\mathbf{W}\cup\{t_{i}\}\). 4.