## 2 Proposed Method
### Binary Weighted Interpolation Maps (B-WIM)  
The occurrence of variations changes the pixel values of the neutral image. Figure 2 shows the difference between \(\mathbf{I}\) and \(\hat{\mathbf{I}}\), caused by facial expression variations. In the aligned face images, the area around the mouth where the smile occurs significantly changes the pixel value of \(\mathbf{I}\). Absolute difference is the difference between \((\mathbf{I}\cup\mathbf{I})\) and \((\mathbf{I}\cap\mathbf{I})\). The standard deviation represents the degree of statistical variation from the difference between the pixels of these images  
\[ \begin{split}\mathcal{M}(u,v)&=log\left(\frac{1}{ m-1}\sqrt{\sum_{i=1}^{m}(|\hat{I}_{i}(u,v)-I_{i}(u,v)|-\mu(u,v))^{2}}\right),\\ \mu(u,v)&=\frac{1}{m}\sum_{i=1}^{m}|\hat{I}_{i}(u,v) -I_{i}(u,v)|\end{split}  \tag{3}\]  
where subscripts \(i(=1,2,..,m)\) denote the \(i\)th individual. In Equation (3), when the value is very large according to the degree of change, some pixels are saturated in the generated image. Therefore, the normalized \(\mathbf{M}\) is calculated as follows:  
\[\mathbf{M}=\frac{\mathbf{M}-\mathbf{min}(\mathbf{M})}{\mathbf{max}(\mathbf{M} )-\mathbf{min}(\mathbf{M})}. \tag{4}\]  
Figure 3a-f shows each \(\mathbf{M}\) for facial expressions, such as angry, afraid, disgusted, sad, smiling, and surprised. Facial expressions are related to the activation of a distinct set of facial muscles [32; 33]. When smiling, pixels around the mouth, which are related to the levator anguli oris muscle, change significantly when compared with the neutral image [34; 35]. Figure 3: Weighted interpolation maps (\(\mathbf{M}\)) and binary weighted interpolation maps (\(\mathbf{B}\)) for facial expressions. (_AN_: angry, _AF_: afraid, _DI_: disgusted, _SA_: sad, _SM_: smiling, _SU_: surprised). Figure 2: (a) Neutral image; (b) variational image; (c) absolute difference between neutral and variational image. In a previous work [31], the WIM was extracted to measure the degree of change between neutral and variational images in pixels, and the query image and mean-variational image from the WIM were combined. However, the WIM has a problem in that the weight values of the locations associated with variations may be relatively lower in the normalization process when the maximum value obtained from Equation (3) is too large. If \(\mathcal{M}(u,v)\) is 0.5, the location where the variation has occurred is not replaced by the pixel value of \(\hat{I}(u,v)\). In this case, the pixel values for \(I(u,v)\) and \(\hat{I}(u,v)\) will be mixed equally at \(I_{new}(u,v)\). This is a type of noise. To overcome this, we define B-WIMs **B** as follows:  
\[\mathcal{B}(u,v)= \begin{cases}1&\mathcal{M}(u,v)>\theta\\ 0,&\mathcal{M}(u,v)\leq\theta\end{cases} . \tag{5}\]  
In Equation (5), depending on the threshold, (\(\theta\)), \(\mathcal{B}(u,v)\) has a logical value of 0 or 1 (Figure 3g-l). The pixel value of \(I(u,v)\) is fully reflected when the value of \(\mathcal{B}(u,v)\) is 0. Conversely, if \(\mathcal{B}(u,v)\) becomes 1, the pixel value of \(I_{new}(u,v)\) will completely replace the pixel value of \(\hat{I}(u,v)\). Accordingly, the WIM problem can be solved. We use the structural similarity (SSIM) index [36] to find the optimal \(\theta\). The SSIM index evaluates a distorted image with respect to a reference image to quantify their structural similarity [37]. If \(\theta\) is 0, all elements of **B** have a value of 1. Thus, \(\textbf{I}_{new}\) is obtained from Equation (2), which becomes the same as the variational image (\(\hat{\textbf{I}}\)) in the auxiliary set. However, if \(\theta\) is 1, the \(\textbf{I}_{new}\) is identical to the query image (**J**).