# Robust Face Recognition in Low Resolution and Blurred Image Using Joint Information in Space and Frequency

Guoqing Li, Guangling Sun, Xinpeng Zhang

► To cite this version:

Guoqing Li, Guangling Sun, Xinpeng Zhang. Robust Face Recognition in Low Resolution and Blurred Image Using Joint Information in Space and Frequency. 9th International Conference on Network and Parallel Computing (NPC), Sep 2012, Gwangju, South Korea. pp.616-624, 10.1007/978-3-642-35606-3_73. hal-01551341

HAL Id: hal-01551341

https://inria.hal.science/hal-01551341

Submitted on 30 Jun 2017

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
Robust Face Recognition in Low Resolution and Blurred
Image Using Joint Information in Space and Frequency

Guoqing Li, Guangling Sun, Xinpeng Zhang

School of Communication and Information Engineering, Shanghai University, Shanghai, China

sunguangling@shu.edu.cn

**Abstract.** Recognizing faces in low resolution and blurred images is common yet challenging task. Local Frequency Descriptor (LFD) has been proved to be effective for this problem and is extracted from a spatial neighborhood of each pixel of a frequency plane regardless of correlations between frequencies. To explore the frequency correlations and preserve low resolution and blur insensitive simultaneously, we propose Enhanced LFD (ELFD) in which information in space and frequency is jointly utilized so as to be more descriptive and discriminative than LFD. The selection of window size of short-term of Fourier transform adaptive to the testing image is also analyzed. In addition, linear weighting fusion of recognition results given by magnitude and phase is proposed. The experiments conducted on Yale and FERET databases demonstrate that promising results have been achieved by the proposed ELFD, adaptive window size selection and fusion scheme.

**Keywords:** face recognition, low resolution and blur, Enhanced Local Frequency Descriptor, frequency correlation, fusion.

# 1 Introduction

Due to a wide range of potential applications as well as academic challenges, face recognition has attracted much attention during the last decade. Despite great progress has been made in design of scheme robust to expressions and aging of subjects, partial occlusions, illuminations and inaccurate registrations, most of them aimed at recognizing faces in high quality image. Once coping with degraded images caused by such as blur, low resolution, noise etc, the performance will decline dramatically. Hence, in this paper, we will focus on robust blurred and low resolution face recognition.

There roughly exist three categories of ways in literature to handle face recognition from blurred and low resolution image. The first category is to deblur or super-resolve an image, then feed the restored image to the recognition engine [1, 2]. While the separated scheme is straightforward, the goal of image restoration is not consistent with recognition. And even worse, especially for blurred image, if the blur model is unknown or complex, notable artifacts introduced by deblurring will in fact decline the recognition performance. The second category is to do a direct recognition from blurred or low resolution image without deblurring or super resolution. Zhang et al [3] presented a joint blind restoration and recognition framework based on sparse repre-
sentation. Once blur kernel is estimated, it is applied to blur the training set to produce a blur dictionary and the sparse coding of the blurred face using the blur dictionary is determined to give recognition result. Moreover, the kernel is estimated iteratively in a loop. We also have explored the blind blurred image recognition in which two frameworks are investigated [4]. One is first to infer the kernel as a separate step, then the kernel is used to generate a data dictionary and an adaptive SIFT feature dictionary is also obtained accordingly. The other is to integrate the kernel estimation and the adaptive SIFT dictionary inference into a common model. The two steps are alternatively executed until stop criterion is reached. The drawback of works in [3] and [4] is the low efficiency since the time consumption of blurring operation is too heavy. Li et al [5] learned two coupled mapping matrix that mapped a pair of high and low resolution image to a unique feature space. The target of the couple mapping matrix is to make the distance between two points in feature space as close as possible provide that they are corresponding to a pair of high and low resolution version of a same image. The approach is high efficiency and also without super resolution, but the mapped feature is global. The last category is to extract blur invariant or insensitive features. Heikkilä et al analyzed Local Phase Quantization (LPQ) descriptor robust to centrally symmetric blur [6]. LPQ relied on short-term of Fourier transform (STFT). They noticed that the local quantized phase information is nearly invariant in low frequency band. Clearly, only phase information is not appropriate since magnitude is also very useful for recognition shown by work [7]. Lei et al proposed Local Frequency Descriptor (LFD) that both magnitude and phase are extracted [8]. Similar to Local Binary Pattern (LBP) encoding relative relations between two pixels [9], LFD is defined in terms of relations of two local Fourier transforms of neighboring pixels of blurred image and proved to be insensitive to arbitrary type of blur kernel.

Our idea stems from the work in [8]. It has been shown that LFD is effective for recognizing low resolution face to a certain extent. We notice that the correlations between frequencies are beneficial for improving recognition performance particularly for low resolution and blurred degradations. Further, an adaptive selection of window size of STFT should be adopted. The paper is structured as follows: Section 2 reviews the LFD. Section 3 gives a detail discussion of the Enhanced LFD and the adaptive selection of window size of STFT. Sections 4 demonstrate good experimental results on Yale and FERET databases. Conclusion and future work are provided in section 5.

## 2 Review on Local Frequency Descriptor

LFD is based on STFT, which is calculated over a local area $N_x$ centered at $x$ of an image $f(x)$ as follows:

$$F_x(\mathbf{u}) = \sum_{y_i \in N_x} f(y_i) \omega^*(y_i - \mathbf{x}) e^{-j2\pi w^T y_i} \quad (1)$$

where $\mathbf{u} = \{\mathbf{u}_1, \mathbf{u}_2, \dots, \mathbf{u}_L\}$ denote a set of 2D frequencies, $\omega(\mathbf{x})$ denote a window function and $\omega^*(\mathbf{x})$ is the conjugate of it. An example of size 5×5 and 4 selected frequencies is shown in figure 1. The STFT of an image using size 5×5 and the 4 selected frequencies is demonstrated in figure 2.
<figure><img src="image_2.png" /><figcaption>Fig. 1. $u_1=(1/5,0)$; $u_2=(0,1/5)$; $u_3=(1/5,1/5)$; $u_4=(1/5,-1/5)$;</figcaption></figure>

<figure><img src="image_3.png" /><figcaption>Fig. 2. (a) Original face image. (b)-(i) magnitudes and phases of STFT at frequencies $u_1, u_2, u_3$ and $u_4$ from left to right.</figcaption></figure>

Then local magnitude descriptor (lmd) and local phase descriptor (lpd) are extracted from magnitude and phase of STFT respectively. Similar to LBP, lmd and lpd are both dependent on binary strings describing relative relations between value of a position and its 8 neighbors. Once a binary string is obtained, it will be encoded a decimal integer.

# 3 Adaptive Window Size Selection and Enhanced LFD

## 3.1 Adaptive Window Size Selection

A low resolution or blurred image $g(\mathbf{x})$ is modeled as a convolution between a high quality image $f(\mathbf{x})$ and a blur kernel $k(\mathbf{x})$:

$$g(\mathbf{x}) = f(\mathbf{x}) \otimes k(\mathbf{x}) \qquad (2)$$

Assume we focus on two positions $\mathbf{x}_i$ and $\mathbf{x}_j$ and two local regions centered at the two positions. In terms of definition of STFT, the Fourier transforms of the two local regions are as follows:

$$F_{\mathbf{x}}(\mathbf{u}) = F[\omega(\mathbf{x} - \mathbf{x}_i)f(\mathbf{x})] \qquad (3) \\ F_{\mathbf{x}_j}(\mathbf{u}) = F[\omega(\mathbf{x} - \mathbf{x}_j)f(\mathbf{x})]$$

Suppose the size of $\omega(\mathbf{x})$ is W. Clearly, for the two local images blurred by blur kernel $k(\mathbf{x})$, the corresponding Fourier transforms are as follows:

$$\tilde{G}_{\mathbf{x}_i}(\mathbf{u}) = F[k(\mathbf{x}) \otimes [\omega(\mathbf{x} - \mathbf{x}_i)f(\mathbf{x})]] = K(\mathbf{u}) \bullet F_{\mathbf{x}_i}(\mathbf{u}) \\ \tilde{G}_{\mathbf{x}_j}(\mathbf{u}) = F[k(\mathbf{x}) \otimes [\omega(\mathbf{x} - \mathbf{x}_j)f(\mathbf{x})]] = K(\mathbf{u}) \bullet F_{\mathbf{x}_j}(\mathbf{u}) \qquad (4)$$

Suppose the size of $k(\mathbf{x})$ is K, the size of blurred local image will be the larger value between K and W. For a same frequency $\mathbf{u}_k$, a blur invariant is true due to $\tilde{G}_{\mathbf{x}_i}(\mathbf{u}_k) / \tilde{G}_{\mathbf{x}_j}(\mathbf{u}_k) = F_{\mathbf{x}_i}(\mathbf{u}_k) / F_{\mathbf{x}_j}(\mathbf{u}_k)$. Nevertheless, the blur operation is first done
then a local area is extracted so that for low resolution and blurred image, the STFT is as follows:

$$ 
\begin{aligned}
\mathbf{G}_{x_i}(\mathbf{u}) &= F[\omega(\mathbf{x}-\mathbf{x}_i)g(\mathbf{x})] = F[\omega(\mathbf{x}-\mathbf{x}_i)[k(\mathbf{x}) \otimes f(\mathbf{x})]] \\
\mathbf{G}_{x_j}(\mathbf{u}) &= F[\omega(\mathbf{x}-\mathbf{x}_j)g(\mathbf{x})] = F[\omega(\mathbf{x}-\mathbf{x}_j)[k(\mathbf{x}) \otimes f(\mathbf{x})]]
\end{aligned}
\qquad (5) $$

Similar to (3), the size of local image $\omega(\mathbf{x}-\mathbf{x}_i)[k(\mathbf{x}) \otimes f(\mathbf{x})]$ and $\omega(\mathbf{x}-\mathbf{x}_j)[k(\mathbf{x}) \otimes f(\mathbf{x})]$ are both W. Obviously, the blur insensitive will be compromised. However, we can make $\mathbf{G}_{x_i}(\mathbf{u})$ and $\mathbf{G}_{x_j}(\mathbf{u})$ approximate $\tilde{\mathbf{G}}_{x_i}(\mathbf{u})$ and $\tilde{\mathbf{G}}_{x_j}(\mathbf{u})$ respectively as close as possible by changing W. Being adaptive to the size of blur kernel of current testing image, W must be larger or at least equal to K; otherwise the blur insensitive is completely impossible. Given a general blur kernel of size 19×19, two STFT windows of sizes of 11×11 and 21×21 are utilized on high quality and corresponding blurred image, the magnitude discrepancy between high quality and blurred image corresponding to window sizes of 11×11 and 21×21 respectively have been illustrated in figure 3. We can see that the discrepancy of size of 21×21 is reduced to certain extend compared with that of size of 11×11. This result is just consistent with our analysis.

Certainly, the window size should not be too large yet for the ultimate goal of the descriptor is to characterize texture and structure of a local area. If the size is too large, the descriptor will be meaningless. Hence blur insensitive and local property is in fact a trade-off. In experiment, we choose a slightly larger window size than K.

<figure><img src="image_4.png" /><figcaption>Fig. 3. Magnitude discrepancies between high quality and blurred image. The left four figures correspond window size of 11×11, the right four figures correspond window size of 21×21.</figcaption></figure>

## 3.2 Enhanced Local Frequency Descriptor Using Joint Information in Space and Frequency

While LFD descriptor has been demonstrated to be effective for recognizing low resolution and blurred face, correlations among frequencies has not been explored since LFD only encoded the spatial neighboring relation in each single frequency plane (FP) independently. In fact, the joint representation in space and frequency is more descriptive and discriminative for recognition. To accomplish the joint representation, we propose to concatenate the binary relation of respective correlated frequencies at the same spatial locations named Enhanced LFD (ELFD). For the sake of recognition performance and efficiency, we choose arbitrary two frequencies from all frequencies as correlated frequencies. As mentioned in section 2, 4 frequencies $u_1, u_2, u_3$ and $u_4$ are selected. Accordingly, a total of 12 2-frequency combinations are pro-
duced: $(\mathbf{u}_1, \mathbf{u}_2), (\mathbf{u}_1, \mathbf{u}_3), (\mathbf{u}_1, \mathbf{u}_4), (\mathbf{u}_2, \mathbf{u}_1), (\mathbf{u}_2, \mathbf{u}_3), (\mathbf{u}_2, \mathbf{u}_4), (\mathbf{u}_3, \mathbf{u}_1), (\mathbf{u}_3, \mathbf{u}_2), (\mathbf{u}_3, \mathbf{u}_4), (\mathbf{u}_4, \mathbf{u}_1), (\mathbf{u}_4, \mathbf{u}_2), (\mathbf{u}_4, \mathbf{u}_3)$. Of a couple of correlated

frequencies, the former is principal FP, the latter is its correlated FP. For arbitrary a couple of correlated frequencies and a spatial location, the extended binary relations contain the 8-neighborings at the principal FP and 4-neighborings at the correlated FP. Detailed description will be given in following.

Based on the magnitude of STFT $M(\mathbf{u}, \mathbf{x})$ at $\mathbf{u}$ and $\mathbf{x}$, the enhance lmd (elmd) is defined as follows:

$$T(M(\mathbf{u}, \mathbf{k}), M(\mathbf{u}, \mathbf{m})) = \begin{cases} 1, & \text{if } M(\mathbf{u}, \mathbf{k}) \ge M(\mathbf{u}, \mathbf{m}) \\ 0, & \text{otherwise} \end{cases} \qquad (6)$$

where $\mathbf{k}$ denotes the focused position and $\mathbf{m}$ denotes the position of one of neighbors of pixel positioned at $\mathbf{k}$. Depending on the binary relations, elmd is encoded as a decimal integer:

$$h_{\text{elmd}}(u_p, u_c, k) = \sum_{w=1}^{4} T(M(u_c, k), M(u_c, \mathbf{m})) 2^{w-1} + \sum_{w=5}^{12} T(M(u_p, k), M(u_p, \mathbf{m})) 2^{w-1} \qquad (7)$$

where $u_p$ denotes principal FP and $u_c$ denotes its correlated FP.

Similarly, based on the phase of STFT $P(\mathbf{u}, \mathbf{x})$ at $\mathbf{u}$ and $\mathbf{x}$, the enhance lpd (elpd) is defined as follows:

$$T(P(\mathbf{u}, \mathbf{k}), P(\mathbf{u}, \mathbf{m})) = \begin{cases} 1, & \text{if } P(\mathbf{u}, \mathbf{k}) \text{ and } P(\mathbf{u}, \mathbf{m}) \text{ are in the same quadrant} \\ 0, & \text{otherwise} \end{cases} \qquad (8)$$

Depending on the binary relations, elpd is also encoded as a decimal integer:

$$h_{\text{elpd}}(u_p, u_c, k) = \sum_{w=1}^{4} T(P(u_c, k), P(u_c, \mathbf{m})) 2^{w-1} + \sum_{w=5}^{12} T(P(u_p, k), P(u_p, \mathbf{m})) 2^{w-1} \qquad (9)$$

An example has been illustrated in figure 4 where $U_1$ is principal FP and $U_2$ is its correlated FP. The encoded integers of all positions compose a labeled image and the 12 labeled magnitude and phase images are shown in figure 5.

<figure><img src="image_5.png" /></figure>

<figure><img src="image_6.png" /></figure>

$$1 \times 2^{11} + 1 \times 2^{10} + 1 \times 2^{5} + 1 \times 2^{4} + 1 \times 2^{3} + 1 \times 2^{2} = 3132$$

$$1 \times 2^{11} + 1 \times 2^{10} + 1 \times 2^{9} + 1 \times 2^{5} + 1 \times 2^{4} + 1 \times 2^{3} + 1 \times 2^{2} + 1 \times 2^{1} + 1 \times 2^{0} = 3643$$

<caption>Fig. 4. The ELFD at a location and two correlated FPs.</caption>
<figure><img src="image_7.png" /><figcaption>Fig. 5. Labeled magnitude and phase images of ELFD</figcaption></figure>

Next, each of the 24 labeled images is divided empirically into 4×4=16 non-overlapping sub-regions. Thereafter a total of 16×12=192 regional label histograms are generated and concatenated into a long feature vector for both magnitude and phase. To declare the low resolution and blur insensitive properties of ELFD, for a sub-region indicated by red rectangle, two histograms of 4096 bins corresponding to original and low resolution with scale of 2 have been demonstrated in figure 6. It can be seen that the two histograms of original and low resolution image indeed stay consistently on the whole though there exists slight variations between them.

Naturally, the main drawback of ELFD is the substantially increased number of bins being 2^l, where l is the length of encoded binary string. The dimensionality of feature vector will be much higher with bins number increasing. It will introduce curse of dimensionality and make the feature unstable. We tackle it with a learning scheme: depending on training set, a global label histogram is obtained first. In a sequel, percentages of all bins are ordered and two bins of least percentage are combined into one bin and the two percentages are summed as the percentage of the combined bin. Ordering and combinations of bins are executed alternatively and iteratively until a satisfied number of bins are achieved. The final kept and combined bins are called valid bins. The original bins of a tested sub-region histogram will be adjusted and combined into a much lower number of bins in terms of the learned valid bins. The specific valid number of bins will be explained in experiment section.
<figure><img src="image_8.png" /><figcaption>Fig. 6. Histograms of a sub-region</figcaption></figure>

## 4 Experimental Results and Analysis

Our proposed method ELFD and adaptive scheme are evaluated on two databases: Yale and FERET. FERET database used here is a random subset of original FERET containing 40 persons. Two low resolution degradations with down sample scale of 2 and 4, two central symmetric blur kernels including Gaussian kernel (standard deviation 3 and size 7×7) and linear motion kernel (7 pixel-length with 45 degrees), 8 complex non-parametric kernels [10] are tested. The 12 degradations of original high resolution image in figure 2(a) are depicted in figure 7.

<figure><img src="image_9.png" /><figcaption>Fig. 7. 12 degradations of original high resolution image</figcaption></figure>

The optimal valid number of bins is 48, Gaussian window of STFT and linear SVM classifier are adopted. Half of samples are training and the rest are testing. In addition, 5 cross validation is conducted. For magnitude and phase, all feature planes (for instance, 4 feature planes for LFD, 12 feature planes for ELFD) are concatenated to compose a complete feature vector to feed the linear SVM classifier. Fix and adaptive window size of STFT, LFD and ELFD are compared respectively. Finally, recognition results of magnitude of ELFD and phase of LFD are fused by an optimal linear weighting
strategy to improve the recognition performance. All results have been listed in table 1. In this table, lmd/lpd with suffix “f” refers to fixed size and “a” refers to adaptive size. Accordingly, elmda and elpda refer to elmd and elpd with adaptive size respectively.

<table><caption>**Table 1.** Accuracy Rates of Yale and FERET (%)</caption><thead><tr><th rowspan="2">Type</th><th colspan="7">Yale</th><th colspan="7">FERET</th></tr><tr><th>lmdf</th><th>lpdf</th><th>lmda</th><th>lpda</th><th>elmda</th><th>elpda</th><th>fusion</th><th>lmdf</th><th>lpdf</th><th>lmda</th><th>lpda</th><th>elmda</th><th>elpda</th><th>fusion</th></tr></thead><tbody><tr><td>LR2</td><td>98.67</td><td>94.67</td><td>98.67</td><td>95.33</td><td>99.33</td><td>94.67</td><td>99.56</td><td>95.33</td><td>91.83</td><td>95.33</td><td>92.83</td><td>96.67</td><td>93.33</td><td>96.83</td></tr><tr><td>LR4</td><td>96.67</td><td>81.33</td><td>96.89</td><td>90.44</td><td>99.11</td><td>92.00</td><td>99.56</td><td>91.50</td><td>56.33</td><td>94.00</td><td>88.83</td><td>95.83</td><td>88.17</td><td>95.50</td></tr><tr><td>Gaussian</td><td>94.22</td><td>61.33</td><td>94.22</td><td>82.67</td><td>97.78</td><td>81.11</td><td>98.00</td><td>90.17</td><td>40.33</td><td>90.17</td><td>78.67</td><td>92.83</td><td>71.00</td><td>93.50</td></tr><tr><td>motion</td><td>94.00</td><td>76.44</td><td>94.00</td><td>90.89</td><td>97.78</td><td>89.56</td><td>97.78</td><td>92.33</td><td>81.33</td><td>92.33</td><td>85.50</td><td>94.67</td><td>85.00</td><td>95.83</td></tr><tr><td>kernel1</td><td>83.33</td><td>34.67</td><td>96.00</td><td>80.22</td><td>98.44</td><td>80.44</td><td>98.00</td><td>83.00</td><td>37.67</td><td>91.83</td><td>81.67</td><td>94.17</td><td>83.17</td><td>94.17</td></tr><tr><td>kernel2</td><td>80.89</td><td>48.22</td><td>94.00</td><td>68.22</td><td>97.78</td><td>71.33</td><td>97.33</td><td>78.67</td><td>45.17</td><td>90.67</td><td>75.00</td><td>92.50</td><td>73.83</td><td>91.83</td></tr><tr><td>kernel3</td><td>93.56</td><td>61.11</td><td>96.44</td><td>90.22</td><td>98.67</td><td>90.22</td><td>99.11</td><td>93.00</td><td>55.17</td><td>94.00</td><td>86.17</td><td>95.67</td><td>86.67</td><td>95.50</td></tr><tr><td>kernel4</td><td>67.56</td><td>38.89</td><td>74.22</td><td>67.78</td><td>77.33</td><td>60.44</td><td>81.78</td><td>52.83</td><td>29.17</td><td>70.33</td><td>65.17</td><td>77.50</td><td>59.50</td><td>82.83</td></tr><tr><td>kernel5</td><td>93.78</td><td>76.22</td><td>95.56</td><td>89.78</td><td>98.44</td><td>87.78</td><td>98.67</td><td>94.17</td><td>71.50</td><td>94.00</td><td>86.33</td><td>94.83</td><td>85.00</td><td>95.67</td></tr><tr><td>kernel6</td><td>82.89</td><td>62.44</td><td>91.11</td><td>78.67</td><td>94.89</td><td>78.22</td><td>95.33</td><td>81.33</td><td>57.33</td><td>87.50</td><td>78.00</td><td>90.33</td><td>78.50</td><td>91.00</td></tr><tr><td>kernel7</td><td>83.78</td><td>67.33</td><td>88.22</td><td>72.00</td><td>92.00</td><td>73.78</td><td>93.11</td><td>74.50</td><td>51.83</td><td>84.33</td><td>71.17</td><td>86.67</td><td>68.00</td><td>87.67</td></tr><tr><td>kernel8</td><td>76.67</td><td>67.33</td><td>85.56</td><td>70.89</td><td>88.44</td><td>70.67</td><td>92.67</td><td>68.00</td><td>55.67</td><td>81.17</td><td>65.33</td><td>82.67</td><td>62.67</td><td>86.67</td></tr><tr><td>average</td><td>87.17</td><td>64.17</td><td>92.07</td><td>81.43</td><td>95.00</td><td>80.85</td><td><strong>95.91</strong></td><td>82.90</td><td>56.11</td><td>88.81</td><td>79.56</td><td>91.19</td><td>77.90</td><td><strong>92.25</strong></td></tr></tbody></table>

## Fixed Size versus Adaptive Size

We test a fix size of 11×11. For adaptive size, we empirically set the size for low resolution of 2 scale and 4 scale to 11×11 and 21×21 respectively and slightly larger than kernel size for other degradations. In fact, we do not know blur kernel size in advance in reality, but the problem is simplified by assuming it is known in this paper. The great improvements obtained by the adaptive window size indicate that the feasibility and necessity of it.

## LFD versus ELFD

Results of both databases have shown the performance of elmd is superior to lmd but the performance of elpd is inferior to lpd in contrast. This may ascribe to curse of dimensionality since the dimensions of elmd and elpd are much higher than that of lmd and lpd. This means that if appropriate discriminant analysis is implemented, the performance of elpd will surpass lpd and the advantage of elmd will be further increased.

## Fusion Strategy

Through an optimal weighting value, the recognition performance could be improved by linear fusion of distance given by elmda and lpda. By an empirical search, the optimal weighting of results provided by elmda is 0.8 and the other is 0.2.
# 5 Conclusions and Future Work

A novel local face representation robust to low resolution and blurred degradation and other usual variations called Enhanced LFD is proposed. It improves the performance of LFD by utilizing the correlations among different frequencies so as to present a joint local descriptor of two correlated frequencies at identical spatial locations. In addition, adaptive size selection for window of STFT is proposed and a discussion in depth about it is presented. Last, a linear weighting fusion strategy is implemented. Encouraging results have been obtained on commonly used Yale and FERET database.

Future work will complement discriminate analysis for ELFD instead of direct concatenations and develop methods of automatic inference for blur kernel size of testing image.

## References

1. Masashi Nishiyama, Abdenour Hadid etc. Facial Deblur Inference Using Subspace Analysis for Recognition of Blurred Faces. IEEE Transactions on Pattern Analysis and Machine Intelligence 33(4), 838-845 (2011).

2. C. Liu, H. Y. Shum, and W. T. Freeman. Face hallucination: Theory and Practice. International Journal of Computer Vision 75(1), 115-134 (2007).

3. Haichao Zhang, Jianchao Yang etc. Close the Loop: Joint Blind Image Restoration and Recognition with Sparse Representation Prior. IEEE International Conference on Computer Vision, 770-777 (2011).

4. Guangling Sun, Guoqing Li. Blurred Image Classification Using Adaptive Dictionary. The 3rd IEEE International Conference on Intelligent Computing and Intelligent Systems, Vol3:419-423, oral, (2011).

5. Bo Li, Hong Chang, Shiguang Shan, Xinlin Chen. Low-Resolution Face Recognition via Coupled Locality Preserving Mappings. IEEE Signal Processing Letters. 17(1), 20-23 (2010).

6. Janne Heikkilä and Ville Ojansivu. Methods for Local Phase Quantization in Blur-Insensitive Image Analysis. Local and Non-Local Approximation in Image Processing, 104-111 (2009).

7. Shufu Xie, Shiguang Shan, Xilin Chen, and Jie Chen. Fusing Local Patterns of Gabor Magnitude and Phase for Face Recognition. IEEE Transactions on Image Processing. 19, 1349-1361 (2010).

8. Zhen Lei; Ahonen, T., etc. Local Frequency Descriptor for Low-Resolution Face Recognition. IEEE International Conference on Automatic Face & Gesture Recognition and Workshops, 161-166 (2011).

9. T. Ahonen, A. Hadid, and M. Pietikainen. Face Description with Local Binary Patterns: Application to Face Recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence. 28, 2037-2041 (2006).

10. A. Levin, Y. Weiss, F. Durand, and W. Freeman. Understanding and Evaluating Blind Deconvolution Algorithms. IEEE Conference on Computer Vision and Pattern Recognition, 1964-1971 (2009).