# Face Recognition from Degraded Images – Super Resolution Approach by Non-adaptive Image-Independent Compressive Sensing Dictionaries

Sabah A. Jassim

► To cite this version:

Sabah A. Jassim. Face Recognition from Degraded Images – Super Resolution Approach by Non-adaptive Image-Independent Compressive Sensing Dictionaries. 14th International Conference on Communications and Multimedia Security (CMS), Sep 2013, Magdeburg, Germany. pp.217-232, 10.1007/978-3-642-40779-6_22. hal-01492827

HAL Id: hal-01492827

https://inria.hal.science/hal-01492827v1

Submitted on 20 Mar 2017

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons CC BY 4.0 - Attribution - International License
# Face Recognition from Degraded Images - Super Resolution Approach by Non-Adaptive Image-Independent Compressive Sensing Dictionaries

Sabah A. Jassim

Department of Applied Computing, University of Buckingham, Buckingham, UK

sabah.jassim@buckingham.ac.uk

**Abstract.** In recent years, the emergence of the new paradigm of compressive sensing (CS) has led to the development of innovative image/signal processing and analysis tools that can be exploited to efficiently deal with serious challenges in pattern recognitions. This paper is concerned with the use of CS tools and dictionaries for face recognition, and in particular when dealing with uncontrolled conditions, e.g. faces captured at a distance in surveillance scenarios or in post-rioting forensic, whereby the images are severely degraded/blurred and of low-resolution. We present the results of our recent investigations¹ into the construction of over-complete dictionaries that recover super-resolved face images from any input low-resolution degraded face image. These results demonstrate that non-adaptive image-independent implicitly designed dictionaries that guarantee the recovery of sparse signals achieve face recognition accuracy levels and yield significant recognition rates that are as good as if not better than those achieved by a recently proposed image-based learnt dictionaries. We shall also show that a variety of random dictionaries known to satisfy the Restricted Isometry Property (RIP), achieve similar accuracy rates, and thereby removing the need for training images. The high quality of the super-resolved images provides great potential for forensics and crime/terrorism fighting.

**Keywords:** Compressive Sensing, Super-resolution, RIP dictionaries, Face recognition, CS classifiers

## 1 Introduction

Automatic face recognition in uncontrolled conditions and in particular when dealing with surveillance tasks is far from reliable due to the degraded nature of captured images. Image resolution enhancement is deemed necessary for face recognition, where the camera is at a distant from imaged face yielding small low resolution, blurred and low-quality image for matching. Image degradation results from a variety of recording conditions: subject on the move, unstable sensors, out of focus optical system, or abnormal weather and atmospheric conditions such as thermal waves.

¹Conducted as part of the PhD research project of Nadia Al-Hassan supervised jointly with Harin Sellahewa.
Recognising faces when matching low-resolution (LR) degraded small images against a gallery of high-resolution good size face images, is traditionally dealt with by pre-processing procedures primarily using the so called super-resolution methods which aims to reconstruct a higher resolution version of the LR image. Hennings-Yeomans et al [1] proposed to perform super-resolution and recognition simultaneously. The performance of this method depends on the training database. He and Zhang in [2] have developed an SR technique that constructs a high-resolution face image, from a sequence of low-resolution images, to be processed by Gabor feature based recognition.

The emergence of compressive sensing (CS) theory and sparse representation has led to a plethora of image/signal processing and analysis tools that can be exploited to efficiently deal with serious challenges in pattern recognitions including the face recognition in uncontrolled conditioned. In particular, the development of efficient $l_1$-minimization procedures to obtain sparse solutions of certain underdetermined linear systems has led to the emergence of new SR schemes that aim recover high quality super-resolved images from low resolution degraded images ([3], [4], [5], [6], & [7]). Such approaches are motivated by the fact that images in general, and more so degraded ones, can be well-approximated by a sparse expansion in terms of suitable bases such as wavelets. Yang et al in [4], [5] proposed a method to reconstruct super-resolved image from a single low-resolution image using a pair of overcomplete dictionaries $D_H$ and $D_L$ whose columns are constructed, through a learning process, from a number of randomly selected patches of high and low resolution training datasets of face images. This pair of image-trained dictionaries is referred to as the LD system.

In this paper, we briefly discuss CS dictionary construction for various purposes in pattern recognition, but our main focus will on CS based image SR approach for face recognition in uncontrolled conditions. We shall demonstrate that non-adaptive dictionaries, implicitly constructed without using images, perform as well as the LD dictionary, if not better. We introduce an implicit approach to CS dictionary construction, an example of which was developed by the author's team at Buckingham, and investigate its performance in comparison to that of the LD scheme as well as a number of different random dictionaries in terms of the quality of their super-resolved images, face recognition accuracy and CS relevant statistical parameters. For completion, we also present the performance of a non-CS based iterative SR method [8] and of matching in low-resolution.

The rest of the paper is organized as follows. Sections 2 and 3 provide a brief review of Super resolution and Compressive Sensing respectively. In section 4, we shall discuss a recently designed CS approach to image RS using different types of dictionaries, and discuss the properties of these dictionaries that are relevant to the recovery of a sparse signal from a down-sampled degraded version of images. In section 5, we shall conduct experiments to compare the performance of a known face recognition scheme when applied to super-resolved images using the different types of dictionaries as well as to the original LR images. In the conclusion, section 6, we shall briefly describe the contribution of the paper and also highlight benefits of using certain types of implicitly constructed CS dictionaries conclusions.
## 2 Super Resolution

Super-resolution (SR) is an inverse problem used as a pre-processing technique to recover a high-resolution (HR) image from one or more low-resolution (LR) images. Generally, SR techniques to obtain a HR image from an observed LR input image $y$ may be modelled as a solution $x$ of the matrix equation:

$$y = SBx + \eta \quad (1)$$

where $B$ is a point-spread function with a blurring effect, $S$ is a down sampling function, and $\eta$ is additive noise. Various traditional non-CS based super-resolution techniques have been developed, and the most common of these are variants of the Iterative Back Projection (IBP) SR scheme which can super-resolve a single or multiple input LR image(s). The standard single LR image IBP scheme generates the initial HR image $x_0$ simply by decimating the pixels of the LR image $y$ and using Bi-cubic interpolation. At the nth iteration, $n>0$, an error image $x_e$ of the size of the $x_{(n-1)}$ image is calculated by: (1) convoluting the $x_{(n-1)}$ image with an appropriate degradation function, (2) down sample the resulting image to obtain $y_{(n)}$, and (3) $x_e$ is obtained from $(y - y_{(n)})$ by up-sampling. The nth iteration output the nth version of the HR image simply by calculating $x_{(n)} = (x_{(n-1)} + x_e)$, which represents the back projection of the difference $(y - y_{(n)})$ onto $x_{(n-1)}$. The iteration procedure terminates either when the energy of the error term $(y - y_{(n)})$ is reduced below a certain threshold or the number of iterations reached a fixed maximum number. In [9], interesting variants of the IBP have been proposed that simply pack-project additional terms in each iteration, representing high frequency information in $x_0$. These variants include the use of Canny edge information and the Gabor filter to preserve edges in different directions.

The main challenge in recovering $x$ is the modelling of the unknown blurring function. Gaussian functions with different blurring effect have been considered as a suitable model for use in SR procedures, but they do not reflect severe degradation conditions seen in surveillance scenarios. A suitable model can be based on the use of atmospheric turbulence functions of different strengths (i.e. degradation functions that model environmental conditions caused by variation in temperature, wind speed and exposure time) which extends the effect of the Gaussian functions. In the frequency domain such functions are of the form:

$$H(u, v) = e^{-k(u^2+v^2)^{5/6}} \quad (2)$$

where $k$ is a constant that reflects severity of blurring. We label degradation as severe if $k \epsilon$]0.045, 0.09]; mild (similar to most Gaussian blurring functions) if $k \epsilon$]0.02, 0.04[; and low if $k \epsilon$]0, 0.02[, Figure 8 illustrates the effect for different values of $k$. In what follows, we shall adopt this model of degradation for a number of $k$ values in these ranges to test performance of face recognition from LR images.

## 3 Compressive Sensing

Compressive sensing, also known as sparse recovery, is a novel paradigm of signal sampling that greatly relaxes the stringent limitations of the conventional Shannon-
Nyquist Sampling Theorem, for signals that can be approximated by a sparse expansion in terms of a suitable basis. Image compression tools (e.g. JPEG and JPEG2000) use a DCT or wavelet transforms to obtain different approximately sparse representation of any input image. The concatenation of 2 bases one constructed from wavelet functions and the other from sinusoid functions is expected to be of benefits for image processing/analysis tasks, [10]. Each of these bases provides. The underlying principle of CS is that the number of linear measurements needed to reconstruct a compressed signal should be proportional to the compressed size of the signal, not the uncompressed size. The central challenge for CS is the construction of preferably non-adaptive relatively small number of linear measurements that can guarantee the reconstruction of a sparse or approximately sparse signal. Such a set of linear measurements are represented by rows of an over complete dictionary, [11], i.e. an mxn matrix whose columns form a spanning set of m-dimensional vectors to be used to decompose the signal. Dictionaries generalize vector space basis, and are represented by overcomplete m × n matrices, (m << n), whose columns are expected to form a pool of R^m bases. In this case, any vector in R^m can have multiple representations in terms of the different bases each capturing different features perhaps at different scales. A main premise of this work is that good CS dictionaries can be constructed implicitly from certain pools of bases by concatenation.

Once a suitable underdetermined dictionary $D = \{d_1, d_2, \dots, d_n\} \in \mathbb{R}^{m \times n}$ is created, the main step in CS based tools is then the recovery of the sparsest solution of the equation: $\mathbf{y} = \mathbf{D}\mathbf{x}$, where $\mathbf{y}$ is the observed vector, i.e. finding $\hat{\mathbf{x}} \in \mathbb{R}^n$ such that:

$$ \hat{\mathbf{x}} = \min_{y} \|\mathbf{x}\|_{0} \quad \text{subject to} \quad \|\mathbf{y} - \mathbf{D}\mathbf{x}\|_{2} \qquad (3) $$

This $l_0$-minimization problem, known as the (P$_0$) problem, is computationally NP-hard. If $\mathbf{x}$ is sparse and $D$ is suitably selected, then we can find a unique solution of the $l_1$- minimisation (P$_1$) problem:

$$ \hat{\mathbf{x}} = \min_{y} \|\mathbf{x}\|_{1} \quad \text{subject to} \quad \|\mathbf{y} - \mathbf{D}\mathbf{x}\|_{2}. \qquad (4) $$

This is a convex optimisation problem which is amenable to linear programming. Note that, if $\mathbf{x}$ is small, then the Least Square (LS) method can be used to solve the corresponding the $l_2$- minimisation (P$_2$) problem:

$$ \hat{\mathbf{x}} = (\arg \min_{x: \|x\|_2 = b} \|x\|_2) = \mathbf{D}^*(\mathbf{D}\mathbf{D}^*)^{-1}\mathbf{b}. $$

However, the LS solution is not desired in many applications such as when $\mathbf{x}$ is spiky. Therefore, the use of the $l_1$- minimisation to recover the solution of (P$_0$) problem have been the subject of intense investigations. Bruckstein et al, [10], discuss two basic questions about (P$_0$): (1) Under what conditions, does it have a unique solution? and (2) Given a feasible solution, is there a simple test to verify that is a global minimizer? We now discuss the dependence of these uniqueness requirements on certain parameters and properties of the matrix $D$.
The **sparke** an mxn matrix $D$, denoted by $sp(D)$ is the minimum number of linearly dependent columns of $D$. It is clear that $sp(D) \le m+1$. Equality occurs when $D$ has a full row rank, and then $D$ is said to be of **full sparke**.

**Theorem 1:** (see [10]) If every $(sp(D)-1)$ columns of $D$ are linearly independent then every $(sp(D)/2)$-sparse $x$ can be recovered uniquely from $Dx$. //

This theorem provides an efficient strategy, that we adopt here, for the implicit construction of suitable CS-dictionaries by concatenating certain sets of $\mathbb{R}^m$ bases.

An $m \times n$ dictionary $D$ satisfies the Null Space Property (NSP) of order $k$ if for each size $k$ set $\Omega \subset \{1,...,n\}$ and nonzero vector $z \in \text{Ker}(D)$,

$$ \|z_{\Omega}\|_1 \le \|z_{\Omega^c}\|_1, $$

where $z_A$ is obtained from $z$ by making 0 all coordinates not indexed by $A \subset \{1,...,n\}$.

**Theorem 2:** ([12]) An mxn dictionary $D$ satisfies NSP of order $k$ iff every $k$-sparse solution $x$ can be recovered by $l_1$-minimization. //

It is not difficult to show that if $D$ satisfies NSP of order $k$ then every $k$ columns of $D$ are linearly independent. Consequently, NSP of order $2k$ guarantee uniqueness by Theorem 1 while Theorem 2 then showing the way of recovering the sparsest solution.

Candes and Tao, [13], introduced Restricted Isometry Property (RIP) as sufficient for $l_1$- recovery: an $m \times n$ dictionary $D$, $m \ll n$, satisfy the RIP of order $k$ if there is a constant $0 < \delta_k < 1$, such that for any $k$-sparse signal $x \in \mathbb{R}^n$:

$$ (1 - \delta_k) \|x\|_2^2 \le \|Dx\|_2^2 \le (1 + \delta_k) \|x\|_2^2 \quad (5) $$

The smallest $\delta_k$ is called the restricted Isometry constant (RIC) of order $k$, and if $D$ satisfies RIP of order $k$, then any $2k$-columns sub-matrix of $D$ must be well- conditioned, [14]. The condition number of a matrix is the ratio of its maximum to the minimum singular values. Checking this property for all $2k$- columns submatrices is computationally infeasible as it requires exhaustive check of all $\binom{n}{2k}$ submatrices. The statistical version of the restricted Isometry property (STRIP) provides a computationally easier to check version of the RIP property. It requires computation of condition numbers of sufficiently large uniformly randomly selected such submatrices. Gan et al [15], developed a STRIP performance bound in terms of the mutual coherence $\mu$ of the dictionary which is an indicator of the dependence between columns of the matrix. The coherence of a matrix provides information about the likelihood of guaranteed recovery of the sparse solution, and is defined as the largest absolute normalized inner product of pairs of columns $a_i$ and $a_j$, i.e.
$$ \mu(A) = \max_{1 \le i < j \le n} \frac{|\langle a_i, a_j \rangle|}{\|a_i\|_2 \|a_j\|_2} \quad (6) $$

It is not difficult to show that if D is a dictionary with unit column vector and coherence $\mu$ then D satisfies RIP of order k with $\delta_k \le (k-1)\mu$, (see [15]). When $n \gg m$, the coherence value has been shown, [15], to be bounded below by $\frac{1}{\sqrt{m}}$, which is reasonably tighter than the Welch bound $\sqrt{\frac{n-m}{m(n-1)}}$.

Due to unfeasibility of exhaustive search, we shall follow a statistical sampling approach when estimating the strength of RIP of the various dictionaries in terms of the condition number and coherence, or when testing for linear independence of the 2k columns submatrices of the dictionary. There are a number of efficient sparse recovery algorithms that have been developed including the Homotopy method (LARS) and the Iteratively Reweighted Least Square method (IRLA), [12].

# 4 CS-based Superresolution

Here, we briefly describe a recently developed CS-based method to super-resolve low resolution degraded images which uses underdetermined dictionaries that are assumed to satisfy RIP. We list a number of approaches to dictionary construction including the LD approach, random constructions, and a new construction strategy that is independent of training images but designed to implicitly be of full sparke. We shall test the strength of RIP, as described above, for the constructed dictionaries and use the corresponding pairs of dictionaries to reconstruct super resolved images from low resolution degraded face images.

## 4.1 CS approach for image SR

It requires the use of 2 CS dictionaries: a Low resolution matrix $D_L$ of size 100x512 and a High resolution matrix $D_H$ of size 25x512. The input to this scheme is a degraded low row resolution small image Lr, and the output is super-resolved to double the size image that is meant to be of “high quality”. The Lr image is resized by decimating its pixels and Bi-cubic interpolation to obtain double the size image LR which is still degraded. Three more versions of the LR image are created by applying 3 spatial filters to highlights edges in different directions. These four images are subdivided into blocks of size 5, and the pixel values in the 4 versions are turned into a column vector of 100=4x25 by concatenation. In order to avoid the appearance of blocking artefacts, the LR image will be subdivided into overlapping blocks.

Initialize a HR image of the same size of the LR image for the super-resolved image. The 5x5 blocks are then processed iteratively as follows:
1. Let $\mathbf{y}$ be corresponding 100-dimensional vector.

2. Find the sparse solution $\mathbf{z}$ of the underdetermined equation $\mathbf{y} = D_L \mathbf{z}$

3. Calculate the 25-dimensional HR patch $\mathbf{x}$ using the matrix multiplication $\mathbf{x} = D_H \mathbf{z}$,

4. Back-projection the 2-dimensional 5x5 version of $\mathbf{x}$ onto the existing HR.

In the rest of this section we describe the pairs of Dictionaries $D_L$ and $D_H$ for the various dictionary construction strategies adopted in this paper.

## 4.2 The Image-based Learnt Dictionary

This construction was proposed by Yang et al, (see [4], [5], & [6]) and used for super-resolution based face recognition. It is based on learning dictionaries using patches from a large training set of high resolution images of good quality that exhibit similar statistical characteristics of the pattern recognition task under investigation. We shall refer to this construction as LD dictionary. The $D_H$ and $D_L$ dictionaries are created as follows:

1. A sufficiently large number of high resolution (HR) images (here Face images) are selected and each divided into patches of 5x5 pixels. Patches overlap.

2. Randomly sampling raw patches from a training HR images, and each selected patch is transformed into a normalised vector that is added as a column to the $D_H$.

3. Generate a set LR of blurred versions of the HR images, and create 3 other filtered versions, and the columns of $D_L$ are constructed in a similar way as in above, but by concatenating the patches from the LR images and their 3 filtered version. Again the columns are to be normalised.

## 4.3 Random Dictionaries

CS randomly constructed measurement matrices that satisfy the Restricted Isometry Condition include Gaussian, Toeplitz and Circular random Matrices. For Gaussian Random Matrix (GRM), the entries $x_{i,j}$ of the CS matrix of size $m \times n$ are independently sampled from a normal distribution $x_{i,j} \sim N(0,1/m)$, the $l_2$-norm was used to normalize each columns in the dictionary. In order to recover super resolved image from a single LR image for face recognition via sparse representation, two overcomplete dictionaries $D_H$, $D_L$ of size $25 \times 512$ and $100 \times 512$ respectively have been generated from a zero mean Gaussian distribution with variance $1/25$.

Toeplitz-Circular Random measurement matrix (TCRM) are another class of RIP dictionaries that have been widely used. Bajwa et al. [16], have shown that Toeplitz-structured matrices are sufficient to recover undersampled sparse signals. Toeplitz and Circular matrices of the size $k \times n$ are respectively of the form:

$$T = \begin{bmatrix} t_n & t_{n-1} & \dots & t_1 \\ t_{n+1} & t_n & \dots & t_2 \\ \vdots & \vdots & \ddots & \vdots \\ t_{n+k-1} & t_{n+k-2} & \dots & t_k \end{bmatrix}, \text{ and } C = \begin{bmatrix} t_n & t_{n-1} & \dots & t_1 \\ t_1 & t_n & \dots & t_2 \\ \vdots & \vdots & \ddots & \vdots \\ t_{n-1} & t_{n-2} & \dots & t_k \end{bmatrix}$$
For image reconstruction, $D_H$ and $D_L$ are generated as TCRM matrices, where the first row consists of standard Gaussian random variables, and the rest of the rows are permuted versions of the first row as shown above.

## 4.4 Iteratively Constructed Full Spark Dictionaries

Full-sparke dictionaries is class of full row rank overcomplete $m \times n$ dictionaries, where $m \ll n$, so that each m-columns sub-matrix is a basis of $\mathbb{R}^m$. Here we describe an example on how to construct such matrices by starting with an invertible mxm matrix and iteratively appending a set of image independent linearly independent m-column vectors in $\mathbb{R}^m$ while maintaining the full sparke property after every addition. One way to maintain the full sparke is to insist that every new column can only be generated by the full columns of the previous inserted submatrices. In this, paper we present a simple example of such a dictionary, but in the future we shall investigate algebraic construction method using group finite actions on $\mathbb{R}^m$.

Our example of full sparke dictionaries, referred to as LID, is of the form:

$$D = [A_{p_1}, A_{p_2}, \dots, A_{p_k}, C(A_{p_{k+1}})].$$

For $i=1, \dots, k+1$, the $p_i$'s, are distinct real numbers $>1$, and

$$A_{p_i} = \begin{pmatrix} 1 & \frac{1}{p_i} & & \frac{1}{p_i^2} & \dots & \frac{1}{p_i^{m-1}} \\ \frac{1}{p_i} & 1 & & \frac{1}{p_i} & \dots & \frac{1}{p_i^{m-2}} \\ & \frac{1}{p_i^{m-1}} & \frac{1}{p_i^{m-2}} & \vdots & \frac{1}{p_i^{m-3}} & \dots & 1 \end{pmatrix}.$$

Note that $k = \lfloor n/m \rfloor$ and the last sub-matrix of $D$ is simply the first (n-km) columns. Then, the $m \times n$ LID dictionary is obtained from the following matrix after normalising its columns using the $l_2$-norm.

For our experimental purposes we the LID high-dictionary $D_H$ is generated from using integers $p_i > 1$. For simplicity, the low-dictionary $D_L$ was created from a Standard Gaussian Random Matrix (GRM).

## 4.5 Comparison of RIP parameters for Different $D_H$ dictionaries

Here we present some comparisons of the “strength” of the RIP for the LD and LID dictionaries. Whenever exhaustive search is infeasible we conducted a statistical testing by taking random sample of 100 cases. To test for full sparke property, we evaluated the determinants, as indicator of linear independence, for more than a hundred randomly selected sample of 25 × 25 submatrices of the corresponding $D_H$ dictionaries. Although in theory, the LD dictionary may statistically satisfy NSP of order 12, Figure 1 shows that the determinant of most 25x25 submatrices is so small (almost zero) and hence the full sparke property is not satisfied. In contrast, figure 2 confirm that the LID is indeed fully sparke.
<figure><img src="image_2.png" /><figcaption>Figure1. Determinant of submatrices from the High-resolution LD Dictionary</figcaption></figure>

<figure><img src="image_3.png" /><figcaption>Figure2. Determinant of submatrices from the High-resolution LID dictionary</figcaption></figure>

The next experiment to calculate another RIP indicator, namely the condition number of 25x25 submatrices of 4 of the DH dictionaries for LD, LID. GRM and the TCRM. These condition numbers are expected to be bounded by RIC of order 2k, with k=12. Table 1, below, displays the mean and standard deviation of the condition numbers for 100 randomly selected submatrices and the condition number of the full size 25 × 512 matrix.

<table><caption>Table 1. Mean and Standard deviation for CN for a hundred random sub-matrices of different sizes.</caption><thead><tr><th rowspan="3">submatrices</th><th colspan="8">Dictionaries</th></tr><tr><th colspan="2">LID</th><th colspan="2">LD</th><th colspan="2">GRM</th><th colspan="2">TCRM</th></tr><tr><th>mean</th><th>std</th><th>mean</th><th>std</th><th>mean</th><th>std</th><th>mean</th><th>std</th></tr></thead><tbody><tr><td>25x25</td><td>3.08</td><td>3.14</td><td>3.34E+16</td><td>1.79 E+17</td><td>279.36</td><td>597.79</td><td>85.19</td><td>155.68</td></tr><tr><td>Full matrix</td><td>1.977</td><td></td><td>1.00E+15</td><td></td><td>1.43</td><td></td><td>1.453</td><td></td></tr></tbody></table>

These results again demonstrate that the overcomplete LID dictionary is well-conditioned in comparison to all others for the various submatrices but for the full matrix GRM and TCRM have similar condition numbers that are better than the LID. Moreover, the condition number of the LD is extremely large for all cases, which make these dictionaries very ill conditioned.

Another test relates to calculating the row-rank and coherence values for the various dictionaries. It is well known that the highest sparsity recovered signal for any dictionary = (1+ row rank)/2, and coherence µ must satisfy 0.2 =1/√m ≤ µ ≤ 1. Again, results in Table 2 highlight the superiority of the LID dictionary.
<table><caption>Table 2. Row-rank and Coherence</caption><thead><tr><th>Dictionary</th><th>LID</th><th>LD</th><th>GRM</th><th>TCRM</th></tr></thead><tbody><tr><td>Row Rank</td><td>25</td><td>24</td><td>25</td><td>25</td></tr><tr><td>Coherence</td><td>0.9958</td><td>&lt; 0.2</td><td>0.7438</td><td>0.7318</td></tr></tbody></table>

Finally, in order to test the viability of the CS-based SR scheme, we compared the visual quality of large set of SR enhanced images obtained from the application of the various discussed dictionaries, as well as the IISR scheme and interpolation methods. The results of the visual inspection reveal that there is little difference in the quality of the recovered HR image using the various dictionaries, but a noticeable improvement that can be noticed in SR methods, including IISR, over the low-resolution images and interpolation methods at every level of degradation. Figure 3, below, show one example but this pattern was repeated over all the images. PSNR valuescalculated between the output SR image and the original images confirm the same pattern, but we omit these results. Unsurprisingly and regardless of the method used in the SR procedure, the quality of SR images decreases as the level of blurring increases. With increased level of blurring there is no difference in image quality obtained by different dictionary methods. But the dictionary methods produced slight improvement on the IISR method, and superiority over the interpolation methods at every level of blurring.

## 5 Face Recognition – Experimental Results

In this section, we shall compare the performance of face recognition using the corresponding super-resolved images. We test the performance of different dictionary methods as well as a state-of-the-art methods to reconstruct super resolved face image from a single LR image with different magnification blur. The image sets were sampled from a publicly available face database and, comparisons recognition rates presented with matching in low-resolution domain. We use a simple but efficient wavelet-based face recognition scheme, whereby the training as well the matching image are wavelet decomposed to level 3 and each of the each of the subbands at level 3 (i.e. LL3, HL3, LH3 and HH3) is used as a face feature vector and Euclidian distance is used for matching against the face feature vectors of the templates.

To test the performance of the wavelet face recognition schemes post the CS-based SR preprocessing schemes, we used face images from the Extended Yale B database. This database consists of 2,414 frontal-face images of 38 individuals. The cropped and normalized 192 × 168 face images were captured under various laboratory-controlled lighting conditions. For each subject, we selected the P00A+00E+00 image for the gallery set and the other images for testing. To construct LD that depends on images, three images for each subject were selected from the well-lit face images in set 1 that were not included in the gallery/test images.

To simulate face recognition under the recording condition of low resolution degraded quality images, we first apply the degradation function defined by equation 2
for different values of k on the high resolution images in the reasonably lit sets (1 and 2) of the database. The low resolution degraded Lr images are finally obtained by down-sampling the degraded images by a factor of 2.

Each test Lr face image is first super-resolved using each of the built dictionaries, the IISR scheme or simply by up-sampling and interpolation.

For feature extraction, on the resulting HR templates as well as the SR-resolved test images, we use the Z-score normalized coefficients of the subbands of the Haar wavelet decomposed face images at level three. Matching will be based on the City Block distance between a probe image and a gallery image.

The experimental results are shown in Figure 5 for the different subbands and differently illuminated sets of images. The various charts display the accuracy rate at each level of degradation function. As can be seen, there is no significant difference in identification accuracy rates, between the different dictionaries methods. Moreover, the accuracy rates seem to be maintained at the same level for different degradation level. In comparisons to the method of matching the LR images with down-sampled gallery images, the performance of the dictionary based methods are far more superior and much more apparent as the image quality deteriorates from mild to severe degradation. However, the picture is surprisingly different for mild image degradation (i.e. $k_1 < 0.7$) when we use the HL3 or HH3 subband as feature vectors for set 1 images which are recorded in a slightly better illumination condition. Note that the full set of experiments, part of which are not included here, indicate similar patterns of performance, i.e. a much better accuracy are achieved by all the dictionaries than that achieved by the various interpolation methods such as nearest, bilinear and Bi-cubic and the difference in performance increases as the of degradation get more severe.

The observed pattern of variation in the performance of face recognition, using different sub-bands is consistent with known results for wavelet-based face recognition without degradation reported in the literature, (see [17]). For this work we also conducted face recognition for exactly the same conditions on resolution and blurring but using the Principle Component Analysis technique (PCA). The results exhibit similar patterns and are omitted here.

Subset 1

<figure><img src="image_4.png" /><figcaption>LL<sub>3</sub>-wavelet subband</figcaption></figure>

Subset 2

<figure><img src="image_5.png" /><figcaption>LL<sub>3</sub>-wavelet subband</figcaption></figure>
<figure><img src="image_6.png" /><figcaption>LH<sub>3</sub>-wavelet subband</figcaption></figure>

<figure><img src="image_7.png" /><figcaption>LH<sub>3</sub>-wavelet subband</figcaption></figure>

<figure><img src="image_8.png" /><figcaption>HL<sub>3</sub>-wavelet subband</figcaption></figure>

<figure><img src="image_9.png" /><figcaption>HL<sub>3</sub>-wavelet subband</figcaption></figure>

<figure><img src="image_10.png" /><figcaption>HH<sub>3</sub>-wavelet subband</figcaption></figure>

<figure><img src="image_11.png" /><figcaption>HH<sub>3</sub>-wavelet subband</figcaption></figure>

<caption>Figure 3. Recognition accuracy rates using different dictionaries and in comparisons with matching in low-resolution</caption>

# 6 Conclusion

We have studied the RIP property for random, and not so random, constructions of CS related overcomplete dictionaries as well as an existing dictionary that trained on a set of high-resolution face images. These dictionaries were used to generate super resolved image with the aim of using for face recognition in uncontrolled conditions
where the input is degraded blurred LR image with a wide range of degradation. This
results effectively support the use of SR based techniques that employ CS dictionaries
for recovering super-resolved images that are suitable for face recognition. More im-
portantly, that there is no need for using image sets for training dictionaries, because
non-adaptive dictionaries perform equally well if not better in some cases. In an
attempt to find possible explanation, we conducted a number of tests of numerical
matrix parameters relevant to the RIP condition. We note that the learning image-
based dictionary is highly ill conditioned and far from satisfying the RIP related con-
ditions discussed in the literature. Perhaps the use of image patches with the same
statistical parameters of general face image patches compensate for the lack of RIP
properties.

Further studies are needed to test other implicit construction of RIP dictionaries.
Indeed, we have developed a new method which aims to implicitly satisfy the known
bounds on singular values. Such approaches can be exploited to use RIP dictionaries
for revocable face biometric instead of the traditional random projection.

<figure><img src="image_12.png" /><figcaption>Original
HR image</figcaption></figure>

<figure><img src="image_13.png" /><figcaption>LR,
Without
Blurred</figcaption></figure>

<figure><img src="image_14.png" /><figcaption>GRM</figcaption></figure>

<caption>TCRM</caption>

<caption>LID1</caption>

<caption>LID2</caption>

<caption>LD</caption>

<figure><img src="image_15.png" /><figcaption>BC</figcaption></figure>

<caption>BL</caption>

<figure><img src="image_16.png" /><figcaption>LR,
k₁=0.0
1</figcaption></figure>

<figure><img src="image_17.png" /><figcaption>GRM</figcaption></figure>

<caption>TCRM</caption>

<caption>LID1</caption>

<caption>LID2</caption>

<caption>LD</caption>

<caption>IISR</caption>
<figure><img src="image_18.png" /></figure>

LR,
k₁=0.0
3

<figure><img src="image_19.png" /><figcaption>BC</figcaption></figure>

<figure><img src="image_20.png" /><figcaption>BL</figcaption></figure>

<figure><img src="image_21.png" /><figcaption>NE</figcaption></figure>

<figure><img src="image_22.png" /><figcaption>GRM</figcaption></figure>

<figure><img src="image_23.png" /><figcaption>TCRM</figcaption></figure>

<figure><img src="image_24.png" /><figcaption>LID1</figcaption></figure>

<figure><img src="image_25.png" /><figcaption>LID2</figcaption></figure>

<figure><img src="image_26.png" /><figcaption>LD</figcaption></figure>

<figure><img src="image_27.png" /><figcaption>IISR</figcaption></figure>

<figure><img src="image_28.png" /><figcaption>BC</figcaption></figure>

<figure><img src="image_29.png" /><figcaption>BL</figcaption></figure>

<figure><img src="image_30.png" /><figcaption>NE</figcaption></figure>

<figure><img src="image_31.png" /><figcaption>GRM</figcaption></figure>

<figure><img src="image_32.png" /><figcaption>TCRM</figcaption></figure>

<figure><img src="image_33.png" /><figcaption>L.I.D1</figcaption></figure>

<figure><img src="image_34.png" /><figcaption>L.I.D2</figcaption></figure>

<figure><img src="image_35.png" /><figcaption>LD</figcaption></figure>

<figure><img src="image_36.png" /><figcaption>IISR</figcaption></figure>

<figure><img src="image_37.png" /></figure>

LR,
k₁=0.0
5

<figure><img src="image_38.png" /><figcaption>BC</figcaption></figure>

<figure><img src="image_39.png" /><figcaption>BL</figcaption></figure>

<figure><img src="image_40.png" /><figcaption>NE</figcaption></figure>

Restoration
on-BC-image

Restoration
on-BL-image

Restoration
on-NE-image

<figure><img src="image_41.png" /><figcaption>GRM</figcaption></figure>

<figure><img src="image_42.png" /><figcaption>TCRM</figcaption></figure>

<figure><img src="image_43.png" /><figcaption>L.I.D1</figcaption></figure>

<figure><img src="image_44.png" /><figcaption>L.I.D2</figcaption></figure>

<figure><img src="image_45.png" /><figcaption>LD</figcaption></figure>

<figure><img src="image_46.png" /><figcaption>IISR</figcaption></figure>

<figure><img src="image_47.png" /></figure>

LR,
k₁=0.0
7
<figure><img src="image_48.png" /><figcaption>Figure 4. Super Resolved image by different SR methods and well non-interpolation methods</figcaption></figure>

REFERNCES

[1] Hennings-Yeomans, P.H., Baker, S. and Kumar, B.V., "Simultaneous super-resolution and feature extraction for recognition of low-resolution faces," In proc. IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 1-8 (2008).

[2] He, J. and Zhang, D., "Face super-resolution reconstruction and recognition from low-resolution image sequences," Computer Engineering and Technology (ICCET), 2nd International Conference on, Vol. 2, V2-620 (2010).

[3] Zeyde, R., Protter, M. and Elad, M., "On Single Image Scale-Up using Sparse-Representation," Springer, Curves and Surfaces, 711-730 (2010).

[4] Yang, J. et al., "Image super-resolution as sparse representation of raw image patches," In Proc. IEEE Computer Society Conference on Computer Vision and Pattern Recognition. 1-8(2008).

[5] Yang, J. et al., "Image super-resolution via sparse representation," IEEE Transactions on Image Processing, 11(19), 2861-2873 (2010).

[6] Yang, J. et al., "Face hallucination via sparse coding. Image Processing," 15th IEEE International Conference on. 1264-1267 (2008).

[7] Wang, S., Zhang, L., Liang, Y. and Pan, Q., "Semi-coupled dictionary learning with applications to image super-resolution and photo-sketch synthesis," 2216-2223(2012).

[8] AL-Hassan, N., Jassim, S.A. and Sellahewa, H., "Enhancing face recognition at a distance using super resolution," ACM, MMSec, 123-132 (2012).
[9] R. R Makwana and N. D Mehta, "Single Image Super-Resolution VIA Iterative Back Projection Based Canny Edge Detection and a Gabor Filter Prior", International Journal of Soft Computing and Engineering (IJSCE), 2231-2307, Volume-3, Issue-1, March 2013.

[10] A. M. Bruckstein, D. L. Donoho, and M. Elad, "From sparse solutions of systems of equations to sparse modeling of signals and images", SIAM Review, vol. 51, no. 1, pp. 34–81, 2009.

[11] Rubinstein, R., Bruckstein, A.M. and Elad, M., "Dictionaries for sparse representation modeling," Proceedings of the IEEE, 6(98), 1045-1057 (2010).

[12] M. Fornasier and H. Rauhut. Compressive Sensing. In O. Scherzer, editor, Handbook of Mathematical Methods in Imaging, pages 187–228. Springer, 2011.

[13] Candes, E.J. and Tao, T., "Near-Optimal Signal Recovery from Random Projections: Universal Encoding Strategies?" IEEE Transaction on Information Theory. 12(52), 5406-5425 (2006).

[14] Baraniuk, R., Davenport, M., DeVore, R. and Wakin, M., "A simple proof of the restricted isometry property for random matrices," Constructive Approximation, 3(28), 253-263 (2008).

[15] Gan, L., Ling, C., Do, T.T., and Tran, T.D., "Analysis of the statistical restricted isometry property for deterministic sensing matrices using Stein's method," Citeseer, 2009.

[16] Bajwa, W.U., Haupt, J.D. and Raz, G.M., "Toeplitz-Structured Compressed Sensing Matrices," IEEE 14th Workshop on SSP'07. 294-298 (2007).

[17] W. Al-Obaydy, and H. and Sellahewa. On using high-definition body worn cameras for face recognition from a distance. Springer-Verlag, 193-204, 2011