# An Efficient Multiscale Scheme Using Local Zernike Moments for Face Recognition
## 1 Introduction
Similar to [39], in MS-LZM, the LZM transformation is applied twice in succession. With the first transformation, the effects of lighting and blurring on the images are removed. The outputs of the second transformation are used to create feature vectors for face images. In most studies using local methods [20; 46; 47], the images generated by encoding with the local operators are divided into sub-regions and histograms are calculated in each sub-region to obtain the features. Recently, methods that use features around facial points have become popular [2; 48; 49], and we prefer to use this approach. For each complex LZM component, we calculate the phase-magnitude histograms (PMHs) on the patches around the facial points. Then, these PMHs calculated on all components are concatenated to form a feature vector for each facial point. Using both local and global shape properties improves performance [50]. Hence, we build an image pyramid and calculate the features using the images of each different scale. In this way, the local and global patterns are extracted by using large- and small-scale images, respectively. In the last step of MS-LZM, the dimension of the feature vectors obtained for each facial point is reduced using whitened principal component analysis (WPCA) [51]. In recent years, high-performance face recognition methods [7; 8; 9; 52] have been developed using deep learning techniques. For the training of the proposed models in these supervised methods, a large number of labeled images are needed. Models that are trained in some studies are shared with the public. However, when these studies are to be repeated for any real-world problem, both data and some high-quality hardware are needed. MS-LZM proposed in this study does not require any labeled images, since it is an unsupervised method. In addition, PCA matrices can be easily learned using very few images. In our previous work [53], we used multiple grids of different sizes to extract features around facial points from single-scale LZM components, and we showed the performance of the proposed method on images obtained from controlled environments. In this study, face images are encoded with the LZM transformation in a multi-scale manner, and the features are extracted from the resulting multi-scale LZM components using a single grid. To evaluate the method, we used three different datasets with different properties. The contributions of this study are as follows: (1) In previous studies using LZM transformation, methods have been proposed for only face identification [39; 42; 43; 53] or verification [41]. In this study, we develop an unsupervised face recognition scheme for both identification and verification; (2) In most of the LZM-based studies [39; 41; 42; 53], only one dataset is used to evaluate the methods. In this study, using Face Recognition Technology (FERET) [54], Labeled Faces in the Wild (LFW) [55], and Surveillance Cameras Face (SCface) [56] datasets, which have very different characteristics from each other, we show the performance of MS-LZM (the code is available at [[https://github.com/ebsrn/lzm-face](https://github.com/ebsrn/lzm-face)]([https://github.com/ebsrn/lzm-face](https://github.com/ebsrn/lzm-face))) on unconstrained and low-resolution images acquired in uncontrolled environments or in infrared spectrum. Therefore, this work is important to show that the LZM transformation can be used successfully in very different real-world conditions; (3) The experimental results demonstrate that the proposed approach outperformed the state-of-the-art for FERET and SCface. In addition, the results for LFW were comparable to the best known results in literature.