## 3 Proposed Framework  
In this section, we first describe the main intuition behind our PSI-CNN architecture. Conventional CNN models for face recognition comprise fewer number of filters in the shallower-side layers than those in the deeper-end convolutional layers. For example, the VGG face model only uses 64 filters in the first layer although the last convolutional layer utilizes 512 filters. The extracted feature map is passed onto the next convolution step. As mentioned in Section 2, utilizing the characteristics of local patterns can improve the recognition performance. However, since the sizes of input images are typically over \(200\times 200\), the number of convolutional filters in the shallower layers are limited by available memory and computation power. Consequently, some patterns are excluded from the network despite these local characteristics being potentially useful for further enhancing the recognition performance. Throughout this paper, we will refer to these excluded but potentially useful patterns as _untrained_ patterns.  
As shown in Figure 2, PSI-CNN works by additionally extracting the aforementioned _untrained_ feature maps from a set of downsampled images (of the original input image) and fusing this information during the convolution stages. We will use the schematic diagram of the PSI-CNN architecture drawn in Figure 2 for further illustration of the model included in Section 3.1. (In this diagram, \((w,h,d)\) in Figure 2 refer to width, height, and number of feature maps respectively.)  
\begin{table}
\begin{tabular}{c c c} \hline \hline
**Approaches** & **Methods** & **Characteristics** \\ \hline \multirow{4}{*}{Holistic-based} & \multirow{2}{*}{Eigenfaces (PCA) [5]} & Generates Eigenfaces which represent the global characteristics of faces from training data by extracting principal components. \\ \cline{2-3}  & & Computes representative axes which minimize intra-class distances and maximizes inter-class distances simultaneously from the global features of the training data. \\ \hline \multirow{4}{*}{Partial-based} & \multirow{2}{*}{Local Binary Patterns [7]} & Describes the characteristics of local textures as binary patterns by comparing the center pixel value with that of its neighbors in each local window. \\ \cline{2-3}  & & SIFT [10] & Applies the SIFT algorithm to extract local facial features which are scale and rotation-invariant. \\ \cline{2-3}  & \multirow{2}{*}{HoG Descriptor [11]} & Extracts local HOG feature from each divided grid and concatenates all feature vectors. \\ \cline{2-3}  & & High-dimensional LBP [12] & Extracts 100K-dimensional LBP feature vector from multiple patches by considering scale variations. \\ \hline \multirow{4}{*}{Deep learning} & \multirow{2}{*}{DeepFace [1]} & Normalizes input face image based on a 3D face model and extracts feature vectors by using a 9-layer CNN. \\ \cline{2-3}  & & \multirow{2}{*}{DeepID [2]} & Generates 60 patches from input image and extracts a 160-dimensional feature vector from each patch. Also, a joint-Bayesian method is applied to verify whether the same person is observed or not. \\ \cline{2-3}  & & \multirow{2}{*}{FaceNet [3]} & Applies triplet loss to minimize distances between images of the same identities and maximize distances between images of different identities. Embeds each feature vector in the Euclidean space. \\ \cline{2-3}  & & \multirow{2}{*}{VGGFace [4]} & Consists of 13 convolutional layers and 3 fully connected layers in series. Extracts feature vectors by propagating through the network. \\ \hline \hline \end{tabular}
\end{table}
Table 1: A summary of past and current face recognition methods