## 1 Introduction  
Image classification and clustering problems are topics fundamental to various areas of machine learning [1; 2; 3] including image recognition and image clustering. Principal component analysis (PCA) has been widely used to perform dimensionality reduction and extract useful information for these problems [4; 5; 6; 7; 8; 9; 10; 11]. One of the common objectives of dimensionality reduction is to retain the most important information that is beneficial to data processing tasks and meanwhile filter out corrupted and noisy information from the dataset. One example is the face recognition problem. A face database may contain face images with occlusions such as scarves, sunglasses and so forth. Obviously, the key information that can recognize a person is the facial features not the scarves nor the sunglasses. A good dimension reduction method can effectively extract the key facial features and, at the same time, ignore the occluded information. Another example is the image clustering problem. The purpose of clustering is to partition the data into different clusters and group the similar data objects together. However, similar to the above face recognition problem, the corrupted and noisy information such as scarves can make two facial images very different even they are taken from the same person. This makes the clustering task very challenging. Again, a good dimension reduction method can filter out the scars and retain the key facial features and gives a more accurate clustering result. The major challenge of PCA for supervised classification and unsupervised clustering problems is how to extract important information that can distinguish the characteristics of different classes/clusters from a corrupted and noisy dataset. For a face recognition problem, the facial features of different persons are the key to distinguish them. However, the corrupted and noisy information can be a factor that cause the difference among different facial images. Figure 1 shows two frontal face images and two facial images with scares. Obviously, facial features such as eyes of these persons are a bit different. They can be used to distinguish these persons. However, the face with scarves and the face without scarves are different. Also, the two scarves are different too. If the dimension reduction method wrongly identifies the scarves as the key features that cause the difference, the accuracy of the trained recognition system may heavily depend on this occluded information and lead to a poor recognition result. In image recognition and clustering problem, this kind of data object is known as outlier. This means the data object carry corrupted and noisy information. Many different methods have been proposed to solve the above outlier problem. However, these techniques still have evident defects when applied to real-world classification and clustering problems. The very first method used to extract the important information of the data is to apply eigenvalue decomposition to the covariance matrix of the data [12]. The eigenvectors with small eigenvalues are discarded and the eigenvectors with large eigenvalues are retained for classification or clustering. This method works well only if the data do not contain any outlier. This is because an eigenvector of a covariance matrix represents the variance of the data in a specific direction. However, the occlusion of a facial image can cause large variation as well. The eigenvectors with large eigenvalues can wrongly identify the occluded information as important information. To address this problem, different \(l_{p}\) norm-based methods have been proposed. One example is the \(l_{1}\)-PCA, which adopts the \(l_{1}\) norm to measure the variation of features of the data [13; 14]. This approach replaces the squared \(l_{2}\) norm that is adopted by the classical PCA by a \(l_{1}\) norm. The \(l_{1}\) norm measure usually gives a much lower weight to the data objects that are far away from the majority.