We propose Image-Semantic-Transformation-Reconstruction-Circle(ISTRC) model, a novel and powerful method using facenet's Euclidean latent space to understand the images. As the name suggests, ISTRC construct the circle, able to perfectly reconstruct images. One powerful Euclidean latent space embedded in ISTRC is FaceNet's last layer with the power of distinguishing and understanding images. Our model will reconstruct the images and manipulate Euclidean latent vectors to achieve semantic transformations and semantic images arthimetic calculations. In this paper, we show that ISTRC performs 10 high-level semantic transformations like "Male and female","add smile","open mouth", "deduct beard or add mustache", "bigger smaller nose", "make older and younger", "bigger lips", "bigger eyes", "bigger smaller mouths" and "more attractive". It just takes 3 hours(GTX 1080) to train the models of 10 semantic transformations.
FaceNet @cite_12 can directly learn the mapping from images to their latent vector in Euclidean space end-to-end. It is used for face verification since we could compare the similarity between two face images by their Euclidean distances in this Euclidean space. In this paper, we directly use the pre-trained FaceNet model for cropped CelebA images.
@cite_12
2096733369
Despite significant recent advances in the field of face recognition [10, 14, 15, 17], implementing face verification and recognition efficiently at scale presents serious challenges to current approaches. In this paper we present a system, called FaceNet, that directly learns a mapping from face images to a compact Euclidean space where distances directly correspond to a measure of face similarity. Once this space has been produced, tasks such as face recognition, verification and clustering can be easily implemented using standard techniques with FaceNet embeddings as feature vectors. We propose a novel discriminative learning approach to image set classification by modeling the image set with its natural second-order statistic, i.e. covariance matrix. Since nonsingular covariance matrices, a.k.a. symmetric positive definite (SPD) matrices, lie on a Riemannian manifold, classical learning algorithms cannot be directly utilized to classify points on the manifold. By exploring an efficient metric for the SPD matrices, i.e., Log-Euclidean Distance (LED), we derive a kernel function that explicitly maps the covariance matrix from the Riemannian manifold to a Euclidean space. With this explicit mapping, any learning method devoted to vector space can be exploited in either its linear or kernel formulation. Linear Discriminant Analysis (LDA) and Partial Least Squares (PLS) are considered in this paper for their feasibility for our specific problem. We further investigate the conventional linear subspace based set modeling technique and cast it in a unified framework with our covariance matrix based modeling. The proposed method is evaluated on two tasks: face recognition and object categorization. Extensive experimental results show not only the superiority of our method over state-of-the-art ones in both accuracy and efficiency, but also its stability to two real challenges: noisy set data and varying set size. Retrieving videos of a specific person given his her face image as query becomes more and more appealing for applications like smart movie fast-forwards and suspect searching. It also forms an interesting but challenging computer vision task, as the visual data to match, i.e., still image and video clip are usually represented quite differently. Typically, face image is represented as point (i.e., vector) in Euclidean space, while video clip is seemingly modeled as a point (e.g., covariance matrix) on some particular Riemannian manifold in the light of its recent promising success. It thus incurs a new hashing-based retrieval problem of matching two heterogeneous representations, respectively in Euclidean space and Riemannian manifold. This work makes the first attempt to embed the two heterogeneous spaces into a common discriminant Hamming space. Specifically, we propose Hashing across Euclidean space and Riemannian manifold (HER) by deriving a unified framework to firstly embed the two spaces into corresponding reproducing kernel Hilbert spaces, and then iteratively optimize the intra- and inter-space Hamming distances in a max-margin framework to learn the hash functions for the two spaces. Extensive experiments demonstrate the impressive superiority of our method over the state-of-the-art competitive hash learning methods. Even if, most of 2D face recognition approaches reached recognition rate more than 90 in controlled environment, current days face recognition systems degrade their performance in case of uncontrolled environment which includes pose variations, illumination variations, expression variations and ageing effect etc. Inclusion of 3D face analysis gives an age over 2D face recognition as they give vital informations such as 3D shape, texture and depth which improve discrimination power of an algorithm. In this paper, we have investigated different 3D face recognition approaches that are robust to changes in facial expressions and illumination variations. 2D-PCA and 2D-LDA approaches have been extended to 3D face recognition because they can directly work on 2D depth image matrices rather than 1D vectors without need for transformations before feature extraction. In turn, this reduces storage space and time required for computations. 2D depth image is extracted from 3D face model and nose region from depth mapped image has been detected as a reference point for cropping stage to convert model into a standard size. Two Dimensional Principal Component Analysis (2D-PCA) and Two Dimensional Linear Discriminant analysis (2D-LDA) are employed to obtain feature vectors globally compared to feature vectors obtained locally using PCA or LDA. Finally, euclidean distance classifier is applied for comparison of extracted features. A set of experiments on GavabDB 3D face database, which has 61 individuals in total, demonstrated that 3D face recognition using 2D-LDA method has achieved recognition accuracy of 93.3 and EER of 8.96 over database, which is higher compared to 2D-PCA. So, more optimized performance has been achieved using 2D-LDA for 3D face recognition analysis.