Search based face annotation

In the era of big data today, a large amount of facial images ar hared over the internet, which have posed a lot of research challenges and created a quantit y of opportunities in various realworld applications. An important emerging research proble m in this area is automatic face annotation, which aims to automatically tag human names to a query facial image. Unlike the conventional “model-based face annotation” techniques, some recent emerging studies have attempted to tackle this problem in the framework “Sear ch-based Face Annotation” (SBFA), which mines massive web facial images that are freely a vail ble on the world wide web (WWW) by exploring content-based image retrieval (CBIR ) techniques. Due to the noisy nature of web images, the raw labels of web facial images are o ften noisy, in which some facial images are associated with incorrect/incomplete name s. Such kind of raw facial image database is referred to as “weakly labeled web facial image d tabase” in this thesis. This thesis investigates a comprehensive framework of the “ Search-based Face Annotation” paradigm, which generally consists of three main stages: fir stly, given a query facial image, it typically involves a pre-processing stage, including face detection, face alignment, and facial feature extraction; subsequently, the input facial image i s aligned into a consistent position and represented as a feature vector in the facial feature space; se ond, it retrieves the top-ranking similar instances of the query facial image from a large-sca le weakly labeled web facial image database by using content-based image retrieval technique s; and thirdly, it names the query image by mining the top-ranking similar images and the corre sponding weak label information. In this thesis, we mainly focus on tackling three open challe nging tasks: (i) how to effectively enhance the initial weak name labels, (ii) how to effectivel y xploit a short list of candidate facial images and their weak label information for the face n ame annotation task, and (iii) how to effectively boost the annotation performance of SBFA pro blem by adopting sophisticated machine learning techniques in a unified framework. i This thesis investigates a family of effective and efficient algorithms to tackle the aforementioned challenges. In particular, the main contributio ns f this thesis are listed as follows: • To enhance the initial weak label information, we propose an Unsupervised Label Refinement (ULR) approach by adopting graph-based learning te chniques. An effective optimization algorithm is proposed to solve the large-scale l earning task and a Clusteringbased Approximation (CBA) algorithm is proposed to improve th system scalability. • To fully mine the top-ranking similar images and the corresp onding weak label information, we propose a Weak Label Regularized Local Coordinat e Coding (WLRLCC) algorithm, which exploits the principle of local coordinat e coding to learn more discriminative facial features for efficient label propagation in a sp rse reconstruction scheme. • By combining two different learning schemes: “transductiv e learning” and “inductive learning” in the same framework based on the entropy informa tion, we propose a “Unified Learning Framework for Auto Face Annotation” (UTIL), wh ic efficiently improve the annotation performance without introduce much trainin g effort. • To go beyond the limitation of single facial feature represe ntation and general Euclidean distance space, we propose a multi-model based “Learn to Nam e Faces” (L2NF) algorithm, which optimize the fusion of multiple annotation mod alities in a learning to rank framework. • Based on search-based face annotation framework, we built a real world demonstration system: FANS, which is available at h tp://msm.cais. ntu.edu. sg/fans . It can be used to predict the celebrity name, or find the most simi lar celebrity image of the query images. In addition to the website interface, we also d eploy an Android APP for the mobile devices, which popularizes the usage of our syste m. This thesis has constructed an extensive set of experiments by evaluating the performance of the proposed techniques in comparison to the state-of-th e-art methods on large-scale realworld facial image databases. Encouraging results illustr ate that the proposed techniques significantly boost the state-of-the-art annotation performa nce.