Largescale Multimodal Person Identification in Real Unconstrained Environments Person identification (PID) under real unconstrained noisy environments is a huge challenge. In multiplefeature learning with Deep Convolutional Neural Networks (DCNNs) or Machine Learning method for largescale person identification in the wild, the key is to design an appropriate strategy for decision layer fusion or feature layer fusion which can enhance discriminative power. It is necessary to extract different types of valid features and establish a reasonable framework to fuse different types of information. In traditional methods, different persons are identified based on single modal features to identify, such as face feature, audio feature, and head feature. These traditional methods cannot realize a highly accurate level of person identification in real unconstrained environments. The study aims to propose a fusion module to fuse multimodal features for person identification in real unconstrained environments.

Deep Multibiometric Fusion for AudioVisual User ReIdentification and Verification From border controls to personal devices, from online exam proctoring to humanrobot interaction, biometric technologies are empowering individuals and organizations with convenient and secure authentication and identification services. However, most biometric systems leverage only a single modality, and may face challenges related to acquisition distance, environmental conditions, data quality, and computational resources. Combining evidence from multiple sources at a certain level (e.g., sensor, feature, score, or decision) of the recognition pipeline may mitigate some limitations of the common unibiometric systems. Such a fusion has been rarely investigated at intermediate level, i.e., when unibiometric model parameters are jointly optimized during training. In this chapter, we propose a multibiometric model training strategy that digests face and voice traits in parallel, and we explore how it helps to improve recognition performance in reidentification and verification scenarios. To this end, we design a neural architecture for jointly embedding face and voice data, and we experiment with several training losses and audiovisual datasets. The idea is to exploit the relation between voice characteristics and facial morphology, so that face and voice unibiometric models help each other to recognize people when trained jointly. Extensive experiments on four realworld datasets show that the biometric feature representation of a unibiometric model jointly trained performs better than the one computed by the same unibiometric model trained alone. Moreover, the recognition results are further improved by embedding face and voice data into a single shared representation of the two modalities. The proposed fusion strategy generalizes well on unseen and unheard users, and should be considered as a feasible solution that improves model performance. We expect that this chapter will support the biometric community to shape the research on deep audiovisual fusion in realworld contexts.