Deep Multibiometric Fusion for AudioVisual User ReIdentification and Verification From border controls to personal devices, from online exam proctoring to humanrobot interaction, biometric technologies are empowering individuals and organizations with convenient and secure authentication and identification services. However, most biometric systems leverage only a single modality, and may face challenges related to acquisition distance, environmental conditions, data quality, and computational resources. Combining evidence from multiple sources at a certain level (e.g., sensor, feature, score, or decision) of the recognition pipeline may mitigate some limitations of the common unibiometric systems. Such a fusion has been rarely investigated at intermediate level, i.e., when unibiometric model parameters are jointly optimized during training. In this chapter, we propose a multibiometric model training strategy that digests face and voice traits in parallel, and we explore how it helps to improve recognition performance in reidentification and verification scenarios. To this end, we design a neural architecture for jointly embedding face and voice data, and we experiment with several training losses and audiovisual datasets. The idea is to exploit the relation between voice characteristics and facial morphology, so that face and voice unibiometric models help each other to recognize people when trained jointly. Extensive experiments on four realworld datasets show that the biometric feature representation of a unibiometric model jointly trained performs better than the one computed by the same unibiometric model trained alone. Moreover, the recognition results are further improved by embedding face and voice data into a single shared representation of the two modalities. The proposed fusion strategy generalizes well on unseen and unheard users, and should be considered as a feasible solution that improves model performance. We expect that this chapter will support the biometric community to shape the research on deep audiovisual fusion in realworld contexts.

Multibiometrics User Recognition using Adaptive Cohort Ranking Personal identification using multibiometrics is desirable in a wide range of highsecurity andor forensic application as it can address performance limitations from unimodal biometrics systems. This paper presents a new scheme the multibiometrics fusion to achieve performance improvement for the user identificationrecognition. We model the biometric identification solution using an adaptive cohort ranking approach, which can more effectively utilize the cohort information for maximizing the true positive identification rates. In contrast to the tradition cohortbased methods, the proposed cohort ranking approach offers merit of being matcher independence as it does not make any assumption on the nature of score distributions from any of the biometric matcher(s). In addition, our scheme is adaptive and can be incorporated for any biometric matchertechnologies. The proposed approach is evaluated on publicly available unimodal and multimodal biometrics databases, i.e., BSSR1 multimodal matching scores for fingerprint and face matchers and XM2VTS matching scores from synchronize databases of face and voice. In both the unimodal and multimodal databases, our results indicate that the proposed approach can outperform the conventional adaptive identification approaches. The experimental results from both public databases are quite promising and validate the contributions from this work.