# Multimodal Biometric Template Protection Based on a
## 1 Introduction
### Motivations and Contributions  
In this paper, we propose an end-to-end CSMoFN scheme for multimodal biometric systems. Our network fuses face and periocular biometric traits at the feature level, producing a single CB template as output. More specifically, CSMoFN is composed of two components, the first of which transforms the fused biometric vectors based on the notion of random permutation maxout transform (RPMoT), which was proposed in [27]. Although RPMoT is a CB transformation scheme, it is not learnable and is data-agnostic, and hence barely meets the performance requirements for the CB scheme. RPMoT is reformulated as a part of the deep neural network and is referred to here as the permutation SoftmaxOut transform (PSMoT). PSMoT is data-driven, learnable, and parameterized by user-specific permutation seeds to satisfy the revocability and unlinkability criteria. PSMoT can be viewed as a locality-sensitive hashing process that transforms biometric data from a high-dimensional input space to a relatively low-dimensional hash space [28]. The PSMoT comes with a customized layer called permutation SoftmaxOut layer. The layer composed of maxout units and a modified softmax function to approximate the permutation and the winner-takes-all operations in the RPMoT. Apart from that, the modified softmax function is also useful to minimize the quantization errors introduced by the SoftmaxOut approximation. Since PSMoT aims to produce a discrete hash vector from the network directly, it is a representation learning problem. Hence, a pairwise distance-based loss called Pairwise Angular (PA) loss, is introduced to optimize the margin between intra- and inter-class distances. The output of PSMoT (i.e., a hash vector) is immediately followed by the second component of the network, called the multiplication-diagonal compression (MDC) module. The MDC module is designed to further enhance the security and compress the PSMoT hash vector, and offers a user-specific seeded binary random projection mechanism as a means to enhance the revocability and unlinkability of the proposed method. Both the PSMoT and MDC transformation follow the many-to-one mapping principle attributed to their hashing trait, meaning that inversion (one-to-many mapping) of the terminal hash output is theoretically impossible and computationally hard in practice. This is essential to satisfy the non-invertibility requirement for a CB scheme. In this paper, we opt to fuse the face and periocular features. The periocular region, also known as the periphery of the ocular area, includes the vicinity of a person's eyes and contains information on the subject's eyebrows, eyelashes, and skin texture. The periocular region is a complementary biometric of the face that is helpful in terms of enhancing the biometric performance of the face alone. It is particularly useful in situations such as when a mask is worn, where the face is occluded and a performance degradation is expected. As we will show in the experiment section, fusion of the face and periocular region outperforms the respective unimodal counterparts, i.e., the face or periocular region alone. We can summarize our contributions as follows:  
* A deep-learning-based CB scheme for multimodal biometrics is proposed. Although the face and periocular biometrics form the focus of this paper, our proposed method can also be applied to other biometrics modalities, provided the input is a raw image. * A deep network, CSMoFN is composed of three modules: a feature extraction and fusion module, a PSMoT module, and an MDC module is proposed to realize the above proposal. The first module is responsible for performing feature extraction and fusion and the latter two are cancelable transformation functions, which are devised with respect to the four CB design criteria. * The three modules are trained in an end-to-end manner with a combination of classification loss and representation learning, namely ArcFace loss and PA loss.