## 3 Structure of the Proposed Biomodal Multitask Network
### Network Training  
The total population we considered for our experiments comprised 51 individuals, which were divided into 2 sets: 45 target clients and 6 impostors. Consequently, the biometric output layer of the network is composed of 45 fully connected neurons, as described in Table 2. After we applied the data augmentation to the voice and image datasets, we split the target clients' data as follows: 65% for training, 5% for validation, and 30% for testing. Each target client had 100 voice samples and 150 face images. The impostors' data was used exclusively to test the text-prompted verification task with each impostor made up of 100 voice samples and 150 face images. Keras libraries with Tensorflow backend was used to train and test the network. We used the Adam optimizer with a 0.0001 learning rate and a training process with 1000 epochs. Data batches were created by randomly pairing MFCC features extracted from sequences of spoken digits with face images at each epoch, using a Python generator to ensure large variability. Training and validation batches were composed of 16 and 8 pairs, respectively. We configured a Keras callback that saved the parameter values only if they decreased the validation loss value in each epoch. Since our network is designed to perform two different tasks simultaneously, we defined a joint loss function that allows the network to learn each task. This function was defined as the sum of the Sparse Categorical Crossentropy (SCC) and Connectionist Temporal Classification (CTC) loss functions. SCC is a commonly used loss function in supervised multi-class classification [30]; it helps to reduce the error of the identification task during the training stage. CTC loss function was proposed to solve temporal classification problems, it allows to reduce the error when labelling sequences where the alignment between the input sequences and the target labels is unknown [57]. CTC aimed to reduce the error of the speech recognition task. The joint loss function has two parameters that scale the values of SCC and CTC. This scaling results in mutual regularization and may accelerate the convergence of the network [10]. The joint loss function is shown in Equations (2) and (3). \[L_{joint}=\alpha_{1}L_{SCC}+\alpha_{2}L_{CTC} \tag{2}\]\[ \begin{split} L_{joint}=\alpha_{1}\Bigg{(}-\sum_{i=1}^{C}y_{i}log(P(y_{i }=i|x_{FSi}))\Bigg{)}+\\ \alpha_{2}\Bigg{(}-\sum_{\pi\in\mathcal{B}(x_{5},y)}P(\pi|x_{5}) \Bigg{)}\end{split}  \tag{3}\]  
The first term corresponds to SCC function where \(y_{i}\) represents the true label assigned to target client \(i\), \(C\) is the total number of target clients in our population, and \(P(y_{i}=i|x_{FSi})\) is the normalized probability distribution predicted with the softmax classifier of the biometric output given the bimodal input data \(x_{FS}\) belonging to target client \(i\). In CTC loss, \(\pi\) represents all possible character sequences given an input of MFCC features \(x_{S}\), \(B\) is the search space of our speech recognition task, and \(y\) is the target sequence. Equation (4) describes the conditional probability \(P(\pi|x_{S})\) that a character \(l_{t}\) is predicted in the label sequence \(y\) at time \(t\) given the MFCC features \(x_{S}\), across all time steps. \[P(\pi|x_{S})=\prod_{t=1}^{T}P(l_{t},t|x_{S}) \tag{4}\]  
The probability \(P(l_{t},t|x_{S})\) is calculated for each time step with the softmax function of the speech recognition output. The CTC loss function maps MFCC speech features to character sequences following the procedure proposed in Ref. [58]. In order to produce a refined transcription from the CTC characters sequence we employed the greedy decoder implemented in Keras. A linguistic model to decode the character sequences was not required since the vocabulary of BIOMEX-DB comprises random digit pronuncations [59]. [PERSON] et al. [11] proposed an audiovisual approach for speech recognition and speech detection tasks.