# A Multi-Modal Person Recognition System for Social Robots
## 3 Architecture of the Person Recognition System
### Dedicated Processing Units and Generation of the Intermediate Outputs
#### 3.3.2 Dedicated Processing Units for Voice-Based Feature Vector  
For the speech-based feature vector, MFCCs (mel-frequency cepstral coefficients) are used as, an input to K-NN classifier with the aforementioned distance measures. Also, we used MFCCs that extracted from speech data of all of the speakers in the training data (gallery set) to create speaker-independent world model or a well-known universal background model (UBM). The UBM is estimated by training M-component GMM with the popular expectation-maximization (EM) algorithm [51]. The UBM represents speaker-independent distribution of the feature vectors. Here, we use 32-compnenent GMM to build the UBM. The UBM is represented by a GMM with 32-compnents, as denoted by \(\lambda_{UBM}\), that characterized by its probability density function as (14). \[p(\overrightarrow{x}|\lambda)=\sum_{i=1}^{M}w_{i}p_{i}(\overrightarrow{x}) \tag{14}\]The model is estimated by the weighted linear combination of D-variate Gaussian density function \(p_{i}(\overrightarrow{x})\), each parameterized by a mean \(D\times 1\) vector, \(\mu_{i}\), mixing weights, which is constrained by \(w_{i}\geq 0\), \(\sum\limits_{i=1}^{M}w_{i}=1\), and a \(D\times D\) covariance matrix, \(\Sigma_{i}\) as (15). \[p_{i}(\overrightarrow{x})=\frac{1}{2\pi^{D/2}|\Sigma_{i}|^{1/2}}exp\{\frac{1} {2}(x-\mu_{i})^{\prime}(\Sigma_{i}{}^{-1})(x-\mu_{i})\} \tag{15}\]  
The purpose of training the UBM is to estimate the parameters of 32-component GMM, \(\lambda_{UBM}=\{w_{i}\mu_{i},\Sigma_{i}\}_{i=1}^{M}\), from the training samples. The next step is to estimate specific GMM from UBM-GMM for each speaker in the gallery set using maximum a posteriori (MAP) estimation. The key difference between estimating the parameters of UBM and estimating the specific GMM parameters for each speaker is that the UBM uses standard iterative expectation-maximization (EM) algorithm for parameter estimation. On the other hand, specific GMM parameters are estimated by adapting the well-trained parameters in the UBM to fit a specific speaker model. Since the UBM represents speaker-independent distribution of the feature vectors, the adaptation approach facilitates the fast scoring, as there is a strong coupling between speaker's model and the UBM. It should be noted that all or some of the GMM's parameters (\(\lambda_{UBM}=\{w,\mu,\Sigma\}\) can be adapted by MAP. Here, we adapted only the mean \(\mu\) to represent specific speaker's model. Now, Let us assume a group of speakers \(s=1,2,3,\dots,\ S\) represented by GMMs \(\lambda_{s}=\lambda_{1},\lambda_{2},\lambda_{3},\dots,\lambda_{S}\). The goal is to find the speaker identity \(\hat{s}\) whose model has the maximum a posteriori probability for a given observation \(X_{k}=\{x_{1},\dots,x_{T}\}\) (MFCCs feature vector). We calculate the posteriori probability of all of the observations \(X_{k}=X_{1},X_{2},X_{3},\dots,X_{K}\) in probe set against all of the speakers models \(\lambda_{s}=\lambda_{1},\lambda_{2},\lambda_{3},\dots,\lambda_{S}\) in gallery set as (16). As \(s\) and \(k\) vary from 1 to number of speakers in the gallery set and the number of utterances in probe set, respectively, the result from (16) is \(S\times K\) matrix, namely \(IO\_FV_{voice\_based}\). This matrix represents the IOs that are generated from speech-based feature vector and it will be integrated with other matrices that represent IOs generated from vision-based feature vectors. \[IO\_FV_{voice\_based}|_{\{s,k\}}=\left.P_{r}(\lambda_{s}|X_{k})=\frac{p(X_{k} |\lambda_{s})}{p(X_{k})}P_{r}(\lambda_{s})\right| \begin{array}{l}1\leq s \leq S\\ 1\leq k\leq K\end{array}  \tag{16}\]  
Assuming equal prior probabilities of all the speakers, the terms \(P_{r}(\lambda_{s})\) and \(p(X_{k})\) are constant for all speaker, thus both terms can be ignored in (16). Since each subject in the probe set is represented as \(X_{k}=\{x_{1},\dots,x_{T}\}\), thus by using logarithmic and assume independence between observations, calculation of \(IO\_FV_{voice\_based}|_{\{s,k\}}\) can be simplified as (17).