## 6 Integration of GEFA and GEFA
### Decision Rule  
In Figure 6, the process to determine the main user emotion during the CA can be observed. It has been considered that the visual emotion detection system does not work properly when the user is talking. Therefore, when the user is quiet, just the output of the GEFA (visual) module is considered, and in the other case, only the GEVA module, based on the user voice, is taken into account. Once the vocal activity finishes, the CA also ends. Considering this temporal constraint, it is necessary to define a decision rule to combine the information given by each module to generate a single output. The final definition of this rule depends on the success rate (accuracy) of each individual module in relation to each emotion. Hence, the decision rule is settled after the results of the experiments of each module, GEVA and GEVA, are obtained.  
Once the success rate of each module (with two classifiers each) is estimated, then the decision rule is defined. In order to do that, it is necessary to determine the confidence degree of each one of the outputs (the detected emotion) of the classifiers.  
In this work, the _confidence degree_ of each output is calculated using the Bayes theorem and the confusion matrices. The Bayes theorem is applied as follows:Let us assume that the current state of the system is \(S\) and that the list of states is finite: \(\mathbf{S}=\{s_{1},\ldots,s_{n}\}\) where \(n\in\mathbb{R}\).  
Then, let us suppose that there is a classifier, \(C\), the output of which is \(S_{C}\in\mathbf{S}\), which is used to estimate \(S\in\mathbf{S}\).  
The _confusion matrix_, \(M\in\mathbb{N}^{n\times n}\), of the classifier, \(C\), is defined as follows:  
\[\forall i,j\in[1,n],M_{ij}=p(S_{C}=s_{j}|S=s_{i}) \tag{1}\]  
where \(s_{C}\) is the state estimated by the classifier, \(C\).  
As such, the value in row \(i\) and column \(j\) corresponds to the probability that the classifier estimates the state \(S_{C}=s_{j}\) given the real state \(S=s_{i}\).  
Therefore, an ideal classifier would have a diagonal confusion matrix, so \(M_{ij}\
eq 0\) if \(i=j\) and \(M_{ij}=0\), if \(i\
eq j\). That is, the detected states are always equal to the real ones.  
In order to determine the confidence degree of the classifier, \(C\), we need to estimate the probability of the system being in a real state, \(s_{i}\), when the classifier output is \(s_{j}\). In other words, we want to compute \(p(S=s_{i}|S_{C}=s_{j})\).  
\[p(S=s_{i}|S_{C}=s_{j}) = \tag{2}\] \[= \frac{p(S=s_{i})p(S_{C}=s_{j}|S=s_{i})}{\sum_{k\in[1,n]}p(S_{C}=s _{j}\cap S=s_{k})}\] (3) \[= \frac{p(S=s_{i})p(S_{C}=s_{j}|S=s_{i})}{\sum_{k\in[1,n]}p(S_{C}=s _{j}|S=s_{k})p(S=s_{k})}\] (4) \[= \frac{p(S=s_{i})M_{ij}}{\sum_{k\in[1,n]}M_{kj}p(S=s_{k})} \tag{5}\]  
Figure 6: Scheme of the process for determining the main user emotion in each communicative act (CA).  
In the case where all states are equally probable, the probability of \(s_{i}\) given \(s_{j}\) is:  
\[p(S=s_{i}|S_{C}=s_{j})=\frac{M_{ij}}{\sum_{k\in[1,n]}M_{kj}} \tag{6}\]  
Let us now suppose we have a collection of \(m\) classifiers \(\mathbf{C}=C_{1},\ldots,C_{m},m\in\mathbb{R}\).  
We want to estimate \(S\), the current state of the system. We know the output of each classifier, \(S_{C}\in\mathbf{S},\forall C\in\mathbf{C}\).  
As seen, for each possible state of the system, \(s_{i}\in\mathbf{S}\), and for each classifier, \(C\in\mathbf{C}\), we can estimate \(p(S=s_{i}|S_{C})\).  
**Decision rule:** knowing all these conditional probabilities, an easy-to-estimate state is the state, \(\hat{s}\), that has the highest probability among all classifiers:  
\[\hat{s}\in\mathbf{S},\hat{C}\in\mathbf{C}\mid\forall s_{i}\in\mathbf{S}, \forall C\in\mathbf{C}\,\ p(S=s_{i}|S_{C})\leq p(S=\hat{s}|S_{\hat{C}}) \tag{7}\]