# A Multi-Modal Person Recognition System for Social Robots
## 3 Architecture of the Person Recognition System
### Temporal Binding via Spiking Neural Networks  
It is known that humans interact with their environment by processing the available information through multisensory modality streams over time with fading memory property. The same process is emulated here. In the context of this algorithm, fading memory implies that the effect of stimuli excitation (represented by IOs) deteriorates moderately if it is not reinforced or refreshed. We implement this feature through the Leaky Integrate-and-Fire neuron (LIF) model [52] to manifest the integration of IOs that are generated in the previous stage in the hierarchical architecture of the proposed system. Figure 8 depicts one block of the spiking neural network (SNN) that is used to perform the integration process. The overall SNN that is used to integrate the information from various biometric modalities is constructed by laterally connecting \(N\) blocks from the circuit, as shown in Figure 8, where \(N\) represents number of subjects in gallery sets. The IOs vectors are fed to LIF neurons in SNN by means of pre-synaptic input spikes, as shown in Figure 8. IOs vectors, which are generated based on different feature vectors, are fed to independent branch in the dendritic tree. On the other hand, IOs vectors that are generated based on same feature vectors are fed to same branch in the dendritic tree. Inspired by neuroscience research studies [53; 54], we suggest that the effect of presynaptic inputs on postsynaptic potential is either sublinear, super linear, or linear. The effects sum sub-linearly, linearly, or super-linearly if they are delivered to the same dendritic branch (within-branch) and sum linearly if they are delivered to different dendritic branches (between-branch). Equation (18) describes the dynamic of postsynaptic potential of LIF neuron. The dynamic of this neuron can be described as follows: initially at time \(t=0\), \(Vm\) is set to _Vinit_. If \(Vm\) exceeds the threshold voltage _Vthresh_, then it fires a spike and it is reset to _Vreset_ and held there for the length _Trefact_ of the absolute refractory period. The total response of postsynaptic potential due to different presynaptic inputs within-branch (\(Syn_{WB}\)) and between-branch (\(Syn_{BB}\)) is computed using (18) to (20). \[\tau_{m}\frac{dV_{m}}{dt}=-(V_{m}-V_{resting})+R_{m}\cdot(I_{syn}(t)+I_{ noise}) \tag{18}\]  
Figure 8: Spiking neural network (SNN) circuit and it dendritic structure that are used as a block to construct the overall SNN. LIF: leaky integrate-and-fire. where \(\tau_{m}=C_{m}\cdot R_{m}\) is the membrane time constant, \(R_{m}\) is the membrane resistance, \(I_{syn}(t)\) is the current supplied by the synapses, \(I_{noise}\) is a Gaussian random variable with zero mean and a given variance noise, \(Vm\) is the membrane potential of LIF neuron, \(Vinit\) is the initial condition for \(Vm\) at time \(t=0\), and \(Vthresh\) is the threshold value. If \(Vm\) exceeds \(Vthresh\), then a spike is emitted, \(Vreset\) is the voltage to reset \(Vm\) to after a spike, and \(V_{resting}\) is the membrane potential of LIF neuron at no activity. \[Syn_{WB}=\sum_{i=1}^{K}\alpha_{i}\;Sigmoid(IO_{i}) \tag{19}\]  
\[Syn_{BB}=\sum_{i=1}^{K}\alpha_{i}\;IO_{i} \tag{20}\]  
where \(IO_{i}\) represents the total input to the \(i\)th dendritic branch, \(\alpha_{i}\) is the \(i\)th dendritic branch weight, \(K\) is the number of dendritic branches. Note that the sigmoid function is one possible choice of synaptic integration function within-branch and can be replaced with other functions, such as hyperbolic tangent sigmoid function. It can be noted from (19) and (20) that the balanced IOs that are delivered to same dendritic branch will sum as follows: (1) small IOs will sum nearly linearly, (2) around average IOs will sum super-linearly, (3) large IOs will sum sub-linearly. Unbalanced IOs fed to the same branch generate near-linear summation over the entire range of IOs intensities.