# Dynamic Gesture Recognition System with Gesture Spotting Based on Self-Organizing Maps
## 3 Gesture Recognition System
### Sequence Vector Generator  
The SOM-1 in the sequence vector generator quantizes the input vectors, and the quantization results are sequentially stored in the shift registers. The contents of the shift registers form the sequence vector, which represents temporal change of the input posture, and is fed to the next SOM-Hebb classifier. The SOM-1 includes \(M_{1}\times M_{1}\) neurons, and \(D\)-dimensional vector \(\overrightarrow{m_{j}}\) that is called a weight vector is included in each neuron. \[\overrightarrow{m_{j}}=\bigl{\{}\mu_{j0},\mu_{j1},\cdots,\mu_{jD-1}\bigr{\}} \in\Re^{D}, \tag{10}\]  
where, \(j\) is the neuron number. Operation of the SOM is divided into learning and recall phases. The weight vectors of the neurons are trained with a set of input vectors in the learning phase. The learning phase is made of a winner search and weight update. During the recall phase, only the winner search is carried out by using the map of the trained weight vectors. The winner neuron has the weight vector that is the nearest to the input vector. Euclidean distance \(V_{1}(j)\) between the input vector and weight vector of neuron-\(j\), is calculated for the winner search. \[V_{1}(j)=\sqrt{\sum\limits_{i=0}^{D-1}\left(\mu_{ji}-\lambda_{i}\right)^{2}} \tag{11}\]  
The winner neuron-\(c\) is then determined. \[c=\arg\min_{j}V_{1}(j) \tag{12}\]  
In the weight update, weight vectors of the winner and its neighborhood neurons are updated to be closer to the input vector as;  
\[\overrightarrow{m_{j}}(t+1)=\overrightarrow{m_{j}}(t)+h(c,j,t)\cdot\left[ \overset{\rightarrow}{G}(t)-\overrightarrow{m_{j}}(t)\right], \tag{13}\]  
where \(t\) is time index, and \(h(c,j,t)\) is a function called neighborhood function, which is defined as;  
\[h(c,j,t)=\alpha(t)\cdot\exp\left(-\frac{\parallel\overrightarrow{r}_{c}- \overrightarrow{r}_{j}\parallel}{2\sigma(t)^{2}}\right), \tag{14}\]  
where \(\alpha(t)\) is a learning coefficient, (\(0<\alpha(t)<1\)). The \(\overrightarrow{r_{c}}\) and \(\overrightarrow{r_{j}}\) are the coordinate vectors of the winner neuron-\(c\), and a neuron-\(j\), respectively. The \(\sigma(t)\) represents the neighborhood radius, and the weight vectors within the radius from the winner neuron are updated. After the learning phase, all weight vectors are kept unchanged and the weight map is used in the recall phase. The winner neuron represents the cluster to which the input vector belongs, and the coordinates \((w_{X},w_{Y})\) of the winner neuron for the input vector are treated as the quantization result. These coordinates are stored sequentially in the shift registers, so their contents represent the sequence of the input video frames. In this paper, this vector is called the sequence vector, \(\overset{\rightarrow}{X}\), which is a \(2F\)-dimensional vector. Its vector element \(\xi_{m}\) is defined as:  
\[\overset{\rightarrow}{X}=\{\xi_{0},\xi_{1},\cdots,\xi_{2F-1}\} \tag{15}\]  
\[\xi_{m} = \left\{ \begin{array}{ll}w_{X}(m)&(0\leq m<F)\\ w_{Y}(m-F)&(F\leq m<2F).\end{array} \right. \tag{16}\]  
Figure 3 shows examples explaining operation of the system. As shown in Figure 3A, a gesture is made of 10 posture images in different video frames. In the example, SOM-1 is composed of \(8\times 8\) neurons and Figure 3B shows the transition of the winner neuron with respect to the input video frames. Posture in the first video frame (\(f=0\)) makes a neuron at \((w_{X},w_{Y})=(0,0)\) the winner. Then the coordinates of the winner neuron are stored in the registers in the sequence vector generator as shown in Figure 3C. Question marks in Figure 3C are the coordinates of the winner neurons of the previous gesture, which are not related to the current gesture. The winner for the second posture (\(f=1\)) is a neuron at \((4,2)\). The registers are shifted to the right and the new winner coordinates \((4,2)\) is stored into the registers' most left position. For the third posture at \(f=2\), \(w_{X}=6\) and \(w_{Y}=6\) are loaded into the registers.