# Skeleton-Based Dynamic Hand Gesture Recognition Using an Enhanced Network with One-Shot Learning
## 3 Dynamic Hand Gesture Recognition with the GREN Network
```
Input: Given N samples \(\{X_{1},X_{2},\ldots,X_{N}\}\) belonging to C classes with Sample-classes \(y_{t}\in Y=\{1,\ldots,C_{1},\;for\;t=1,\ldots,N\); Output: A softmax layer for class prediction;
1 Initialization:
2\(prev\_state\leftarrow\)\(init\_state(N)\){
3\(c_{0}\leftarrow\)\(C_{lstm}(N)\);
4\(r_{0}\gets 0_{N\times({\it head\_num\_memory\_size})}\);
5\(w_{0}^{\prime}\leftarrow\)\(one\_hot\_weight\_vector(N,{\it head\_num\_memory\_slots})\);
6\(w_{0}^{lu}\leftarrow\)\(one\_hot\_weight\_vector(N,{\it memory\_slots})\);
7\(M_{0}\leftarrow\)\(\epsilon_{N\_memory\_slotsotsome\_size}\);
8 return\(\{c_{0},r_{0},w_{0}^{\prime},w_{0}^{\prime},M_{0}\}\);
9];
10\(o=[\;]\);
11 for\(t\gets 1\)to\(N\)do
12\(h_{t},c_{t}\gets C_{lstm}((X_{t},y_{t}),{\it pre\_state})\);
13 for\(i\gets 0\)to\(X_{t}\).lengthdo
14\(output,curr\_state\gets\)\(gen((X_{t}(i),x\_table_{t}(i)),prev\_state)\){
15 Memory Retrieval:
16\(K(k_{t},M_{t}(i))\leftarrow\)\(cosine\_similarity(k_{t},M_{t}(i))\);
17\(w_{t}^{\prime}(i)\gets softmax(K(k_{t},M_{t}(i)))\);
18\(r_{t+}=w_{t}^{\prime}(i)\cdot M_{t}(i)\);
19 Memory Encoding (LRUA):
20\(w_{t}^{lu}\leftarrow\)\(\gamma\cdot w_{t-1}^{lu}+w_{t}^{lu}+w_{t}^{lu}\);
21 if\(w_{t}^{lu}\leq\)\(m\)\(w_{t}^{lu},n\)then\(w_{t}^{lu}(i)=1\)else\(w_{t}^{lu}(i)=0\);
22\(w_{t}^{lu}\leftarrow\)\(sigmoid(\cdot)w_{t-1}^{lu}+(1-sigmoid(\cdot))\cdot w_{t-1}^{lu}\);
23\(M_{t}(i)\gets M_{t-1}(i)+w_{t}^{lu}(i)\cdot k_{t}\);
24 return\(\{h_{t},r_{t}\}\),\(\{c_{t},r_{t},w_{t}^{lu},M_{t}\}\);
25\(\}\);
26\(prev\_state=curr\_state\);
27 if\(i==0\)then
28\(o2\_w\leftarrow(output\_length,M_{\rm class}),rand\_unit\_init(\_minv,{\it max })\);
29\(o2\_b\leftarrow(M_{\rm class}),rand\_unit\_init(\_minv,{\it max})\);
30 endif;
31\(output=output\)-\(o2\_w+o2\_b\);
32\(output=softmax(output)\);
33\(oappend(output)\);
34\(\mathit{append}(output)\);
35\(\mathit{\it end}\)
36\(\mathit{learning\_loss}=-cross\_entropy\_cost(y_{t},o)\);
37\(\mathit{optimizer}=AdamOptimizer(learning\_rate)\);
38\(\mathit{train\_op}=\mathit{optimizer.minimize}(learning\_loss)\);
39
40 end
```  
**Algorithm 1**GREN  
In the algorithm, the \(one\_hot\_weigh\_vector(a,b,c)\) function generates a tensor of shape \(a\times b\times c\) with \([:,:,0]\) set to _one_ (or \([:,0]\), if the \(one\_hot\_weigh\_vector(a,b)\) function generates a tensor of shape \(a\times b\)); \(\{(a,b),rand\_unif\_init(\mathit{\it minv},{\it max})\}\) generates a tensor of shape \(a\times b\) (or\(\{(a),rand\_unif\_init(minv,maxv)\}\) generates a tensor of shape \(a\times 1\)) with a uniform distribution, and the value of all elements is set between \(minv\) and \(maxv\). In general, for the current time-step \(t\), the sample data \(X_{t}\) and the corresponding sample-class \(y_{t}\) will be received by the controller \(C_{lstm}\). The current state of the GREN network \(curr\_state\) is used by the controller as an additional input for the next time-step. According to each sequence of the sample, the GREN algorithm randomly generates the class label \(x\_label_{t}\). If the sample date \(X_{t}\) comes from a never-before-seen class, it will be bound to the appropriate sample-class \(y_{t}\) and stored by the write heads in the external memory, which is presented in the subsequent time-step (see Figure 1). Later, once a sample from an already-seen class is presented, the controller will retrieve the bound sample-class information by the read heads from the external memory for class prediction. A softmax layer, \(softmax(\cdot)\), is selected to output the standardized probability distribution of the model prediction, and combined with the cross-entropy cost function, \(cross\_entropy\_cost(\cdot)\), to measure the loss between the predicted value and correct class label. Then, the adaptive moment estimation (Adam) [43], \(AdamOptimizer(\cdot)\), is adopted to minimize the loss, and the back-propagated error signal from the current prediction updates those previous weights, which is followed by the updating of the external memory. Those processes would be repeated until the model converges.