# Hand Gesture Recognition via Lightweight VGG16 and Ensemble Classifier
## 3 Hand Gesture Recognition Technique
### Lightweight VGG16 as Feature Extractor  
While a network such as ResNet and its variant improves gradient flow and feature propagation through the summation of an identity function, a recent variant of ResNet [15] shows that a huge number of convolutional layers contribute very little to the results. Needless to say, this induces a huge amount of trainable parameters. On the other hand, DenseNet emphasizes feature reuse via dense connectivity to mitigate the problem and improve parameter efficiency. In the meantime, dense connectivity further improves the gradient flow and feature propagation. Figure 2 illustrates the proposed lightweight VGG16 network architecture for feature extraction. Figure 3 shows a comparison of the original VGG16 model and the proposed lightweight VGG16 model. The architecture consists of an input layer, four convolution blocks and a single batch normalization layer. The first two convolution blocks consist of two convolution layers and a max pooling layer, whereas the third and fourth convolution blocks have three convolution layers and one max pooling layer. The number of channels of the image increases continuously after passing through each of the convolution blocks while the width and height are halved. The channel first increases from 3 to 64 and then 128, 256 and 512 channels, respectively. As the number of channels increases, the size of the width and height reduces by half continuously as the image goes through the series of max pooling layers. The max pooling operation can be denoted as taking the input channel's width, \(N_{w}\), and height, \(N_{h}\), and partitioning it into adjacent pixels of \(2\times 2\) size which results in the output image of \(N_{w}/2\) and \(N_{h}/2\). The proposed VGG16 architecture was slightly modified compared to the original VGG16 structure where the 5 th convolution block was removed from the pre-trained VGG16 model to ease the memory load as the image size was increased from 32 \(\times\) 32 \(\times\) 3 to 48 \(\times\) 48 \(\times\) 3 at the input layer. Weights for the VGG16 layers were adopted from the pre-trained VGG16 model on ImageNet which consists of millions of images instead of training the model from scratch. This allows us to utilize much more optimized weights for feature extraction. Figure 4 provides an illustration of the feature extracted from different convolution layers. It can be seen that, as the layers go deeper, more specific features are extracted from the image. However, a good architecture needs to be defined to ensure an appropriate learning capacity without overfitting. A layer of batch normalization is added after the 4 th convolution block to normalize the output of the layers and prevent the model overfitting. The batch normalization process can be applied to any layer of the neural network with the main purpose of having a stable activation value distribution, which will reduce the internal covariate shift and suppress the over-fitting problem [16]. With the \(d\)-dimensional input, \(x=(x^{(1)}\dots x^{(d)})\), each dimension is normalized by  
\[\hat{x}^{(k)}=\frac{x^{(k)}-\text{E}\Big{[}x^{(k)}\Big{]}}{\sqrt{\text{Var} \big{[}x^{(k)}\big{]}}} \tag{1}\]  
where \(x^{(k)}\) represents each particular activation while \(E\Big{[}x^{(k)}\Big{]}=\frac{1}{m}\sum_{i=1}^{m}x_{i}^{(k)}\), \(\text{Var}\Big{[}x^{(k)}\Big{]}=\frac{1}{m}\sum_{i=1}^{m}\Big{(}x_{i}^{(k)}-E \Big{[}x^{(k)}\Big{]}\Big{)}^{2}+\epsilon\) represents the mean, variance and \(\epsilon\) is a constant added for numerical stability. To ensure that the input value is not limited to a narrow range, the normalized value is generally multiplied with the scaling amount \(\gamma^{(k)}\) and the offset amount \(\beta^{(k)}\) :  
\[y^{(k)}=\gamma^{(k)}\hat{x}^{(k)}+\beta^{(k)} \tag{2}\]  
Conventionally, a typical VGG16 model will have three fully connected layers at the end of the VGG16 architecture for classification, but there is none for this case. The features extracted from the convolution layers are prepared and fed into the ensemble classifier.