# Image-Based Feature Representation for Insider Threat Classification
## 3 Proposed Method
### Transfer Learning
The purpose of this layer is to utilize the features from the output of the previous layer to perform classification on the input image-based on the training data. In a fully connected layer, neurons have full connections to all activation in the previous layer similar to the normal neural networks. Fully-connected layer calculates the class probability scores, producing the volume of size \([1\times 1\times 2]\), in which both the numbers map to a class score. This class score is used for the classification. In the proposed problem, there are two classes, Malicious and Non-Malicious. The number of neurons in the last layer of the network should be same as the number of classes to be identified in the dataset. In our last layer, we have two neurons as there are two classes (Malicious and Non-Malicious). The Softmax activation function is used in the final layer to predict the class probabilities. We used the MobileNetV2 [37], VGG19 [38], and ResNet50 [39] to demonstrate the proposed approach. The base models are instantiated and loaded with the initial weights. Then, the fully connected model is appended to this loaded model and train the network. The details for each model are explained in detail in the following sections. We only fine-tuned the last convolutional block instead of the whole network so that it helps in preventing overfitting. The low-level convolutional blocks learn features that are less abstract and more general than the ones found from higher layers, so it is sensible to keep the first few blocks fixed (more general features) and only fine-tune the last one (more specialized features). Fine-tuning is done with a very slow learning rate to ensure that the updates stay very small, such that it does not affect the previously learned features. The more layers are frozen, the less effective the network capacity becomes. So, we started from freezing all the layers and only learning the final classifier. Then, we considered unfreezing more layers progressively from the top to achieve a better result. Networks trained on relatively small datasets can overfit the training data. Dropout layers drop out a random set of activations in that layer by setting them to zero. This layer ensures that the network does not get too much fitted to the training data, thereby helping decrease the issue of overfitting. Dropout layer is only used while training. The outputs of a layer under dropout undergo random subsampling. This has the effect of thinning the network or reducing the capacity during training. Fine tuning the models included addition of a global average pooling layer followed by fully connected layers as the classification layers. In all the three models, we used a Global Average Pooling after the base mode. Unlike the ordinary pooling layer that makes small changes to the images to reduce the size, the global average pooling layer is used to fully or partially replace the fully connected layers used as the highest ones in the convolutional neural networks. The fine-tuning includes the addition of global average pooling layer, which helps to reduce the size of the tensor and speeds up the calculations. The global average pooling layer does a more intense dimensionality reduction, such that a tensor of dimensions h x w x d is reduced to dimensions 1 x 1 x d. It is a simple operation that has no trainable parameters, hence speeding up the training process. In a classification task, Softmax function is applied to classify an image with probabilistic values ranging between 0 and 1. The summed-up values of a feature map from the global average pooling layer is fed into the Softmax layer. The activation values are added up and divided by the sum produced to give the probability values. Figure 6 gives the details of these models. Figure 6: Transfer learning using MobilenetV2, VGG16, and ResNet50. For the MobilenetV2 [37] model, weights are initialized with the weights used in the training of the original model. The global average pooling layer is followed by a dropout layer.