## 3 Methodologies and Research Framework
### Deep Neural Networks and Fully Connected Network  
Before discussing our chosen deep learning model, we examined a variety of DNNs. The first DNN model we examined is a FCN, which is the simplest DNN model. The reason for calling it the simplest DNN model is that the DNN can be implemented by solely fully-connected layers [26]. All the output activations in fully-connected layers consist of a weighted sum of all the input activations, thus all the outputs are connected to all the inputs [26]. However, the well-known disadvantage of this model is that it requires a large amount of storage and computation, but this issue could be solved by removing some connections between the activations [26]. In [27], the authors proposed deep Convolutional Neural Networks (CNNs), which are composed of five convolutional layers followed by max pooling layers and three fully-connected layers with a final 1000-way softmax. Besides, Rectified Linear Units (ReLus) were applied to the output of every convolutional and fully-connected layer. One interesting point in this research was that two GPUs were employed to train the proposed model due to the size of the adopted dataset. By doing so, the authors mentioned that it was possible to reduce error rates and achieve less training time than when using a single GPU. Note that other concepts such as local response normalization, two forms of data augmentation and dropout to avoid an overfitting issue, etc. were employed. The study, which began with a strong motivation that the deeper the network, the better the accuracy, discussed the deep residual learning methodology, also known as the ResNet [28]. In this research, however, the authors pointed out a degradation problem caused by vanishing or exploding gradient if a network gets deeper. To avoid such a problem, the authors inserted shortcut connections into the plain network to turn the network into its counterpart residual version. Two advantages were claimed: (i) easy to optimize; and (ii) gain accuracy from considerably increased depth, but the disadvantage of residual networks is that the computational and memory costs could be linearly increased to improve performance [29]. Figure 7: Example of HOG feature extraction for the vehicle and non-vehicle object. In [30], the authors proposed a deep learning model that combines two Recurrent Neural Networks (RNNs) consisting of an encoder and a decoder. This model is also called Sequence to Sequence (Seq2 Seq). The encoder basically creates a fixed size context vector from the input sequence of the model, and the decoder uses this context vector as an input to create a translated output sequence. The Seq2 Seq model is very effective in generating responses, but the dialogue response generated by the Seq2 Seq model tends to be less diverse [31]. The authors of [32] proposed the abstractive text summarization method based on Generative Adversarial Networks (GANs) consisting of the generator and discriminator. In this research, the authors pointed out that GAN is very powerful in terms of computer vision and image generation areas, but it is challenging to generate discrete outputs. As described above, DNNs can be organized into different forms depending on how they are applied: (i) the FCN is suitable when all the input data have to be jointly considered; (ii) the CNN is optimized to extract spatial features; and (iii) the RNN is appropriate for sequential data such as natural language [33]. Back to a discussion of our chosen deep learning model, the basic idea in this research is a simple neural network based on the following equation:  
\[H(x)=Wx+b \tag{1}\]  
where \(H(x)\) is a hypothesis, \(x\) indicates input data variables or data, and \(W\) and \(b\) denote weights and bias, respectively. This equation refers to neural network learning as a linear regression model, and the main objective is to find a set of the optimized network parameters by minimizing a cost function [34].