## 5 Machine Learning Model Designs
### Regression and Classification Problems
#### 5.1.7 Neural Networks  
Neural networks can be trained to classify based on supervised or unsupervised learning, and specialized neural network models are the leading approach in the visual classification of objects. Every neural network contains at least three layers: (1) Input Layer, (2) Hidden Layer, and (3) Output Layer. These networks' generalizability makes them well-suited for all types of classification problems, from binary to multi-class, with all types of data. Some of the best computer vision models include u-networks, so called based on the 'U' shaped structure of their design. Another type of network is a recurrent neural network. One popular form of this network uses so-called long short-term memory cells in its hidden layers. These types of cells are able to capture or learn relations between events that are occurring over more extended time separations. This is applied to time-series classification. The neural network can also work for numerous regression problems with higher accuracy than linear regression, decision trees, random forests, etc. In this article, we have explained six main types of neural networks. Feed Forward: This is the simplest form of all neural work. The most basic feed-forward neural network contains one input layer, one hidden layer, and one output layer. The processing of data only occurs in one way. Although the data could travel via a number of the hidden layers. The output of the neuron is represented by Equation (18)  
\[z=\sum_{i=1}^{n}(a_{i}*w_{i})+b \tag{18}\]  
where \(z\) is the output of the neuron, \(n\) is the number of neurons in the previous layer, \(a\) in the input vector, \(w\) is the weight vector, and \(b\) is the bias. Figure 9 illustrates the workings of a feed-forward neural network. \[h_{1}=a_{1}*w_{11}+a_{2}*w_{21}+a_{3}*w_{31} \tag{19}\]  
\[h_{2}=a_{1}*w_{12}+a_{2}*w_{22}+a-3*w_{32} \tag{20}\]  
Figure 8: Elbow method. \[t=h_{1}*w^{\prime}_{1}+h_{2}*w^{\prime}_{2} \tag{21}\]  
The output of the hidden neuron is the sum of the dot product of the corresponding weight and the input. Similarly the the output,t, if the summation of the dot product of the hidden neuron output and its corresponding weight. 1. Multi Layer Perceptron(MLP)/ Deep Neural Network: Multi layer perceptrons are similar to feed-forward neural network, but in this network there are more than one hidden layer. In the backpropogation algorithm, the output from the output layer is compared with the real value, and then weights are adjusted according to the residual. This step is repeated until there is no improvement in the output. Figure 10 shows a typical example of a deep neural network. In the real world, the data is rarely linear, so to handle non-linear data, non-linear activation functions are required. Usually two types of activation functions are applied in the MLP model: one for the hidden layer, which is generally the same for all the layers, but there is no compulsion, and one for the output layer. The most commonly used activation functions are (1) linear activation function, (2) sigmoid, and (3) relu. The output of these activation functions decides whether the neuron will fire or not. For example, in one of the most common activation functions denoted by Equation (11) the neuron is activated or fired only when the output of the neuron is greater than or equal to zero. Figure 11 shows the graphical representation of the Relu activation function. And Figure 12 representation of Sigmoid activation function. \[f(x)= \begin{cases}x&x>=0\\ 0&x<0\end{cases} \] (22)
2. Recurrent Neural Networks (RNN): Recurrent Neural Networks are derived from the feed-forward neural network. These networks have additional connections in their layers that enhance an ANN's ability to learn from sequential data. This can be any form of sequential data, such as numerical time-series data or video and audio. As seen in Figure 13, a hidden layer of RNN cells passes the output to the next layer and to the adjacent cell in series. This enables the network to learn from patterns that appear in sequence.