# MR Images, Brain Lesions, and Deep Learning
## 2 The Literature Review: Selection Criteria
### Machine Learning and Deep Learning
#### 3.1.2 Deep Learning Methods  
Deep learning (DL) is a subfield of ML [110] that uses artificial neural networks (ANNs) to develop decision-making algorithms [90]. Artificial neural networks are neural networks that employ learning algorithms [111] and infer rules for learning. To do so, a set of training data examples is needed. The idea is derived from the concept of the biological neuron (Figure 9e). An artificial neuron receives inputs from other neurons, integrates the inputs with weights, and activates (or fires in the language of biology) when a pre-defined condition is satisfied [92]. There are many books describing ANNs; see, for example, [93]. Figure 9: Graphical representation of some machine learning (ML) algorithms and the representations of an artificial neural network (ANN) and a DL neural network: (**a**) corresponds to the \(k\)-nearest neighbor (\(k\)-NN) algorithm and a representation of \(k=5\) (the number of nearest neighbors); (**b**) represents the \(k\)-means clustering algorithm, also represented by \(k=2\) clusters, with the blue circle representing the cluster centroid; (**c**) is the representation of the support vector machine (SVM) algorithm with the optimal separation by a hyperplane between classes; (**d**) corresponds to a random forest (RF) algorithm and represents a forest of classification trees; and finally (**e**) represents the similarity between concepts used between an artificial neuron and a true neuron with inputs and outputs. It also shows the architecture of an ANN and a DL neural network, where IL is the input layers, HL the hidden layers, and OL the output layer. This figure was created and adapted using references [112; 113; 114]. The fundamental unit of a neural network is the neuron, which has a bias \(w_{0}\) and a weight vector \(w=(w_{1}\),..., \(w_{n})\) as parameters \(\theta=(w_{0}\),..., \(w_{n})\) to model a decision using a non-linear activation function \(h(x)\)[115]. \[f(x)=h\Big{(}w^{T}x+\ w_{0}\Big{)} \tag{4}\]  
The activation functions commonly used are: _sign_(\(x\)), _sigmoid_\(\sigma(x)\), and _tanh_(\(x\)):  
\[sign(x) \tag{5}\]  
\[\sigma(x)=\frac{1}{1+e^{-x}} \tag{6}\]  
\[tanh(x)=\frac{e^{x}-e^{-x}}{e^{x}+e^{-x}} \tag{7}\]  
An interconnected group of nodes comprise the ANN, with each node representing a neuron arranged in layers [16] and the arrow representing a connection from the output of one neuron to the input of another [103]. ANNs have an input layer, which receives observed values, and an output layer, which represents the target (a value or class), and the layers between input and output layers are called hidden layers [92]. There are different types of ANNs [116], and the most common types are convolutional neural networks (CNNs) [117], recurrent neural networks (RNNs) [118], long short-term memory (LSTM) [119], and generative adversarial networks (GANs) [120]. In practice, these types of networks can be combined [116] between themselves and with classical machine learning algorithms. CNNs are most commonly used for the processing of medical images because of their success in processing and recognition of patterns in vision systems [49]. CNNs are inspired by the biological visual cortex and also called multi-layer perceptrons (MLPs) [49; 121; 122]. An MLP consists of a stack of layers: convolutional, max pooling, and fully connected layers. The intermediate layer is fed by the output of the previous layer, e.g., the convolutional layer creates feature maps of different sizes, and the pooling layers reduce the sizes of the feature maps to be fed to the following layers. The final fully connected layers produce the specified class prediction at the output [49]. The general CNN architecture is presented in Figure 10. There is a compromise between the number of neurons in each layer, the connection between them, and the number of layers with the number of parameters that defines the network [49]. Table 3 presents a summary of the principal structures of a CNN and the commonly used DL libraries.