# Assessment of Deep Learning Methodology for Self-Organizing 5G Networks
## 3 From Machine Learning to Deep Learning
### What Is Deep Learning? Deep learning is based on almost the same concepts as machine learning; however, all the recent success in deep learning is mainly based on the artificial neural networks (ANN). Research on artificial neural networks can be traced back to as early as the 1940s [12]. For many years, ANNs were less popular, and that is why the current popularity appears new. Roughly, there have been three eras of research in deep learning: the early 1940s-1960s, when the field was known as cybernetics; the 1980s-1990s, when it was called connection-ism; and the current surge started in 2006, which is called deep learning. Early development on deep learning algorithms was inspired by computational models of how the brain works or learns. This is the reason why, for a long time, research in deep learning went by the name of artificial neural networks (ANN). The first neuron model was presented in [15]. Later, the first perceptron learning algorithm was presented in [16]. Since then, many varied models and learning algorithms have been introduced such as the Hopfield networks [17], self-organizing maps [18], Boltzmann machines [19], multi-layer perceptrons [20], radial-basis function networks [21], autoencoders [22], and sigmoid belief networks [23]. Deep learning solves the problem of representation learning by enabling computers to build complex concepts out of simpler concepts. Figure 1 (www.iro.umontreal.ca/bengioy/talks/icml2012-YB-tutorial.pdf) shows an example of representation learning, as discussed in the introduction section. The goal of modern deep learning is to make algorithms that are successful in solving tasks that are easy for humans. Humans are good at tasks such as recognizing objects and understanding language [12]. Previously, many machine learning methods solved such tasks by hand-crafted features and by providing prior knowledge to the system. On the other hand, deep learning motivates the learning of features from the data so that the model learns to do what is expected from it. This is more like how humans learn at a young age. In this kind of machine learning approach, the system improves its performance through increased experience or more training. Therefore, deep learning is essentially machine learning that is tailored for deep models. The recent progress in deep learning has been exclusively based on deep neural network models. The concept of depth or the determination of whether a model is deep is based on the fact that the structure of the model should consist of multiple layers that represent the level of abstraction, and each layer should adapt to model training. The features in the lower levels (layer) of the model (neural network) should progressively combine to form higher level (layer) features [12]. There are three types of DL techniques:  
Supervised Learning: In supervised learning, the network is fed example inputs with labels and their desired labeled outputs. The main target of this method is to get a result that can better generalize the unseen data according to the training process and map inputs to outputs well. Supervised learning has been widely applied to solve channel estimation issues in cellular networks. Multiple layer  
Figure 1: The lower level features progressively combine to form higher layer features in deep learning. perceptron (MLP), convolutional neural network (CNN), and recurrent neural networks can be used as supervised learning techniques. Unsupervised Learning: In unsupervised learning, unlabeled input data is fed to the network; therefore, the network has to learn the hidden patterns in the input data on its own to produce a generalized output. In unsupervised learning, the model or network aims to find patterns or representations in the input data. In deep learning, the restricted Boltzmann machines (RBMs), autoencoders (AE), and generative adversarial networks (GAN) are unsupervised techniques while CNN, RNN, and MLP can also be unsupervised.