# Deep Learning and Big Data in Healthcare: A Double Review for Critical Beginners
## 3 Deep Learning Fundamentals and Elements
It is normally used with images, by rotating different angles or adding different levels of noise, cropping, and geometrically transforming (e.g., rotation, translation, and scaling) [48; 52]. Special mention is deserved by the so-called _transfer learning_, which involves transferring the knowledge gained by a DL architecture in one dataset onto another dataset with a different kind of data [20]. It has been shown that DL models are highly reusable, especially convolutional architectures where inputs are very high dimensional, such as images, audio, or text data models. A model trained on a large-scale dataset can be reused on significantly different small-dataset problems. This kind of model used in transfer learning are called pretrained networks, and there are two ways to use them, namely, feature extraction and fine-tuning. The first one consists of using a pretrained network to extract features from samples of a new dataset. These features are then run through a new ML algorithm in the output layer, which is trained from scratch. The second one consists of slightly adjusting the pretrained network together with the new classifier in the top layers [47]. Another type of transfer learning, mainly applied to categorical variables and especially in natural language processing, is the use of precomputed embeddings [60; 61], which are network-yielded vectors used to represent discrete variables as low-dimensional vectors of continuous values. The use of embedding vectors from a precomputed embedding space, like Word2 vec [62] or GloVe [63], is very widespread today. Usually, large DL networks are needed to solve real problems. Training these networks is very challenging and computationally expensive, and the results are models with lots of parameters and operations. Weight pruning is used to decrease the size of the trained model and makes the inference phase more efficiency without compromising accuracy [64]. A recent publication [54] articulates the hypothesis that dense, randomly-initialized, feed-forward networks contain some subnetworks that get an accuracy similar to the original network in the same number of iterations when they are trained in isolation. Most of the famous DL frameworks, like Tensorflow, Keras, or Pytorch, have primary support for Python. However, they are normally multilanguage supported. For example, Tensorflow can be used in C++, JavaScript, or Swift; Keras can be used in R; and Pytorch can be used in C++. Reasons why Python is popular and widely used in DL communities are: (1) Its simple and readability syntax that provides rapid testing of complex algorithms using less code and less time than with other programming languages; (2) The large numbers of libraries and frameworks for scientific computation, advanced computing, visualization, and ML, among others; and (3) the great amount of developers around the world that offer comprehensive support and assistance via forums and tutorials. This is because Python is today the language used in nearly all research and commercial applications of DL. Nevertheless, Python has some problems, namely, it is not designed to be fast and it has poor support for parallel processing. For these reasons, DL core frameworks are written in other languages (generally C/C++ libraries like CUDA), and provide Python (and other languages) wrappers. To avoid all of these problems of combining different languages to solve speed problems, a very promising programming language, called Julia, is emerging strongly due to its ease of use (similar to MATLAB and Python languages), and due to its speed (comparable in most solutions to C languages). Although Julia is still not yet as mature as Python is, it already proposes very interesting libraries for DL like Flux. The blast that DL has witnessed in recent years has generated an unmanageable amount of resources, including books, papers, courses, websites, or blogs. It can be hard work for the beginner to choose which resources to start with.