## 3 What Is Transfer Learning?
The key assumptions are reasonable degrees of similarity between the source and target domains [31] and prior knowledge regarding what knowledge is to be transferred and when [20; 31]. The goal is that the transfer is \"positive,\" meaning the knowledge acquired from learning in a source domain improves the algorithm performance for the target domain, but there is always a risk of \"negative\" transfer where the source knowledge does not help and may even hurt performance in the target domain [15; 30; 31; 32]. The probability of \"negative\" transfer is higher for \"far transfer\" (source and target are farther away semantically) problems compared to \"near transfer\" (source and target are closer semantically) [8]. Taking a simple example presented in [33], a near transfer analogy example would be \"Blindness is to sight as deafness is to hearing\" all of which occurs within the general domain of \"human senses.\" Using the same source words, a far transfer analogy would be \"Blindness is to sight as poverty is to money,\" where the former word pair is still from the domain of human senses, but the latter word pair is from the different domain of economics [33]. Despite the relation between the word pairs remaining constant, the far transfer analogy is often more difficult to infer in comparison to the near transfer analogy problem. Figure 3 shows an example architecture of a transfer learning algorithm, which uses an artificial neural network (ANN) consisting of convolutional and dense (i.e., \"fully connected\" and/or \"classification\") layers. In this example, a source dataset, which is usually quite large, is used to train a model to make predictions using traditional methodology. When a target dataset is selected, instead of repeating this training process for the target dataset, the feature extraction elements of the source model (such as the number of layers and their weights) are utilized within the target model architecture. The layers and weights re-utilized from the source model are \"frozen.\" Note that there are no expectations of how many layers or which types of layers are frozen. However, depending on whether the source and target dataset have different classes, the target model might need to add new dense layers that align with the different (if any) prediction classification categories. There can also be the addition of new layers; however, there are many combinations of frozen, unfrozen, and new layers that will yield different results. Figure 2: Difference between traditional and transfer ML methods: (**a**) source and target datasets are used to train separate models that do not interact; (**b**) source dataset is used to train a model and the derived knowledge is used to influence the target model for the target dataset. One question to be answered within every transfer learning problem is how to balance the ratio between frozen, unfrozen, and new layers. This is often derived from the size and similarity differences (if any) between the source and target datasets. Consider the various architectures of a theoretical ANN in Figure 4. Figure 4a represents the source ANN architecture, which, at its most basic level, consists of convolutional and dense layers. Regardless of the input dataset, the ANN in Figure 4a will always be agnostic and produce predictions based on the classes associated with the source dataset. Many applications will implement architectures as shown in Figure 4b,c, which involve allowing some dense layers to be unfrozen and/or added in the case of Figure 4c specifically. Though the convolutional layers of the model stay the same, the classification (dense) layer parameters can fluctuate. Convolutional layers can also be unfrozen (see Figure 4d) or added (see Figure 4e). Finally, for completeness, if the entire architecture is new, the model is completely different and no longer deemed transfer learning. As one allows more layers to be unfrozen or adds new layers, the resulting model adapts more to the target dataset.