## 3 What Is Transfer Learning?
At its instantiation, data are considered to be \"raw\" in the sense that they are typically in need of some pre-processing (value validation, removal of duplicates, standardization, etc.) and considered to be highly dimensional (instances are described by many variables) [34; 35]. Dimensionality reduction (assumed to encompass feature selection (see [36; 37; 38]) and extraction (see [39])) are often used to represent the \"raw\" data using \"features,\" [39; 40]. These features are often vectors, which can be represented (and sometimes visualized) within a \"feature space.\" This is an important step of representation to cope with computational resource, mathematical feasibility, and algorithm complexity when many dimensions are involved [39; 40]. However, as these raw inputs are manipulated and turned into \"features,\" they need to be represented in a new way because they have changed from their original \"raw\" state. These concepts of \"feature spaces\" are important when discussing transfer learning. For example, one of the most popular types of transfer learning is domain adaptation, which is where the feature spaces of the source and target domains remain the same [15; 30]. A \"domain shift\" (or gap) exists when these feature spaces are different [41; 42]. Figure 3: General transfer learning architecture; example optional elements have a dashed outline. There are many ways to categorize transfer learning algorithms and methods. Some surveys classify transfer learning into two primary types depending on the relatedness between the source and target domains: homogenous (source and target domains share the same feature space) and heterogenous (source and target domains have different feature spaces) [41; 43]. One study additionally included negative transfer learning, aiming to understand the association between the source and target domains and narrowing the gap in association (if any) through vector space manipulation [30]. Other surveys classify types based on the availability of data labels: inductive (only target domain has labels), transductive (only source domain has labels), or unsupervised (neither domain has labels) [31; 44; 45]. A more recent survey proposed two overlapping hierarchies, within which transfer learning algorithms can be viewed as data-based (focused on minimizing differences between the features of the source and target data) or model-based (focused on accurate predictions on the target data given the source data and model) approaches [15]. Figure 4: Varying amounts of “knowledge” transferred in a transfer learning application involving ANNs: (**a**) original source mode; (**b**) frozen convolutional layers with unfrozen dense layers; (**c**) frozen convolutional layers with a combination of frozen, unfrozen, and new dense layers; (**d**) combination of frozen and unfrozen convolutional layers and a mix of unfrozen and new dense layers; (**e**) combination of frozen and new convolutional layers with all new dense layers; (**f**) all new convolutional and dense layers, i.e., a completely separate model. Transfer learning is a large and quickly growing domain; in contrast, analogical inference is a smaller field with steadier growth over time.