## 5 Comparisons of Transfer Learning and Analogical Inference in Two Application Domains
### Natural Language Processing  
Natural language processing (NLP) focuses on allowing machines to produce human-like text and language. Natural language processing is a realm in which machines learn to process unstructured textual data in intelligent ways. Language inputs consist of pre-defined consistent building blocks--words--which can be tokenized and vectorized. Compared to computer vision applications, the consistent building blocks make applications of transfer learning and analogical inference in NLP more similar to one another. In general, text-based algorithms/models can be grouped into three categories: cognitive-science-inspired architectures, vector space models, and language transformer models (summarized in Table 3). Major pioneering work on textual analogical inference includes the \"structure mapping engine\" (SME) [92]. SME is a symbolist algorithm, considered to be top-down in the sense an analogy is broken into its individual components (similar to Legos). Relations between elements (e.g., \"a part of,\" \"a type of\") are developed to understand the underlying meaning of an analogy [92]. Many subsequent algorithms focus on connectionist (bottom-up approach) or hybrid approaches to map elements within a textual analogy problem (summarized in Figure 5c-e) [59]. Connectionist algorithms also break analogies into their individual components but place emphasis on the connectivity of these elements, not necessarily in specific relationships as symbolist algorithms. Hybrid algorithms combine elements of these two primary approaches (symbolist and connectionist). More recently, vector space models, such as Word2 Vec [96], Global Vectors (GloVe) [97], and FastText [98], have been applied to a wide range of semantics tasks, including verbal analogy (see reviews in [99; 100; 101]). Vector space models are a subset of connectionist algorithms, but unlike most cognitive-science-inspired algorithms, vector space models have been limited to four-term analogy problems, such as A:B:C:D-type analogies (e.g., Figure 6c), and have completely ignored the problem of mapping in analogy. Though vector space models can be trained, many are pre-trained on a large corpus of text from which word similarity is derived (often through cosine similarity). Figure 7 shows a taxonomy of analogical inference \"families\" and their respective algorithms, presented previously in [59]. Most recently, language transformer models have been in the spotlight, including the Bidirectional Encoder Representations from Transformers (BERT) family [102; 103; 104; 105], the Generative Pretrained Transfer (GPT) family of algorithms [106; 107; 108]), and XLNet [109] (see reviews in [110; 111]). Language transformer models, especially within the GPT family, can perform realistic human-like language generation in addition to simple classification based on text inputs. Although these models are not explicitly trained to solve analogy problems, they have exhibited human-like performance in four-term analogy problems; vector space models and language transformer models are usually pre-trained on a corpus and are ready to be applied to language tasks, either within or outside of the pre-trained domain, making them frequently used for transfer learning [112]. The tasks for which analogical inference algorithms, vector space models, and transfer models are used primarily only overlap for _A:B:C:D_ analogy problems, due to the limitations of vector space model. Language transformer models can be used in a wider variety of problems such as analogy/story generation. One of the few applications of transfer learning that utilizes analogical inference maps similar, but different, domains with low-level representations [113], where \"low-level\" representations mean relatively basic relations. One application from transfer learning applies to story-based cross-domain analogies through Reproducing Kernel Hilbert Space.