Contributions on Deep Transfer Learning

Machine Learning (ML) is reshaping our world by building machines that can see, read, listen, talk, and write (Marr, 2015). For example, imagine you want to eat your favorite food and ask a mobile app what are the nearby restaurants that serve your favorite food. The app uses a ML algorithm to analyze the information of your location and time to provide better service. Imagine a startup that wants to use ML algorithm optimize their resources and requirements. Envision a scenario where a breast cancer patient and a surgeon want to plan a surgery. Generally, it takes a highly trained surgeon’s subjective evaluation of the breast cancer surgery to make treatment decisions. Employing ML algorithms allows an objective evaluation of the surgery with a sense of femininity (aesthetic metric by (Beadle et al., 1984)), using 3D imaging that helps the patient and the doctor to visualize and collectively decide what they are seeing, and then make a informed decision about the surgery (Oliveira, 2013). The ability of ML algorithms to automatically understand the contents of images opened new ways to tackle prominent computer vision challenges like view point variation, illumination, occlusion, scale, deformation, background clutter, and intra-class variation. Especially, Deep learning algorithms like Stacked Denoising Autoencoders, Convolutional Neural Networks, and Deep Belief Nets have addressed these challenges with state-of-the-art results and have surpassed human performance. The salient attribute of Deep learning algorithms is that they automatically learn features from a large number of data samples without overfitting the model. Deep learning provides a partial solution to the research statement how to get computer programs to self-learn patterns from data. We consider machine learning as a broadening field which integrates interdisciplinary knowledge of learning processes from other fields such as psychology, biology, neuroscience, and economy. From this we attempt to answer questions such as: what kind of process can lead to learning, under what conditions, and for what kind of data? (Mitchell, 2006) In this thesis we integrated knowledge of deep learning with transfer learning where the learning is inspired from human-like learning processes to build our Deep Transfer Learning framework: “how to get computer programs to self-learn patterns in data-efficient and domaingeneral way? ” Our survey on the state-of-the-art machine learning methods that use self-learn patterns lead to deep learning models. The subsequent question is about how learning algorithms can be inspired from human-like learning processes. For the sake of simplicity, we narrowed our scope down to human like knowledge transfer from one scenario to another. It is interesting to study the various possible cases of transfer learning settings based on the distributions, posterior probabilities, learning function and the classification tasks. We have designed three main mechanisms that harness the advantages of our proposed Deep Transfer Learning (DTL) framework: 1) Layerwise Transfer Learning (LTL), 2) Source-Target-Source (STS) and 3) Deep Transfer Learning Ensemble. First, we developed two approaches based on LTL mechanism : a) Transfer Learning unsupervised (TLu) and b) Transfer Learning supervised (TLs). On