Chapter 1

Introduction
Currently machine learning (ML) and data mining (DM) are the largest partof artiﬁcial intelligence (AI) research, and AI itself would be one of the largest
research area in computer science and also being emerged as an important part ofthe entire science and more generally our society. Behind the technologies of AI,
there have been developed various approaches and methods of ML for a varietyof applications, although the history of ML itself is not so long, comparing withother research ﬁelds, like basic sciences and engineering.
ML research started in 1970s to 80s, and the community was not becoming
so large for a long time, even after the DM research started around 1980s to 90s.
However after around 2005 to 2010, the research area related with ML and DM
has become drastically huge, aﬀecting a lot of impacts on a variety of scienceand engineering ﬁelds, and also our society and life. Originally the applicationof ML was very limited and closely related with scientiﬁc or academic researchareas, such as natural language processing, speech recognition, computer vision,
robotics and medical engineering. However due to the development of internet, theapplications were becoming more diverse, such as web searching, recommendation,
fraud detection, etc. Also ML has accelerated pure science research, including drug
discovery process, biological experiment management and physical or chemical
simulations. Furthermore by the recent explosive data increase, the applicationsof ML are not limited to science and engineering but also abundant ﬁelds ofour society, such as autonomous driving, industrial applications, ﬁnance, gaming,
cooperate management, construction and agriculture.
There would be a lot of ways to classify a variety of ML and DM methods
into some sort of order. In this book, we ﬁrst focus on input data types. We
segment ML and DM methods into seven categories, from a viewpoint of input
data types. More in detail we categorize data types into the following sevens: vectors, sets, sequences (strings), trees, graphs, nodes in a graph and integrated data.
The last integrated data means that data can be a combination of one or moredatasets, such as sequences plus graphs. We will deﬁne and also explain each ofthese seven types in Chapter 2. Then each of the seven data types is describedby one chapter: Chapters 3 to 9, in the above order. In each chapter, we further

CHAPTER 1. INTRODUCTION

2classify the methods for the corresponding data type, by either problem settingsor their techniques. That is, problem settings are supervised, unsupervised andsemi-supervised learning. This was done for vectors and nodes in a graph (Chapters 3 and 8). On the other hand, the techniques are frequent pattern mining,probabilistic models (statistical learning) and kernel learning. This way of classiﬁcation by techniques was done for sets, sequences, trees, graphs (Chapters 4to 7). Then we explain one or more standard methods for each classiﬁed setting,focusing on already established methods. Also in the last of each chapter, we raise
one or more application examples of bioinformatics.
These data types, such as vectors and graphs, look totally diﬀerent from eachother, and also maybe because of this, ML methods for diﬀerent data types would
be thought to be diﬀerent from each other. They however deﬁnitely have somecommon ideas, motivations and derivation, etc., especially techniques behind different methods. One example is the formulation of principal component analysis,
kernel K-means clustering (both for vectors), spectral clustering (for graphs), andcanonical correlation analysis (for integrated data), all take the form of so-called
Rayleigh quotient and eventually their setting can be solved by a (generalized)
eigenvalue problem. Also there is a reverse case, in which diﬀerent methods havebeen developed for the same problem setting for the same data type, while they
have been developed by diﬀerent ideas. We will explain these cases on what point
they are diﬀerent and also how reasonably they have been developed. We thinkthat describing such shared points of diﬀerent methods or unique features of eachmethod will clarify each corresponding method.
Theoretical work are usually classiﬁed mainly by their signiﬁcance into diﬀerent levels, such as Theorems, Lemmas, etc. In this book theoretical results are allpresented as Propositions, since they are rather already established.
After the seven chapters corresponding to the seven data types, we extensively explain practical manners of evaluating the results obtained by applying
ML methods to actual data. Even if ML methods can be analyzed theoretically,
it would be reasonable to investigate how well predictions by ML methods canbe succeeded, to understand the performance of the ML methods themselves andalso the hardness of the problem or the data, to which the methods are applied.
The last chapter is Appendix, which describes the terms/methods used in thisbook. Although the corresponding sections of Appendix are cited in the main
text as much as possible, readers can refer Appendix if coming across the terms
not described well in the main text. The last part of Appendix is the detail ofseveral derivations in the main text.
The idea of this book is to cover major and standard approaches of alreadyestablished problem settings in ML and DM extensively, showing their relevanceand diﬀerences. Then ML and DM areas covered by this book are major, whilemuch more ML topics are not considered in this book, because of rather being
new or too complicated for an introductory text book. These uncovered topics
include reinforcement learning, transfer learning, query learning, learning to rank,
statistical Bayesian learning for numerous settings, such as Bayesiann belief networks, etc. There exist good books and reviews already for each of these problem
settings, and interested readers can refer to them.