Learning to interact with high-dimensional data

Artificial intelligence techniques and humans have skills that complement each other; the first being good in computation at lowest level (e.g. matrix multiplications) whereas people are better at abstracting and transferring knowledge from their experience. This thesis examines how we can combine the two aforementioned strengths in order to create robust, efficient, visual and interpretable machine learning. In particular, we research on dimensionality reduction techniques, which provide ways of projecting high-dimensional data in 2D. While dimensionality reduction is used for many reasons such as to reduce storage space and processing time, we focus on its usage for visualization. By visualizing the features in a two dimensional (2D) or three dimensional (3D) space, we make it easier for the human perception to understand the structure of data in a manner that feels natural. In layman terms, we examine what happens when a user interacts (moves, drags etc.) with some data points in 2D, which correspond to high-dimensional data. Most modern dimensionality reduction techniques are based in distance metrics, which are prone to outliers and crowding issues. By extending a recently proposed generic framework called Similarity Embedding Framework (SEF) which minimizes the objective function of the difference between the projection and a target, we define that target similarity matrix as the outcome of user interaction. Then, the framework learns iteratively the optimal projection with gradient descent. In essence, we just optimize two similarity matrices. Fast linear and kernel versions are proposed. The experimental procedure covers two interaction scenarios. The first one is a quite common in multi-dimensional projection literature, so that users are provided with a subset of data points (also known as control points) and they are free to rearrange it as they wish, usually to cluster them better. Based, on that interaction, techniques have been proposed that perform interpolation so that they are able to project the rest of unseen dataset, in a kind of semi-supervised learning. While most techniques of this kind rely on feeding coordinates to least-squares, bayesian modeling, eigendecomposition, or other kinds of solvers, this thesis suggests an end-toend optimization algorithm that models user interaction as a target similarity matrix. Extensive evaluations are performed where we report results that outperform competitive baselines in a wide range of datasets (numerical, image, text). The second interaction scenario involves questions like "what happens if I move that class away? ". This scenario involves a modification of SEF in which the target matrix of user interaction is enriched with the information of high-dimensional neighbors. In essence, when a whole dataset is projected, we drag a class away and we set the target matrix so as that the points we moved to be similar with their high-dimensional neighbors. This procedure results in some surprising observations. Apart from improving classification precision and clustering, the new emerging structure of the projection unveils semantic manifolds. For example, on a Head Pose dataset, by just dragging the faces looking far left to the left and those looking far right to the right, all faces are re-arranged on a continuum even on the vertical axis (face up and down). This methodology could be used in domain adaptation of dense embeddings and transfer learning. Αριστοτέλειο Πανεπιστήμιο Θεσσαλονίκης