Learning Low-Dimensional Representations With Application to Classification and Decision-Making

Many signal processing and machine learning algorithms perform poorly when applied to high-dimensional data, as is known by the phenomenon of the curse of dimensionality. Learning low-dimensional representations aims at reducing the dimensionality of the observation space while maintaining the characteristics of the data. Further, low-dimensional representations can help to reveal latent structures, allowing for deeper insights into the observations. For these reasons, models are proposed that allow to learn low-dimensional representations of the observations, providing means for the analysis of the observed data. In particular, approaches for efficient data acquisition and classification and for the inference of the structure of the observed data are presented. First, low-dimensional methods for classification are proposed with application to hyperspectral imaging. In remote sensing, hyperspectral imaging provides an efficient means for the analysis of vast areas. As each element of the captured image represents the spectrum of the visible and infra-red light, the acquired data allows for effective discrimination between different materials. For classification, a feature selection approach as well as a sparse acquisition scheme are presented. The goal of both methods is to reduce the amount of data that needs to be evaluated during classification, while maintaining high classification accuracies. In the first approach, a clustering-based method for selecting the bands of a hyperspectral image, which can be considered as features for classification, is proposed. However, removing costly acquired data during feature selection is clearly resource-inefficient. For this reason, further a sparse acquisition approach based on the Compressive Sensing framework is proposed. The key idea of this approach is to capture the data in a low-dimensional representation, which is interpreted as being embedded in a feature space for the classification problem. As we are interested in the classification result directly, costly reconstruction of the data is not required and can be avoided. Second, a feature-based approach to learn the structure of the spectra is proposed, revealing the materials present in a hyperspectral image. Hyperspectral images often suffer from low spatial resolutions such that each element of an image represents a large area, often in the range from 2m² to 400m². However, many algorithms, such as for classification, assume that each element of the image represents a single material only. Thus, learning the structure is an important task in the analysis of hyperspectral images, which is also known as spectral unmixing. For this, a Bayesian nonparametric formulation of the problem is proposed. A significant advantage of this model, in comparison to existing approaches, is that the number of materials is inferred from the data and, hence, is not required to be known a priori. The proposed formulation results in a Bayesian nonparametric unmixing algorithm which enables to learn the number of latent features, the actual features, and their coefficients jointly. Third, a model for decision-making based on a feature representation of the observations is proposed. In particular, the problem of learning from observations is considered, in which we aim at learning a behavior from observations which are provided by an experienced agent. A key difference to existing approaches consists in the assumption that the agent makes its decision based on latent features of the observations, where each feature indicates a certain action. Learning the features and their policies enables to reason about the observed behavior. Further, actions for new situations can be predicted, from which a policy can be derived for other agents. Using the developed algorithm, a driver's behavior is analyzed, which is a typical task in advanced driver assistance systems, in order to show the performance of the model in a real-world problem. The algorithms based on the proposed models are evaluated on simulated data to proof the concepts. Further, all methods are applied to real data, demonstrating the high performance of the developed approaches.