  Low-dimensional embeddings for data from disparate sources play critical
roles in multi-modal machine learning, multimedia information retrieval, and
bioinformatics. In this paper, we propose a supervised dimensionality reduction
method that learns linear embeddings jointly for two feature vectors
representing data of different modalities or data from distinct types of
entities. We also propose an efficient feature selection method that
complements, and can be applied prior to, our joint dimensionality reduction
method. Assuming that there exist true linear embeddings for these features,
our analysis of the error in the learned linear embeddings provides theoretical
guarantees that the dimensionality reduction method accurately estimates the
true embeddings when certain technical conditions are satisfied and the number
of samples is sufficiently large. The derived sample complexity results are
echoed by numerical experiments. We apply the proposed dimensionality reduction
method to gene-disease association, and predict unknown associations using
kernel regression on the dimension-reduced feature vectors. Our approach
compares favorably against other dimensionality reduction methods, and against
a state-of-the-art method of bilinear regression for predicting gene-disease
associations.
