Computational modelling of music similarity constitutes a key element for music information retrieval and recommendation systems. Similarity models and their analysis are also important for research in musicology and music perception. In this study, we test feature preprocessing with Restricted Boltzmann Machines in combination with established methods for learning distance measures. Our experiments show that this preprocessing improves the overall generalisation results of the trained models. We compare the eects of feature preprocessing on distance function learning using gradient ascent and support vector machines. The evaluation is performed using similarity data from the MagnaTagATune dataset, which allows a comparison of our results with previous studies