Spectral Clustering for dynamic data using Multiple Kernel Learning

In this paper we propose a kernel-spectralclustering-based technique to catch the different regimes experienced by a time-varying system. Our method is based on a multiple kernel learning approach, which is a linear combination of kernels. The calculation of the linear combination coefficients is done by determining a ranking vector that quantifies the overall dynamical behavior of the data sequence analyzed over time. This vector can be calculated from the eigenvectors provided by the solution of the kernel spectral clustering problem. We apply the proposed technique to a trial from the Graphics Lab Motion Capture Database from Carnegie Mellon University, as well as to a synthetic database, namely three moving Gaussian clouds. For comparison purposes, some conventional spectral clustering techniques are also considered, namely, kernel kmeans, min-cuts as well as standard k-means. The normalized mutual information and adjusted random index metrics are used to quantify the clustering performance. Results show the usefulness and applicability of proposed technique to track dynamic data, even being able to detect hidden objects. I. I NTRODUCTION Spectral clustering has taken an important place in the pattern recognition field due to its capability of accurately grouping data having complex structure. There are several spectral clustering approaches mainly related to graph partitioning [1]. The most suitable techniques are those based on kernels. Nevertheless, one of the biggest disadvantages of spectral clustering techniques is that most of them have been designed for static data analysis, that is to say, without taking into consideration the changes along time. Some works have been developed taking into account the temporal information for the clustering task, mainly in segmentation of human motion [2], [3]. Other approaches include either the design of dynamic kernels for clustering [4], [5] or a dynamic kernel principal component analysis (KPCA) based model [6], [7]. Another study [8] modifies the primal functional of a KPCA formulation for spectral clustering to add the memory effect. Another approach, known as multiple kernel learning (MKL), has emerged to deal with different issues in machine learning, mainly, regarding support vector machines (SVM) [9], [10]. The intuitive idea of MKL is that learning can be enhanced when using different kernels instead of an unique kernel. Indeed, local analysis provided by each kernel is of benefit to examine the structure of the whole data. From this idea, in this work, we introduce a dynamic kernel spectral clustering (DKSC) approach based on MKL. D. Peluffo-Ordóñez, S. Garcı́a-Vega and G. Castellanos-Domı́nguez are with the Department of Electric Engineering, Electronics and Computer Science, Universidad Nacional de Colombia Manizales (email: {dhpeluffoo, segarciave, cgcastallanosd }@unal.edu.co). R. Langone and J. A. K. Suykens are with the Katholieke Universiteit Leuven, Belgium. {rocco.langone, Johan.Suykens }@esat.kuleuven. be Our approach uses the so-called kernel spectral clustering (KSC), introduced in [11], which is founded on KPCA formulation from least-square support vector machines and has shown to be a powerful tool for clustering hardly separable data allowing also out-of-samples extensions [12]. MKL is used in such a manner that kernel matrices are computed from an input data sequence, in which each data matrix represents a frame in a different time instance. Afterwards, an accumulative kernel is calculated as a linear combination of the previously obtained kernels where the weighting factors are estimated by ranking each sample contained in the frame. Such ranking is done by combining the relevance procedure proposed in [13] and the MKL approach presented in [14]. Experiments are carried out using two databases. On one hand, a subject from Graphics Lab Motion Capture Database from Carnegie Mellon University, here called Motion Caption. On the other hand, an artificial three-moving Gaussian clouds in which the mean of each cloud is changed along the frames. For comparison purposes, some conventional spectral clustering techniques are also considered, namely, kernel kmeans (KKM) and min-cuts (MC) [1]. Also, standard kmeans is considered. The normalized mutual information [15] and adjusted random index [16] metrics are used to quantify the clustering performance. II. T HEORETICAL BACKGROUND A. Kernel Spectral Clustering The kernel spectral clustering (KSC), introduced in [11], is aimed to split a data set of N samples intoK homogeneous and disjoint subsets. Define a data matrix X ∈ R N×d in the form X = [x⊤1 , . ,x ⊤ N ] , where xi ∈ R d is a sample vector. Data are first mapped and then projected. Assume a mapping function φ(·), which maps data from the original dimension to a higher one dh, so: φ(·) : R → Rh ,xi 7→ φ(xi). Then, the mapping matrix Φ ∈ Rh is Φ = [ φ(x1) , . ,φ(xN ) ⊤ ]⊤ . The projection matrixE ∈ Re follows a latent variable model in the form: E = ΦW + 1N ⊗ b , (1) Notation ⊗ denotes the Kronecker product and term ne denotes the number of considered support vectors. Then, according to [11] within a least-square-support vector machine framework, a matrix primal formulation of KSC can be stated Proceedings of International Joint Conference on Neural Networks, Dallas, Texas, USA, August 4-9, 2013 978-1-4673-6129-3/13/$31.00 ©2013 IEEE 1085