Extensions of dominant sets and their applications in computer vision

Many problems in computer vision can be formulated as a clustering problem, a problem that aims to organize a collection of data objects into groups or clusters, such that objects within a cluster are more “similar” to each other than they are to objects in the other groups. Assuming the feature-based representation, a computer vision problem can be formulated as follows: Given a set of data points in a ’d’ dimensional space, find the best partition of the space which gives us meaningful groups. The points in the representative metric space correspond to the feature vectors extracted from the object with their distances reflecting the dissimilarity relations. On the other hand, objects could also be described indirectly by their respective similarity relations, an approach which is more natural than feature-based technique as there are numerous application domains where it is not possible to find satisfactory features, but it is more natural to provide a measure of similarity. This work proposes similarity based data clustering framework, based on extensions of the dominant sets framework using theories and mathematical tools inherited from graph theory, optimization theory and game theory, that could be adapted “flexibly” in a wide range of vision applications, thereby combining the research domain of computer vision and that of machine learning. In our system, clusters are in one-to-one correspondence with Evolutionary Stable Strategies (ESS) a classic notion of equilibrium in evolutionary game theory field of a so-called “clustering game”. The clustering game is a non-cooperative game between two-players, where the objects to cluster form the set of strategies, while the affinity matrix provides the players’ payoffs. The dominant sets framework, a well-known graph-theoretic notion of a cluster which generalizes the concept of a maximal clique to edge-weighted graphs, has proven itself to be relevant in many computer vision problems such as action recognition, image segmentation, tracking, group detection and others. Its regularized counterpart, determining the global shape of the energy landscape as well as the location of its extrema, is able to organize the data to be clustered in a hierarchical manner. It generalizes the dominant sets framework in that putting the regularization parameter to zero results local solutions that are in one-to-one correspondence with dominant sets. In this thesis we propose constrained dominant sets, parameterized family of quadratic programs that generalizes both formulations, the dominant sets framework and its regularized counterpart, in that here, only a subset of elements in the main diagonal is allowed to take the parameter, the other ones being set to zero. In particular, we show that by properly controlling a regularization parameter which determines the structure and the scale of the underlying problem, we are in a position to extract groups of dominant sets clusters which are constrained to contain user-specified elements. We provide bounds that allow us to control this process, which are based on the spectral properties of certain submatrices of the original affinity matrix. Thanks to the many sensors, which generate a large amount of data every day, distributed in the society, there is a large amount of data for training and testing many computer vision systems. However, real data collected through those sensors is contaminated by outliers, and many computer vision tasks involve processing the available large amounts of data without any assumption on the existence of outliers. Recently, very few computer vision systems have shown that considering presence of outliers while solving computer vision problems help boosting the state-of-the-art results. However, most of the systems either try just to detect outliers from the computer vision datasets or solve their problems by detecting and rejecting outliers before applying the method on the dataset. Our proposed work is robust to outliers, and since we also believe that it is important for clustering methods to detect data consensuses and isolated outliers simultaneously in a clustering task, this thesis proposes a method for simultaneous clustering and outliers detection. Evolutionary game theory offers a whole class of simple dynamical systems to solve quadratic (constrained) optimization problems like ours. It envisages a scenario in which pairs of players are repeatedly drawn at random from a large population of individuals to play a symmetric two player game. One of the best known class of game dynamics to extract (constrained) dominant set from a graph is the so-called replicator dynamics whose computational complexity is quadratic per step which makes it handicapped for large-scale applications. In this thesis, we propose a fast algorithm, based on dynamics from evolutionary game theory, which is efficient and scalable to large-scale real-world applications. In general, the clustering algorithm proposed in this thesis has many interesting properties, suitable for solving many computer vision problems, such as: it does clustering while obliterating outliers in simultaneous fashion, it doesn’t need any a prior knowledge on the number of clusters, able to deal with compact clusters and with situations involving arbitrarily-shaped clusters in a context of heavy background noise, able to extract clusters which are constrained to contain user-specified elements, does not have any assumptions with the structure of the affinity matrix, it is fast and scalable to large scale problems, and others. This increased flexibility leads to an efficient method that we apply on different computer vision problems: interactive image segmentation, co-segmentation, multi-target multi-camera tracking in multiple non overlapping cameras, to extract dense neighbors which we have used to constrain the diffusion process locally, geo-localization and for person reidentification.