Classification, Novelty Detection and Clustering for Point Pattern Data

A point pattern (PP) is a set or multi-set of unordered points, where each point is a (fixed-length) vector representing the state or features of the object of study. PP data are abundant in nature and applications. For example, locations of trees, positions of stars and galaxies, neurons in brain tissue can be considered as PPs. In machine learning, where PP data are more commonly known as bags or multiple instance data, many data sources are indeed PP data or can be represented by PPs, such as point cloud data, transactional data, text data (in bag-of-words representation), images (bag-of-visual-words), audio signals (bag-of-audio-words), and sparse data. Despite the abundance of PP data, the fundamental machine learning problems for PPs — including classification (supervised learning), novelty detection (semi-supervised learning), and clustering (unsupervised learning) — have not received much attention. In classification, of the three approaches for this learning task, namely the Instance-Space, Embedded-Space, and Bag-Space paradigm, only the Bag-Space paradigm solves PP classification at its fundamental level. The other approaches (i.e., Instance-Space, and Embedded-Space) attempt to convert the PP learning problem into a point (or vector) learning problem, hence the potential for information loss and additional computation. In clustering, only two algorithms have been developed for PP data, namely BAMIC and M3IC, but none of them are model-based, hence they lack the ability to exploit statistical trends in the data, not to mention computational problems with high dimensional inputs and large datasets. For novelty detection, there are no solutions for PP data in the literature. In this work, we solve three fundamental learning problems, namely classification, novelty detection, and clustering, for PP data using two approaches: one with knowledge of the underlying data model (model-based approach), vii and one without (distance-based approach). In particular, for model-based classification, we propose to model PP data using point processes and apply the maximum likelihood estimation method to learn the data model. For distancebased classification, we use the Optimal Sub-Pattern Assignment (OSPA) distance with the k-nearest neighbour (k-NN) algorithm to classify PP data. For model-based novelty detection, the likelihood evaluation of PP data is fundamentally different from that of point data. To this end, we propose a novel ranking function for PP data based on the IID-cluster point process density. This ranking function outperforms the naive Bayes likelihood as well as the IID-cluster point process probability density in PP novelty detection. For distance-based novelty detection, we propose a solution based on the set distance between the candidate PP and its nearest neighbour in the normal training set. For model-based clustering, we propose to model PP clusters by mixture of point processes. Then the expectation-maximization (EM) technique is applied to learn the mixture and cluster the data. For distance-based clustering, we combine the Affinity Propagation (AP) clustering algorithm with set distances as dissimilarity measures. The proposed learning methods perform well for both simulated and real data.