nils-herrmann/clean_scirepeval huggingface dataset:
Spectral Embedded Clustering: A Framework for In-Sample and Out-of-Sample Spectral Clustering

Spectral clustering (SC) methods have been successfully applied to many real-world applications. The success of these SC methods is largely based on the manifold assumption, namely, that two nearby data points in the high-density region of a low-dimensional data manifold have the same cluster label. However, such an assumption might not always hold on high-dimensional data. When the data do not exhibit a clear low-dimensional manifold structure (e.g., high-dimensional and sparse data), the clustering performance of SC will be degraded and become even worse than K -means clustering. In this paper, motivated by the observation that the true cluster assignment matrix for high-dimensional data can be always embedded in a linear space spanned by the data, we propose the spectral embedded clustering (SEC) framework, in which a linearity regularization is explicitly added into the objective function of SC methods. More importantly, the proposed SEC framework can naturally deal with out-of-sample data. We also present a new Laplacian matrix constructed from a local regression of each pattern and incorporate it into our SEC framework to capture both local and global discriminative information for clustering. Comprehensive experiments on eight real-world high-dimensional datasets demonstrate the effectiveness and advantages of our SEC framework over existing SC methods and K-means-based clustering methods. Our SEC framework significantly outperforms SC using the Nyström algorithm on unseen data.
New boosting methods of Gaussian processes for regression

Feedforward neural networks are popular tools for nonlinear regression and classification problems. Gaussian Process (GP) can be viewed as an RBF neural network which have infinite number of hidden neurons. On regression problems, they can predict both the mean value and the variance of the given sample. Boosting is one of the most important recent developments in machine learning. Classification problems have dominated research on boosting to date. On the other hand, the application of boosting of regression has received less investigation. In this paper, we develop two boosting methods of GPs for regression according to the characteristic of them. We compare the performance of our ensembles with other boosting algorithms and find that our methods are more stable and essentially have less over-fitting problems than the other methods.
Linear Neighborhood Propagation and Its Applications

In this paper, a novel graph-based transductive classification approach, called linear neighborhood propagation, is proposed. The basic idea is to predict the label of a data point according to its neighbors in a linear way. This method can be cast into a second-order intrinsic Gaussian Markov random field framework. Its result corresponds to a solution to an approximate inhomogeneous biharmonic equation with Dirichlet boundary conditions. Different from existing approaches, our approach provides a novel graph structure construction method by introducing multiple-wise edges instead of pairwise edges, and presents an effective scheme to estimate the weights for such multiple-wise edges. To the best of our knowledge, these two contributions are novel for semi-supervised classification. The experimental results on image segmentation and transductive classification demonstrate the effectiveness and efficiency of the proposed approach.
Hand segmentation with metric learning superpixels

In this paper, we propose a novel method for hand segmentation. To improve the robustness to the wide range of hand appearances and illuminations, we segment the hand area with a superpixels based method instead of a general color model. With the exploitation of the distribution of hand pixels in color space, a distance metric learning stage is designed to promote the segmentation performance. This stage makes the points in hand areas more concentrate and pulls away from the points of background in color space. The comparisons with several widely used algorithms are made on both public available and our own datasets. The experimental results show the superior performance of our method.
Short-term traffic flow forecasting using Sampling Markov Chain method with incomplete data

Short-term traffic flow forecasting is an important problem in the research area of intelligent transportation system. In practical situations, flow data may be incomplete, that is, partially missing or unavailable, where few methods could implement forecasting successfully. A method called Sampling Markov Chain is proposed to deal with this circumstance. In this paper, the traffic flow is modeled as a high order Markov Chain; and the transition probability from one state to the other state is approximated by Gaussian Mixture Model (GMM) whose parameters are estimated with Competitive Expectation Maximum (CEM) algorithm. The incomplete data in forecasting the trend of Markov Chain is represented by enough points sampled using the idea of Monte Carlo integration. Experimental results show that the Sampling Markov Chain method is applicable and effective for short-term traffic flow forecasting in case of incomplete data.
Evolutionary mechanisms in self-organizing map

In this paper a new model of self-organizing neural networks is proposed, which has realized the ability to project data from high dimensional space into low dimensional space, as well as to facilitate the visual inspection in the inherent topological structure of the projected data. In this model, a series of evolutionary working mechanisms of neurons have been introduced to overcome the weakness of conventional learning algorithm. The empirical evidences show the proposed algorithm is adaptive and robust. This work can be viewed as a development of classic Kohonen self-organizing feature map.
Feature Extraction by Maximizing the Average Neighborhood Margin

A novel algorithm called Average Neighborhood Margin Maximization (ANMM) is proposed for supervised linear feature extraction. For each data point, ANMM aims at pulling the neighboring points with the same class label towards it as near as possible, while simultaneously pushing the neighboring points with different labels away from it as far as possible. We will show that features extracted from ANMM can separate the data from different classes well, and it avoids the small sample size problem existed in traditional Linear Discriminant Analysis (LDA). The kernelized (nonlinear) counterpart of ANMM is also established in this paper. Moreover, as in many computer vision applications the data are more naturally represented by higher order tensors (e.g. images and videos), we develop a tensorized (multilinear) form of ANMM, which can directly extract features from tensors. The experimental results of applying ANMM to face recognition are presented to show the effectiveness of our method.
Label Propagation through Linear Neighborhoods

In many practical data mining applications such as text classification, unlabeled training examples are readily available, but labeled ones are fairly expensive to obtain. Therefore, semi supervised learning algorithms have aroused considerable interests from the data mining and machine learning fields. In recent years, graph-based semi supervised learning has been becoming one of the most active research areas in the semi supervised learning community. In this paper, a novel graph-based semi supervised learning approach is proposed based on a linear neighborhood model, which assumes that each data point can be linearly reconstructed from its neighborhood. Our algorithm, named linear neighborhood propagation (LNP), can propagate the labels from the labeled points to the whole data set using these linear neighborhoods with sufficient smoothness. A theoretical analysis of the properties of LNP is presented in this paper. Furthermore, we also derive an easy way to extend LNP to out-of-sample data. Promising experimental results are presented for synthetic data, digit, and text classification tasks.
Mamdanian logic

In the framework of stratified fuzzy propositional logic F, the regular harmonious neighborhood structure determined by the Mamdanian approximate function Hgt is the typical and best paradigm of approximate reasoning neighborhood algorithms. The advantage of Mamdanian algorithm comes from the regular harmoniousness property of Hgt presented by us in this paper, which guarantees that when the input of the obtained new approximate knowledge is close enough to that of the standard knowledge, not only the approximate reasoning consequence of the new knowledge is adequately close to that of the standard knowledge, but also the new knowledge itself is close enough to the standard knowledge.
Music playlist prediction via detecting song moods

Modern internet technologies make people easily access millions of songs. However it also forms a huge barrier between customers and those songs people truely want, due to the difficulty to explore this large collections. In this paper, we propose a novel music playlist prediction algorithm to facilitate this process for users. This method captures the moods expressed by songs in playlist context and also models the nature of composing a playlist. With the help of captured moods, personalized predictions can be achieved. We offer two ways to represent these hidden song moods and the transitions between them. The empirical evaluations show that our method outperforms other state-of-the-art methods in terms of perplexity.
Localized content based image retrieval by multiple instance active learning

In this paper, we propose two general multiple instance active learning (MIAL) algorithms, multiple-instance active learning with a simple margin strategy (S-MIAL) and multiple- instance active learning with fisher information (F-MIAL), and apply them to the relevance feedback in localized content based image retrieval (LCBIR). S-MIAL considers the most ambiguous picture as the most valuable one, while F-MIAL can utilize the fisher information and analyze the value of the unlabeled pictures by assigning different labels to them. We show that F-MIAL can be integrated more naturally into the multiple instance learning scenario. In experiments, we will show their superior performances on some real-world image datasets.
Solve JIGSAW puzzle problems with Co-Normalization

Recently a graphic model method has been developed to reconstruct an image from a bag of square, non-overlapping patches. Markov Random Field (MRF) is used to model this type of jigsaw puzzle problem. In this model, each node represents a spatial position where patches are placed. Belief Propagation (BP) algorithm is used to solve the model. BP naturally fulfills the fact that each node should be assigned a patch with probability one. However, the fact that all patches should be equally used in the reconstructed image with probability one is neglected. As a result, there is little chance for some patches to be used in the reconstructed image. In this paper, we propose a Co-Normalization method to encourage equal treatment toward patches and nodes. Our method naturally yields a new update formula for the exclusivity term introduced by previous work. This term requires each patch to be used only once in the reconstructed image. Experiments show that our method preserves exclusivity term's requirement and adds a new requirement of equal usage of patches. Image reconstruction results show that our method solves the jigsaw puzzle problem much better.
Trace Ratio Problem Revisited

Dimensionality reduction is an important issue in many machine learning and pattern recognition applications, and the trace ratio (TR) problem is an optimization problem involved in many dimensionality reduction algorithms. Conventionally, the solution is approximated via generalized eigenvalue decomposition due to the difficulty of the original problem. However, prior works have indicated that it is more reasonable to solve it directly than via the conventional way. In this brief, we propose a theoretical overview of the global optimum solution to the TR problem via the equivalent trace difference problem. Eigenvalue perturbation theory is introduced to derive an efficient algorithm based on the Newton-Raphson method. Theoretical issues on the convergence and efficiency of our algorithm compared with prior literature are proposed, and are further supported by extensive empirical results.
Regularized tree partitioning and its application to unsupervised image segmentation.

In this paper, we propose regularized tree partitioning approaches. We study normalized cut (NCut) and average cut (ACut) criteria over a tree, forming two approaches: 1) normalized tree partitioning (NTP) and 2) average tree partitioning (ATP). We give the properties that result in an efficient algorithm for NTP and ATP. In addition, we present the relations between the solutions of NTP and ATP over the maximum weight spanning tree of a graph and NCut and ACut over this graph. To demonstrate the effectiveness of the proposed approaches, we show its application to image segmentation over the Berkeley image segmentation data set and present qualitative and quantitative comparisons with state-of-the-art methods.
Special Article Series on Signal Processing Education via Hands-On and Design Projects [From the Guest Editors]

As professionals in the signal and information processing field, we build on the tools and theories we learned in our undergraduate studies, adding knowledge and skills over the years. For many of us, it has been a while since our undergraduate studies, yet we can probably recall some of these “Ahha” moments when a full meaning of some fundamental theory sank in. As educators and mentors, we seize on these moments and our own experiences to implement new successful teaching methods that advance effective learning and skill development.
Unsupervised Maximum Margin Feature Selection with manifold regularization

Feature selection plays a fundamental role in many pattern recognition problems. However, most efforts have been focused on the supervised scenario, while unsupervised feature selection remains as a rarely touched research topic. In this paper, we propose manifold-based maximum margin feature selection (M3FS) to select the most discriminative features for clustering. M3FS targets to find those features that would result in the maximal separation of different clusters and incorporates manifold information by enforcing smoothness constraint on the clustering function. Specifically, we define scale factor for each feature to measure its relevance to clustering, and irrelevant features are identified by assigning zero weights. Feature selection is then achieved by the sparsity constraints on scale factors. Computationally, M3FS is formulated as an integer programming problem and we propose a cutting plane algorithm to efficiently solve it. Experimental results on both toy and real-world data sets demonstrate its effectiveness.
Fast Mining of Complex Spatial Co-location Patterns Using GLIMIT

Most algorithms for mining interesting spatial co- locations integrate the co-location / clique generation task with the interesting pattern mining task, and are usually based on the Apriori algorithm. This has two downsides. First, it makes it difficult to meaningfully include certain types of complex relationships ­ especially negative rela- tionships ­ in the patterns. Secondly, the Apriori algorithm is slow. In this paper, we consider maximal cliques ­ cliques that are not contained in any other clique. We use these to extract complex maximal cliques and subsequently mine these for interesting sets of object types (including complex types). That is, we mine interesting complex relationships. We show that applying the GLIMIT itemset mining algo- rithm to this task leads to far superior performance than using an Apriori style approach.
Deep ranking: Triplet MatchNet for music metric learning

Metric learning for music is an important problem for many music information retrieval (MIR) applications such as music generation, analysis, retrieval, classification and recommendation. Traditional music metrics are mostly defined on linear transformations of handcrafted audio features, and may be improper in many situations given the large variety of music styles and instrumentations. In this paper, we propose a deep neural network named Triplet MatchNet to learn metrics directly from raw audio signals of triplets of music excerpts with human-annotated relative similarity in a supervised fashion. It has the advantage of learning highly nonlinear feature representations and metrics in this end-to-end architecture. Experiments on a widely used music similarity measure dataset show that our method significantly outperforms three state-of-the-art music metric learning methods. Experiments also show that the learned features better preserve the partial orders of the relative similarity than handcrafted features.
Learning No-Reference Quality Metric by Examples

In this paper, a novel learning based method is proposed for No-Reference image quality assessment. Instead of examining the exact prior knowledge for the given type of distortion and finding a suitable way to represent it, our method aims to directly get the quality metric by means of learning. At first, some training examples are prepared for both high-quality and low-quality classes; then a binary classifier is built on the training set; finally the quality metric of an un-labeled example is denoted by the extent to which it belongs to these two classes. Different schemes to acquire examples from a given image, to build the binary classifier and to model the quality metric are proposed and investigated. While most existing methods are tailored for some specific distortion type, the proposed method might provide a general solution for No-Reference image quality assessment. Experimental results on JPEG and JPEG2000 compressed images validate the effectiveness of the proposed method.
Learning with only multiple instance positive bags

In traditional multiple instance learning (MIL), both positive and negative bags are required to learn a prediction function. However, a high human cost is needed to know the label of each bag-positive or negative. Only positive bags contain our focus (positive instances) while negative bags consist of noise or background (negative instances). So we do not expect to spend too much to label the negative bags. Contrary to our expectation, nearly all existing MIL methods require enough negative bags besides positive ones. In this paper we propose an algorithm called “Positive Multiple Instance” (PMI), which learns a classifier given only a set of positive bags. So the annotation of negative bags becomes unnecessary in our method. PMI is constructed based on the assumption that the unknown positive instances in positive bags be similar each other and constitute one compact cluster in feature space and the negative instances locate outside this cluster. The experimental results demonstrate that PMI achieves the performances close to or a little worse than those of the traditional MIL algorithms on benchmark and real data sets. However, the number of training bags in PMI is reduced significantly compared with traditional MIL algorithms.
Poster paper: Predicting seizures from electroencephalography recordings: A knowledge transfer strategy

Epilepsy, a brain disorder afflicts nearly 1% of the world's population, is characterized by the occurrence of spontaneous seizures. For most epilepsy patients, the drugs are either not effective or produce severe side-effects. Seizure forecasting systems have the potential to help patients with epilepsy lead more normal lives. Recently multi-center clinical studies showed evidence of premonitory symptoms in 6.2% of 500 patients with epilepsy, and some interviews of epilepsy patients also found that a certain amount of patients felt "auras". All these are promising signs suggesting that seizure might be predictable. In this paper, we will study the application of deep learning techniques for seizure prediction with EEG signals. Deep learning methods have been shown to be very effective on exploring the latent structures from continuous signals and they have achieved state-of-the-art performance on speech analysis. One potential requirement for deep learning algorithms to work is a huge training set, which could be difficult for a specific medical problem. Therefore we specifically investigated a transfer learning strategy: we performed the major seizure prediction task on the data from American Epilepsy Society Seizure Prediction Challenge1, and we adopted another 6 publicly available EEG datasets2, which are not directly related to seizure prediction, as auxiliary information to pre-train the deep neural network for getting a good initial point. Our results show that with those auxiliary information, the prediction performance can be boosted. This observation is validated with different predictive models, which opens another gate for effective integration and utilization of medical data resources.
Learning probabilistic kernel feature subspace with side-information for classification

Kernel PCA is an efficient method for nonlinear feature extraction. We address two issues in kernel PCA based feature extraction and classification. First, it extracts features without utilizing sample label information. Second, it does not provide a practical means to choose the dimensionality for principal subspace. In this paper, one kind of side-information is incorporated into kernel PCA to solve the first problem. And a complete probabilistic density function is estimated in kernel space so that the choice of dimensionality for principal subspace becomes less important. The proposed model is named probabilistic kernel feature subspace (PKFS). Experiments show that it achieves promising performance and outperforms many other algorithms in classification.
Cost-sensitive boosting algorithms as gradient descent

AdaBoost is a well known boosting method for generating strong ensemble of weak base learners. The procedure of AdaBoost can be fitted in a gradient descent optimization framework, which is important for analyzing and devising its procedure. Cost sensitive boosting (CSB) is an emerging subject extending the boosting methods for cost sensitive classification applications. Most CSB methods are performed by directly modifying the original AdaBoost procedure. Unfortunately, the effectiveness of most cost sensitive boosting methods are checked only by experiments. It remains unclear whether these methods can be viewed as gradient descent procedures like AdaBoost. In this paper, we show that several typical CSB methods can also be view as gradient descent for minimizing a unified objective function. We then deduce a general greedy boosting procedure. Experimental results also validate the effectiveness of the proposed procedure.
A novel approach in off-line handwritten Chinese character stroke segmentation

In recognition of hand-written characters, stroke segmentation often serves as a crucial step. In this paper, we introduce a new method called manifold extraction to solve this problem. The basic idea of manifold extraction is: first build a neighborhood graph to capture the intrinsic topological structure of the sampled characters, then analyze the dimensional uniformity of neighboring points to discover the segments of strokes, finally combine the segments that are possibly from the same stroke and get the more informative structures of the characters. In this way, manifold extraction identifies the interlacing strokes in a complicated background and accomplishes the step of stroke segmentation. The experimental results show the effectiveness of this method in stroke segmentation as well as in exploratory data analysis.