nils-herrmann/clean_scirepeval huggingface dataset:
WebSuite: A Tool Suite for Harnessing Web Data

We present a system for searching, collecting, and integrating Web-resident data. The system consists of five tools, where each tool provides a specific functionality aimed at solving one aspect of the complex task of using and managing Web data. Each tool can be used in a stand-alone mode, in combination with the other tools, or even in conjunction with other systems. Together, the tools offer a wide range of capabilities that overcome many of the limitations in existing systems for harnessing Web data. The paper describes each tool, possible ways of combining the tools, and the architecture of the combined system.
Multi-Electrode Spike Sorting by Clustering Transfer Functions

A new paradigm is proposed for sorting spikes in multielectrode data using ratios of transfer functions between cells and electrodes. It is assumed that for every cell and electrode there is a stable linear relation. These are dictated by the properties of the tissue, the electrodes and their relative geometries. The main advantage of the method is that it is insensitive to variations in the shape and amplitude of a spike. Spike sorting is carried out in two separate steps. First, templates describing the statistics of each spike type are generated by clustering transfer function ratios then spikes are detected in the data using the spike statistics. These techniques were applied to data generated in the escape response system of the cockroach.
Agnostic Classi cation of MarkovianSequencesRan

Classiication of nite sequences without explicit knowledge of their statistical nature is a fundamental problem with many important applications. We propose a new information theoretic approach to this problem which is based on the following ingredients: (i) sequences are similar when they are likely to be generated by the same source; (ii) cross entropies can be estimated via \universal compression"; (iii) Markovian sequences can be optimally merged. With these ingredients we can classify discrete sequences whenever they can be compressed. We introduce the method and illustrate its application for hierarchical clustering of languages and for estimating similarities of protein sequences.
A Map of the Protein Space: An Automatic Hierarchical Classification of all Protein Sequences

We investigate the space of all protein sequences. We combine the standard measures of similarity (SW, FASTA, BLAST), to associate with each sequence an exhaustive list of neighboring sequences. These lists induce a (weighted directed) graph whose vertices are the sequences. The weight of an edge connecting two sequences represents their degree of similarity. This graph encodes much of the fundamental properties of the sequence space. We look for clusters of related proteins in this graph. These clusters correspond to strongly connected sets of vertices. Two main ideas underlie our work: i) Interesting homologies among proteins can be deduced by transitivity. ii) Transitivity should be applied restrictively in order to prevent unrelated proteins from clustering together. Our analysis starts from a very conservative classification, based on very significant similarities, that has many classes. Subsequently, classes are merged to include less significant similarities. Merging is performed via a novel two phase algorithm. First, the algorithm identifies groups of possibly related clusters (based on transitivity and strong connectivity) using local considerations, and merges them. Then, a global test is applied to identify nuclei of strong relationships within these groups of clusters, and the classification is refined accordingly. This process takes place at varying thresholds of statistical significance, where at each step the algorithm is applied on the classes of the previous classification, to obtain the next one, at the more permissive threshold. Consequently, a hierarchical organization of all proteins is obtained. The resulting classification splits the space of all protein sequences into well defined groups of proteins. The results show that the automatically induced sets of proteins are closely correlated with natural biological families and super families. The hierarchical organization reveals finer sub-families that make up known families of proteins as well as many interesting relations between protein families. The hierarchical organization proposed may be considered as the first map of the space of all protein sequences. An interactive web site including the results of our analysis has been constructed, and is now accessible through http:/(/)www.protomap.cs.huji.ac.il
Agnostic Classiication of Markovian Sequences

Classiication of nite sequences without explicit knowledge of their statistical nature is a fundamental problem with many important applications. We propose a new information theoretic approach to this problem which is based on the following ingredients: (i) sequences are similar when they are likely to be generated by the same source; (ii) cross entropies can be estimated via \universal compres-sion"; (iii) Markovian sequences can be asymptotically-optimally merged. With these ingredients we design a method for the classiication of discrete sequences whenever they can be compressed. We introduce the method and illustrate its application for hierarchical clustering of languages and for estimating similarities of protein sequences.
Synergy and Redundancy among Brain Cells of Behaving Monkeys

Determining the relationship between the activity of a single nerve cell to that of an entire population is a fundamental question that bears on the basic neural computation paradigms. In this paper we apply an information theoretic approach to quantify the level of cooperative activity among cells in a behavioral context. It is possible to discriminate between synergetic activity of the cells vs. redundant activity, depending on the difference between the information they provide when measured jointly and the information they provide independently. We define a synergy value that is positive in the first case and negative in the second and show that the synergy value can be measured by detecting the behavioral mode of the animal from simultaneously recorded activity of the cells. We observe that among cortical cells positive synergy can be found, while cells from the basal ganglia, active during the same task, do not exhibit similar synergetic activity.
Approximate Faithful Embedding in Learning

In this paper we consider the problem of embedding the input and hypotheses of boolean function classes in other classes, such that the natural metric structure of the two spaces is approximately preserved. We rst prove some general properties of such embedding and then suggest and discuss possible approximate embedding in the class of \half-spaces" (single layer perceptrons) with dimension polynomial in the VC dimension of the original problem. Our main result is that such an approximate embedding by half-spaces is possible for a class of problems, which we call \informative". These are homogeneous learnable concept classes for which the dual problem (learning the input from labels of random hypotheses) has a similar VC-dimension, and the variance of the generalization error is bounded from zero. This is a distribution dependent property which we need to hold for most distributions on instances and hypotheses. We argue that many important learning classes are \informative" for typical distributions, e.g. geometric concepts, neural networks, decision trees, etc.
Agnostic Classification of Markovian Sequences

Classification of finite sequences without explicit knowledge of their statistical nature is a fundamental problem with many important applications. We propose a new information theoretic approach to this problem which is based on the following ingredients: (i) sequences are similar when they are likely to be generated by the same source; (ii) cross entropies can be estimated via "universal compression"; (iii) Markovian sequences can be asymptotically-optimally merged. 
 
With these ingredients we design a method for the classification of discrete sequences whenever they can be compressed. We introduce the method and illustrate its application for hierarchical clustering of languages and for estimating similarities of protein sequences.
Polyspectra as measures of sound texture and timbre

In this work aperiodicities in the waveform of musical sounds, are investigated, with a focus on fluctuations (jitter) in frequency that occur during the stable (sustained) portion of the sound. The existence of statistical independence between fluctuations of different harmonics brings a sense of separability among the sound components, and this puts the problem in the border area between timbre and texture. In the paper it is shown that coherence of fluctuations is strongly related to nonlinear properties of the time series model of the sound. By application of higher order statistical (HOS, polyspectral) methods, we analyze and model this phenomena. Our main result is demonstrating the important correspondence between HOS features, and texture and timbre properties, specifically the classification into instrument families of string, woodwind, and brass instruments. It is shown that the appearance of higher order cumulants is directly related to the number of statistically dependent partiais, a...
Hidden Markov modelling of simultaneously recorded cells in the associative cortex of behaving monkeys

A widely held idea regarding information processing in the brain is the cell-assembly hypothesis suggested by Hebb in 1949. According to this hypothesis, the basic unit of information processing in the brain is an assembly of cells, which can act briefly as a closed system, in response to a specific stimulus. This work presents a novel method of characterizing this supposed activity using a hidden Markov model. This model is able to reveal some of the underlying cortical network activity of behavioural processes. In our study the process in hand was the simultaneous activity of several cells recorded from the frontal cortex of behaving monkeys. Using such a model we were able to identify the behavioural mode of the animal and directly identify the corresponding collective network activity. Furthermore, the segmentation of the data into the discrete states also provides direct evidence for the state dependence of the short-time correlation functions between the same pair of cells. Thus, this cross-correlat...
Analysis of sound textures in musical and machine sounds by means of higher order statistical features

In this paper we describe a sound classification method, which seems to be applicable to a broad domain of stationary, non-musical sounds, such as machine noises and other man made non-periodic sounds. The method is based on matching higher order spectra (HOS) of the acoustic signals and it generalizes our earlier results on classification of sustained musical sounds by higher order moments. An efficient "decorrelated matched filter" implemetation is presented. The results show good sound classification statistics and a comparison to spectral matching methods is also discussed.
Global self-organization of all known protein sequences reveals inherent biological signatures.

A global classification of all currently known protein sequences is performed. Every protein sequence is partitioned into segments of 50 amino acid residues and a dynamic programming distance is calculated between each pair of segments. This space of segments is initially embedded into Euclidean space. The algorithm that we apply embeds every finite metric space into Euclidean space so that (1) the dimension of the host space is small, (2) the metric distortion is small. A novel self-organized, cross-validated clustering algorithm is then applied to the embedded space with Euclidean distances. We monitor the validity of our clustering by randomly splitting the data into two parts and performing an hierarchical clustering algorithm independently on each part. At every level of the hierarchy we cross-validate the clusters in one part with the clusters in the other. The resulting hierarchical tree of clusters offers a new representation of protein sequences and families, which compares favorably with the most updated classifications based on functional and structural data about proteins. Some of the known families clustered into well distinct clusters. Motifs and domains such as the zinc finger, EF hand, homeobox, EGF-like and others are automatically correctly identified, and relations between protein families are revealed by examining the splits along the tree. This clustering leads to a novel representation of protein families, from which functional biological kinship of protein families can be deduced, as demonstrated for the transporter family. Finally, we introduce a new concise representation for complete proteins that is very useful in presenting multiple alignments, and in searching for close relatives in the database. The self-organization method presented is very general and applies to any data with a consistent and computable measure of similarity between data items.
Beyond Word N-Grams Fernando C. Pereira Yoram Singer AT&T Research

We describe, analyze, and evaluate experimentally a new probabilistic model for word-sequence prediction in natural language based on prediction suffix trees(PSTs). By using efficient data structures, we extend the notion of PST to unbounded vocabularies. We also show how to use a Bayesian approach based on recursive priors over all possible PSTs to efficiently maintain tree mixtures. These mixtures have provably and practically better performance than almost any single model. We evaluate the model on several corpora. The low perplexity achieved by relatively small PST mixture models suggests that they may be an advantageous alternative, both theoretically and practically, to the widely used n-gram models.
Innuence of Frequency Modulating Jitter on Higher Order Moments of Sound Residual with Applications to Synthesis and Classiication

In this paper we provide a simple model for musical sounds that accounts for timbre properties due to mi-croouctuations in the harmonics of the signal. When considering a sound model that consists of an excita-tion signal passing through a resonator lter, we nd, by means of higher order statistical analysis of the ex-citation, a grouping of sounds according to common instrumental families of string, woodwind and brass sounds. For resynthesis purposes we model the exci-tation by a family of stochastic, pulse train like functions whose statistical properties resemble those found in real signals. By introducing an idea of \eeective number of harmonics" that represents the number of coupled, or statistically dependent harmonics among the complete set of partials present in the signal, we show that this number can be calculated directly form the 3rd and 4th moments of the residual. Musically speaking we suggest that microouctuations administer a sense of texture within timbre and these texture properties depend upon the concurrence/non concurrence parameter of the random frequency deviations caused by the jitter.
Testing for Gaussianity and Non Linearity in the sustained portion of musical sounds

Higher order spectra of a signal contain information about the non Gaussian and non Linear properties of the system that created it. Since the non linearity in musical signal usually originate in the excitation signal while the linear spectral characteristics are attributed to the resonant chambers, we discard the spectral information by looking at the higher order statistical properties of the residual signal, i.e. the estimated input signal obtained by inverse ltering of the sound. In the current paper we show that the skewness and kurtosis values of the residual could be used for characterization of such important sound properties as belonging to families of strings, woodwind and brass instrumental timbres. The skewness parameter is shown to be closely related to the bicoherence function calculated over the original signal and as such it is succinct to an interpretation as statistical test for the signal conforming to a linear non Gaussian model. The above results are compared to the Hinich bispectral tests for Gaussianity and non Linearity of time series and exhibit a similar classiication results. Finally, regarding the higher order statistics of a signal as a feature vector, a statistical distance measure for the cumulant space is suggested.
Rigorous learning curve bounds from statistical mechanics

In this paper we introduce and investigate a mathematically rigorous theory of learning curves that is based on ideas from statistical mechanics. The advantage of our theory over the well-established Vapnik-Chervonenkis theory is that our bounds can be considerably tighter in many cases, and are also more reflective of the true behavior of learning curves. This behavior can often exhibit dramatic properties such as phase transitions, as well as power law asymptotics not explained by the VC theory. The disadvantages of our theory are that its application requires knowledge of the input distribution, and it is limited so far to finite cardinality function classes.We illustrate our results with many concrete examples of learning curve bounds derived from our theory.
Hearing beyond the spectrum

In this work we focus on the problem of acoustic signals modeling and analysis, with particular interest in models that can capture the timbre of musical sounds. Traditional methods usually relate to several “dimensions” which represent the spectral properties of the signal and their change in time. Here we confine ourselves to the stationary portion of the sound signal, the analysis of which is generalized by incorporating polyspectral techniques. We suggest that by looking at the higher order statistics of the signal we obtain additional information not present in the standard autocorrelation or its Fourier related power‐spectra. It is shown that over the bispectral plane several acoustically meaningful measures could be devised, which are sensitive to properties such as harmonicity and phase coherence among the harmonics. Effects such as reverberation and chorusing are demonstrated to be clearly detected by the above measures. In the second part of the paper we perform an information theoretic...
Learning blind source separation

The problem of recovering signals from linear mixtures, with only partial knowledge of the mixing process and the signals---a problem often referred to as blind source separation---is a central problem in signal processing. It has applications in many fields, including speech processing, network tomography and biomedical imaging. 
Current algorithms can only be applied successfully with specific assumptions regarding the mixing process and the underlying sources. In this thesis, we proposed three sets of algorithms that allow to relax some of those assumptions, using tools from machine learning. 
In the situation where sources are assumed statistically independent and there are as many sources as sensors, a problem usually referred to as independent component analysis (ICA), we propose an algorithm that is robust to source densities, that make use of a measure of independence based on reproducing kernel Hilbert spaces. 
We also propose an algorithm to relax the assumption of independence made by most ICA algorithms, by using statistical dependencies based on probabilistic graphical models. By using tree-structured graphical models, we are able to estimate jointly the mixing process parameters and the pattern of dependencies among the underlying sources. 
Finally, we present an algorithm dedicated to blind separation of speech signals from only one sensor. We formulate the problem of speech separation as a problem in segmenting the spectrogram of the signal into two or more disjoint sets. We then build feature sets for our segmenter using classical cues from speech psychophysics. By taking advantage of the fact that we can generate training examples for segmentation by artificially superposing separately-recorded signals, we can learn parameters of similarity matrices that can be used for spectral clustering.
Discriminative training of hidden Markov models for multiple pitch tracking [speech processing examples]

We present a multiple pitch tracking algorithm that is based on direct probabilistic modeling of the spectrogram of the signal. The model is a factorial hidden Markov model whose parameters are learned discriminatively from the Keele pitch database. Our algorithm can track several pitches and determines the number of pitches that are active at any given time. We present simulation results on mixtures of several speech signals and noise, showing the robustness of our approach.
Predictive low-rank decomposition for kernel methods

Low-rank matrix decompositions are essential tools in the application of kernel methods to large-scale learning problems. These decompositions have generally been treated as black boxes---the decomposition of the kernel matrix that they deliver is independent of the specific learning task at hand---and this is a potentially significant source of inefficiency. In this paper, we present an algorithm that can exploit side information (e.g., classification labels, regression responses) in the computation of low-rank decompositions for kernel matrices. Our algorithm has the same favorable scaling as state-of-the-art methods such as incomplete Cholesky decomposition---it is linear in the number of data points and quadratic in the rank of the approximation. We present simulation results that show that our algorithm yields decompositions of significantly smaller rank than those found by incomplete Cholesky decomposition.
Computing regularization paths for learning multiple kernels

The problem of learning a sparse conic combination of kernel functions or kernel matrices for classification or regression can be achieved via the regularization by a block 1-norm . In this paper, we present an algorithm that computes the entire regularization path for these problems. The path is obtained by using numerical continuation techniques, and involves a running time complexity that is a constant times the complexity of solving the problem for one value of the regularization parameter. Working in the setting of kernel linear regression and kernel logistic regression, we show empirically that the effect of the block 1-norm regularization differs notably from the (non-block) 1-norm regularization commonly used for variable selection, and that the regularization path is of particular value in the block case.
Learning graphical models for stationary time series

Probabilistic graphical models can be extended to time series by considering probabilistic dependencies between entire time series. For stationary Gaussian time series, the graphical model semantics can be expressed naturally in the frequency domain, leading to interesting families of structured time series models that are complementary to families defined in the time domain. In this paper, we present an algorithm to learn the structure from data for directed graphical models for stationary Gaussian time series. We describe an algorithm for efficient forecasting for stationary Gaussian time series whose spectral densities factorize in a graphical model. We also explore the relationships between graphical model structure and sparsity, comparing and contrasting the notions of sparsity in the time domain and the frequency domain. Finally, we show how to make use of Mercer kernels in this setting, allowing our ideas to be extended to nonlinear models.