nils-herrmann/clean_scirepeval huggingface dataset:
Online calibration of floating gate detectors for RFID sensors

As the use of radio frequency identification (RFID) technology becomes widespread, one of the actively pursued research areas has been integration of smart sensors with RFID tags. These miniaturized devices when queried, transmit an identifier, subject to a specific condition of their environment. Applications of the sensors range from surveillance to biomedical systems. In this paper, we present a floating gate classifier that detects patterns of interest and enables or disables RF transmission on the tag. These classifiers consume minimal amount of energy and can be directly operated by scavenging power through inductive coupling. Applying online learning and calibration techniques to the classifier can compensate analog imperfections and mismatches in sensors. Results obtained from a prototype fabricated using standard 0.5mum CMOS process, are presented and the utility of learning on silicon is validated by demonstrated improvements in detection rates
Margin normalization and propagation in analog VLSI

We propose a margin propagation algorithm as an alternate decoding method to usual probability propagation algorithm. The decoding based on margins depends only on basic additions and subtractions and thus can be easily implemented in analog VLSI in a manner that is independent of the device characteristics. We present an analog VLSI implementation of the margin propagation network using CMOS transistors and provide experimental results obtained from a system fabricated in 0.5/spl mu/ CMOS process. Preliminary results indicate that margin based decoding is robust to input noise and parameter mismatches and can provide superior performance as compared to usual probability propagation.
Sub-Microwatt Analog VLSI Support Vector Machine for Pattern Classification and Sequence Estimation

An analog system-on-chip for kernel-based pattern classification and sequence estimation is presented. State transition probabilities conditioned on input data are generated by an integrated support vector machine. Dot product based kernels and support vector coefficients are implemented in analog programmable floating gate translinear circuits, and probabilities are propagated and normalized using sub-threshold current-mode circuits. A 14-input, 24-state, and 720-support vector forward decoding kernel machine is integrated on a 3mm × 3mm chip in 0.5μm CMOS technology. Experiments with the processor trained for speaker verification and phoneme sequence estimation demonstrate real-time recognition accuracy at par with floating-point software, at sub-microwatt power.
Three-decade programmable fully differential linear OTA

Acoustic and sonar analog signal processing applications require design operational transconductance amplifiers (OTAs) that can be configured over wide frequency range in multiple bands and yet achieve low power consumption and low harmonic distortion. A fully differential, linear OTA is presented with digitally programmable transconductance ranging over three-decades of dynamic range. Measurements from a prototype fabricated in a 0.5 /spl mu/m CMOS process demonstrate a 0.4 nA/V to 0.8 /spl mu/A/V transconductance range, 40 dB common-mode rejection ratio (CMRR),and -48 dB third-order harmonic distortion, at 12 /spl mu/W power dissipation.
Analog auditory perception model for robust speech recognition

An auditory perception model for noise-robust speech feature extraction is presented. The model assumes continuous-time filtering and rectification, amenable to real-time, low-power analog VLSI implementation. A 3 mm/spl times/3 mm CMOS chip in 0.5 /spl mu/m CMOS technology implements the general form of the model with digitally programmable filter parameters. Experiments on the TI-DIGIT database demonstrate consistent robustness of the new features to noise of various statistics, yielding significant improvements in digit recognition accuracy over models identically trained using Mel-scale frequency cepstral coefficient (MFCC) features.
Robust cephalometric landmark identification using support vector machines

A robust and accurate image recognizer for cephalometric landmarking is presented. The recognizer uses Gini support vector machine (SVM) to model discrimination boundaries between different landmarks and also between the background frames. Large margin classification with non-linear kernels allows to extract relevant details from the landmarks, approaching human expert levels of recognition. In conjunction with projected principal-edge distribution (PPED) representation as feature vectors, GiniSVM. is able to demonstrate more than 95% accuracy for landmark detection on medical cephalograms within a reasonable location tolerance value.
Silicon Support Vector Machine with On-Line Learning

Training of support vector machines (SVMs) amounts to solving a quadratic programming problem over the training data. We present a simple on-line SVM training algorithm of complexity approximately linear in the number of training vectors, and linear in the number of support vectors. The algorithm implements an on-line variant of sequential minimum optimization (SMO) that avoids the need for adjusting select pairs of training coefficients by adjusting the bias term along with the coefficient of the currently presented training vector. The coefficient assignment is a function of the margin returned by the SVM classifier prior to assignment, subject to inequality constraints. The training scheme lends efficiently to dedicated SVM hardware for real-time pattern recognition, implemented using resources already provided for run-time operation. Performance gains are illustrated using the Kerneltron, a massively parallel mixed-signal VLSI processor for kernel-based real-time video recognition.
Support vector machines for segmental minimum Bayes risk decoding of continuous speech

Segmental minimum Bayes risk (SMBR) decoding involves the refinement of the search space into sequences of small sets of confusable words. We describe the application of support vector machines (SVMs) as discriminative models for the refined search spaces. We show that SVMs, which in their basic formulation are binary classifiers of fixed dimensional observations, can be used for continuous speech recognition. We also study the use of GiniSVMs, which is a variant of the basic SVM. On a small vocabulary task, we show this two pass scheme outperforms MMI (maximum mutual information) trained HMMs. Using system combination we also obtain further improvements over discriminatively trained HMMs.
Power dissipation limits and large margin in wireless sensors

Wireless smart sensors impose severe power constraints that call for power budget optimization at all levels in the design hierarchy. We elucidate a connection between statistical learning theory and rate distortion theory that allows to operate a wireless sensor array at fundamental limits of power dissipation. GiniSVM, a support vector machine kernel-based classifier based on quadratic entropy, is shown to encode the sensor data with maximum fidelity for a given constraint on transmission budget. The transmission power is minimized by GiniSVM in the form of a quadratic cost function under linear constraints. A classifier architecture that implements these principles is presented.
Robust cephalometric landmark identification using support vector machines

A robust and accurate image recognizer for cephalometric landmarking is presented. The recognizer uses Gini support vector machine (SVM) to model discrimination boundaries between different landmarks and also between the background frames. Large margin classification with non-linear kernels allows to extract relevant details from the landmarks, approaching human expert levels of recognition. In conjunction with projected principal-edge distribution (PPED) representation as feature vectors, GiniSVM is able to demonstrate more than 95% accuracy for landmark detection on medical cephalograms within a reasonable location tolerance value.
MARGIN PROPAGATION AND FORWARD DECODING IN ANALOG VLSI

We propose margin propagation as an alternative to probability propagation in forward decoding. In contrast to sumproduct probability propagation, margin propagation only incurs addition and subtraction in the computation and thus leads to reduced complexity of implementation. Simulations indicate that margin based forward decoding is more robust to input noise and parameter mismatch than sumproduct probability decoding, and offers superior decoding performance. We also present an analog VLSI implementation of the margin propagation network independent of MOS device models, and provide experimental results from a prototype fabricated in a process.
Sparse Probability Regression by Label Partitioning

A large-margin learning machine for sparse probability regression is presented. Unlike support vector machines and other forms of kernel machines, nonlinear features are obtained by transforming labels into higher-dimensional label space rather than transforming data vectors into feature space. Linear multi-class logistic regression with partitioned classes of labels yields a nonlinear classifier in the original labels. With a linear kernel in data space, storage and run-time requirements are reduced from the number of support vectors to the number of partitioned labels. Using the partitioning property of KL-divergence in label space, an iterative alignment procedure produces sparse training coefficients. Experiments show that label partitioning is effective in modeling non-linear decision boundaries with same, and in some cases superior, generalization performance to Support Vector Machines with significantly reduced memory and run-time requirements.
Forward-decoding kernel-based phone sequence recognition

Forward decoding kernel machines (FDKM) combine large-margin classifiers with hidden Markov models (HMM) for maximum a posteriori (MAP) adaptive sequence estimation. State transitions in the sequence are conditioned on observed data using a kernel-based probability model trained with a recursive scheme that deals effectively with noisy and partially labeled data. Training over very large datasets is accomplished using a sparse probabilistic support vector machine (SVM) model based on quadratic entropy, and an on-line stochastic steepest descent algorithm. For speaker-independent continuous phone recognition, FDKM trained over 177,080 samples of the TIMIT database achieves 80.6% recognition accuracy over the full test set. without use of a prior phonetic language model.
Sequence estimation and channel equalization using forward decoding kernel machines

A forward decoding approach to kernel machine learning is presented. The method combines concepts from Markovian dynamics, large margin classifiers and reproducing kernels for robust sequence detection by learning inter-data dependencies. A MAP (maximum a posteriori) sequence estimator is obtained by regressing transition probabilities between symbols as a function of received data. The training procedure involves maximizing a lower bound of a regularized cross-entropy on the posterior probabilities, which simplifies into direct estimation of transition probabilities using kernel logistic regression. Applied to channel equalization, forward decoding kernel machines outperform support vector machines and other techniques by about 5dB in SNR for given BER, within 1 dB of theoretical limits.
Forward-Decoding Kernel-Based Phone Recognition

Forward decoding kernel machines (FDKM) combine large-margin classifiers with hidden Markov models (HMM) for maximum a posteriori (MAP) adaptive sequence estimation. State transitions in the sequence are conditioned on observed data using a kernel-based probability model trained with a recursive scheme that deals effectively with noisy and partially labeled data. Training over very large data sets is accomplished using a sparse probabilistic support vector machine (SVM) model based on quadratic entropy, and an on-line stochastic steepest descent algorithm. For speaker-independent continuous phone recognition, FDKM trained over 177 ,080 samples of the TlMIT database achieves 80.6% recognition accuracy over the full test set, without use of a prior phonetic language model.
Forward Decoding Kernel Machines: A Hybrid HMM/SVM Approach to Sequence Recognition

Forward Decoding Kernel Machines (FDKM) combine largemargin classifiers with Hidden Markov Models (HMM) for Maximum a Posteriori (MAP) adaptive sequence estimation. State transitions in the sequence are conditioned on observed data using a kernel-based probability model, and forward decoding of the state transition probabilities with the sum-product algorithm directly produces the MAP sequence. The parameters in the probabilistic model are trained using a recursive scheme that maximizes a lower bound on the regularized cross-entropy. The recursion performs an expectation step on the outgoing state of the transition probability model, using the posterior probabilities produced by the previous maximization step. Similar to Expectation-Maximization (EM), the FDKM recursion deals effectively with noisy and partially labeled data.We also introduce a multi-class support vector machine for sparse conditional probability regression, GiniSVM based on a quadratic formulation of entropy. Experiments with benchmark classification data show that GiniSVM generalizes better than other multi-class SVM techniques. In conjunction with FDKM, GiniSVM produces a sparse kernel expansion of state transition probabilities, with drastically fewer non-zero coefficients than kernel logistic regression. Preliminary evaluation of FDKM with GiniSVM on a subset of the TIMIT speech database reveals significant improvements in phoneme recognition accuracy over other SVM and HMM techniques.
Hybrid support vector machine/hidden Markov model approach for continuous speech recognition

A hybrid support vector machine (SVM) and hidden Markov model (HMM) approach is proposed for designing continuous speech recognition systems. Using novel properties of SVMs and combining them with HMMs one can obtain models that map easily to hardware and lead to more modular and scalable design. The overall architecture of the proposed system is based on the MAP (maximum a posteriori) framework which offers a direct, feedforward recognition model. The SVMs generate smooth estimates of local transition probabilities in the HMM, conditioned on the acoustic inputs. The transition probabilities are then used to estimate the global posterior probabilities of HMM state sequences. A parallel architecture that implements a simple speech recognition model in real-time is presented.
Adaptive Gaussian Kernel SVMs

We consider binary classification using Support Vector Machines with Gaussian kernels: KΣ(xi, xj) = e −(xi−xj)′Σ−1(xi−xj) and address the problem of selecting a covariance matrix Σ which gives good classification performance. As with other nonparametric classification methods based on distances between data points (such as nearest neighbor and Parzen methods), the choice of distance function (Σ) can be crucial to the success of learning. A good choice of Σ accounts for differences in the scales of different features (coordinates of the input vectors x), can remove global correlations between features and can adapt to the fact that some features may be much more informative about the class labels than others. We suggest an approach which ties training the SVM and choosing the distance function, as opposed to the more standard practice of feature normalization as a separate pre-processing step.
Loss Functions for Preference Levels : Regression with Discrete Ordered Labels

We consider different types of loss functions for discrete ordinal regression, i.e. fitting labels that may take one of several discrete, but ordered, values. These types of labels arise when preferences are specified by selecting, for each item, one of several rating “levels”, e.g. one through five stars. We present two general threshold-based constructions which can be used to generalize loss functions for binary labels, such as the logistic and hinge loss, and another generalization of the logistic loss based on a probabilistic model for discrete ordered labels. Experiments on the 1 Million MovieLens data set indicate that one of our construction is a significant improvement over previous classificationand regression-based approaches.
Fast maximum margin matrix factorization for collaborative prediction

Maximum Margin Matrix Factorization (MMMF) was recently suggested (Srebro et al., 2005) as a convex, infinite dimensional alternative to low-rank approximations and standard factor models. MMMF can be formulated as a semi-definite programming (SDP) and learned using standard SDP solvers. However, current SDP solvers can only handle MMMF problems on matrices of dimensionality up to a few hundred. Here, we investigate a direct gradient-based optimization method for MMMF and demonstrate it on large collaborative prediction problems. We compare against results obtained by Marlin (2004) and find that MMMF substantially outperforms all nine methods he tested.
When is Clustering Hard ?

of Work in Progress PASCAL Workshop on Statistics and Optimization of Clustering, July 2005 We propose questions regarding the informational and computational limits of learning a mixture of Gaussians—the sample sizes necessary in order to recover the generating mixture with unbounded, and bounded, computation. We would like to quantify the gap between the two, i.e. the excess information required for tractable estimation. We report on an empirical study attempting to shed light on these
Time-Varying Topic Models using Dependent Dirichlet Processes

We lay the ground for extending Dirichlet Processes based clustering and factor models to explicitly include variability as a function of time (or other known covariates) by integrating a Dependent Dirichlet Processes into existing hierarchical topic models. Time-Varying Topic Models using Dependent Dirichlet Processes Nathan Srebro Sam Roweis Dept. of Computer Science, University of Toronto, Canada {nati,roweis}@cs.toronto.edu Abstract We lay the ground for extending Dirichlet Processes based clustering and factor models to explicitly include variability as a function of time (or other known covariates) by integrating a Dependent Dirichlet Processes into existing hierarchical topic models.We lay the ground for extending Dirichlet Processes based clustering and factor models to explicitly include variability as a function of time (or other known covariates) by integrating a Dependent Dirichlet Processes into existing hierarchical topic models.
Distribution of short paired duplications in mammalian genomes.

Mammalian genomes are densely populated with long duplicated sequences. In this paper, we demonstrate the existence of doublets, short duplications between 25 and 100 bp, distinct from previously described repeats. Each doublet is a pair of exact matches, separated by some distance. The distribution of these intermatch distances is strikingly nonrandom. An unexpectedly high number of doublets have matches either within 100 bp (adjacent) or at distances tightly concentrated approximately 1,000 bp apart (nearby). We focus our study on these proximate doublets. First, they tend to have both matches on the same strand. By comparing nearby doublets shared in human and chimpanzee, we can also see that these doublets seem to arise by an insertion event that produces a copy without markedly affecting the surrounding sequence. Most doublets in humans are shared with chimpanzee, but many new pairs arose after the divergence of the species. Doublets found in human but not chimpanzee are most often composed of almost tandem matches, whereas older doublets (found in both species) are more likely to have matches spaced by approximately 1 kb, indicating that the nearly tandem doublets may be more dynamic. The spacing of doublets is highly conserved. So far, we have found clearly recognizable doublets in the following genomes: Homo sapiens, Mus musculus, Arabidopsis thaliana, and Caenorhabditis elegans, indicating that the mechanism generating these doublets is widespread. A mechanism that generates short local duplications while conserving polarity could have a profound impact on the evolution of regulatory and protein-coding sequences.
Methods and Experiments With Bounded Tree-width Markov Networks

Markov trees generalize naturally to bounded tree-width Markov networks, on which exact computations can still be done efficiently. However, learning the maximum likelihood Markov network with tree-width greater than 1 is NP-hard, so we discuss a few algorithms for approximating the optimal Markov network. We present a set of methods for training a density estimator. Each method is specified by three arguments: tree-width, model scoring metric (maximum likelihood or minimum description length), and model representation (using one joint distribution or several class-conditional distributions). On these methods, we give empirical results on density estimation and classification tasks and explore the implications of these arguments.
Learning with matrix factorizations

Matrices that can be factored into a product of two simpler matrices can serve as a useful and often natural model in the analysis of tabulated or high-dimensional data. Models based on matrix factorization (Factor Analysis, PCA) have been extensively used in statistical analysis and machine learning for over a century, with many new formulations and models suggested in recent years (Latent Semantic Indexing, Aspect Models, Probabilistic PCA, Exponential PCA, Non-Negative Matrix Factorization and others). In this thesis we address several issues related to learning with matrix factorizations: we study the asymptotic behavior and generalization ability of existing methods, suggest new optimization methods, and present a novel maximum-margin high-dimensional matrix factorization formulation.
Maximum likelihood Markov networks : an algorithmic approach

We show how a graphical model learning problem can be presented as a purely combinatorial problem. This allows us to analyze the computational hardness of the learning problem, and devise global optimization algorithms with proven performance guarantees. Markov networks are a class of graphical models that use an undirected graph to capture dependency information among random variables. Of particular interest are Markov networks over low treewidth graphs, under which many operations are tractable. We study the problem of finding a maximum likelihood distribution among Markov networks over graphs of bounded treewidth. We define the maximum hypertree problem, which is the problem of finding an acyclic hypergraph of bounded width, that maximizes the weight of hyperedges it covers (or equivalently, a triangulated graph of bounded clique size maximizing the weight of its cliques). We show that the maximum likelihood Markov network problem can be formulated as a maximum hypertree problem, and in fact the two problems are equivalent. This extends the work of Chow and Liu (1968) who considered the case where k = 1 (i.e. trees). We show that the maximum hypertree problem is NP-hard even for k = 2 and give the first constant factor approximation algorithm for it. More precisely, for any fixed treewidth objective k, we find a k-hypertree with an f(k) fraction of the maximum possible weight of any k-hypertree graph.  Associate Professor