nils-herrmann/clean_scirepeval huggingface dataset:
Weak Learners and Improved Rates of Convergence in Boosting

The problem of constructing weak classifiers for boosting algorithms is studied. We present an algorithm that produces a linear classifier that is guaranteed to achieve an error better than random guessing for any distribution on the data. While this weak learner is not useful for learning in general, we show that under reasonable conditions on the distribution it yields an effective weak learner for one-dimensional problems. Preliminary simulations suggest that similar behavior can be expected in higher dimensions, a result which is corroborated by some recent theoretical bounds. Additionally, we provide improved convergence rate bounds for the generalization error in situations where the empirical error can be made small, which is exactly the situation that occurs if weak learners with guaranteed performance that is better than random guessing can be established.
Regret Minimization in Signal Space for Repeated Matrix Games with Partial Observations

The Bayes Envelope of a repeated matrix game traces the maximal payoo rate that a player (say P1) could secure for herself had she known in advance the empirical frequency of P2's actions. It was shown by J. Hannan (1957) that Regret Minimizing policies for P1 exist which asymptotically attain the Bayes Envelope even without such prior knowledge, but assuming a complete observation of the P2's action at each stage. Recently, these results have been extended to the case where the other player's action is not fully observed but rather some signal that is related to it. In this paper we consider a simpliied concept of the Bayesian envelope for the incomplete observation case, which is constrained to rely only on the relative frequencies of the observed signals and rewards. An important class of games for which this type of constraint seems to arise naturally is when the stage game itself involves sequential moves, so that P2's normal-form action actually corresponds to an extensive form strategy, and the unobserved part of this \action" actually corresponds to decisions that have never materialized. We ooer a solution which naturally extends the Bayes envelope concept in the reward-signals space by taking worst-case view over missing information, and show that taking the convex hull of the resulting envelope is required in order to obtain an achievable envelope. A detailed study of a speciic (Stackelberg-type) stage game in which P1 moves before P2 is used to highlight several basic properties of the proposed envelope. The analysis relies on appropriate applications of Blackwell's approachability theory.
Moment Kernels for Regular Distributions

Many machine learning problems in natural language processing, transaction-log analysis, or computational biology, require the analysis of variable-length sequences, or, more generally, distributions of variable-length sequences.Kernel methods introduced for fixed-size vectors have proven very successful in a variety of machine learning tasks. We recently introduced a new and general kernel framework, rational kernels, to extend these methods to the analysis of variable-length sequences or more generally distributions given by weighted automata. These kernels are efficient to compute and have been successfully used in applications such as spoken-dialog classification with Support Vector Machines.However, the rational kernels previously introduced in these applications do not fully encompass distributions over alternate sequences. They are based only on the counts of co-occurring subsequences averaged over the alternate paths without taking into accounts information about the higher-order moments of the distributions of these counts.In this paper, we introduce a new family of rational kernels, moment kernels, that precisely exploits this additional information. These kernels are distribution kernels based on moments of counts of strings. We describe efficient algorithms to compute moment kernels and apply them to several difficult spoken-dialog classification tasks. Our experiments show that using the second moment of the counts of n-gram sequences consistently improves the classification accuracy in these tasks.
Tutorial HandOuts – Weighted Finite-State Transducers in Computational Biology

Weighted finite-state transducers are used in many applications such as text, speech and image processing. This chapter gives an overview of several recent weighted transducer algorithms, including composition of weighted transducers, determinization of weighted automata, a weight pushing algorithm, and minimization of weighted automata. It briefly describes these algorithms, discusses their running time complexity and conditions of application, and shows examples illustrating their application.
The mnist database of handwritten digits

Disclosed is an improved articulated bar flail having shearing edges for efficiently shredding materials. An improved shredder cylinder is disclosed with a plurality of these flails circumferentially spaced and pivotally attached to the periphery of a rotatable shaft. Also disclosed is an improved shredder apparatus which has a pair of these shredder cylinders mounted to rotate about spaced parallel axes which cooperates with a conveyer apparatus which has a pair of inclined converging conveyer belts with one of the belts mounted to move with respect to the other belt to allow the transport of articles of various sizes therethrough.
Signature-Based Methods for Data Streams

We have been developing signature-based methods in the telecommunications industry for the past 5 years. In this paper, we describe our work as it evolved due to improvements in technology and our aggressive attitude toward scale. We discuss the types of features that our signatures contain, nuances of how these are updated through time, our treatment of outliers, and the trade-off between time-driven and event-driven processing. We provide a number of examples, all drawn from the application of signatures to toll fraud detection.
Linear and non linear models for ozone generators considering electrodes losses

Linear and non linear models of ozone generators are presented in this paper. The non linear model includes the losses in the electrodes and the linear model is obtained for nominal operation conditions of the ozone generator. The linear model is proposed as a tool that simplifies the design of power supplies for ozone generators and the non linear model is proposed as a tool to simulate the operation of ozone generators including the electrode losses. Simulation results of the standard non linear model, proposed non linear model and the linear model are presented with excellent results.
Distribution kernels based on moments of counts

Many applications in text and speech processing require the analysis of distributions of variable-length sequences. We recently introduced a general kernel framework, rational kernels, to extend kernel methods to the analysis of such variable-length sequences or more generally weighted automata. These kernels are efficient to compute and have been successfully used in applications such as spoken-dialog classification using Support Vector Machines.However, the rational kernels previously introduced do not fully encompass distributions over alternate sequences. Prior similarity measures between two weighted automata are based only on the expected counts of co-occurring subsequences and ignore similarities (or dissimilarities) in higher order moments of the distributions of these counts.In this paper, we introduce a new family of rational kernels, moment kernels, that precisely exploit this additional information. These kernels are distribution kernels based on moments of counts of strings. We describe efficient algorithms to compute moment kernels and apply them to several difficult spoken-dialog classification tasks. Our experiments show that using the second moment of the counts of n-gram sequences consistently improves the classification accuracy in these tasks.
Support-Vector Networks

The support-vector network is a new learning machine for two-group classification problems. The machine conceptually implements the following idea: input vectors are non-linearly mapped to a very high-dimension feature space. In this feature space a linear decision surface is constructed. Special properties of the decision surface ensures high generalization ability of the learning machine. The idea behind the support-vector network was previously implemented for the restricted case where the training data can be separated without errors. We here extend this result to non-separable training data.High generalization ability of support-vector networks utilizing polynomial input transformations is demonstrated. We also compare the performance of the support-vector network to various classical learning algorithms that all took part in a benchmark study of Optical Character Recognition.
Hancock: A language for analyzing transactional data streams

Massive transaction streams present a number of opportunities for data mining techniques. The transactions in such streams might represent calls on a telephone network, commercial credit card purchases, stock market trades, or HTTP requests to a web server. While historically such data have been collected for billing or security purposes, they are now being used to discover how the transactors, for example, credit-card numbers or IP addresses, use the associated services.Over the past 5 years, we have computed evolving profiles (called signatures) of transactors in several very large data streams. The signature for each transactor captures the salient features of his or her behavior through time. Programs for processing signatures must be highly optimized because of the size of the data stream (several gigabytes per day) and the number of signatures to maintain (hundreds of millions). Originally, we wrote such programs directly in C, but because these programs often sacrificed readability for performance, they were difficult to verify and maintain.Hancock is a domain-specific language we created to express computationally efficient signature programs cleanly. In this paper, we describe the obstacles to computing signatures from massive streams and explain how Hancock addresses these problems. For expository purposes, we present Hancock using a running example from the telecommunications industry; however, the language itself is general and applies equally well to other data sources.
Confidence Intervals for the Area Under the ROC Curve

In many applications, good ranking is a highly desirable performance for a classifier. The criterion commonly used to measure the ranking quality of a classification algorithm is the area under the ROC curve (AUC). To report it properly, it is crucial to determine an interval of confidence for its value. This paper provides confidence intervals for the AUC based on a statistical and combinatorial analysis using only simple parameters such as the error rate and the number of positive and negative examples. The analysis is distribution-independent, it makes no assumption about the distribution of the scores of negative or positive examples. The results are of practical use and can be viewed as the equivalent for AUC of the standard confidence intervals given in the case of the error rate. They are compared with previous approaches in several standard classification tasks demonstrating the benefits of our analysis.
AUC Optimization vs. Error Rate Minimization

The area under an ROC curve (AUC) is a criterion used in many applications to measure the quality of a classification algorithm. However, the objective function optimized in most of these algorithms is the error rate and not the AUC value. We give a detailed statistical analysis of the relationship between the AUC and the error rate, including the first exact expression of the expected value and the variance of the AUC for a fixed error rate. Our results show that the average AUC is monotonically increasing as a function of the classification accuracy, but that the standard deviation for uneven distributions and higher error rates is noticeable. Thus, algorithms designed to minimize the error rate may not lead to the best possible AUC values. We show that, under certain conditions, the global function optimized by the RankBoost algorithm is exactly the AUC. We report the results of our experiments with RankBoost in several datasets demonstrating the benefits of an algorithm specifically designed to globally optimize the AUC over other existing algorithms optimizing an approximation of the AUC or only locally optimizing the AUC.
Computational Methods for Dynamic Graphs

This article considers problems that can be characterized by large dynamic graphs. Communication networks provide the prototypical example of such problems where nodes in the graph are network IDs and the edges represent communication between pairs of network IDs. In such graphs, nodes and edges appear and disappear through time so that methods that apply to static graphs are not sufficient. Our definition of a dynamic graph is procedural. We introduce a data structure and an updating scheme that captures, in an approximate sense, the graph and its evolution through time. The data structure arises from a bottom-up representation of the large graph as the union of small subgraphs centered on every node. These subgraphs are interesting in their own right and can be enhanced to form what we call communities of interest (COI). We discuss an application in the area of telecommunications fraud detection to help motivate the ideas.
Air carrier liability and automation issues

Our intended topic is a general discussion of the basic elements of liability related to airline accidents to which fully automated cockpits have constituted an associated contributory factor. In addition we addressed the liability of air carriers arising from injuries or death caused to passengers traveling on international flights. For this purpose, we reviewed the Warsaw System and the different international instruments that constitute it. We also reviewed principles of common law applicable to aircraft manufacturers and the "Free Flight" as an example of the growing automation environment, which is a general benefit to commercial aviation but also a likely contributory cause for accidents in particular cases. In the last part we briefly discuss a personal view regarding the interplay between manufacturers and airlines under the 1999 Montreal Convention, which is an international treaty unifying the desegregated Warsaw System into one single instrument that is expected to enter into force in a few years.
Communities of interest

We consider problems that can be characterized by large dynamic graphs. Communication networks provide the prototypical example of such problems where nodes in the graph are network IDs and the edges represent communication between pairs of network IDs. In such graphs, nodes and edges appear and disappear through time so that methods that apply to static graphs are not sufficient. We introduce a data structure that captures, in an approximate sense, the graph and its evolution through time. The data structure arises from a bottom-up representation of the large graph as the union of small subgraphs, called Communities of Interest (COI), centered on every node. These subgraphs are interesting in their own right and we discuss two applications in the area of telecommunications fraud detection to help motivate the ideas.
Hancock: a language for extracting signatures from data streams

" # %$ '&( ) *+ , ( $ . / 0*1 32 4 , ,5 6, 87!$ 93:; , 5 6! ) ! < = = < > = 6, , 0 @?A B(C, / =D E 2 . F 2 $ 6G C ! EB 4 H B A 2 C JI0:J:JKL 7!$, M ' "?+ & E 9ANO6, = 6, =/=.PQ $ E6 2, 6, &( = = 2R*S & =/= ,5> J $ . @P ,$ ( C% 6, P ) ?T&( ,53$, 2 >2 +6 H?O $, A A 6, A % E 2 / <UV = = 2QWVX Y Z [EY \ ] W_^ XE[@`J$, 0 6, )$, G2 =.P / 5' % 9 a b E =!P! H C ?A +6G c $ 2 = 5) dG= JUV = =/ 2 [ e/f ZGY WVg%X hE[@` *0 6 R i j=8 H 5 k2 $, 5 6, ,2 ?J . #lm %2 9M:+6, F /5 G H $ n*1 A 6 + $ 6, = / % "*S $ " *J6 / , E / , 6 $,5 6o 9 Kp 5 k*1 ! ,5q 5 , $ o >$, k&( L6, 5 6,=.Pr .s 2m&( $, k * 6 Q /s t * 6, k2 u vUV =<5 .5 &!P% n ( <2, P,`+ ,2# 6, %$, >&( + *b 5 G H $ n 4 / ! / UV6%$ G2 2 ) *+ = = / , `E9 lw 5 ) R ,$ > 5 G $ *S o .d, 2o 2, &, = xPk*1 3 ( *S 4 , 9 yA 7!$, ! =.P!C 6 P# "2 z $,=. n .*8PR G2R 4 % ;9 I) ! EB{ R L2 4 / -_ ( .dG Q=/ 5 $, 5 k 2m L | ,$ , = =.P3 z ! p 5 , $ + 5 p =/ ,=.P!9~}@ " 6, , ( CJ?A t2 &( Q 6, Q &, = i L ,$ ,5 5 G $ *8  4 4 ' G2u | ,=/ / L6 ?I0 , ! Bu 2,2% 6 &,= 9 a, 3 | ( Pt ,$ ( C~?A ! "I0 ! EB4$, 5 > $ , , 5> | ,= <*8  6, F = '$, ,2 $, P%6 H?A C% 6 )=/ 5 $G 5 0 . =.*p J5 , =D G2i ,= 7%$, = =.P#?A = =; > 6, E )2 ' $ 9 1.   , =,2, F  b ) 7%$ , A *, 2 p 6G b= 5 ! , ~&( @?A " ! . / 9ba, ~ | =/ C n r *, ! B 4 H B < , 0 *c&,$ P,H = =p 2 E )*1 G H / $  G H ! J *byA ,$ < / , CGyA = =~< , . @P!9 Appears in the  X ^ \ h h  e1Z%f [i^@ WS h4,e  WSt Z WxhEXEZGY WVe^ ZGY <M^ ZG]  hEX hEZ(\ hi^ Z "Z(^ n/h  f!h'0e1[E\ ^  hEXEtY ZG#3Y W_Y#ke1Z e1Z%f
Context-Free Recognition with Weighted Automata

We introduce the definition of language recognition with weighted automata, a generalization of the classical definition of recognition with unweighted acceptors. We show that, with our definition of recognition, weighted automata can be used to recognize a class of languages that strictly includes regular languages. The class of languages accepted depends on the weight set which has the algebraic structure of a semiring. We give a generic linear time algorithm for recognition with weighted automata and describe examples with various weight sets illustrating the recognition of several classes of context-free languages. We prove, in particular, that the class of languages equivalent to the language of palindromes can be recognized by weighted automata over the (+,ċ)-semiring, and that the class of languages equivalent to the Dyck language of first order D1'*can be recognized by weighted automata over the real tropical semiring. We also prove that weighted automata over the real tropical semiring can be used to recognize regular expressions.
Information mining platforms: an infrastructure for KDD rapid deployment

Experience has shown that the data extraction, parsing, cleaning and analysis steps of a KDD problem account for a much larger expenditure of resources (time and money) than the statistical modeling or machine learning part. Couple this statement with the need for fast turn-around time in commercial applications, and the obvious conclusion is that it is impractical to start from scratch for each new KDD application. To deal with this situation, we propose the use of an information mining platform that amortizes several of the critical preand post-processing steps needed to apply KDD. Thus, new KDD applications can leverage the platform for efficiency and robustness.
Squashing flat files flatter

A feature of data mining that distinguishes it from “classical” machine learning (ML) and statistical modeling (SM) is scale. The community seems to agree on this yet progress to this point has been limited. We present a methodology that addresses scale in a novel fashion that has the potential for revolutionizing the field. While the methodology applies most directly to flat (row by column) data sets we believe that it can be adapted to other representations. Our approach to the problem is not to scale up individual ML and SM methods. Rather we prefer to leverage the entire collection of existing methods by scaling down the data set. We call the method squashing. Our method demonstrably outperforms random sampling and a theoretical argument suggests how and why it works well. Squashing consists of three modular steps: grouping, momentizing, and generating (GMG). These three steps describe the squashing pipeline whereby the original (very large data set) is sectioned off into mutually exclusive groups (or bins); within each group a series of low-order moments are computed; and finally these moments are passed off to a routine that generates pseudo data that accurately reproduce the moments. The result of the GMG squashing pipeline is a squashed data set that has the same structure as the original data with the addition of a weight for each pseudo data point that reflects the distribution of the original data into the initial groups. Any ML or SM method that accepts weights can be used to analyze the weighted pseudo data. By construction the resulting analyses will mimic the corresponding analyses on the original data set. Squashing should appeal to many of the sub-disciplines of
Giga-Mining

We describe an industrial-strength data mining application in telecommunications. The application requires building a short (7 byte) profile for all telephone numbers seen on a large telecom network. By large, we mean very large: we maintain approximately 350 million profiles. In addition, the procedure for updating these profiles is based on processing approximately 275 million call records per day. We discuss the motivation for massive tracking and fully describe the definition and the computation of one of the more interesting bytes in the profile.
Capacity and Complexity Control in Predicting the Spread Between Borrowing and Lending Interest Rates

This is a problem in financial prediction with a small number of data points and high dimensionality in which classical economic forecasting techniques do not work. Many commercial institutions including banks, department stores, and credit companies charge their customers one interest rate (termed the lending rate) while they can borrow at a lower rate, termed the borrowing rate. The spread (difference) between these two rates can be a major profit center. If a commercial institution forecasts that the spread (and hence profit) will decrease, they can "hedge" by buying insurance against that decrease thereby locking in a profit. We used a variety of techniques that trade off training error against model complexity using the concept of capacity control for dimensionality reduction. We minimized the mean squared error of prediction and confirmed statistical validity using bootstrap techniques to predict that the spread will increase and hence one should not hedge. We briefly discuss the classical economic forecasting techniques which are not correct because the data is not independent, stationary, or normally distributed. Our predictions of the spread are consistent with the actual spread subsequent to the original analysis.
Boosting Decision Trees

We introduce a constructive, incremental learning system for regression problems that models data by means of locally linear experts. In contrast to other approaches, the experts are trained independently and do not compete for data during learning. Only when a prediction for a query is required do the experts cooperate by blending their individual predictions. Each expert is trained by minimizing a penalized local cross validation error using second order methods. In this way, an expert is able to find a local distance metric by adjusting the size and shape of the receptive field in which its predictions are valid, and also to detect relevant input features by adjusting its bias on the importance of individual input dimensions. We derive asymptotic results for our method. In a variety of simulations the properties of the algorithm are demonstrated with respect to interference, learning speed, prediction accuracy, feature detection, and task oriented incremental learning.