nils-herrmann/clean_scirepeval huggingface dataset:
Small Sample Statistics for Classi cation Error Rates II : Con dence Intervals and Signi cance Tests

Several techniques for estimating the reliability of estimated error rates and for estimating the signicance of observed dierences in error rates are explored in this paper. Textbook formulas which assume a large test set, i.e., a normal distribution, are commonly used to approximate the condence limits of error rates or as an approximate signicance test for comparing error rates. Expressions for determining more exact limits and signicance levels for small samples are given here, and criteria are also given for determining when these more exact methods should be used. The assumed normal distribution gives a poor approximation to the condence interval in most cases, but is usually useful for signicance tests when the proper mean and variance expressions are used. A commonly used 62 signicance test uses an improper expression for , which is too low and leads to a high likelihood of Type I errors. Common machine learning methods for estimating signicance from observations on a single sample may be unreliable.
Reactive planning idioms for multi-scale game AI

Many modern games provide environments in which agents perform decision making at several levels of granularity. In the domain of real-time strategy games, an effective agent must make high-level strategic decisions while simultaneously controlling individual units in battle. We advocate reactive planning as a powerful technique for building multi-scale game AI and demonstrate that it enables the specification of complex, real-time agents in a unified agent architecture. We present several idioms used to enable authoring of an agent that concurrently pursues strategic and tactical goals, and an agent for playing the real-time strategy game StarCraft that uses these design patterns.
A Meaning-based English Math Word Problem Solver with Understanding, Reasoning and Explanation

This paper presents a meaning-based statistical math word problem (MWP) solver with understanding, reasoning and explanation. It comprises a web user interface and pipelined modules for analysing the text, transforming both body and question parts into their logic forms, and then performing inference on them. The associated context of each quantity is represented with proposed role-tags (e.g., nsubj, verb, etc.), which provides the flexibility for annotating the extracted math quantity with its associated syntactic and semantic information (which specifies the physical meaning of that quantity). Those role-tags are then used to identify the desired operands and filter out irrelevant quantities (so that the answer can be obtained precisely). Since the physical meaning of each quantity is explicitly represented with those role-tags and used in the inference process, the proposed approach could explain how the answer is obtained in a human comprehensible way.
Automatic socio-economic classification of households using electricity consumption data

Interest in analyzing electricity consumption data of private households has grown steadily in the last years. Several authors have for instance focused on identifying groups of households with similar consumption patterns or on providing feedback to consumers in order to motivate them to reduce their energy consumption. In this paper, we propose to use electricity consumption data to classify households according to pre-defined "properties" of interest. Examples of these properties include the floor area of a household or the number of its occupants. Energy providers can leverage knowledge of such household properties to shape premium services (e.g., energy consulting) for their customers. We present a classification system - called CLASS - that takes as input electricity consumption data of a private household and provides as output the estimated values of its properties. We describe the design and implementation of CLASS and evaluate its performance. To this end, we rely on electricity consumption traces from 3,488 private households, collected at a 30-minute granularity and for a period of more than 1.5 years. Our evaluation shows that CLASS - relying on electricity consumption data only - can estimate the majority of the considered household properties with more than 70% accuracy. For some of the properties, CLASS's accuracy exceeds 80%. Furthermore, we show that for selected properties the use of a priori information can increase classification accuracy by up to 11%.
Reasoning about Temporal Relations: A Maximal Tractable Subclass of Allen's Interval Algebra

We introduce a new subclass of Allen's interval algebra we call “ORD-Horn subclass,” which is a strict superset of the “pointisable subclass.” We prove that reasoning in the ORD-Horn subclass is a polynomial-time problem and show that the path-consistency method is sufficient for deciding satisfiability. Further, using an extensive machine-generated case analysis, we show that the ORD-Horn subclass is a maximal tractable subclass of the full algebra (assuming P <inline-equation> <f> ≠</f> </inline-equation> NP). In fact, it is the unique greatest tractable subclass amongst the subclasses that contain all basic relations.
CBCD: Cloned buggy code detector

Developers often copy, or clone, code in order to reuse or modify functionality. When they do so, they also clone any bugs in the original code. Or, different developers may independently make the same mistake. As one example of a bug, multiple products in a product line may use a component in a similar wrong way. This paper makes two contributions. First, it presents an empirical study of cloned buggy code. In a large industrial product line, about 4% of the bugs are duplicated across more than one product or file. In three open source projects (the Linux kernel, the Git version control system, and the PostgreSQL database) we found 282, 33, and 33 duplicated bugs, respectively. Second, this paper presents a tool, CBCD, that searches for code that is semantically identical to given buggy code. CBCD tests graph isomorphism over the Program Dependency Graph (PDG) representation and uses four optimizations. We evaluated CBCD by searching for known clones of buggy code segments in the three projects and compared the results with text-based, token-based, and AST-based code clone detectors, namely Simian, CCFinder, Deckard, and CloneDR. The evaluation shows that CBCD is fast when searching for possible clones of the buggy code in a large system, and it is more precise for this purpose than the other code clone detectors.
Vehicle Logo Recognition and Classification : Feature Descriptors vs . Shape Descriptors

We explore several image processing methods to automatically identify the make of a vehicle based focused on the manufacturer’s iconic logo. Our findings reveal that large variations in brightness, vehicle features in the foreground, and specular reflections render the scale-invariant feature transform (SIFT) approach practically useless. Methods such as Fourier shape descriptors and inner structure mean square error analysis are able to achieve more reliable results.
A 12b 22.5/45MS/s 3.0mW 0.059mm2 CMOS SAR ADC achieving over 90dB SFDR

CMOS technology scaling has opened a pathway to high-performance analog-to-digital conversion in the nanometer regime, where switching is preferred over amplifying. Successive-approximation-register (SAR) is one of the conversion architectures that rely on the high switching speed of process technology, and is thus distinctively known for its superior energy efficiency, small chip area, and good digital compatibility. When properly implemented, a SAR ADC also benefits from a potential rail-to-rail input swing, 100% capacitance utilization during input sampling (thus low kT/C noise), and insensitivity to comparator offsets during the conversion process. The linearity-limiting factors for SAR ADC are capacitor mismatch, sampling switch non-idealities, as well as the reference voltage settling issue due to the high internal switching speed of the DAC. In this work, a sub-radix-2 SAR ADC is presented, which employs a perturbation-based digital background calibration scheme and a dynamic-threshold-comparison (DTC) technique to overcome some of these performance-limiting factors.
Cross-Lingual Named Entity Recognition via Wikification

Named Entity Recognition (NER) models for language L are typically trained using annotated data in that language. We study cross-lingual NER, where a model for NER in L is trained on another, source, language (or multiple source languages). We introduce a language independent method for NER, building on cross-lingual wikification, a technique that grounds words and phrases in nonEnglish text into English Wikipedia entries. Thus, mentions in any language can be described using a set of categories and FreeBase types, yielding, as we show, strong language-independent features. With this insight, we propose an NER model that can be applied to all languages in Wikipedia. When trained on English, our model outperforms comparable approaches on the standard CoNLL datasets (Spanish, German, and Dutch) and also performs very well on lowresource languages (e.g., Turkish, Tagalog, Yoruba, Bengali, and Tamil) that have significantly smaller Wikipedia. Moreover, our method allows us to train on multiple source languages, typically improving NER results on the target languages. Finally, we show that our languageindependent features can be used also to enhance monolingual NER systems, yielding improved results for all 9 languages.
Exploiting Sentence and Context Representations in Deep Neural Models for Spoken Language Understanding

This paper presents a deep learning architecture for the semantic decoder component of a Statistical Spoken Dialogue System. In a slot-filling dialogue, the semantic decoder predicts the dialogue act and a set of slot-value pairs from a set of n-best hypotheses returned by the Automatic Speech Recognition. Most current models for spoken language understanding assume (i) word-aligned semantic annotations as in sequence taggers and (ii) delexicalisation, or a mapping of input words to domain-specific concepts using heuristics that try to capture morphological variation but that do not scale to other domains nor to language variation (e.g., morphology, synonyms, paraphrasing ). In this work the semantic decoder is trained using unaligned semantic annotations and it uses distributed semantic representation learning to overcome the limitations of explicit delexicalisation. The proposed architecture uses a convolutional neural network for the sentence representation and a long-short term memory network for the context representation. Results are presented for the publicly available DSTC2 corpus and an In-car corpus which is similar to DSTC2 but has a significantly higher word error rate (WER).
Centiman: elastic, high performance optimistic concurrency control by watermarking

We present Centiman, a system for high performance, elastic transaction processing in the cloud. Centiman provides serializability on top of a key-value store with a lightweight protocol based on optimistic concurrency control (OCC).
 Centiman is designed for the cloud setting, with an architecture that is loosely coupled and avoids synchronization wherever possible. Centiman supports sharded transaction validation; validators can be added or removed on-the-fly in an elastic manner. Processors and validators scale independently of each other and recover from failure transparently to each other. Centiman's loosely coupled design creates some challenges: it can cause spurious aborts and it makes it difficult to implement common performance optimizations for read-only transactions. To deal with these issues, Centiman uses a watermark abstraction to asynchronously propagate information about transaction commits through the system.
 In an extensive evaluation we show that Centiman provides fast elastic scaling, low-overhead serializability for read-heavy workloads, and scales to millions of operations per second.
Stock Prediction by Searching for Similarities in Candlestick Charts

The aim of stock prediction is to effectively predict future stock market trends (or stock prices), which can lead to increased profit. One major stock analysis method is the use of candlestick charts. However, candlestick chart analysis has usually been based on the utilization of numerical formulas. There has been no work taking advantage of an image processing technique to directly analyze the visual content of the candlestick charts for stock prediction. Therefore, in this study we apply the concept of image retrieval to extract seven different wavelet-based texture features from candlestick charts. Then, similar historical candlestick charts are retrieved based on different texture features related to the query chart, and the “future” stock movements of the retrieved charts are used for stock prediction. To assess the applicability of this approach to stock prediction, two datasets are used, containing 5-year and 10-year training and testing sets, collected from the Dow Jones Industrial Average Index (INDU) for the period between 1990 and 2009. Moreover, two datasets (2010 and 2011) are used to further validate the proposed approach. The experimental results show that visual content extraction and similarity matching of candlestick charts is a new and useful analytical method for stock prediction. More specifically, we found that the extracted feature vectors of 30, 90, and 120, the number of textual features extracted from the candlestick charts in the BMP format, are more suitable for predicting stock movements, while the 90 feature vector offers the best performance for predicting short- and medium-term stock movements. That is, using the 90 feature vector provides the lowest MAPE (3.031%) and Theil’s U (1.988%) rates in the twenty-year dataset, and the best MAPE (2.625%, 2.945%) and Theil’s U (1.622%, 1.972%) rates in the two validation datasets (2010 and 2011).
The Crossfire Attack

We present the Crossfire attack -- a powerful attack that degrades and often cuts off network connections to a variety of selected server targets (e.g., servers of an enterprise, a city, a state, or a small country) by flooding only a few network links. In Crossfire, a small set of bots directs low intensity flows to a large number of publicly accessible servers. The concentration of these flows on the small set of carefully chosen links floods these links and effectively disconnects selected target servers from the Internet. The sources of the Crossfire attack are undetectable by any targeted servers, since they no longer receive any messages, and by network routers, since they receive only low-intensity, individual flows that are indistinguishable from legitimate flows. The attack persistence can be extended virtually indefinitely by changing the set of bots, publicly accessible servers, and target links while maintaining the same disconnection targets. We demonstrate the attack feasibility using Internet experiments, show its effects on a variety of chosen targets (e.g., servers of universities, US states, East and West Coasts of the US), and explore several countermeasures.
KNIME - the Konstanz information miner: version 2.0 and beyond

The Konstanz Information Miner is a modular environment, which enables easy visual assembly and interactive execution of a data pipeline. It is designed as a teaching, research and collaboration platform, which enables simple integration of new algorithms and tools as well as data manipulation or visualization methods in the form of new modules or nodes. In this paper we describe some of the design aspects of the underlying architecture, briey sketch how new nodes can be incorporated, and highlight some of the new features of version 2.0.
Practical Recommendations on Crawling Online Social Networks

Our goal in this paper is to develop a practical framework for obtaining a uniform sample of users in an online social network (OSN) by crawling its social graph. Such a sample allows to estimate any user property and some topological properties as well. To this end, first, we consider and compare several candidate crawling techniques. Two approaches that can produce approximately uniform samples are the Metropolis-Hasting random walk (MHRW) and a re-weighted random walk (RWRW). Both have pros and cons, which we demonstrate through a comparison to each other as well as to the "ground truth." In contrast, using Breadth-First-Search (BFS) or an unadjusted Random Walk (RW) leads to substantially biased results. Second, and in addition to offline performance assessment, we introduce online formal convergence diagnostics to assess sample quality during the data collection process. We show how these diagnostics can be used to effectively determine when a random walk sample is of adequate size and quality. Third, as a case study, we apply the above methods to Facebook and we collect the first, to the best of our knowledge, representative sample of Facebook users. We make it publicly available and employ it to characterize several key properties of Facebook.
Parallel Selective Algorithms for Nonconvex Big Data Optimization

We propose a decomposition framework for the parallel optimization of the sum of a differentiable (possibly nonconvex) function and a (block) separable nonsmooth, convex one. The latter term is usually employed to enforce structure in the solution, typically sparsity. Our framework is very flexible and includes both fully parallel Jacobi schemes and Gauss-Seidel (i.e., sequential) ones, as well as virtually all possibilities “in between” with only a subset of variables updated at each iteration. Our theoretical convergence results improve on existing ones, and numerical results on LASSO, logistic regression, and some nonconvex quadratic problems show that the new method consistently outperforms existing algorithms.
Graph Isomorphism in Quasipolynomial Time

We show that the Graph Isomorphism (GI) problem and its generalizations, the String Isomorphism (SI) and Coset Intersection (CI) problems, can be solved in quasipolynomial (exp ( (log n) ) ) time. The best previous bound for GI was exp(O( √ n log n)), where n is the number of vertices (Luks, 1983); for SI and CI, the bound was similar, exp(Õ( √ n)), where n is the size of the permutation domain (Babai, 1983). The SI problem takes as input two strings, x and y, of length n, and a permutation group G of degree n and asks if some element of G transforms x into y. Our algorithm builds on Luks’s SI framework and attacks its bottleneck, characterized by an epimorphism φ of G onto the alternating group acting on a set Γ of size k > c log n. Our goal is to break this symmetry. The crucial first step is to find a canonical t-ary relational structure on Γ, with not too much symmetry, for some t = O(log n). We say that an element x in the domain of G is affected by φ if φ maps the stabilizer of x to a proper subgroup of Ak. The affected/unaffected dichotomy provides a device to construct global symmetry from local information through the core group-theoretic “local certificates” routine. This algorithm in turn produces the required t-ary structure and thereby sets the stage for symmetry breaking via combinatorial methods of canonical partitioning. The latter lead to the emergence of the Johnson graphs as the sole obstructions to effective canonical partitioning. For a list of updates compared to the first two arXiv versions, see the Acknowledgments (.  While the present version fills significant gaps of the previous versions and improves the presentation of some components of the paper, the revision is incomplete; at the current stage, it includes notational, conceptual, and organizational inconsistencies. A fuller explanation of this disclaimer appears in the Acknowledgments (Sec. 18.1) at the end of the paper. ∗ Research supported in part by NSF Grants CCF-7443327 (2014-current), CCF-1017781 (2010-2014), and CCF-0830370 (2008–2010). Any opinions, findings, and conclusions or recommendations expressed in this paper are those of the author and do not necessarily reflect the views of the National Science Foundation (NSF).
Making Gradient Descent Optimal for Strongly Convex Stochastic Optimization

Stochastic gradient descent (SGD) is a simple and popular method to solve stochastic optimization problems which arise in machine learning. For strongly convex problems, its convergence rate was known to be O(log(T )/T ), by running SGD for T iterations and returning the average point. However, recent results showed that using a different algorithm, one can get an optimal O(1/T ) rate. This might lead one to believe that standard SGD is suboptimal, and maybe should even be replaced as a method of choice. In this paper, we investigate the optimality of SGD in a stochastic setting. We show that for smooth problems, the algorithm attains the optimal O(1/T ) rate. However, for non-smooth problems, the convergence rate with averaging might really be Ω(log(T )/T ), and this is not just an artifact of the analysis. On the flip side, we show that a simple modification of the averaging step suffices to recover the O(1/T ) rate, and no other change of the algorithm is necessary. We also present experimental results which support our findings, and point out open problems.
An Online Algorithm for Segmenting Time Series

In recent years, there has been an explosion of interest in mining time series databases. As with most computer science problems, representation of the data is the key to efficient and effective solutions. One of the most commonly used representations is piecewise linear approximation. This representation has been used by various researchers to support clustering, classification, indexing and association rule mining of time series data. A variety of algorithms have been proposed to obtain this representation, with several algorithms having been independently rediscovered several times. In this paper, we undertake the first extensive review and empirical comparison of all proposed techniques. We show that all these algorithms have fatal flaws from a data mining perspective. We introduce a novel algorithm that we empirically show to be superior to all others in the literature.
ConvNets with Smooth Adaptive Activation Functions for Regression

Within Neural Networks (NN), the parameters of Adaptive Activation Functions (AAF) control the shapes of activation functions. These parameters are trained along with other parameters in the NN. AAFs have improved performance of Convolutional Neural Networks (CNN) in multiple classification tasks. In this paper, we propose and apply AAFs on CNNs for regression tasks. We argue that applying AAFs in the regression (second-tolast) layer of a NN can significantly decrease the bias of the regression NN. However, using existing AAFs may lead to overfitting. To address this problem, we propose a Smooth Adaptive Activation Function (SAAF) with a piecewise polynomial form which can approximate any continuous function to arbitrary degree of error, while having a bounded Lipschitz constant for given bounded model parameters. As a result, NNs with SAAF can avoid overfitting by simply regularizing model parameters. We empirically evaluated CNNs with SAAFs and achieved state-of-the-art results on age and pose estimation datasets.