nils-herrmann/clean_scirepeval huggingface dataset:
A New Paradigm for Collision-free Hashing: Incrementality at Reduced Cost

We present a simple, new paradigm for the design of collision-free hash functions. Any function emanating from this paradigm is incremental. (This means that if a message x which I have previously hashed is modi ed to x0 then rather than having to re-compute the hash of x 0 from scratch, I can quickly \update" the old hash value to the new one, in time proportional to the amount of modi cation made in x to get x.) Also any function emanating from this paradigm is parallelizable, useful for hardware implementation. We derive several speci c functions from our paradigm. All use a standard hash function, assumed ideal, and some algebraic operations. The rst function, MuHASH, uses one modular multiplication per block of the message, making it reasonably e cient, and signi cantly faster than previous incremental hash functions. Its security is proven, based on the hardness of the discrete logarithm problem. A second function, AdHASH, is even faster, using additions instead of multiplications, with security proven given either that approximation of the length of shortest lattice vectors is hard or that the weighted subset sum problem is hard. A third function, LtHASH, is a practical variant of recent lattice based functions, with security proven based, again on the hardness of shortest lattice vector approximation. Dept. of Computer Science & Engineering, University of California at San Diego, 9500 Gilman Drive, La Jolla, California 92093, USA.  Technology Square, Cambridge, MA 02139, USA.  miccianc@theory.lcs.mit.edu. Supported in part by DARPA contract DABT63-96-C-0018.
KNADIA: Enterprise KNowledge Assisted DIAlogue Systems Using Deep Learning

In this paper we present the design, architecture and implementation of KNADIA, a conversational dialogue system for intra-enterprise use, providing knowledge-assisted question answering and transactional assistance to employees of a large organization. KNADIA has been deployed in production in TCS, a large organization with over 380,000 employees distributed globally; the system is currently supporting a few thousand active users making hundreds of queries per day. We identify, define and distinguish two distinct classes of use-cases: virtual assistance and knowledge synthesis, which we have found to cover a variety of enterprise needs. KNADIA supports both types of conversational agents, with multiple instances of each, while presenting a common digital persona. KNADIA disambiguates which agent should respond to each query. Further, since individual components use deep learning algorithms giving probabilistic outputs, the confidence with which different components answer is often required before answering or acting. The user's dialogue context also needs to be maintained judiciously. Due to these and many other challenges, the overall architecture of KNADIA is non-trivial. We present instances of HR-assistance and technical knowledge synthesis that are in production use in TCS along with accuracy figures and key performance metrics. Finally, we suggest that many elements of our architecture are also generally applicable to other complex deep learning systems.
IL-Miner: Instance-Level Discovery of Complex Event Patterns

Complex event processing (CEP) matches patterns over a continuous stream of events to detect situations of interest. Yet, the definition of an event pattern that precisely characterises a particular situation is challenging: there are manifold dimensions to correlate events, including time windows and value predicates. In the presence of historic event data that is labelled with the situation to detect, event patterns can be learned automatically. To cope with the combinatorial explosion of pattern candidates, existing approaches work on a type-level and discover patterns based on predefined event abstractions, aka event types. Hence, discovery is limited to patterns of a fixed granularity and users face the burden to manually select appropriate event abstractions. We present IL-MINER, a system that discovers event patterns by genuinely working on the instance-level, not assuming a priori knowledge on event abstractions. In a multi-phase process, ILMINER first identifies relevant abstractions for the construction of event patterns. The set of events explored for pattern discovery is thereby reduced, while still providing formal guarantees on correctness, minimality, and completeness of the discovery result. Experiments using real-world datasets from diverse domains show that IL-MINER discovers a much broader range of event patterns compared to the state-of-the-art in the field.
A high-level 3D visualization API for Java and ImageJ

Current imaging methods such as Magnetic Resonance Imaging (MRI), Confocal microscopy, Electron Microscopy (EM) or Selective Plane Illumination Microscopy (SPIM) yield three-dimensional (3D) data sets in need of appropriate computational methods for their analysis. The reconstruction, segmentation and registration are best approached from the 3D representation of the data set. Here we present a platform-independent framework based on Java and Java 3D for accelerated rendering of biological images. Our framework is seamlessly integrated into ImageJ, a free image processing package with a vast collection of community-developed biological image analysis tools. Our framework enriches the ImageJ software libraries with methods that greatly reduce the complexity of developing image analysis tools in an interactive 3D visualization environment. In particular, we provide high-level access to volume rendering, volume editing, surface extraction, and image annotation. The ability to rely on a library that removes the low-level details enables concentrating software development efforts on the algorithm implementation parts. Our framework enables biomedical image software development to be built with 3D visualization capabilities with very little effort. We offer the source code and convenient binary packages along with extensive documentation at http://3dviewer.neurofly.de .
State Detection Using Adaptive Human Sensor Sampling

With the massive prevalence of smartphones, mobile social sensing systems in which humans acting as social sensors respond to geo-located crowdsourcing tasks, became extremely popular. Such systems can provide significant benefits particularly during crisis management and emergency situations. However, not only querying users can be extremely costly but also human sensors are mobile, subjective and their response delays can highly vary. In this paper we develop a social sensing system that performs sampling on mobile social sensors to achieve accurate and real-time detection of the state of emergency events. Our contributions are two-fold: (i) our approach can capture well emergencies even in large geographical regions, and (ii) our sampling approach considers the individual characteristics of the social sensors to maximize the probability of receiving accurate responses in a timely manner. We provide comprehensive experiments that indicate that our approach accurately identifies critical real-world events, has low overhead and reduces the classification error up to 90% compared to traditional approaches.
Characterizing conflicts in fair division of indivisible goods using a scale of criteria

We investigate five different fairness criteria in a simple model of fair resource allocation of indivisible goods based on additive preferences. We show how these criteria are connected to each other, forming an ordered scale that can be used to characterize how conflicting the agents’ preferences are: for a given instance of a resource allocation problem, the less conflicting the agents’ preferences are, the more demanding criterion this instance is able to satisfy, and the more satisfactory the allocation can be. We analyze the computational properties of the five criteria, give some experimental results about them, and further investigate a slightly richer model with $$k$$ k -additive preferences.
Finding a Maximum Clique using Ant Colony Optimization and Particle Swarm Optimization in Social Networks

Interaction between users in online social networks plays a key role in social network analysis. One on important types of social group is full connected relation between some users, which known as clique structure. Therefore finding a maximum clique is essential for some analysis. In this paper, we proposed a new method using ant colony optimization algorithm and particle swarm optimization algorithm. In the proposed method, in order to attain better results, it is improved process of pheromone update by particle swarm optimization. Simulation results on popular standard social network benchmarks in comparison standard ant colony optimization algorithm are shown a relative enhancement of proposed algorithm. Keywordssocial network analysis; clique problem; ACO; PSO.
Reviving the Method of Particular Solutions

Fox, Henrici, and Moler made famous a “method of particular solutions” for computing eigenvalues and eigenmodes of the Laplacian in planar regions such as polygons. We explain why their formulation of this method breaks down when applied to regions that are insufficiently simple and propose a modification that avoids these difficulties. The crucial changes are to introduce points in the interior of the region as well as on the boundary and to minimize a subspace angle rather than just a singular value or a determinant. Similar methods may be used to improve other “mesh-free” algorithms for a variety of computational problems.
A robust FISTA-like algorithm

The Fast Iterative Shrinkage-Thresholding Algorithm (FISTA) is regarded as the state-of-the-art among a number of proximal gradient-based methods used for addressing large-scale optimization problems with simple but non-differentiable objective functions. However, the efficiency of FISTA in a wide range of applications is hampered by a simple drawback in the line search scheme. The local estimate of the Lipschitz constant, the inverse of which gives the step size, can only increase while the algorithm is running. As a result, FISTA can slow down significantly if the initial estimate of the Lipschitz constant is excessively large or if the local Lipschitz constant decreases in the vicinity of the optimal point. We propose a new FISTA-like method endowed with a robust step size search procedure and demonstrate its effectiveness by means of a rigorous theoretical convergence analysis and simulations.
Deploying Microservice Based Applications with Kubernetes: Experiments and Lessons Learned

Microservices represent a new architectural style where small and loosely coupled modules can be developed and deployed independently to compose an application. This architectural style brings various benefits such as maintainability and flexibility in scaling and aims at decreasing downtime in case of failure or upgrade. One of the enablers is Kubernetes, an open source platform that provides mechanisms for deploying, maintaining, and scaling containerized applications across a cluster of hosts. Moreover, Kubernetes enables healing through failure recovery actions to improve the availability of applications. As our ultimate goal is to devise architectures to enable high availability (HA) with Kubernetes for microservice based applications, in this paper we examine the availability achievable through Kubernetes under its default configuration. We have conducted a set of experiments which show that the service outage can be significantly higher than expected.
Adopting Code Reviews for Agile Software Development

Code reviews have many benefits, most importantly to find bugs early in the development phase and to enforce coding standards. Still, it is widely accepted that formal code reviews are time-consuming and the practical applicability in agile development is controversial. This work presents a continuous differential-based method and tool for code reviews. By using a continuous approach to code reviews, the review overhead can be reduced and the effectiveness and applicability in agile environments shall be improved.
Zero-Shot Learning Posed as a Missing Data Problem

This paper presents a method of zero-shot learning (ZSL) which poses ZSL as the missing data problem, rather than the missing label problem. Specifically, most existing ZSL methods focus on learning mapping functions from the image feature space to the label embedding space. Whereas, the proposed method explores a simple yet effective transductive framework in the reverse way - our method estimates data distribution of unseen classes in the image feature space by transferring knowledge from the label embedding space. Following the transductive setting, we leverage unlabeled data to refine the initial estimation. In experiments, our method achieves the highest classification accuracies on two popular datasets, namely, 96.00% on AwA and 60.24% on CUB.
qBase relative quantification framework and software for management and automated analysis of real-time quantitative PCR data

Although quantitative PCR (qPCR) is becoming the method of choice for expression profiling of selected genes, accurate and straightforward processing of the raw measurements remains a major hurdle. Here we outline advanced and universally applicable models for relative quantification and inter-run calibration with proper error propagation along the entire calculation track. These models and algorithms are implemented in qBase, a free program for the management and automated analysis of qPCR data.
Color image processing [basics and special issue overview]

umans have always seen the world in color but only recently have we been able to generate vast quantities of color images with such ease. In the last three decades, we have seen a rapid and enormous transition from grayscale images to color ones. Today, we are exposed to color images on a daily basis in print, photographs, television, computer displays, and cinema movies, where color now plays a vital role in the advertising and dissemination of information throughout the world. Color monitors, printers, and copiers now dominate the office and home environments, with color becoming increasingly cheaper and easier to generate and reproduce. Color demands have soared in the marketplace and are projected to do so for years to come. With this rapid progression, color and multispectral properties of images are becoming increasingly crucial to the field of image processing, often extending and/or replacing previously known grayscale techniques. We have seen the birth of color algorithms that range from direct extensions of grayscale ones, where images are treated as three monochrome separations, to more sophisticated approaches that exploit the correlations among the color bands, yielding more accurate results. Hence, it is becoming increasingly necessary for the signal processing community to understand the fundamental differences between color and grayscale imaging. There are more than a few extensions of concepts
Efficient trajectory extraction and parameter learning for data-driven crowd simulation

We present a trajectory extraction and behavior-learning algorithm for data-driven crowd simulation. Our formulation is based on incrementally learning pedestrian motion models and behaviors from crowd videos. We combine this learned crowd-simulation model with an online tracker based on particle filtering to compute accurate, smooth pedestrian trajectories. We refine this motion model using an optimization technique to estimate the agents’ simulation parameters. We highlight the benefits of our approach for improved data-driven crowd simulation, including crowd replication from videos and merging the behavior of pedestrians from multiple videos. We highlight our algorithm’s performance in various test scenarios containing tens of human-like agents.
Analyzing the Held-Karp TSP Bound: A Monotonicity Property with Application

In their 1971 paper on the Traveling Salesman Problem and Minimum Spanning Trees, Held and Karp showed that finding an optimally weighted 1-tree is equivalent to solving a linear program for the Traveling Salesman Problem (TSP) with only node-degree constraints and subtour elimination constraints. In this paper we show that the Held-Karp 1-trees have a certain monotonicity property: given a particular instance of the symmetric TSP with triangle inequality, the cost of the minimum weighted 1-tree is monotonic with respect to the set of nodes included. As a consequence, we obtain an alternate proof of a result of Wolsey and show that linear programs with node-degree and subtour elimination constraints must have a cost at least 23OPT , where OPT is the cost of the optimum solution to the TSP instance. The traveling salesman problem is one of the most notorious in the field of combinatorial optimization, and one of the most well-studied [7]. Currently, the most successful approach to finding optimal solutions to large-scale problems is based on formulating the problem as a linear program and finding explicit partial descriptions of this linear polytope [5], . The most natural constraints are derived from an integer linear programming formulation that uses nodedegree constraints and subtour elimination constraints. We focus our attention on symmetric instances of the TSP that obey the triangle inequality. Let V = {1, 2, . . . , n} denote the set of nodes. For any distinct i and j, assign a cost cij such that cij = cji, and for any k distinct from i and j, cij ≤ cik + ckj . Then the Subtour LP on this instance is B = min ∑ 1≤i<j≤n cijxij subject to: ∑ j>i xij + ∑ j<i xji = 2, i = 1, 2, . . . , n, ∑ i∈S,j∈S,i<j xij ≤ |S|− 1, for any proper subsetS ⊂ V, 0 ≤ xij ≤ 1, 1 ≤ i < j ≤ n. (1) ∗This research was supported in part by Air Force Contract AFOSR-86-0078 and by the National Science Foundation under a Presidential Young Investigator Award CCR-8996272 with matching support from IBM, Sun, and UPS.
Evolving neural networks to play checkers without relying on expert knowledge

An experiment was conducted where neural networks compete for survival in an evolving population based on their ability to play checkers. More specifically, multilayer feedforward neural networks were used to evaluate alternative board positions and games were played using a minimax search strategy. At each generation, the extant neural networks were paired in competitions and selection was used to eliminate those that performed poorly relative to other networks. Offspring neural networks were created from the survivors using random variation of all weights and bias terms. After a series of 250 generations, the best-evolved neural network was played against human opponents in a series of 90 games on an internet website. The neural network was able to defeat two expert-level players and played to a draw against a master. The final rating of the neural network placed it in the "Class A" category using a standard rating system. Of particular importance in the design of the experiment was the fact that no features beyond the piece differential were given to the neural networks as a priori knowledge. The process of evolution was able to extract all of the additional information required to play at this level of competency. It accomplished this based almost solely on the feedback offered in the final aggregated outcome of each game played (i.e., win, lose, or draw). This procedure stands in marked contrast to the typical artifice of explicitly injecting expert knowledge into a game-playing program.
WOVSQLI: Detection of SQL Injection Behaviors Using Word Vector and LSTM

The Structured Query Language Injection Attack (SQLIA) is one of the most serious and popular threats of web applications. The results of SQLIA include the data loss or complete host takeover. Detection of SQLIA is always an intractable challenge because of the heterogeneity of the attack payloads. In this paper, a novel method to detect SQLIA based on word vector of SQL tokens and LSTM neural networks is described. In the proposed method, SQL query strings were firstly syntactically analyzed into tokens, and then likelihood ratio test is used to build the word vector of SQL tokens, ultimately, an LSTM model is trained with sequences of token word vectors. We developed a tool named WOVSQLI, which implements the proposed technique, and it was evaluated with a dataset from several sources. The results of experiments demonstrate that WOVSQLI can effectively identify SQLIA.
Fast linear iterations for distributed averaging

We consider the problem of finding a linear iteration that yields distributed averaging consensus over a network, i.e., that asymptotically computes the average of some initial values given at the nodes. When the iteration is assumed symmetric, the problem of finding the fastest converging linear iteration can be cast as a semidefinite program, and therefore efficiently and globally solved. These optimal linear iterations are often substantially faster than several common heuristics that are based on the Laplacian of the associated graph. We show how problem structure can be exploited to speed up interior-point methods for solving the fastest distributed linear iteration problem, for networks with up to a thousand or so edges. We also describe a simple subgradient method that handles far larger problems, with up to one hundred thousand edges. We give several extensions and variations on the basic problem.
Adapting to a Changing Environment: the Brownian Restless Bandits

In the multi-armed bandit (MAB) problem there are k distributions associated with the rewards of playing each of k strategies (slot machine arms). The reward distributions are initially unknown to the player. The player iteratively plays one strategy per round, observes the associated reward, and decides on the strategy for the next iteration. The goal is to maximize the reward by balancing exploitation: the use of acquired information, with exploration: learning new information. We introduce and study a dynamic MAB problem in which the reward functions stochastically and gradually change in time. Specifically, the expected reward of each arm follows a Brownian motion, a discrete random walk, or similar processes. In this setting a player has to continuously keep exploring in order to adapt to the changing environment. Our formulation is (roughly) a special case of the notoriously intractable restless MAB problem. Our goal here is to characterize the cost of learning and adapting to the changing environment, in terms of the stochastic rate of the change. We consider an infinite time horizon, and strive to minimize the average cost per step which we define with respect to a hypothetical algorithm that at every step plays the arm with the maximum expected reward at this step. A related line of work on the adversarial MAB problem used a significantly weaker benchmark, the best time-invariant policy. The dynamic MAB problem models a variety of practical online, game-againstnature type optimization settings. While building on prior work, algorithms and steady-state analysis for the dynamic setting require a novel approach based on different stochastic tools. ∗Microsoft Research,  slivkins at microsoft.com. Parts of this work has been completed while A. Slivkins was a postdoc at Brown University. †Computer Science Department, Brown University, Providence RI. E-mail: eli at cs.brown.edu. Supported in part by NSF awards CCR-0121154 and DMI-0600384, and ONR Award N000140610607.