nils-herrmann/clean_scirepeval huggingface dataset:
SentiMATE: Learning to play Chess through Natural Language Processing

We present SentiMATE, a novel end-to-end Deep Learning model for Chess, employing Natural Language Processing that aims to learn an effective evaluation function assessing move quality. This function is pre-trained on the sentiment of commentary associated with the training moves and is used to guide and optimize the agent's game-playing decision making. The contributions of this research are three-fold: we build and put forward both a classifier which extracts commentary describing the quality of Chess moves in vast commentary datasets, and a Sentiment Analysis model trained on Chess commentary to accurately predict the quality of said moves, to then use those predictions to evaluate the optimal next move of a Chess agent. Both classifiers achieve over 90 % classification accuracy. Lastly, we present a Chess engine, SentiMATE, which evaluates Chess moves based on a pre-trained sentiment evaluation function. Our results exhibit strong evidence to support our initial hypothesis - "Can Natural Language Processing be used to train a novel and sample efficient evaluation function in Chess Engines?" - as we integrate our evaluation function into modern Chess engines and play against agents with traditional Chess move evaluation functions, beating both random agents and a DeepChess implementation at a level-one search depth - representing the number of moves a traditional Chess agent (employing the alpha-beta search algorithm) looks ahead in order to evaluate a given chess state.
MIMIC-Extract: a data extraction, preprocessing, and representation pipeline for MIMIC-III

Machine learning for healthcare researchers face challenges to progress and reproducibility due to a lack of standardized processing frameworks for public datasets. We present MIMIC-Extract, an open source pipeline for transforming the raw electronic health record (EHR) data of critical care patients from the publicly-available MIMIC-III database into data structures that are directly usable in common time-series prediction pipelines. Extract addresses three challenges in making complex EHR data accessible to the broader machine learning community. First, Extract transforms raw vital sign and laboratory measurements into usable hourly time series, performing essential steps such as unit conversion, outlier handling, and aggregation of semantically similar features to reduce missingness and improve robustness. Second, MIMIC-Extract extracts and makes prediction of clinically-relevant targets possible, including outcomes such as mortality and length-of-stay as well as comprehensive hourly intervention signals for ventilators, vasopressors, and fluid therapies. Finally, the pipeline emphasizes reproducibility and extensibility to future research questions. We demonstrate the pipeline's effectiveness by developing several benchmark tasks for outcome and intervention forecasting and assessing the performance of competitive models.
Scalable Topological Data Analysis and Visualization for Evaluating Data-Driven Models in Scientific Applications

With the rapid adoption of machine learning techniques for large-scale applications in science and engineering comes the convergence of two grand challenges in visualization. First, the utilization of black box models (e.g., deep neural networks) calls for advanced techniques in exploring and interpreting model behaviors. Second, the rapid growth in computing has produced enormous datasets that require techniques that can handle millions or more samples. Although some solutions to these interpretability challenges have been proposed, they typically do not scale beyond thousands of samples, nor do they provide the high-level intuition scientists are looking for. Here, we present the first scalable solution to explore and analyze high-dimensional functions often encountered in the scientific data analysis pipeline. By combining a new streaming neighborhood graph construction, the corresponding topology computation, and a novel data aggregation scheme, namely topology aware datacubes, we enable interactive exploration of both the topological and the geometric aspect of high-dimensional data. Following two use cases from high-energy-density (HED) physics and computational biology, we demonstrate how these capabilities have led to crucial new insights in both applications.
I Stand With You: Using Emojis to Study Solidarity in Crisis Events

We study how emojis are used to express solidarity in social media in the context of two major crisis events - a natural disaster, Hurricane Irma in 2017 and terrorist attacks that occurred on November 2015 in Paris. Using annotated corpora, we first train a recurrent neural network model to classify expressions of solidarity in text. Next, we use these expressions of solidarity to characterize human behavior in online social networks, through the temporal and geospatial diffusion of emojis. Our analysis reveals that emojis are a powerful indicator of sociolinguistic behaviors (solidarity) that are exhibited on social media as the crisis events unfold.
Estimating the Moon-to-Earth Radius Ratio with a Smartphone, a Telescope, and an Eclipse

On January 20th, 2019, a total lunar eclipse was possible to be observed in Santiago, Chile. Using a smartphone attached to a telescope, photographs of the phenomenon were taken. With Earth's shadow on those images, and using textbook geometry, a simple open-source software and analytical procedures, we were allowed to calculate the ratio between the radii of the Moon and the Earth. The results are in very good agreement with the correct value for such ratio. This shows the strength of the smartphone technology to get powerful astronomical results in a very simple way and in a very short amount of time.
Open Set Domain Adaptation: Theoretical Bound and Algorithm

The aim of unsupervised domain adaptation is to leverage the knowledge in a labeled (source) domain to improve a model’s learning performance with an unlabeled (target) domain—the basic strategy being to mitigate the effects of discrepancies between the two distributions. Most existing algorithms can only handle unsupervised closed set domain adaptation (UCSDA), i.e., where the source and target domains are assumed to share the same label set. In this article, we target a more challenging but realistic setting: unsupervised open set domain adaptation (UOSDA), where the target domain has unknown classes that are not found in the source domain. This is the first study to provide learning bound for open set domain adaptation, which we do by theoretically investigating the risk of the target classifier on unknown classes. The proposed learning bound has a special term, namely, open set difference, which reflects the risk of the target classifier on unknown classes. Furthermore, we present a novel and theoretically guided unsupervised algorithm for open set domain adaptation, called distribution alignment with open difference (DAOD), which is based on regularizing this open set difference bound. The experiments on several benchmark data sets show the superior performance of the proposed UOSDA method compared with the state-of-the-art methods in the literature.
Robust Real-time RGB-D Visual Odometry in Dynamic Environments via Rigid Motion Model

In the paper, we propose a robust real-time visual odometry in dynamic environments via rigid-motion model updated by scene flow. The proposed algorithm consists of spatial motion segmentation and temporal motion tracking. The spatial segmentation first generates several motion hypotheses by using a grid-based scene flow and clusters the extracted motion hypotheses, separating objects that move independently of one another. Further, we use a dual-mode motion model to consistently distinguish between the static and dynamic parts in the temporal motion tracking stage. Finally, the proposed algorithm estimates the pose of a camera by taking advantage of the region classified as static parts. In order to evaluate the performance of visual odometry under the existence of dynamic rigid objects, we use self-collected dataset containing RGB-D images and motion capture data for ground-truth. We compare our algorithm with state-of-the-art visual odometry algorithms. The validation results suggest that the proposed algorithm can estimate the pose of a camera robustly and accurately in dynamic environments.
Automated Machine Learning in Practice: State of the Art and Recent Results

A main driver behind the digitization of industry and society is the belief that data-driven model building and decision making can contribute to higher degrees of automation and more informed decisions. Building such models from data often involves the application of some form of machine learning. Thus, there is an ever growing demand in work force with the necessary skill set to do so. This demand has given rise to a new research topic concerned with fitting machine learning models fully automatically—AutoML. This paper gives an overview of the state of the art in AutoML with a focus on practical applicability in a business context, and provides recent benchmark results of the most important AutoML algorithms.
Refractive Interstellar Scintillation of Extra-galactic Radio Sources I: Expectations

Surveys for transient and variable phenomena can be confounded by the presence of extrinsic variability such as refractive interstellar scintillation (RISS). We have developed an all-sky model for RISS which can predict variability on a variety of timescales, survey locations, and observing frequencies. The model makes use of Halpha intensity maps to probe the emission measure along the line of sight, convert this to a scattering measure, and finally a scintillation strength. The model uses previously developed and long understood physics along with (indirect) measurements of the electron content and distribution within the Milky Way. We develop a set of expectations that are useful in the planning of future surveys for transient and radio variability, and demonstrate that the 1-GHz sky is a poor predictor of the variable nature of the $100$-MHz sky. Interestingly, the correlation between the incidence of variability and Galactic latitude which has been seen at 1GHz, is reversed at 100MHz. We compare the predictions of our model to a low-frequency radio survey that was conducted with the Murchison Widefield Array, and find good qualitative agreement. We discuss the implications, current limitations, and future development of the model. The model has been implemented in a Python code and is available on GitHub/Zenodo.
Stochastic Spread Pairs Trading in the Indian Commodity Market

In this study, we applied a stochastic spread pairs trading strategy on the Indian commodity market. The complete set of commodities were taken whose spot price was available for the period of January 1st 2010 to December 31st 2018 including energy, metals and the agricultural commodity sector. Spot data was taken from the MCX pooled spot prices for 17 commodities. The data was split into training period (January 1st 2010 to 14th March 2017) and testing period(15th Match 2017 to 31st December 2018). The splitting was done using a 80:20 split.Johanssen Cointegration tests were done on training data for pairs of commodities to check for long-run relationship and the cointegrated commodities were selected for formation of the trading process. We found a total of 12 cointegrated pairs out of 136 possible pairs. Cointegration was assumed for the testing period. A single-factor stochastic trading approach was applied on the logarithmic spread of the cointegrated pairs for both the training and testing period.The parameters of stochastic spread model were estimated using differential evolution algorithm. Also parameters for the trading rule were optimized by backtesting on the training period and assumed for the testing period. The results show a sharpe ratio of above 1.4 for all the commodity cointegrated pairs in the backtesing period.
Enhancing Magic Sets with an Application to Ontological Reasoning

Magic sets are a Datalog to Datalog rewriting technique to optimize query answering. The rewritten program focuses on a portion of the stable model(s) of the input program which is sufficient to answer the given query. However, the rewriting may introduce new recursive definitions, which can involve even negation and aggregations, and may slow down program evaluation. This paper enhances the magic set technique by preventing the creation of (new) recursive definitions in the rewritten program. It turns out that the new version of magic sets is closed for Datalog programs with stratified negation and aggregations, which is very convenient to obtain efficient computation of the stable model of the rewritten program. Moreover, the rewritten program is further optimized by the elimination of subsumed rules and by the efficient handling of the cases where binding propagation is lost. The research was stimulated by a challenge on the exploitation of Datalog/dlv for efficient reasoning on large ontologies. All proposed techniques have been hence implemented in the dlv system, and tested for ontological reasoning, confirming their effectiveness.
Force feedback effects on single molecule hopping and pulling experiments.

Single-molecule experiments with optical tweezers have become an important tool to study the properties and mechanisms of biological systems, such as cells and nucleic acids. In particular, force unzipping experiments have been used to extract the thermodynamics and kinetics of folding and unfolding reactions. In hopping experiments, a molecule executes transitions between the unfolded and folded states at a preset value of the force [constant force mode (CFM) under force feedback] or trap position [passive mode (PM) without feedback] and the force-dependent kinetic rates extracted from the lifetime of each state (CFM) and the rupture force distributions (PM) using the Bell-Evans model. However, hopping experiments in the CFM are known to overestimate molecular distances and folding free energies for fast transitions compared to the response time of the feedback. In contrast, kinetic rate measurements from pulling experiments have been mostly done in the PM while the CFM is seldom implemented in pulling protocols. Here, we carry out hopping and pulling experiments in a short DNA hairpin in the PM and CFM at three different temperatures (6 °C, 25 °C, and 45 °C) exhibiting largely varying kinetic rates. As expected, we find that equilibrium hopping experiments in the CFM and PM perform well at 6 °C (where kinetics are slow), whereas the CFM overestimates molecular parameters at 45 °C (where kinetics are fast). In contrast, nonequilibrium pulling experiments perform well in both modes at all temperatures. This demonstrates that the same kind of feedback algorithm in the CFM leads to more reliable determination of the folding reaction parameters in irreversible pulling experiments.
Towards learning universal, regional, and local hydrological behaviors via machine learning applied to large-sample datasets

Regional rainfall–runoff modeling is an old but still mostly outstanding problem in the hydrological sciences. The problem currently is that traditional hydrological models degrade significantly in performance when calibrated for multiple basins together instead of for a single basin alone. In this paper, we propose a novel, data-driven approach using Long Short-Term Memory networks (LSTMs) and demonstrate that under a “big data” paradigm, this is not necessarily the case. By training a single LSTM model on 531 basins from the CAMELS dataset using meteorological time series data and static catchment attributes, we were able to significantly improve performance compared to a set of several different hydrological benchmark models. Our proposed approach not only significantly outperforms hydrological models that were calibrated regionally, but also achieves better performance than hydrological models that were calibrated for each basin individually. Furthermore, we propose an adaption to the standard LSTM architecture, which we call
an Entity-Aware-LSTM (EA-LSTM), that allows for learning catchment similarities as a
feature layer in a deep learning model. We show that these learned catchment
similarities correspond well to what we would expect from prior hydrological
understanding.
Provenance Analysis for Logic and Games

A model checking computation checks whether a given logical sentence is true in a given finite structure. Provenance analysis abstracts from such a computation mathematical information on how the result depends on the atomic data that describe the structure. In database theory, provenance analysis by interpretations in commutative semirings has been rather succesful for positive query languages (such a unions of conjunctive queries, positive relational algebra, or datalog). However, it did not really offer an adequate treatment of negation or missing information. 
Here we propose a new approach for the provenance analysis of logics with negation, such as first-order logic and fixed-point logics. It is closely related to a provenance analysis of the associated model-checking games, and based on new semirings of dual-indeterminate polynomials or dual-indeterminate formal power series. These are obtained by taking quotients of traditional provenance semirings by congruences that are generated by products of positive and negative provenance tokens. Beyond the use for model-checking problems in logics, provenance analysis of games is of independent interest. Provenance values in games provide detailed information about the number and properties of the strategies of the players, far beyond the question whether or not a player has a winning strategy from a given position.
Representational Capacity of Deep Neural Networks - A Computing Study

There is some theoretical evidence that deep neural networks with multiple hidden layers have a potential for more efficient representation of multidimensional mappings than shallow networks with a single hidden layer. The question is whether it is possible to exploit this theoretical advantage for finding such representations with help of numerical training methods. Tests using prototypical problems with a known mean square minimum did not confirm this hypothesis. Minima found with the help of deep networks have always been worse than those found using shallow networks. This does not directly contradict the theoretical findings---it is possible that the superior representational capacity of deep networks is genuine while finding the mean square minimum of such deep networks is a substantially harder problem than with shallow ones.
Interactive Learning of Environment Dynamics for Sequential Tasks

In order for robots and other artificial agents to efficiently learn to perform useful tasks defined by an end user, they must understand not only the goals of those tasks, but also the structure and dynamics of that user's environment. While existing work has looked at how the goals of a task can be inferred from a human teacher, the agent is often left to learn about the environment on its own. To address this limitation, we develop an algorithm, Behavior Aware Modeling (BAM), which incorporates a teacher's knowledge into a model of the transition dynamics of an agent's environment. We evaluate BAM both in simulation and with real human teachers, learning from a combination of task demonstrations and evaluative feedback, and show that it can outperform approaches which do not explicitly consider this source of dynamics knowledge.
Empowering A* Search Algorithms with Neural Networks for Personalized Route Recommendation

Personalized Route Recommendation (PRR) aims to generate user-specific route suggestions in response to users' route queries. Early studies cast the PRR task as a pathfinding problem on graphs, and adopt adapted search algorithms by integrating heuristic strategies. Although these methods are effective to some extent, they require setting the cost functions with heuristics. In addition, it is difficult to utilize useful context information in the search procedure. To address these issues, we propose using neural networks to automatically learn the cost functions of a classic heuristic algorithm, namely A* algorithm, for the PRR task. Our model consists of two components. First, we employ attention-based Recurrent Neural Networks (RNN) to model the cost from the source to the candidate location by incorporating useful context information. Instead of learning a single cost value, the RNN component is able to learn a time-varying vectorized representation for the moving state of a user. Second, we propose to use a value network for estimating the cost from a candidate location to the destination. For capturing structural characteristics, the value network is built on top of improved graph attention networks by incorporating the moving state of a user and other context information. The two components are integrated in a principled way for deriving a more accurate cost of a candidate location. Extensive experiment results on three real-world datasets have shown the effectiveness and robustness of the proposed model.
Lévy-Ito models in finance

We propose a class of financial models in which the prices of assets are Levy-Ito processes driven by Brownian motion and a dynamic Poisson random measure. Each such model consists of a pricing kernel, a money market account, and one or more risky assets. The Poisson random measure is associated with an $n$-dimensional Levy process. We show that the excess rate of return of a risky asset in a pure-jump model is given by an integral of the product of a term representing the riskiness of the asset and a term representing the level of market risk aversion. The integral is over the state space of the Poisson random measure and is taken with respect to the Levy measure associated with the $n$-dimensional Levy process. The resulting framework is applied to the theory of interest rates and foreign exchange, allowing one to construct new models as well as various generalizations of familiar models.
Multi-Granular Text Encoding for Self-Explaining Categorization

Self-explaining text categorization requires a classifier to make a prediction along with supporting evidence. A popular type of evidence is sub-sequences extracted from the input text which are sufficient for the classifier to make the prediction. In this work, we define multi-granular ngrams as basic units for explanation, and organize all ngrams into a hierarchical structure, so that shorter ngrams can be reused while computing longer ngrams. We leverage the tree-structured LSTM to learn a context-independent representation for each unit via parameter sharing. Experiments on medical disease classification show that our model is more accurate, efficient and compact than the BiLSTM and CNN baselines. More importantly, our model can extract intuitive multi-granular evidence to support its predictions.
Predicting Human Activities from User-Generated Content

The activities we do are linked to our interests, personality, political preferences, and decisions we make about the future. In this paper, we explore the task of predicting human activities from user-generated content. We collect a dataset containing instances of social media users writing about a range of everyday activities. We then use a state-of-the-art sentence embedding framework tailored to recognize the semantics of human activities and perform an automatic clustering of these activities. We train a neural network model to make predictions about which clusters contain activities that were performed by a given user based on the text of their previous posts and self-description. Additionally, we explore the degree to which incorporating inferred user traits into our model helps with this prediction task.