nils-herrmann/clean_scirepeval huggingface dataset:
Question Answering based Clinical Text Structuring Using Pre-trained Language Model

Clinical text structuring is a critical and fundamental task for clinical research. Traditional methods such as task-specific end-to-end models and pipeline models usually suffer from the lack of dataset and error propagation. In this paper, we present a question answering based clinical text structuring (QA-CTS) task to unify different specific CTS tasks and make dataset shareable. A novel model that aims to introduce domain-specific features (e.g., clinical named entity information) into pre-trained language model is also proposed for QA-CTS task. Experimental results on Chinese pathology reports collected from Ruijing Hospital demonstrate our presented QA-CTS task is very effective to improve the performance on specific tasks. Our proposed model also competes favorably with strong baseline models in specific tasks.
RANet: Ranking Attention Network for Fast Video Object Segmentation

Despite online learning (OL) techniques have boosted the performance of semi-supervised video object segmentation (VOS) methods, the huge time costs of OL greatly restricts their practicality. Matching based and propagation based methods run at a faster speed by avoiding OL techniques. However, they are limited by sub-optimal accuracy, due to mismatching and drifting problems. In this paper, we develop a real-time yet very accurate Ranking Attention Network (RANet) for VOS. Specifically, to integrate the insights of matching based and propagation based methods, we employ an encoder-decoder framework to learn pixel-level similarity and segmentation in an end-to-end manner. To better utilize the similarity maps, we propose a novel ranking attention module, which automatically ranks and selects these maps for fine-grained VOS performance. Experiments on DAVIS16 and DAVIS17 datasets show that our RANet achieves the best speed-accuracy trade-off, e.g., with 33 milliseconds per frame and J&F=85.5% on DAVIS16. With OL, our RANet reaches J&F=87.1% on DAVIS16, exceeding state-of-the-art VOS methods. The code can be found at https://github.com/Storife/RANet.
Learning to Play the Chess Variant Crazyhouse Above World Champion Level With Deep Neural Networks and Human Data

Deep neural networks have been successfully applied in learning the board games Go, chess, and shogi without prior knowledge by making use of reinforcement learning. Although starting from zero knowledge has been shown to yield impressive results, it is associated with high computationally costs especially for complex games. With this paper, we present CrazyAra which is a neural network based engine solely trained in supervised manner for the chess variant crazyhouse. Crazyhouse is a game with a higher branching factor than chess and there is only limited data of lower quality available compared to AlphaGo. Therefore, we focus on improving efficiency in multiple aspects while relying on low computational resources. These improvements include modifications in the neural network design and training configuration, the introduction of a data normalization step and a more sample efficient Monte-Carlo tree search which has a lower chance to blunder. After training on 569537 human games for 1.5 days we achieve a move prediction accuracy of 60.4%. During development, versions of CrazyAra played professional human players. Most notably, CrazyAra achieved a four to one win over 2017 crazyhouse world champion Justin Tan (aka LM Jann Lee) who is more than 400 Elo higher rated compared to the average player in our training set. Furthermore, we test the playing strength of CrazyAra on CPU against all participants of the second Crazyhouse Computer Championships 2017, winning against twelve of the thirteen participants. Finally, for CrazyAraFish we continue training our model on generated engine games. In 10 long-time control matches playing Stockfish 10, CrazyAraFish wins three games and draws one out of 10 matches.
Intrinsically Motivated Discovery of Diverse Patterns in Self-Organizing Systems

In many complex dynamical systems, artificial or natural, one can observe self-organization of patterns emerging from local rules. Cellular automata, like the Game of Life (GOL), have been widely used as abstract models enabling the study of various aspects of self-organization and morphogenesis, such as the emergence of spatially localized patterns. However, findings of self-organized patterns in such models have so far relied on manual tuning of parameters and initial states, and on the human eye to identify interesting patterns. In this paper, we formulate the problem of automated discovery of diverse self-organized patterns in such high-dimensional complex dynamical systems, as well as a framework for experimentation and evaluation. Using a continuous GOL as a testbed, we show that recent intrinsically-motivated machine learning algorithms (POP-IMGEPs), initially developed for learning of inverse models in robotics, can be transposed and used in this novel application area. These algorithms combine intrinsically-motivated goal exploration and unsupervised learning of goal space representations. Goal space representations describe the interesting features of patterns for which diverse variations should be discovered. In particular, we compare various approaches to define and learn goal space representations from the perspective of discovering diverse spatially localized patterns. Moreover, we introduce an extension of a state-of-the-art POP-IMGEP algorithm which incrementally learns a goal representation using a deep auto-encoder, and the use of CPPN primitives for generating initialization parameters. We show that it is more efficient than several baselines and equally efficient as a system pre-trained on a hand-made database of patterns identified by human experts.
Towards Assessing the Impact of Bayesian Optimization's Own Hyperparameters

Bayesian Optimization (BO) is a common approach for hyperparameter optimization (HPO) in automated machine learning. Although it is well-accepted that HPO is crucial to obtain well-performing machine learning models, tuning BO's own hyperparameters is often neglected. In this paper, we empirically study the impact of optimizing BO's own hyperparameters and the transferability of the found settings using a wide range of benchmarks, including artificial functions, HPO and HPO combined with neural architecture search. In particular, we show (i) that tuning can improve the any-time performance of different BO approaches, that optimized BO settings also perform well (ii) on similar problems and (iii) partially even on problems from other problem families, and (iv) which BO hyperparameters are most important.
Network constraints in scale free dynamical systems.

Scale free dynamics are observed in a variety of physical and biological systems. These include neural activity in which evidence for scale freeness has been reported using a range of imaging modalities. Here, we derive the ways in which connections within a network must transform - relative to system size - in order to maintain scale freeness and test these theoretical transformations via simulations. First, we explore the known invariance of planetary motion for orbits varying in size. Using parametric empirical Bayesian modelling and a generic dynamical systems model, we show that we recover Kepler's third law from orbital timeseries, using our proposed transformations; thereby providing construct validation. We then demonstrate that the dynamical critical exponent is inversely proportional to the time rescaling exponent, in the context of coarse graining operations. Using murine calcium imaging data, we then show that the dynamical critical exponent can be estimated in an empirical biological setting. Specifically, we compare dynamical critical exponents - associated with spontaneous and task states in two regions of imaged cortex - that are classified as task-relevant and task-irrelevant. We find, consistently across animals, that the task-irrelevant region exhibits higher dynamical critical exponents during spontaneous activity than during task performance. Conversely, the task-relevant region is associated with higher dynamical critical exponents in task vs. spontaneous states. These data support the idea that higher dynamical critical exponents, within relevant cortical structures, underwrite neuronal processing due to the implicit increase in cross-scale information transmission.
Scalar field effects on the orbit of S2 star

Precise measurements of the S-stars orbiting SgrA* have set strong constraints on the nature of the compact object at the centre of the Milky Way. The presence of a black hole in that region is well established, but its neighbouring environment is still an open debate. In that respect, the existence of dark matter in that central region may be detectable due to its strong signatures on the orbits of stars: the main effect is a Newtonian precession which will affect the overall pericentre shift of S2, the latter being a target measurement of the GRAVITY instrument. The exact nature of this dark matter (e.g. stellar dark remnants or diffuse dark matter) is unknown. This article assumes it to be a scalar field of toroidal distribution, associated with ultralight dark matter particles, surrounding the Kerr black hole. Such a field is a form of ‘hair’ expected in the context of superradiance, a mechanism that extracts rotational energy from the black hole. Orbital signatures for the S2 star are computed and shown to be detectable by GRAVITY. The scalar field can be constrained because the variation of orbital elements depends both on the relative mass of the scalar field to the black hole and on the field mass coupling parameter.
Improving multilevel regression and poststratification with structured priors.

A central theme in the field of survey statistics is estimating population-level quantities through data coming from potentially non-representative samples of the population. Multilevel regression and poststratification (MRP), a model-based approach, is gaining traction against the traditional weighted approach for survey estimates. MRP estimates are susceptible to bias if there is an underlying structure that the methodology does not capture. This work aims to provide a new framework for specifying structured prior distributions that lead to bias reduction in MRP estimates. We use simulation studies to explore the benefit of these prior distributions and demonstrate their efficacy on non-representative US survey data. We show that structured prior distributions offer absolute bias reduction and variance reduction for posterior MRP estimates in a large variety of data regimes.
Enhancing the Demand for Labour survey by including skills from online job advertisements using model-assisted calibration

In the article we describe an enhancement to the Demand for Labour (DL) survey conducted by Statistics Poland, which involves the inclusion of skills obtained from online job advertisements. The main goal is to provide estimates of the demand for skills (competences), which is missing in the DL survey. To achieve this, we apply a data integration approach combining traditional calibration with the LASSO-assisted approach to correct representation error in the online data. Faced with the lack of access to unit-level data from the DL survey, we use estimated population totals and propose a~bootstrap approach that accounts for the uncertainty of totals reported by Statistics Poland. We show that the calibration estimator assisted with LASSO outperforms traditional calibration in terms of standard errors and reduces representation bias in skills observed in online job ads. Our empirical results show that online data significantly overestimate interpersonal, managerial and self-organization skills while underestimating technical and physical skills. This is mainly due to the under-representation of occupations categorised as Craft and Related Trades Workers and Plant and Machine Operators and Assemblers.
Maize yield and nitrate loss prediction with machine learning algorithms

Pre-growing season prediction of crop production outcomes such as grain yields and nitrogen (N) losses can provide insights to farmers and agronomists to make decisions. Simulation crop models can assist in scenario planning, but their use is limited because of data requirements and long runtimes. Thus, there is a need for more computationally expedient approaches to scale up predictions. We evaluated the potential of four machine learning (ML) algorithms (LASSO Regression, Ridge Regression, random forests, Extreme Gradient Boosting, and their ensembles) as meta-models for a cropping systems simulator (APSIM) to inform future decision support tool development. We asked: (1) How well do ML meta-models predict maize yield and N losses using pre-season information? (2) How many data are needed to train ML algorithms to achieve acceptable predictions? (3) Which input data variables are most important for accurate prediction? And (4) do ensembles of ML meta-models improve prediction? The simulated dataset included more than three million data including genotype, environment and management scenarios. XGBoost was the most accurate ML model in predicting yields with a relative mean square error (RRMSE) of 13.5%, and Random forests most accurately predicted N loss at planting time, with a RRMSE of 54%. ML meta-models reasonably reproduced simulated maize yields using the information available at planting, but not N loss. They also differed in their sensitivities to the size of the training dataset. Across all ML models, yield prediction error decreased by 10%–40% as the training dataset increased from 0.5 to 1.8 million data points, whereas N loss prediction error showed no consistent pattern. ML models also differed in their sensitivities to input variables (weather, soil properties, management, initial conditions), thus depending on the data availability researchers may use a different ML model. Modest prediction improvements resulted from ML ensembles. These results can help accelerate progress in coupling simulation models and ML toward developing dynamic decision support tools for pre-season management.
BOAH: A Tool Suite for Multi-Fidelity Bayesian Optimization & Analysis of Hyperparameters

Hyperparameter optimization and neural architecture search can become prohibitively expensive for regular black-box Bayesian optimization because the training and evaluation of a single model can easily take several hours. To overcome this, we introduce a comprehensive tool suite for effective multi-fidelity Bayesian optimization and the analysis of its runs. The suite, written in Python, provides a simple way to specify complex design spaces, a robust and efficient combination of Bayesian optimization and HyperBand, and a comprehensive analysis of the optimization process and its outcomes.
Iterative Update and Unified Representation for Multi-Agent Reinforcement Learning

Multi-agent systems have a wide range of applications in cooperative and competitive tasks. As the number of agents increases, nonstationarity gets more serious in multi-agent reinforcement learning (MARL), which brings great difficulties to the learning process. Besides, current mainstream algorithms configure each agent an independent network,so that the memory usage increases linearly with the number of agents which greatly slows down the interaction with the environment. Inspired by Generative Adversarial Networks (GAN), this paper proposes an iterative update method (IU) to stabilize the nonstationary environment. Further, we add first-person perspective and represent all agents by only one network which can change agents' policies from sequential compute to batch compute. Similar to continual lifelong learning, we realize the iterative update method in this unified representative network (IUUR). In this method, iterative update can greatly alleviate the nonstationarity of the environment, unified representation can speed up the interaction with environment and avoid the linear growth of memory usage. Besides, this method does not bother decentralized execution and distributed deployment. Experiments show that compared with MADDPG, our algorithm achieves state-of-the-art performance and saves wall-clock time by a large margin especially with more agents.
Systematic search for stellar pulsators in the eclipsing binaries observed by Kepler

Eclipsing binaries (EBs) are unique targets for measuring precise stellar properties and can be used to constrain stellar evolution models. In particular, it is possible to measure masses and radii of both components of a double-lined spectroscopic EB at the percent level. Since the advent of high-precision photometric space missions (MOST, CoRoT, Kepler, BRITE, TESS), the use of stellar pulsation properties to infer stellar interiors and dynamics constitutes a revolution for studies of low-mass stars. The Kepler mission has led to the discovery of thousands of classical pulsators such as δ Scuti and solar-like oscillators (main sequence and evolved), but also almost 3000 EBs with orbital periods shorter than 1100 days. We report the first systematic search for stellar pulsators in the entire Kepler EB catalog. The focus is mainly aimed at discovering δ Scuti, γ Doradus, red giant, and tidally excited pulsators. We developed a data inspection tool (DIT) that automatically produces a series of plots from the Kepler light curves that allows us to visually identify whether stellar oscillations are present in a given time series. We applied the DIT to the whole Kepler EB database and identified 303 systems whose light curves display oscillations, including 163 new discoveries. A total of 149 stars are flagged as δ Scuti (100 from this paper), 115 as γ Doradus (69 new), 85 as red giants (27 new), and 59 as tidally excited oscillators (29 new). There is some overlap among these groups, as some display several types of oscillations. Despite the likelihood that many of these systems are false positives, for example, when an EB light curve is blended with a pulsator, this catalog gathers a vast sample of systems that are valuable for a better understanding of stellar evolution.
Wi-Fringe: Leveraging Text Semantics in WiFi CSI-Based Device-Free Named Gesture Recognition

The lack of adequate training data is one of the major hurdles in WiFi-based activity recognition systems. In this paper, we propose Wi-Fringe, which is a WiFi CSI-based devicefree human gesture recognition system that recognizes named gestures, i.e., activities and gestures that have a semantically meaningful name in English language, as opposed to arbitrary free-form gestures. Given a list of activities (only their names in English text), along with zero or more training examples (WiFi CSI values) per activity, Wi-Fringe is able to detect all activities at runtime. We show for the first time that by utilizing the state-of-the-art semantic representation of English words, which is learned from datasets like the Wikipedia (e.g., Google’s word-to-vector ) and verb attributes learned from how a word is defined (e.g, American Heritage Dictionary), we can enhance the capability of WiFi-based named gesture recognition systems that lack adequate training examples per class. We propose a novel cross-domain knowledge transfer algorithm between radio frequency (RF) and text to lessen the burden on developers and end-users from the tedious task of data collection for all possible activities. To evaluate Wi-Fringe, we collect data from four volunteers in a multi-person apartment and an office building for a total of 20 activities. We empirically quantify the trade-off between the accuracy and the number of unseen activities.
Style Transfer for Texts: Retrain, Report Errors, Compare with Rewrites

This paper shows that standard assessment methodology for style transfer has several significant problems. First, the standard metrics for style accuracy and semantics preservation vary significantly on different re-runs. Therefore one has to report error margins for the obtained results. Second, starting with certain values of bilingual evaluation understudy (BLEU) between input and output and accuracy of the sentiment transfer the optimization of these two standard metrics diverge from the intuitive goal of the style transfer task. Finally, due to the nature of the task itself, there is a specific dependence between these two metrics that could be easily manipulated. Under these circumstances, we suggest taking BLEU between input and human-written reformulations into consideration for benchmarks. We also propose three new architectures that outperform state of the art in terms of this metric.
GLAMpoints: Greedily Learned Accurate Match Points

We introduce a novel CNN-based feature point detector - Greedily Learned Accurate Match Points (GLAMpoints) - learned in a semi-supervised manner. Our detector extracts repeatable, stable interest points with a dense coverage, specifically designed to maximize the correct matching in a specific domain, which is in contrast to conventional techniques that optimize indirect metrics. In this paper, we apply our method on challenging retinal slitlamp images, for which classical detectors yield unsatisfactory results due to low image quality and insufficient amount of low-level features. We show that GLAMpoints significantly outperforms classical detectors as well as state-of-the-art CNN-based methods in matching and registration quality for retinal images.
Are You for Real? Detecting Identity Fraud via Dialogue Interactions

Identity fraud detection is of great importance in many real-world scenarios such as the financial industry. However, few studies addressed this problem before. In this paper, we focus on identity fraud detection in loan applications and propose to solve this problem with a novel interactive dialogue system which consists of two modules. One is the knowledge graph (KG) constructor organizing the personal information for each loan applicant. The other is structured dialogue management that can dynamically generate a series of questions based on the personal KG to ask the applicants and determine their identity states. We also present a heuristic user simulator based on problem analysis to evaluate our method. Experiments have shown that the trainable dialogue system can effectively detect fraudsters, and achieve higher recognition accuracy compared with rule-based systems. Furthermore, our learned dialogue strategies are interpretable and flexible, which can help promote real-world applications.