nils-herrmann/clean_scirepeval huggingface dataset:
Biomedical relation extraction with pre-trained language representations and minimal task-specific architecture

This paper presents our participation in the AGAC Track from the 2019 BioNLP Open Shared Tasks. We provide a solution for Task 3, which aims to extract “gene - function change - disease” triples, where “gene” and “disease” are mentions of particular genes and diseases respectively and “function change” is one of four pre-defined relationship types. Our system extends BERT (Devlin et al., 2018), a state-of-the-art language model, which learns contextual language representations from a large unlabelled corpus and whose parameters can be fine-tuned to solve specific tasks with minimal additional architecture. We encode the pair of mentions and their textual context as two consecutive sequences in BERT, separated by a special symbol. We then use a single linear layer to classify their relationship into five classes (four pre-defined, as well as ‘no relation’). Despite considerable class imbalance, our system significantly outperforms a random baseline while relying on an extremely simple setup with no specially engineered features.
Constraining a causal dissipative cosmological model

In this paper a cosmological solution of polynomial type $H \approx ( t + const.)^{-1}$ for the causal thermodynamical approach of Isarel-Stewart, found in \cite{MCruz:2017, Cruz2017}, is constrained using the joint of the latest measurements of the Hubble parameter (OHD) and Type Ia Supernovae (SNIa). Since the expansion described by this solution does not present a transition from a decelerated phase to an accelerated one, both phases can be well modeled connecting both phases by requiring the continuity of the Hubble parameter at $z=z_{t}$, the accelerated-decelerated transition redshift. Our best fit constrains the main free parameters of the model to be $A_1= 1.58^{+0.08}_{-0.07}$ ($A_2=0.84^{+0.02}_{-0.02}$) for the accelerated (decelerated) phase. For both phases we obtain $q=-0.37^{+0.03}_{-0.03}$ ($0.19^{+0.03}_{-0.03}$) and $\omega_{eff} = -0.58^{+0.02}_{-0.02}$ ($-0.21^{+0.02}_{-0.02}$) for the deceleration parameter and the effective equation of state, respectively. Comparing our model and LCDM statistically through the Akaike information criterion and the Bayesian information criterion we obtain that the LCDM model is preferred by the OHD+SNIa data. Finally, it is shown that the constrained parameters values satisfy the criterion for a consistent fluid description of a dissipative dark matter component, but with a high value of the speed of sound within the fluid, which is a drawback for a consistent description of the structure formation. We briefly discuss the possibilities to overcome this problem with a non-linear generalization of the causal linear thermodynamics of bulk viscosity and also with the inclusion of some form of dark energy.
NGTS-10b: the shortest period hot Jupiter yet discovered

We report the discovery of a new ultrashort period (USP) transiting hot Jupiter from the Next Generation Transit Survey (NGTS). NGTS-10b has a mass and radius of $2.162\, ^{+0.092}_{-0.107}$ MJ and $1.205\, ^{+0.117}_{-0.083}$ RJ and orbits its host star with a period of 0.7668944 ± 0.0000003 d, making it the shortest period hot Jupiter yet discovered. The host is a 10.4 ± 2.5 Gyr old K5V star (Teff = 4400 ± 100 K) of Solar metallicity ([Fe/H] = −0.02 ± 0.12 dex) showing moderate signs of stellar activity. NGTS-10b joins a short list of USP Jupiters that are prime candidates for the study of star–planet tidal interactions. NGTS-10b orbits its host at just 1.46 ± 0.18 Roche radii, and we calculate a median remaining inspiral time of 38 Myr and a potentially measurable orbital period decay of 7 s over the coming decade, assuming a stellar tidal quality factor $Q^{\prime }_{\rm s}$ =2 × 107.
A Re-classification of Information Seeking Tasks and Their Computational Solutions

This article presents a re-classification of information seeking (IS) tasks, concepts, and algorithms. The proposed taxonomy provides new dimensions to look into information seeking tasks and methods. The new dimensions include number of search iterations, search goal types, and procedures to reach these goals. Differences along these dimensions for the information seeking tasks call for suitable computational solutions. The article then reviews machine learning solutions that match each new category. The article ends with a review of evaluation campaigns for IS systems.
Learning the Difference that Makes a Difference with Counterfactually-Augmented Data

Despite alarm over the reliance of machine learning systems on so-called spurious patterns, the term lacks coherent meaning in standard statistical frameworks. However, the language of causality offers clarity: spurious associations are due to confounding (e.g., a common cause), but not direct or indirect causal effects. In this paper, we focus on natural language processing, introducing methods and resources for training models less sensitive to spurious patterns. Given documents and their initial labels, we task humans with revising each document so that it (i) accords with a counterfactual target label; (ii) retains internal coherence; and (iii) avoids unnecessary changes. Interestingly, on sentiment analysis and natural language inference tasks, classifiers trained on original data fail on their counterfactually-revised counterparts and vice versa. Classifiers trained on combined datasets perform remarkably well, just shy of those specialized to either domain. While classifiers trained on either original or manipulated data alone are sensitive to spurious features (e.g., mentions of genre), models trained on the combined data are less sensitive to this signal. Both datasets are publicly available.
Improving Pre-Trained Multilingual Model with Vocabulary Expansion

Recently, pre-trained language models have achieved remarkable success in a broad range of natural language processing tasks. However, in multilingual setting, it is extremely resource-consuming to pre-train a deep language model over large-scale corpora for each language. Instead of exhaustively pre-training monolingual language models independently, an alternative solution is to pre-train a powerful multilingual deep language model over large-scale corpora in hundreds of languages. However, the vocabulary size for each language in such a model is relatively small, especially for low-resource languages. This limitation inevitably hinders the performance of these multilingual models on tasks such as sequence labeling, wherein in-depth token-level or sentence-level understanding is essential. In this paper, inspired by previous methods designed for monolingual settings, we investigate two approaches (i.e., joint mapping and mixture mapping) based on a pre-trained multilingual model BERT for addressing the out-of-vocabulary (OOV) problem on a variety of tasks, including part-of-speech tagging, named entity recognition, machine translation quality estimation, and machine reading comprehension. Experimental results show that using mixture mapping is more promising. To the best of our knowledge, this is the first work that attempts to address and discuss the OOV issue in multilingual settings.
Total Least Squares Regression in Input Sparsity Time

In the total least squares problem, one is given an $m \times n$ matrix $A$, and an $m \times d$ matrix $B$, and one seeks to ``correct'' both $A$ and $B$, obtaining matrices $\hat{A}$ and $\hat{B}$, so that there exists an $X$ satisfying the equation $\hat{A}X = \hat{B}$. Typically the problem is overconstrained, meaning that $m \gg \max(n,d)$. The cost of the solution $\hat{A}, \hat{B}$ is given by $\|A-\hat{A}\|_F^2 + \|B - \hat{B}\|_F^2$. We give an algorithm for finding a solution $X$ to the linear system $\hat{A}X=\hat{B}$ for which the cost $\|A-\hat{A}\|_F^2 + \|B-\hat{B}\|_F^2$ is at most a multiplicative $(1+\epsilon)$ factor times the optimal cost, up to an additive error $\eta$ that may be an arbitrarily small function of $n$. Importantly, our running time is $\tilde{O}(\nnz(A) + \nnz(B)) + \poly(n/\epsilon) \cdot d$, where for a matrix $C$, $\nnz(C)$ denotes its number of non-zero entries. Importantly, our running time does not directly depend on the large parameter $m$. As total least squares regression is known to be solvable via low rank approximation, a natural approach is to invoke fast algorithms for approximate low rank approximation, obtaining matrices $\hat{A}$ and $\hat{B}$ from this low rank approximation, and then solving for $X$ so that $\hat{A}X = \hat{B}$. However, existing algorithms do not apply since in total least squares the rank of the low rank approximation needs to be $n$, and so the running time of known methods would be at least $mn^2$. In contrast, we are able to achieve a much faster running time for finding $X$ by never explicitly forming the equation $\hat{A} X = \hat{B}$, but instead solving for an $X$ which is a solution to an implicit such equation. Finally, we generalize our algorithm to the total least squares problem with regularization.
Interpreting Undesirable Pixels for Image Classification on Black-Box Models

In an effort to interpret black-box models, researches for developing explanation methods have proceeded in recent years. Most studies have tried to identify input pixels that are crucial to the prediction of a classifier. While this approach is meaningful to analyse the characteristic of black-box models, it is also important to investigate pixels that interfere with the prediction. To tackle this issue, in this paper, we propose an explanation method that visualizes undesirable regions to classify an image as a target class. To be specific, we divide the concept of undesirable regions into two terms: (1) factors for a target class, which hinder that black-box models identify intrinsic characteristics of a target class and (2) factors for non-target classes that are important regions for an image to be classified as other classes. We visualize such undesirable regions on heatmaps to qualitatively validate the proposed method. Furthermore, we present an evaluation metric to provide quantitative results on ImageNet.
Nonlinear Dynamic Models of Conflict via Multiplexed Interaction Networks

The risk of conflict is exasperated by a multitude of internal and external factors. Current multivariate analysis paints diverse causal risk profiles that vary with time. However, these profiles evolve and a universal model to understand that evolution remains absent. Most of the current conflict analysis is data-driven and conducted at the individual country or region level, often in isolation. Consistent consideration of multi-scale interactions and their non-linear dynamics is missing. Here, we develop a multiplexed network model, where each city is modelled as a non-linear bi-stable system with stable states in either war or peace. The causal factor categories which exasperate the risk of conflict are each modelled as a network layer. We consider 3 layers: (1) core geospatial network of interacting cities reflecting ground level interactions, (2) cultural network of interacting countries reflecting cultural groupings, and (3) political network of interacting countries reflecting alliances. Together, they act as drivers to push cities towards or pull cities away from war. Using a variety of data sources relative to 2002-2016, we show, that our model correctly predicts the transitions from war to peace and peace to war with F1 score of 0.78 to 0.92 worldwide at the city scale resolution. As many conflicts during this period are auto-regressive (e.g. the War on Terror in Afghanistan and Iraq, the Narco War across the Americas), we can predict the emergence of new war or new peace. We demonstrate successful predictions across a wide range of conflict genres and we perform causal discovery by identifying which model component led to the correct prediction. In the cases of Somalia (2008-13), Myanmar (2013-15), Colombia (2011-14), Libya (2014-16), and Yemen (2011-13) we identify the set of most likely causal factors and how it may differ across a country and change over time.
Open-source neuronavigation for multimodal non-invasive brain stimulation using 3D Slicer

In recent years, non-invasive neuro-modulation methods such as Focused Ultrasound (FUS) have gained popularity. The aim of this work is to introduce the use of existing open-source technology for surgical navigation to the field of multimodal non-invasive brain stimulation. Unlike homegrown and commercial systems, the use of well-documented, well maintained, and freely available open-source components minimizes the learning curve, maximizes technology transfer outcome, and fosters reproducible science for complex, guided neuromodulation systems. The described system significantly lowers the entry bar to clinical research and experimentation in the field of non-invasive brain stimulation. Our contribution is two-fold. First, a high-level overview of the components of the descried system is given in this manuscript. Second, all files are made available online, with a comprehensive step-by-step manual, quickly allowing researchers to build a custom system. A spatial accuracy of 0.93 mm was found through validation using a robotic positioning system.
Leveraging Multimodal Haptic Sensory Data for Robust Cutting

Cutting is a common form of manipulation when working with divisible objects such as food, rope, or clay. Cooking in particular relies heavily on cutting to divide food items into desired shapes. However, cutting food is a challenging task due to the wide range of material properties exhibited by food items. Due to this variability, the same cutting motions cannot be used for all food items. Sensations from contact events, e.g., when placing the knife on the food item, will also vary depending on the material properties, and the robot will need to adapt accordingly. In this paper, we propose using vibrations and force-torque feedback from the interactions to adapt the slicing motions and monitor for contact events. The robot learns neural networks for performing each of these tasks and generalizing across different material properties. By adapting and monitoring the skill executions, the robot is able to reliably cut through more than 20 different types of food items and even detect whether certain food items are fresh or old.
Playing Atari Ball Games with Hierarchical Reinforcement Learning

Human beings are particularly good at reasoning and inference from just a few examples. When facing new tasks, humans will leverage knowledge and skills learned before, and quickly integrate them with the new task. In addition to learning by experimentation, human also learn socio-culturally through instructions and learning by example. In this way humans can learn much faster compared with most current artificial intelligence algorithms in many tasks. In this paper, we test the idea of speeding up machine learning through social learning. We argue that in solving real-world problems, especially when the task is designed by humans, and/or for humans, there are typically instructions from user manuals and/or human experts which give guidelines on how to better accomplish the tasks. We argue that these instructions have tremendous value in designing a reinforcement learning system which can learn in human fashion, and we test the idea by playing the Atari games Tennis and Pong. We experimentally demonstrate that the instructions provide key information about the task, which can be used to decompose the learning task into sub-systems and construct options for the temporally extended planning, and dramatically accelerate the learning process.
A novel metric for community detection

Detection of dense communities has recently attracted increasing attention within network science and various metrics for detection of such communities have been proposed. The most popular metric —modularity— is based on the rule that the links within communities are denser than external links among communities. However, the principle of this metric suffers from ambiguity, and is based on a narrow intuition of what it means to form a “community”. Instead we propose that the defining characteristic of a community is that links are more predictable within a community rather than between communities. In this letter, based on the effect of communities on link prediction, we propose a novel metric for community detection based directly on this property. We find that our metric is more robust than traditional modularity measures for each specific algorithm. Finally, we provide a measure of the improvement offered by our metric.
DMM-Net: Differentiable Mask-Matching Network for Video Object Segmentation

In this paper, we propose the differentiable mask-matching network (DMM-Net) for solving the video object segmentation problem where the initial object masks are provided. Relying on the Mask R-CNN backbone, we extract mask proposals per frame and formulate the matching between object templates and proposals as a linear assignment problem where thA heading inside a blocke cost matrix is predicted by a deep convolutional neural network. We propose a differentiable matching layer which unrolls a projected gradient descent algorithm in which the projection step exploits the Dykstra's algorithm. We prove that under mild conditions, the matching is guaranteed to converge to the optimal one. In practice, it achieves similar performance compared to the Hungarian algorithm during inference. Meanwhile, we can back-propagate through it to learn the cost matrix. After matching, a U-Net style architecture is exploited to refine the matched mask per time step. On DAVIS 2017 dataset, DMM-Net achieves the best performance without online learning on the first frames and the 2nd best with it. Without any fine-tuning, DMM-Net performs comparably to state-of-the-art methods on SegTrack v2 dataset. At last, our differentiable matching layer is very simple to implement; we attach the PyTorch code in the supplementary material which is less than $50$ lines long.
A Radio Signal Modulation Recognition Algorithm Based on Residual Networks and Attention Mechanisms

To solve the problem of inaccurate recognition of types of communication signal modulation, a RNN neural network recognition algorithm combining residual block network with attention mechanism is proposed. In this method, 10 kinds of communication signals with Gaussian white noise are generated from standard data sets, such as MASK, MPSK, MFSK, OFDM, 16QAM, AM and FM. Based on the original RNN neural network, residual block network is added to solve the problem of gradient disappearance caused by deep network layers. Attention mechanism is added to the network to accelerate the gradient descent. In the experiment, 16QAM, 2FSK and 4FSK are used as actual samples, IQ data frames of signals are used as input, and the RNN neural network combined with residual block network and attention mechanism is trained. The final recognition results show that the average recognition rate of real-time signals is over 93%. The network has high robustness and good use value.
Stratified Space Learning: Reconstructing Embedded Graphs

Many data-rich industries are interested in the efficient discovery and modelling of structures underlying large data sets, as it allows for the fast triage and dimension reduction of large volumes of data embedded in high dimensional spaces. The modelling of these underlying structures is also beneficial for the creation of simulated data that better represents real data. In particular, for systems testing in cases where the use of real data streams might prove impractical or otherwise undesirable. We seek to discover and model the structure by combining methods from topological data analysis with numerical modelling. As a first step in combining these two areas, we examine the recovery of the abstract graph $G$ structure, and model a linear embedding $|G|$ given only a noisy point cloud sample $X$ of $|G|$.
Reweighted Proximal Pruning for Large-Scale Language Representation

Recently, pre-trained language representation flourishes as the mainstay of the natural language understanding community, e.g., BERT. These pre-trained language representations can create state-of-the-art results on a wide range of downstream tasks. Along with continuous significant performance improvement, the size and complexity of these pre-trained neural models continue to increase rapidly. Is it possible to compress these large-scale language representation models? How will the pruned language representation affect the downstream multi-task transfer learning objectives? In this paper, we propose Reweighted Proximal Pruning (RPP), a new pruning method specifically designed for a large-scale language representation model. Through experiments on SQuAD and the GLUE benchmark suite, we show that proximal pruned BERT keeps high accuracy for both the pre-training task and the downstream multiple fine-tuning tasks at high prune ratio. RPP provides a new perspective to help us analyze what large-scale language representation might learn. Additionally, RPP makes it possible to deploy a large state-of-the-art language representation model such as BERT on a series of distinct devices (e.g., online servers, mobile phones, and edge devices).
Classification of time-domain waveforms using a speckle-based optical reservoir computer.

Reservoir computing is a recurrent machine learning framework that expands the dimensionality of a problem by mapping an input signal into a higher-dimension reservoir space that can capture and predict features of complex, non-linear temporal dynamics. Here, we report on a bulk electro-optical demonstration of a reservoir computer using speckles generated by propagating a laser beam modulated with a spatial light modulator through a multimode waveguide. We demonstrate that the hardware can successfully perform a multivariate audio classification task performed using the Japanese vowel speakers public data set. We perform full wave optical calculations of this architecture implemented in a chip-scale platform using an SiO2 waveguide and demonstrate that it performs as well as a fully numerical implementation of reservoir computing. As all the optical components used in the experiment can be fabricated using a commercial photonic integrated circuit foundry, our result demonstrates a framework for building a scalable, chip-scale, reservoir computer capable of performing optical signal processing.
Invisible Marker: Automatic Annotation for Object Manipulation

We propose invisible marker for accurate automatic annotation to manipulate objects. Invisible marker is invisible in visible light, whereas it can be visible by applying ultraviolet light in the dark. By capturing images while alternately switching between visible and invisible light at high speed, massive annotation datasets for objects painted with invisible marker are created quickly and inexpensively. We show comparison with manual annotation and demonstrations of semantic segmentation by deep learning for deformable objects such as cloth, liquid, and powder.