nils-herrmann/clean_scirepeval huggingface dataset:
A review of keyphrase extraction

Keyphrase extraction is a textual information processing task concerned with the automatic extraction of representative and characteristic phrases from a document that express all the key aspects of its content. Keyphrases constitute a succinct conceptual summary of a document, which is very useful in digital information management systems for semantic indexing, faceted search, document clustering and classification. This article introduces keyphrase extraction, provides a well‐structured review of the existing work, offers interesting insights on the different evaluation approaches, highlights open issues and presents a comparative experimental study of popular unsupervised techniques on five datasets.
Spectral Analysis of Kernel and Neural Embeddings: Optimization and Generalization

We extend the recent results of (Arora et al. 2019). by spectral analysis of the representations corresponding to the kernel and neural embeddings. They showed that in a simple single-layer network, the alignment of the labels to the eigenvectors of the corresponding Gram matrix determines both the convergence of the optimization during training as well as the generalization properties. We generalize their result to the kernel and neural representations and show these extensions improve both optimization and generalization of the basic setup studied in (Arora et al. 2019). In particular, we first extend the setup with the Gaussian kernel and the approximations by random Fourier features as well as with the embeddings produced by two-layer networks trained on different tasks. We then study the use of more sophisticated kernels and embeddings, those designed optimally for deep neural networks and those developed for the classification task of interest given the data and the training labels, independent of any specific classification model.
Universal and nonuniversal neural dynamics on small world connectomes: A finite-size scaling analysis.

Evidence of critical dynamics has been found recently in both experiments and models of large-scale brain dynamics. The understanding of the nature and features of such a critical regime is hampered by the relatively small size of the available connectome, which prevents, among other things, the determination of its associated universality class. To circumvent that, here we study a neural model defined on a class of small-world networks that share some topological features with the human connectome. We find that varying the topological parameters can give rise to a scale-invariant behavior either belonging to the mean-field percolation universality class or having nonuniversal critical exponents. In addition, we find certain regions of the topological parameter space where the system presents a discontinuous, i.e., noncritical, dynamical phase transition into a percolated state. Overall, these results shed light on the interplay of dynamical and topological roots of the complex brain dynamics.
LSANet: Feature Learning on Point Sets by Local Spatial Attention

Directly learning features from the point cloud has become an active research direction in 3D understanding. Existing learning-based methods usually construct local regions from the point cloud and extract the corresponding features using shared Multi-Layer Perceptron (MLP) and max pooling. However, most of these processes do not adequately take the spatial distribution of the point cloud into account, limiting the ability to perceive fine-grained patterns. We design a novel Local Spatial Attention (LSA) module to adaptively generate attention maps according to the spatial distribution of local regions. The feature learning process which integrates with these attention maps can effectively capture the local geometric structure. We further propose the Spatial Feature Extractor (SFE), which constructs a branch architecture, to aggregate the spatial information with associated features in each layer of the network better.The experiments show that our network, named LSANet, can achieve on par or better performance than the state-of-the-art methods when evaluating on the challenging benchmark datasets. The source code is available at this https URL.
Improving Neural Conversational Models with Entropy-Based Data Filtering

Current neural network-based conversational models lack diversity and generate boring responses to open-ended utterances. Priors such as persona, emotion, or topic provide additional information to dialog models to aid response generation, but annotating a dataset with priors is expensive and such annotations are rarely available. While previous methods for improving the quality of open-domain response generation focused on either the underlying model or the training objective, we present a method of filtering dialog datasets by removing generic utterances from training data using a simple entropy-based approach that does not require human supervision. We conduct extensive experiments with different variations of our method, and compare dialog models across 17 evaluation metrics to show that training on datasets filtered this way results in better conversational quality as chatbots learn to output more diverse responses.
An excess of excesses examined via dark matter radio emissions from galaxies

Cosmic-ray and gamma-ray observations have yielded several notable excesses that often lend themselves to explanation by various dark matter annihilation/decay models. In particular, the AMS-02 anti-proton and positron excesses have continued to grow more robust with the collection of more data. This is supplemented by gamma-ray excesses in the Galactic Centre and a high-energy break in the spectrum of electron/positron cosmic rays seen by DAMPE . In this work we carefully model the magnetic field environments of M31 and M33 and use this to estimate expected synchrotron emissions from electrons produced via dark matter annihilation. By comparing this to available radio data we review simplifying assumptions used previously for dark matter hunting in these environments and produce novel constraints that are capable of fully ruling out dark matter models proposed to accommodate all the aforementioned excesses barring that of DAMPE . However, we do show that significant constraints can be placed upon the DAMPE parameter space with M31 data. In addition to this we project SKA non-observation constraints for the Reticulum II and Triangulum II dwarf galaxies and find these have potential to rule out cosmic-ray and gamma-ray excess-producing models of dark matter, even when the most conservative assumptions are employed.
Sense Vocabulary Compression through the Semantic Knowledge of WordNet for Neural Word Sense Disambiguation

In this article, we tackle the issue of the limited quantity of manually sense annotated corpora for the task of word sense disambiguation, by exploiting the semantic relationships between senses such as synonymy, hypernymy and hyponymy, in order to compress the sense vocabulary of Princeton WordNet, and thus reduce the number of different sense tags that must be observed to disambiguate all words of the lexical database. We propose two different methods that greatly reduce the size of neural WSD models, with the benefit of improving their coverage without additional training data, and without impacting their precision. In addition to our methods, we present a WSD system which relies on pre-trained BERT word vectors in order to achieve results that significantly outperforms the state of the art on all WSD evaluation tasks.
A Unified Linear-Time Framework for Sentence-Level Discourse Parsing

We propose an efficient neural framework for sentence-level discourse analysis in accordance with Rhetorical Structure Theory (RST). Our framework comprises a discourse segmenter to identify the elementary discourse units (EDU) in a text, and a discourse parser that constructs a discourse tree in a top-down fashion. Both the segmenter and the parser are based on Pointer Networks and operate in linear time. Our segmenter yields an F1 score of 95.4%, and our parser achieves an F1 score of 81.7% on the aggregated labeled (relation) metric, surpassing previous approaches by a good margin and approaching human agreement on both tasks (98.3 and 83.0 F1).
Predicting the density profiles of the first halos

The first dark matter halos form by direct collapse from peaks in the matter density field, and evidence from numerical simulations and other analyses suggests that the dense inner regions of these objects largely persist today. These halos would be the densest dark matter structures in the Universe, and their abundance can probe processes that leave imprints on the primordial density field, such as inflation or an early matter-dominated era. They can also probe dark matter through its free-streaming scale. The first halos are qualitatively different from halos that form by hierarchical clustering, as evidenced by their ρ∝r-3/2 inner density profiles. In this work, we present and tune models that predict the density profiles of these halos from properties of the density peaks from which they collapsed. These models predict the coefficient A of the ρ=Ar-3/2 small-radius asymptote of the density profile along with the maximum circular velocity vmax and associated radius rmax. These models are universal; they can be applied to any cosmology, and we confirm this by validating them using six N-body simulations carried out in wildly disparate cosmological scenarios. We find that these models can even predict the full population of halos with reasonable accuracy in scenarios with narrowly supported power spectra, although for broader power spectra, an understanding of the impact of halo mergers is needed. With their connection to the primordial density field established, the first dark matter halos will serve as probes of the early Universe and the nature of dark matter.
Misleading Failures of Partial-input Baselines

Recent work establishes dataset difficulty and removes annotation artifacts via partial-input baselines (e.g., hypothesis-only model for SNLI or question-only model for VQA). A successful partial-input baseline indicates that the dataset is cheatable. But the converse is not necessarily true: failures of partial-input baselines do not mean the dataset is free of artifacts. We first design artificial datasets to illustrate how the trivial patterns that are only visible in the full input can evade any partial-input baseline. Next, we identify such artifacts in the SNLI dataset—a hypothesis-only model augmented with trivial patterns in the premise can solve 15% of previously-thought “hard” examples. Our work provides a caveat for the use and creation of partial-input baselines for datasets.
Interpretable Deep Neural Networks for Patient Mortality Prediction: A Consensus-based Approach

Deep neural networks have achieved remarkable success in challenging tasks. However, the black-box approach of training and testing of such networks is not acceptable to critical applications. In particular, the existence of adversarial examples and their overgeneralization to irrelevant inputs makes it difficult, if not impossible, to explain decisions by commonly used neural networks. In this paper, we analyze the underlying mechanism of generalization of deep neural networks and propose an ($n$, $k$) consensus algorithm to be insensitive to adversarial examples and at the same time be able to reject irrelevant samples. Furthermore, the consensus algorithm is able to improve classification accuracy by using multiple trained deep neural networks. To handle the complexity of deep neural networks, we cluster linear approximations and use cluster means to capture feature importance. Due to weight symmetry, a small number of clusters are sufficient to produce a robust interpretation. Experimental results on a health dataset show the effectiveness of our algorithm in enhancing the prediction accuracy and interpretability of deep neural network models on one-year patient mortality prediction.
Electric-field-induced Z2 topological phase transition in strained single bilayer Bi(111)

For controlling critical electric fields of the topological phase transition in a single bilayer Bi(111), we investigated topological phases in a strained system through first-principles calculations. We found a quadratic band touching semimetallic state at tensile strain ϵ = 0.5%. Around this strain, the topological phase can be switched to a trivial insulator by an infinitesimal electric field. The momentum positions at which Dirac cones appear in the electric-field-induced topological phase transition changed for the strain ϵ > 0.5% and ϵ < 0.5%. Our results indicate that this topological phase transition could be applied to novel spintronic devices.
Aligning Visual Regions and Textual Concepts for Semantic-Grounded Image Representations

In vision-and-language grounding problems, fine-grained representations of the image are considered to be of paramount importance. Most of the current systems incorporate visual features and textual concepts as a sketch of an image. However, plainly inferred representations are usually undesirable in that they are composed of separate components, the relations of which are elusive. In this work, we aim at representing an image with a set of integrated visual regions and corresponding textual concepts, reflecting certain semantics. To this end, we build the Mutual Iterative Attention (MIA) module, which integrates correlated visual features and textual concepts, respectively, by aligning the two modalities. We evaluate the proposed approach on two representative vision-and-language grounding tasks, i.e., image captioning and visual question answering. In both tasks, the semantic-grounded image representations consistently boost the performance of the baseline models under all metrics across the board. The results demonstrate that our approach is effective and generalizes well to a wide range of models for image-related applications.
Stochastic inflation beyond slow roll

We study the stochastic formalism of inflation beyond the usual slow-roll approximation. We verify that the assumptions on which the stochastic formalism relies still hold even far from the slow-roll attractor. This includes demonstrating the validity of the separate universe approach to evolving long-wavelength scalar field perturbations beyond slow roll. We also explain that, in general, there is a gauge correction to the amplitude of the stochastic noise. This is because the amplitude is usually calculated in the spatially-flat gauge, while the number of e-folds is used as the time variable (hence one works in the uniform-N gauge) in the Langevin equations. We show that these corrections vanish in the slow-roll limit, but we also explain how to calculate them in general. We compute them in difference cases, including ultra-slow roll and the Starobinsky model that interpolates between slow roll and ultra-slow roll, and find the corrections to be negligible in practice. This confirms the validity of the stochastic formalism for studying quantum backreaction effects in the very early universe beyond slow roll.
Neutrino-dark matter connections in gauge theories

We discuss the connection between the origin of neutrino masses and the properties of dark matter candidates in the context of gauge extensions of the Standard Model. We investigate minimal gauge theories for neutrino masses where the neutrinos are predicted to be Dirac or Majorana fermions. We find that the upper bound on the effective number of relativistic species provides a strong constraint in the scenarios with Dirac neutrinos. In the context of theories where the lepton number is a local gauge symmetry spontaneously broken at the low scale, the existence of dark matter is predicted from the condition of anomaly cancellation. Applying the cosmological bound on the dark matter relic density, we find an upper bound on the symmetry breaking scale in the multi-TeV region. These results imply we could hope to test simple gauge theories for neutrino masses at current or future experiments.
Direct detection of strongly interacting sub-GeV dark matter via electron recoils

We consider direct-detection searches for sub-GeV dark matter via electron scatterings in the presence of large interactions between dark and ordinary matter. Scatterings both on electrons and nuclei in the Earth's crust, atmosphere, and shielding material attenuate the expected local dark matter flux at a terrestrial detector, so that such experiments lose sensitivity to dark matter above some critical cross section. We study various models, including dark matter interacting with a heavy and ultralight dark photon, through an electric dipole moment, and exclusively with electrons. For a dark-photon mediator and an electric dipole interaction, the dark matter-electron scattering cross-section is directly linked to the dark matter-nucleus cross section, and nuclear interactions typically dominate the attenuation process. We determine the exclusion bands for the different dark-matter models from several experiments—SENSEI, CDMS-HVeV, XENON10, XENON100, and DarkSide-50—using a combination of Monte Carlo simulations and analytic estimates. We also derive projected sensitivities for a detector located at different depths and for a range of exposures, and calculate the projected sensitivity for SENSEI at SNOLAB and DAMIC-M at Modane. Finally, we discuss the reach to high cross sections and the modulation signature of a small balloon- and satellite-borne detector sensitive to electron recoils, such as a Skipper-CCD. Such a detector could potentially probe unconstrained parameter space at high cross sections for a sub-dominant component of dark matter interacting with a massive, but ultralight, dark photon.
Evidence for a Circumplanetary Disk around Protoplanet PDS 70 b

We present the first observational evidence for a circumplanetary disk (CPD) around the protoplanet PDS 70 b, based on a new spectrum in the K-band acquired with Very Large Telescope/SINFONI. We tested three hypotheses to explain the spectrum: atmospheric emission from the planet with either (1) a single value of extinction, (2) a variable extinction, and (3) a combined atmospheric and CPD model. Goodness-of-fit indicators favor the third option, suggesting that circumplanetary material contributes excess thermal emission—most prominent at λ ≳ 2.3 μm. Inferred accretion rates (∼10−7.8–10−7.3 MJ yr−1) are compatible with observational constraints based on the Hα and Brγ lines. For the planet, we derive an effective temperature of 1500–1600 K, surface gravity , radius ∼1.6RJ, mass ∼10MJ, and possible thick clouds. Models with variable extinction lead to slightly worse fits. However, the amplitude (ΔAV ≳ 3 mag) and timescale of variation (≲years) required for the extinction would also suggest circumplanetary material.
Correlating Neural and Symbolic Representations of Language

Analysis methods which enable us to better understand the representations and functioning of neural models of language are increasingly needed as deep learning becomes the dominant approach in NLP. Here we present two methods based on Representational Similarity Analysis (RSA) and Tree Kernels (TK) which allow us to directly quantify how strongly the information encoded in neural activation patterns corresponds to information represented by symbolic structures such as syntax trees. We first validate our methods on the case of a simple synthetic language for arithmetic expressions with clearly defined syntax and semantics, and show that they exhibit the expected pattern of results. We then our methods to correlate neural representations of English sentences with their constituency parse trees.
Controlled CNN-based Sequence Labeling for Aspect Extraction

One key task of fine-grained sentiment analysis on reviews is to extract aspects or features that users have expressed opinions on. This paper focuses on supervised aspect extraction using a modified CNN called controlled CNN (Ctrl). The modified CNN has two types of control modules. Through asynchronous parameter updating, it prevents over-fitting and boosts CNN's performance significantly. This model achieves state-of-the-art results on standard aspect extraction datasets. To the best of our knowledge, this is the first paper to apply control modules to aspect extraction.
From What to How. An Overview of AI Ethics Tools, Methods and Research to Translate Principles into Practices

The debate about the ethical implications of Artificial Intelligence dates from the 1960s. However, in recent years symbolic AI has been complemented and sometimes replaced by Neural Networks and Machine Learning techniques. This has vastly increased its potential utility and impact on society, with the consequence that the ethical debate has gone mainstream. Such debate has primarily focused on principles - the what of AI ethics - rather than on practices, the how. Awareness of the potential issues is increasing at a fast rate, but the AI community's ability to take action to mitigate the associated risks is still at its infancy. Therefore, our intention in presenting this research is to contribute to closing the gap between principles and practices by constructing a typology that may help practically-minded developers apply ethics at each stage of the pipeline, and to signal to researchers where further work is needed. The focus is exclusively on Machine Learning, but it is hoped that the results of this research may be easily applicable to other branches of AI. The article outlines the research method for creating this typology, the initial findings, and provides a summary of future research needs.
ERNIE: Enhanced Language Representation with Informative Entities

Neural language representation models such as BERT pre-trained on large-scale corpora can well capture rich semantic patterns from plain text, and be fine-tuned to consistently improve the performance of various NLP tasks. However, the existing pre-trained language models rarely consider incorporating knowledge graphs (KGs), which can provide rich structured knowledge facts for better language understanding. We argue that informative entities in KGs can enhance language representation with external knowledge. In this paper, we utilize both large-scale textual corpora and KGs to train an enhanced language representation model (ERNIE), which can take full advantage of lexical, syntactic, and knowledge information simultaneously. The experimental results have demonstrated that ERNIE achieves significant improvements on various knowledge-driven tasks, and meanwhile is comparable with the state-of-the-art model BERT on other common NLP tasks. The code and datasets will be available in the future.
End-to-end Adaptation with Backpropagation through WFST for On-device Speech Recognition System

An on-device DNN-HMM speech recognition system efficiently works with a limited vocabulary in the presence of a variety of predictable noise. In such a case, vocabulary and environment adaptation is highly effective. In this paper, we propose a novel method of end-to-end (E2E) adaptation, which adjusts not only an acoustic model (AM) but also a weighted finite-state transducer (WFST). We convert a pretrained WFST to a trainable neural network and adapt the system to target environments/vocabulary by E2E joint training with an AM. We replicate Viterbi decoding with forward--backward neural network computation, which is similar to recurrent neural networks (RNNs). By pooling output score sequences, a vocabulary posterior for each utterance is obtained and used for discriminative loss computation. Experiments using 2--10 hours of English/Japanese adaptation datasets indicate that the fine-tuning of only WFSTs and that of only AMs are both comparable to a state-of-the-art adaptation method, and E2E joint training of the two components achieves the best recognition performance. We also adapt each language system to the other language using the adaptation data, and the results show that the proposed method also works well for language adaptations.