nils-herrmann/clean_scirepeval huggingface dataset:
Semi-Supervised Neural Text Generation by Joint Learning of Natural Language Generation and Natural Language Understanding Models

In Natural Language Generation (NLG), End-to-End (E2E) systems trained through deep learning have recently gained a strong interest. Such deep models need a large amount of carefully annotated data to reach satisfactory performance. However, acquiring such datasets for every new NLG application is a tedious and time-consuming task. In this paper, we propose a semi-supervised deep learning scheme that can learn from non-annotated data and annotated data when available. It uses a NLG and a Natural Language Understanding (NLU) sequence-to-sequence models which are learned jointly to compensate for the lack of annotation. Experiments on two benchmark datasets show that, with limited amount of annotated data, the method can achieve very competitive results while not using any pre-processing or re-scoring tricks. These findings open the way to the exploitation of non-annotated datasets which is the current bottleneck for the E2E NLG system development to new applications.
Controlled Text Generation for Data Augmentation in Intelligent Artificial Agents

Data availability is a bottleneck during early stages of development of new capabilities for intelligent artificial agents. We investigate the use of text generation techniques to augment the training data of a popular commercial artificial agent across categories of functionality, with the goal of faster development of new functionality. We explore a variety of encoder-decoder generative models for synthetic training data generation and propose using conditional variational auto-encoders. Our approach requires only direct optimization, works well with limited data and significantly outperforms the previous controlled text generation techniques. Further, the generated data are used as additional training samples in an extrinsic intent classification task, leading to improved performance by up to 5% absolute f-score in low-resource cases, validating the usefulness of our approach.
Neural Language Priors

The choice of sentence encoder architecture reflects assumptions about how a sentence's meaning is composed from its constituent words. We examine the contribution of these architectures by holding them randomly initialised and fixed, effectively treating them as as hand-crafted language priors, and evaluating the resulting sentence encoders on downstream language tasks. We find that even when encoders are presented with additional information that can be used to solve tasks, the corresponding priors do not leverage this information, except in an isolated case. We also find that apparently uninformative priors are just as good as seemingly informative priors on almost all tasks, indicating that learning is a necessary component to leverage information provided by architecture choice.
Fake news detection using Deep Learning

News is a routine in everyone's life. It helps in
enhancing the knowledge on what happens around the world.
Fake news is a fictional information madeup with the intension to
delude and hence the knowledge acquired becomes of no use. As
fake news spreads extensively it has a negative impact in the
society and so fake news detection has become an emerging
research area. The paper deals with a solution to fake news
detection using the methods, deep learning and Natural Language
Processing. The dataset is trained using deep neural network. The
dataset needs to be well formatted before given to the network
which is made possible using the technique of Natural Language
Processing and thus predicts whether a news is fake or not.
SentiCite - An Approach for Publication Sentiment Analysis

With the rapid growth in the number of scientific publications, year after year, it is becoming increasingly difficult to identify quality authoritative work on a single topic. Though there is an availability of scientometric measures which promise to offer a solution to this problem, these measures are mostly quantitative and rely, for instance, only on the number of times an article is cited. With this approach, it becomes irrelevant if an article is cited 10 times in a positive, negative or neutral way. In this context, it is quite important to study the qualitative aspect of a citation to understand its significance. This paper presents a novel system for sentiment analysis of citations in scientific documents (SentiCite) and is also capable of detecting nature of citations by targeting the motivation behind a citation, e.g., reference to a dataset, reading reference. Furthermore, the paper also presents two datasets (SentiCiteDB and IntentCiteDB) containing about 2,600 citations with their ground truth for sentiment and nature of citation. SentiCite along with other state-of-the-art methods for sentiment analysis are evaluated on the presented datasets. Evaluation results reveal that SentiCite outperforms state-of-the-art methods for sentiment analysis in scientific publications by achieving a F1-measure of 0.71.
Investigating the Effectiveness of Representations Based on Word-Embeddings in Active Learning for Labelling Text Datasets

Manually labelling large collections of text data is a time-consuming, expensive, and laborious task, but one that is necessary to support machine learning based on text datasets. Active learning has been shown to be an effective way to alleviate some of the effort required in utilising large collections of unlabelled data for machine learning tasks without needing to fully label them. The representation mechanism used to represent text documents when performing active learning, however, has a significant influence on how effective the process will be. While simple vector representations such as bag of words have been shown to be an effective way to represent documents during active learning, the emergence of representation mechanisms based on the word embeddings prevalent in neural network research (e.g. word2vec and transformer-based models like BERT) offer a promising, and as yet not fully explored, alternative. This paper describes a large-scale evaluation of the effectiveness of different text representation mechanisms for active learning across 8 datasets from varied domains. This evaluation shows that using representations based on modern word embeddings---especially BERT---, which have not yet been widely used in active learning, achieves a significant improvement over more commonly used vector-based methods like bag of words.
Sharp nonzero lower bounds for the Schur product theorem

By a result of Schur [], the entrywise product $M \circ N$ of two positive semidefinite matrices $M,N$ is again positive. Vybiral (2019) improved on this by showing the uniform lower bound $M \circ \overline{M} \geq E_n / n$ for all $n \times n$ real or complex correlation matrices $M$, where $E_n$ is the all-ones matrix. This was applied to settle a conjecture of Novak [J. Complexity 1999] and to positive definite functions on groups. Vybiral then asked if one can obtain similar uniform lower bounds for higher entrywise powers of $M$, or for $M \circ N$ when $N \neq M, \overline{M}$. A natural third question is to obtain a tighter lower bound that need not vanish, over infinite-dimensional Hilbert spaces as well. In this note, we affirmatively answer all three questions by extending and refining Vybiral's result to lower bound $M \circ N$ for arbitrary positive $M, N$. As a special case, the above bound of $E_n/n$ can be improved to $E_n / {\rm rk}(M)$. In addition, our lower bounds - which we show are tracial Cauchy-Schwartz inequalities - are sharp. This is followed by a variant for Hilbert-Schmidt operators. We end with consequences for positive definite functions on groups, metric spaces, and Hilbert spaces.
Gravitational lensing of the cosmic neutrino background

We study gravitational lensing of the cosmic neutrino background. This signal is undetectable for the foreseeable future, but there is a rich trove of information available. At least some of the neutrinos from the early universe will be non-relativistic today, with a closer surface of last scattering (compared to the cosmic microwave background) and with larger angles of deflection. Lensing of massive neutrinos is strongly chromatic: both the amplitude of lensing and the cosmic time at which the potential is traversed depend on neutrino momentum, in principle giving access to our entire causal volume, not restricted to the light cone. As a concrete example, we focus on the case where the cosmic neutrino background would be strongly lensed when passing through halos of galaxy clusters and galaxies. We calculate the Einstein radius for cosmic neutrinos and investigate the impact of neutrino mass.
Deep Network classification by Scattering and Homotopy dictionary learning

We introduce a sparse scattering deep convolutional neural network, which provides a simple model to analyze properties of deep representation learning for classification. Learning a single dictionary matrix with a classifier yields a higher classification accuracy than AlexNet over the ImageNet 2012 dataset. The network first applies a scattering transform that linearizes variabilities due to geometric transformations such as translations and small deformations. A sparse $\ell^1$ dictionary coding reduces intra-class variability while preserving class separation through projections over unions of linear spaces. It is implemented in a deep convolutional network with a homotopy algorithm having an exponential convergence. A convergence proof is given in a general framework that includes ALISTA. Classification results are analyzed on ImageNet.
Dynamic Brain Functional Networks Guided By Anatomical Knowledge

Recently, the potential of dynamic brain networks as a neuroimaging biomarkers for mental illnesses is being increasingly recognized. However, there are several unmet challenges in developing such biomarkers, including the need for methods to model rapidly changing network states. In one of the first such efforts, we develop a novel approach for computing dynamic brain functional connectivity (FC), that is guided by brain structural connectivity (SC) computed from diffusion tensor imaging (DTI) data. The proposed approach involving dynamic Gaussian graphical models decomposes the time course into non-overlapping state phases determined by change points, each having a distinct network. We develop an optimization algorithm to implement the method such that the estimation of both the change points and the state-phase specific networks are fully data driven and unsupervised, and guided by SC information. The approach is scalable to large dimensions and extensive simulations illustrate its clear advantages over existing methods in terms of network estimation accuracy and detecting dynamic network changes. An application of the method to a posttraumatic stress disorder (PTSD) study reveals important dynamic resting state connections in regions of the brain previously implicated in PTSD. We also illustrate that the dynamic networks computed under the proposed method are able to better predict psychological resilience among trauma exposed individuals compared to existing dynamic and stationary connectivity approaches, which highlights its potential as a neuroimaging biomarker.
Stack Sorting with Increasing and Decreasing Stacks

We introduce a sorting machine consisting of $k+1$ stacks in series: the first $k$ stacks can only contain elements in decreasing order from top to bottom, while the last one has the opposite restriction. This device generalizes the $\mathfrak{DI}$ machine introduced by Rebecca Smith, which studies the case $k=1$. Here we show that, for $k=2$, the set of sortable  permutations is a class with infinite basis, by explicitly finding an antichain of minimal nonsortable permutations. This construction can easily be adapted to each $k\geqslant 3$. Next we describe an optimal sorting algorithm, again for the case $k=2$. We then analyze two types of left-greedy sorting procedures, obtaining complete results in one case and only some partial results in the other one. We close the paper by discussing a few open questions.
Graph-Based Spatio-Temporal Feature Learning for Neuromorphic Vision Sensing

Neuromorphic vision sensing (NVS) devices represent visual information as sequences of asynchronous discrete events (a.k.a., “spikes”) in response to changes in scene reflectance. Unlike conventional active pixel sensing (APS), NVS allows for significantly higher event sampling rates at substantially increased energy efficiency and robustness to illumination changes. However, feature representation for NVS is far behind its APS-based counterparts, resulting in lower performance in high-level computer vision tasks. To fully utilize its sparse and asynchronous nature, we propose a compact graph representation for NVS, which allows for end-to-end learning with graph convolution neural networks. We couple this with a novel end-to-end feature learning framework that accommodates both appearance-based and motion-based tasks. The core of our framework comprises a spatial feature learning module, which utilizes residual-graph convolutional neural networks (RG-CNN), for end-to-end learning of appearance-based features directly from graphs. We extend this with our proposed Graph2Grid block and temporal feature learning module for efficiently modelling temporal dependencies over multiple graphs and a long temporal extent. We show how our framework can be configured for object classification, action recognition and action similarity labeling. Importantly, our approach preserves the spatial and temporal coherence of spike events, while requiring less computation and memory. The experimental validation shows that our proposed framework outperforms all recent methods on standard datasets. Finally, to address the absence of large real-world NVS datasets for complex recognition tasks, we introduce, evaluate and make available the American Sign Language letters (ASL-DVS), as well as human action dataset (UCF101-DVS, HMDB51-DVS and ASLAN-DVS).
Visualizing dissipative charge-carrier dynamics at the nanoscale with superconducting-charge-qubit microscopy

The investigation of novel electronic phases in low-dimensional quantum materials demands for the concurrent development of measurement techniques that combine surface sensitivity with high spatial resolution and high measurement accuracy. We propose a new quantum sensing imaging modality based on superconducting charge qubits to study dissipative charge carrier dynamics with nanometer spatial and high temporal resolution. Using analytical and numerical calculations we show that superconducting charge qubit microscopy (SCQM) has the potential to resolve temperature and resistivity changes in a sample as small as ∆T ≤ 0.1 mK and ∆ρ ≤ 1 · 10 Ω · cm, respectively. Among other applications, SCQM will be especially suited to study the microscopic mechanisms underlying resistive phase transition, such as the superconductor-insulator-transition in twisted bilayer graphene, to investigate novel topological boundary modes found in higher order topological insulators and to optimize the transport properties of nanoand mesoscopic devices.
The Role of the Parker Instability in Structuring the Interstellar Medium

The Parker instability, a Rayleigh–Taylor-like instability of thermal gas supported against gravity by magnetic fields and cosmic rays, is thought to be dynamically important for galaxy evolution, possibly promoting molecular cloud formation and the galactic dynamo. In previous work, we examined the effect of three different cosmic-ray transport models on the Parker instability: decoupled (γc = 0), locked to the thermal gas (γc = 4/3), and coupled to the gas with streaming by self-confinement. We expand on that work here by considering radiative cooling, a smooth gravitational potential, and simulations into the nonlinear regime. We determine that cosmic-ray transport away from compression points, whether by diffusion or streaming, is the largest driver of the instability. Heating due to cosmic-ray streaming is also destabilizing and especially affects the nonlinear regime. While cooling depressurizes the dense gas, streaming cosmic rays heat and inflate the diffuse extraplanar gas, greatly modifying the phase structure of the medium. In 3D, we find that the fastest growth favors short-wavelength modes in the horizontal direction perpendicular to the background magnetic field; this is imprinted on Faraday rotation measure maps that may be used to detect the Parker instability. The modifications to the Parker instability that we observe in this work have large implications for the structure and evolution of galaxies, and they highlight the major role that cosmic rays play in shaping their environments.
Weighing the stellar constituents of the galactic halo with APOGEE red giant stars

The stellar mass in the halo of the Milky Way is notoriously difficult to determine, owing to the paucity of its stars in the solar neighbourhood. With tentative evidence from Gaia that the nearby stellar halo is dominated by a massive accretion event – referred to as Gaia-Enceladus or Sausage – these constraints are now increasingly urgent. We measure the mass in kinematically selected mono-abundance populations (MAPs) of the stellar halo between −3 < [Fe/H] < −1 and 0.0 < [Mg/Fe] < 0.4 using red giant star counts from APOGEE DR14. We find that MAPs are well fit by single power laws on triaxial ellipsoidal surfaces, and we show that that the power-law slope α changes such that high [Mg/Fe] populations have α ∼ 4, whereas low [Mg/Fe] MAPs are more extended with shallow slopes, α ∼ 2. We estimate the total stellar mass to be $M_{*,\mathrm{tot}} = 1.3^{+0.3}_{-0.2}\times 10^{9}\ \mathrm{M_{\odot}}$, of which we estimate ${\sim}0.9^{+0.2}_{-0.1} \times 10^{9}\ \mathrm{M_{\odot}}$ to be accreted. We estimate that the mass of accreted stars with e > 0.7 is M*,accreted, e > 0.7 = 3 ± 1 (stat.) ± 1 (syst.) × 108 M⊙, or ${\sim}30{-}50{{\ \rm per\ cent}}$ of the accreted halo mass. If the majority of these stars are the progeny of a massive accreted dwarf, this places an upper limit on its stellar mass, and implies a halo mass for the progenitor of ∼1010.2 ± 0.2 M⊙. This constraint not only shows that the Gaia-Enceladus/Sausage progenitor may not be as massive as originally suggested, but that the majority of the Milky Way stellar halo was accreted. These measurements are an important step towards fully reconstructing the assembly history of the Milky Way.
Multi-wavelength Properties of Radio- and Machine-learning-identified Counterparts to Submillimeter Sources in S2COSMOS

We identify multi-wavelength counterparts to 1147 submillimeter sources from the S2COSMOS SCUBA-2 survey of the COSMOS field by employing a recently developed radio+machine-learning method trained on a large sample of Atacama Large Millimeter/submillimeter Array (ALMA)–identified submillimeter galaxies (SMGs), including 260 SMGs identified in the AS2COSMOS pilot survey. In total, we identify 1222 optical/near-infrared (NIR)/radio counterparts to the 897 S2COSMOS submillimeter sources with S850 > 1.6 mJy, yielding an overall identification rate of (78 ± 9)%. We find that (22 ± 5)% of S2COSMOS sources have multiple identified counterparts. We estimate that roughly 27% of these multiple counterparts within the same SCUBA-2 error circles very likely arise from physically associated galaxies rather than line-of-sight projections by chance. The photometric redshift of our radio+machine-learning-identified SMGs ranges from z = 0.2 to 5.7 and peaks at z = 2.3 ± 0.1. The AGN fraction of our sample is (19 ± 4)%, which is consistent with that of ALMA SMGs in the literature. Comparing with radio/NIR-detected field galaxy population in the COSMOS field, our radio+machine-learning-identified counterparts of SMGs have the highest star formation rates and stellar masses. These characteristics suggest that our identified counterparts of S2COSMOS sources are a representative sample of SMGs at z ≲ 3. We employ our machine-learning technique to the whole COSMOS field and identified 6877 potential SMGs, most of which are expected to have submillimeter emission fainter than the confusion limit of our S2COSMOS surveys ( mJy). We study the clustering properties of SMGs based on this statistically large sample, finding that they reside in high-mass dark matter halos ((1.2 ± 0.3) × 1013 h−1 ), which suggests that SMGs may be the progenitors of massive ellipticals we see in the local universe.
The Two LIGO/Virgo Binary Black Hole Mergers on 2019 August 28 Were Not Strongly Lensed

The LIGO/Virgo gravitational wave events S190828j and S190828l were detected only 21 minutes apart, from nearby regions of sky, and with the same source classifications (binary black hole mergers). It is therefore natural to speculate that the two signals are actually strongly lensed images of the same merger. However, an estimate of the separation of the (unknown) positions of the two events requires them to be >10 deg apart, much wider than the arcsecond-scale separations that usually arise in extragalactic lensing. The large separation is much more consistent with two independent, unrelated events that occurred close in time by chance. We quantify the overlap between simulated pairs of lensed events, and use frequentist hypothesis testing to reject S190828j/l as a lensed pair at 99.8% confidence.
Constraining the scatter in the galaxy–halo connection at Milky Way masses

We develop and implement two new methods for constraining the scatter in the relationship between galaxies and dark matter halos. These new techniques are sensitive to the scatter at low halo masses, making them complementary to previous constraints that are dependent on clustering amplitudes or rich galaxy groups, both of which are only sensitive to more massive halos. In both of our methods, we use a galaxy group finder to locate central galaxies in the SDSS main galaxy sample. Our first technique uses the small-scale cross-correlation of central galaxies with all lower mass galaxies. This quantity is sensitive to the satellite fraction of low-mass galaxies, which is in turn driven by the scatter between halos and galaxies. The second technique uses the kurtosis of the distribution of line-of-sight velocities between central galaxies and neighboring galaxies. This quantity is sensitive to the distribution of halo masses that contain the central galaxies at fixed stellar mass. Theoretical models are constructed using peak halo circular velocity, $V_{\rm peak}$, as our property to connect galaxies to halos. The cross-correlation technique yields a constraint of $\sigma[ M_\ast|V_{\rm peak}]=0.27\pm 0.05$ dex, corresponding to a scatter in $\log M_\ast$ at fixed $M_h$ of $\sigma[ M_\ast|M_h]=0.38\pm 0.06$ dex at $M_h=10^{11.8}$ Msun. The kurtosis technique yields $\sigma[ M_\ast|V_{\rm peak}]=0.30\pm0.03$, corresponding to $\sigma[ M_\ast|M_h]=0.34\pm 0.04$ at $M_h=10^{12.2}$ Msun. The values of $\sigma[ M_\ast|M_h]$ are significantly larger than the constraints at higher masses, in agreement with the results of hydrodynamic simulations. This increase is only partly due to the scatter between $V_{\rm peak}$ and $M_h$, and it represents an increase of nearly a factor of two relative to the values inferred from clustering and group studies at high masses.