nils-herrmann/clean_scirepeval huggingface dataset:
Unsupervised Context Retrieval for Long-tail Entities

Monitoring entities in media streams often relies on rich entity representations, like structured information available in a knowledge base (KB). For long-tail entities, such monitoring is highly challenging, due to their limited, if not entirely missing, representation in the reference KB. In this paper, we address the problem of retrieving textual contexts for monitoring long-tail entities. We propose an unsupervised method to overcome the limited representation of long-tail entities by leveraging established entities and their contexts as support information. Evaluation on a purpose-built test collection shows the suitability of our approach and its robustness for out-of-KB entities.
Measurement of the cross sections for inclusive electron scattering in the E12-14-012 experiment at Jefferson Lab

The E12-14-012 experiment performed at Jefferson Lab Hall A has collected inclusive electron-scattering data for different targets at the kinematics corresponding to beam energy 2.222 GeV and scattering angle 15.54 deg. Here we present a comprehensive analysis of the collected data and compare the double-differential cross sections for inclusive scattering of electrons, extracted using solid targets (aluminum, carbon, and titanium) and a closed argon-gas cell. The data extend over broad range of energy transfer, where quasielastic interaction, Delta-resonance excitation, and inelastic scattering yield contributions to the cross section. The double-differential cross sections are reported with high precision (~3%) for all targets over the covered kinematic range.
Scaling relations for globular cluster systems in early-type galaxies

The formation and growth of globular cluster systems (GCSs) is closely related to the evolutionary processes experienced by their host galaxies. In particular, their radial distributions scale with several properties of the galaxies and their haloes. We performed a photometric study, by means of HST/ACS archival data of several intermediate luminosity galaxies located in low-density environments. It was supplemented with available photometric data of GCSs from the Virgo and Fornax clusters, resulting in a sample of almost 30 GCSs for which we fitted their radial profiles. The resulting overall properties agree with those from previous studies, as we found that the effective radius, extension, and concentration of the GCS radial profiles correlate with the stellar mass, effective radius, and number of globular clusters, presenting in some cases a bilinear relation. The extension also correlates with the central velocity dispersion for central galaxies, but not for satellites. From a statistical comparison with numerical simulations we obtained good agreement between the effective radius and extension of the GCS scale with the effective and virial radius of the haloes, respectively. Finally, we analysed these results in the literature context.
Evaluating Pest Management Strategies: A Robust Method and its Application to Strawberry Disease Management

Farmers use pesticides to reduce yield losses. The efficacies of pesticide treatments are often evaluated by analyzing the average treatment effects and risks. The stochastic efficiency with respect to a function is often employed in such evaluations through ranking the certainty equivalents of each treatment. The main challenge of using this method is gathering an adequate number of observations to produce results with statistical power. However, in many cases, only a limited number of trials are replicated in field experiments, leaving an inadequate number of observations. In addition, this method focuses only on the farmer's profit without incorporating the impact of disease pressure on yield and profit. The objective of our study is to propose a methodology to address the issue of an insufficient number of observations using simulations and take into account the effect of disease pressure on yield through a quantile regression model. We apply this method to the case of strawberry disease management in Florida.
Optimal Joins using Compact Data Structures

Worst-case optimal join algorithms have gained a lot of attention in the database literature. We now count with several algorithms that are optimal in the worst case, and many of them have been implemented and validated in practice. However, the implementation of these algorithms often requires an enhanced indexing structure: to achieve optimality we either need to build completely new indexes, or we must populate the database with several instantiations of indexes such as B$+$-trees. Either way, this means spending an extra amount of storage space that may be non-negligible. 
We show that optimal algorithms can be obtained directly from a representation that regards the relations as point sets in variable-dimensional grids, without the need of extra storage. Our representation is a compact quad tree for the static indexes, and a dynamic quadtree sharing subtrees (which we dub a qdag) for intermediate results. We develop a compositional algorithm to process full join queries under this representation, and show that the running time of this algorithm is worst-case optimal in data complexity. Remarkably, we can extend our framework to evaluate more expressive queries from relational algebra by introducing a lazy version of qdags (lqdags). Once again, we can show that the running time of our algorithms is worst-case optimal.
Pars-ABSA: An Aspect-based Sentiment Analysis Dataset in Persian

Due to the increased availability of online reviews, sentiment analysis had been witnessed a booming interest from the researchers. Sentiment analysis is a computational treatment of sentiment used to extract and understand the opinions of authors. While many systems were built to predict the sentiment of a document or a sentence, many others provide the necessary detail on various aspects of the entity (i.e. aspect-based sentiment analysis). Most of the available data resources were tailored to English and the other popular European languages. Although Persian is a language with more than 110 million speakers, to the best of our knowledge, there is a lack of public dataset on aspect-based sentiment analysis for Persian. This paper provides a manually annotated Persian dataset, Pars-ABSA, which is verified by 3 native Persian speakers. The dataset consists of 5,114 positive, 3,061 negative and 1,827 neutral data samples from 5,602 unique reviews. Moreover, as a baseline, this paper reports the performance of some state-of-the-art aspect-based sentiment analysis methods with a focus on deep learning, on Pars-ABSA. The obtained results are impressive compared to similar English state-of-the-art.
Sparsity Emerges Naturally in Neural Language Models

Concerns about interpretability, computational resources, and principled inductive priors have motivated efforts to engineer sparse neural models for NLP tasks. If sparsity is important for NLP, might well-trained neural models naturally become roughly sparse? Using the Taxi-Euclidean norm to measure sparsity, we find that frequent input words are associated with concentrated or sparse activations, while frequent target words are associated with dispersed activations but concentrated gradients. We find that gradients associated with function words are more concentrated than the gradients of content words, even controlling for word frequency.
Dialogue Act Classification in Group Chats with DAG-LSTMs

Dialogue act (DA) classification has been studied for the past two decades and has several key applications such as workflow automation and conversation analytics. Researchers have used, to address this problem, various traditional machine learning models, and more recently deep neural network models such as hierarchical convolutional neural networks (CNNs) and long short-term memory (LSTM) networks. In this paper, we introduce a new model architecture, directed-acyclic-graph LSTM (DAG-LSTM) for DA classification. A DAG-LSTM exploits the turn-taking structure naturally present in a multi-party conversation, and encodes this relation in its model structure. Using the STAC corpus, we show that the proposed method performs roughly 0.8% better in accuracy and 1.2% better in macro-F1 score when compared to existing methods. The proposed method is generic and not limited to conversation applications.
Structured Knowledge Discovery from Massive Text Corpus

Nowadays, with the booming development of the Internet, people benefit from its convenience due to its open and sharing nature. A large volume of natural language texts is being generated by users in various forms, such as search queries, documents, and social media posts. As the unstructured text corpus is usually noisy and messy, it becomes imperative to correctly identify and accurately annotate structured information in order to obtain meaningful insights or better understand unstructured texts. On the other hand, the existing structured information, which embodies our knowledge such as entity or concept relations, often suffers from incompleteness or quality-related issues. Given a gigantic collection of texts which offers rich semantic information, it is also important to harness the massiveness of the unannotated text corpus to expand and refine existing structured knowledge with fewer annotation efforts. 
In this dissertation, I will introduce principles, models, and algorithms for effective structured knowledge discovery from the massive text corpus. We are generally interested in obtaining insights and better understanding unstructured texts with the help of structured annotations or by structure-aware modeling. Also, given the existing structured knowledge, we are interested in expanding its scale and improving its quality harnessing the massiveness of the text corpus. In particular, four problems are studied in this dissertation: Structured Intent Detection for Natural Language Understanding, Structure-aware Natural Language Modeling, Generative Structured Knowledge Expansion, and Synonym Refinement on Structured Knowledge.
Multi-turn Dialogue Response Generation with Autoregressive Transformer Models

Neural dialogue models, despite their successes, still suffer from lack of relevance, diversity, and in many cases coherence in their generated responses. These issues have been attributed to reasons including (1) short-range model architectures that capture limited temporal dependencies, (2) limitations of the maximum likelihood training objective, (3) the concave entropy profile of dialogue datasets resulting into short and generic responses, and (4) out-of-vocabulary problem leading to generation of a large number of <UNK> tokens. Autoregressive transformer models such as GPT-2, although trained with the maximum likelihood objective, do not suffer from the out-of-vocabulary problem and have demonstrated an excellent ability to capture long-range structures in language modeling tasks. In this paper, we examine the use of autoregressive transformer models for multi-turn dialogue response generation. In our experiments, we employ small and medium GPT-2 models (with publicly available pretrained language model parameters) on the opendomain Movie Triples dataset and the closed-domain Ubuntu Dialogue dataset. The models (with and without pretraining) achieve significant improvements over the baselines for multi-turn dialogue response generation. They also produce state-of-the-art performance on the two datasets based on several metrics, including BLEU, ROGUE, and distinct n-gram.
Nonparametric Regression on Low-Dimensional Manifolds using Deep ReLU Networks

Real world data often exhibit low-dimensional geometric structures, and can be viewed as samples near a low-dimensional manifold. This paper studies nonparametric regression of H\"older functions on low-dimensional manifolds using deep ReLU networks. Suppose $n$ training data are sampled from a H\"older function in $\mathcal{H}^{s,\alpha}$ supported on a $d$-dimensional Riemannian manifold isometrically embedded in $\mathbb{R}^D$, with sub-gaussian noise. A deep ReLU network architecture is designed to estimate the underlying function from the training data. The mean squared error of the empirical estimator is proved to converge in the order of $n^{-\frac{2(s+\alpha)}{2(s+\alpha) + d}}\log^3 n$. This result shows that deep ReLU networks give rise to a fast convergence rate depending on the data intrinsic dimension $d$, which is usually much smaller than the ambient dimension $D$. It therefore demonstrates the adaptivity of deep ReLU networks to low-dimensional geometric structures of data, and partially explains the power of deep ReLU networks in tackling high-dimensional data with low-dimensional geometric structures.
Test of lepton flavor universality in $B \to K \ell^{+}\ell^{-}$ decays

We present measurements of the branching fractions for the decays $B\to K \mu^{+}\mu^{-}$ and $B\to K e^{+}e^{-}$, and their ratio ($R_{K}$), using a data sample of 711 $fb^{-1}$ that contains $772 \times 10^{6}$ $B\bar{B}$ events. The data were collected at the $\Upsilon(4S)$ resonance with the Belle detector at the KEKB asymmetric-energy $e^{+}e^{-}$ collider. The ratio $R_{K}$ is measured in four bins of dilepton invariant-mass squared, $q^{2}$; the results are \begin{eqnarray*} R_{K} = \begin{cases} 0.95~ ^{+0.27}_{-0.24} \pm 0.06 & q^{2} \in (0.1,4.0)~\mathrm{\,GeV^2}c^4 \, , 0.81~ ^{+0.28}_{-0.23} \pm 0.05 & q^{2} \in (4.0,8.12)~\mathrm{\,GeV^2}c^4 \, , 0.98~ ^{+0.27}_{-0.23} \pm 0.06 & q^{2} \in (1.0,6.0)~\mathrm{\,GeV^2}c^4 \, , 1.11~ ^{+0.29}_{-0.26} \pm 0.07 & q^{2} > 14.18~\mathrm{\,GeV^2}c^4 \, . \end{cases} \end{eqnarray*} The first uncertainties listed are statistical, and the second uncertainties are systematic. The $R_{K}$ value in the whole $q^2$ range is found to be $1.06~ ^{+0.15}_{-0.14} \pm  We also measure $CP-$averaged isospin asymmetries in the same $q^{2}$ bins; the results are consistent with a null asymmetry with the largest difference of 2.7 standard deviations is found in the $q^{2}\in(1.0,6.0)~\mathrm{\,GeV^2}c^4 \,$ bin in the mode with muon final states. The measured branching fractions are $\cal B\rm{\it(B\to K \mu^{+}\mu^{-})}= (5.5 \pm0.5 \pm0.3) \times 10^{-7}$ and $\cal B\rm{\it(B\to K e^{+}e^{-})} = (5.1 \pm 0.5 \pm0.3) \times 10^{-7}$. These results are compatible with standard model expectations.
Self-Knowledge Distillation in Natural Language Processing

Since deep learning became a key player in natural language processing (NLP), many deep learning models have been showing remarkable performances in a variety of NLP tasks. Such high performance can be explained by efficient knowledge representation of deep learning models. Knowledge distillation from pretrained deep networks suggests that we can use more information from the soft target probability to train other neural networks. In this paper, we propose a self-knowledge distillation method, based on the soft target probabilities of the training model itself, where multimode information is distilled from the word embedding space right below the softmax layer. Due to the time complexity, our method approximates the soft target probabilities. In experiments, we applied the proposed method to two different and fundamental NLP tasks: language model and neural machine translation. The experiment results show that our proposed method improves performance on the tasks.
DELTA: A DEep learning based Language Technology plAtform

In this paper we present DELTA, a deep learning based language technology platform. DELTA is an end-to-end platform designed to solve industry level natural language and speech processing problems. It integrates most popular neural network models for training as well as comprehensive deployment tools for production. DELTA aims to provide easy and fast experiences for using, deploying, and developing natural language processing and speech models for both academia and industry use cases. We demonstrate the reliable performance with DELTA on several natural language processing and speech tasks, including text classification, named entity recognition, natural language inference, speech recognition, speaker verification, etc. DELTA has been used for developing several state-of-the-art algorithms for publications and delivering real production to serve millions of users.
Semiautomatic Labeling for Deep Learning in Robotics

In this article, we propose an augmented reality semiautomatic labeling (ARS), a semiautomatic method which leverages on moving a 2-D camera by means of a robot, proving precise camera tracking, and an augmented reality pen (ARP) to define initial object bounding box, to create large labeled data sets with minimal human intervention. By removing the burden of generating annotated data from humans, we make the deep learning technique applied to computer vision, which typically requires very large data sets, truly automated and reliable. With the ARS pipeline, we created two novel data sets effortlessly, one on electromechanical components (industrial scenario) and other on fruits (daily-living scenario) and trained two state-of-the-art object detectors robustly, based on convolutional neural networks, such as you only look once (YOLO) and single shot detector (SSD). With respect to conventional manual annotation of 1000 frames that takes us slightly more than 10 h, the proposed approach based on ARS allows to annotate 9 sequences of about 35 000 frames in less than 1 h, with a gain factor of about 450. Moreover, both the precision and recall of object detection is increased by about 15% with respect to manual labeling. All our software is available as a robot operating system (ROS) package in a public repository alongside with the novel annotated data sets. Note to Practitioners—This article was motivated by the lack of a simple and effective solution for the generation of data sets usable to train a data-driven model, such as a modern deep neural network, so as to make them accessible in an industrial environment. Specifically, a deep learning robot guidance vision system would require such a large amount of manually labeled images that it would be too expensive and impractical for a real use case, where system reconfigurability is a fundamental requirement. With our system, on the other hand, especially in the field of industrial robotics, the cost of image labeling can be reduced, for the first time, to nearly zero, thus paving the way for self-reconfiguring systems with very high performance (as demonstrated by our experimental results). One of the limitations of this approach is the need to use a manual method for the detection of objects of interest in the preliminary stages of the pipeline (ARP or graphical interface). A feasible extension, related to the field of collaborative robotics, could be used to exploit the robot itself, manually moved by the user, even for this preliminary stage, so as to eliminate any source of inaccuracy.
Unsupervised Representations of Pollen in Bright-Field Microscopy

We present the first unsupervised deep learning method for pollen analysis using bright-field microscopy. Using a modest dataset of 650 images of pollen grains collected from honey, we achieve family level identification of pollen. We embed images of pollen grains into a low-dimensional latent space and compare Euclidean and Riemannian metrics on these spaces for clustering. We propose this system for automated analysis of pollen and other microscopic biological structures which have only small or unlabelled datasets available.
High-Fidelity Measurement of Qubits Encoded in Multilevel Superconducting Circuits

Qubit measurements are central to quantum information processing. In the field of superconducting qubits, standard readout techniques are not only limited by the signal-to-noise ratio, but also by state relaxation during the measurement. In this work, we demonstrate that the limitation due to relaxation can be suppressed by using the many-level Hilbert space of superconducting circuits: in a multilevel encoding, the measurement is only corrupted when multiple errors occur. Employing this technique, we show that we can directly resolve transmon gate errors at the level of one part in $10^3.$ Extending this idea, we apply the same principles to the measurement of a logical qubit encoded in a bosonic mode and detected with a transmon ancilla, implementing a proposal by Hann et al. . Qubit state assignments are made based on a sequence of repeated readouts, further reducing the overall infidelity. This approach is quite general and several encodings are studied; the codewords are more distinguishable when the distance between them is increased with respect to photon loss. The tradeoff between multiple readouts and state relaxation is explored and shown to be consistent with the photon-loss model. We report a logical assignment infidelity of $5.8\times 10^{-5}$ for a Fock-based encoding and $4.2\times 10^{-3}$ for a QEC code (the $S=2,N=1$ binomial code). Our results will not only improve the fidelity of quantum information applications, but also enable more precise characterization of process or gate errors.
Attention Control with Metric Learning Alignment for Image Set-based Recognition

This paper considers the problem of image set-based face verification and identification. Unlike traditional single sample (an image or a video) setting, this situation assumes the availability of a set of heterogeneous collection of orderless images and videos. The samples can be taken at different check points, different identity documents $etc$. The importance of each image is usually considered either equal or based on a quality assessment of that image independent of other images and/or videos in that image set. How to model the relationship of orderless images within a set remains a challenge. We address this problem by formulating it as a Markov Decision Process (MDP) in a latent space. Specifically, we first propose a dependency-aware attention control (DAC) network, which uses actor-critic reinforcement learning for attention decision of each image to exploit the correlations among the unordered images. An off-policy experience replay is introduced to speed up the learning process. Moreover, the DAC is combined with a temporal model for videos using divide and conquer strategies. We also introduce a pose-guided representation (PGR) scheme that can further boost the performance at extreme poses. We propose a parameter-free PGR without the need for training as well as a novel metric learning-based PGR for pose alignment without the need for pose detection in testing stage. Extensive evaluations on IJB-A/B/C, YTF, Celebrity-1000 datasets demonstrate that our method outperforms many state-of-art approaches on the set-based as well as video-based face recognition databases.
Animal Wildlife Population Estimation Using Social Media Images Collections

We are losing biodiversity at an unprecedented scale and in many cases, we do not even know the basic data for the species. Traditional methods for wildlife monitoring are inadequate. Development of new computer vision tools enables the use of images as the source of information about wildlife. Social media is the rich source of wildlife images, which come with a huge bias, thus thwarting traditional population size estimate approaches. Here, we present a new framework to take into account the social media bias when using this data source to provide wildlife population size estimates. We show that, surprisingly, this is a learnable and potentially solvable problem.