nils-herrmann/clean_scirepeval huggingface dataset:
Unsupervised Lemmatization as Embeddings-Based Word Clustering

We focus on the task of unsupervised lemmatization, i.e. grouping together inflected forms of one word under one label (a lemma) without the use of annotated training data. We propose to perform agglomerative clustering of word forms with a novel distance measure. Our distance measure is based on the observation that inflections of the same word tend to be similar both string-wise and in meaning. We therefore combine word embedding cosine similarity, serving as a proxy to the meaning similarity, with Jaro-Winkler edit distance. Our experiments on 23 languages show our approach to be promising, surpassing the baseline on 23 of the 28 evaluation datasets.
VL-BERT: Pre-training of Generic Visual-Linguistic Representations

We introduce a new pre-trainable generic representation for visual-linguistic tasks, called Visual-Linguistic BERT (VL-BERT for short). VL-BERT adopts the simple yet powerful Transformer model as the backbone, and extends it to take both visual and linguistic embedded features as input. In it, each element of the input is either of a word from the input sentence, or a region-of-interest (RoI) from the input image. It is designed to fit for most of the visual-linguistic downstream tasks. To better exploit the generic representation, we pre-train VL-BERT on the massive-scale Conceptual Captions dataset, together with text-only corpus. Extensive empirical analysis demonstrates that the pre-training procedure can better align the visual-linguistic clues and benefit the downstream tasks, such as visual commonsense reasoning, visual question answering and referring expression comprehension. It is worth noting that VL-BERT achieved the first place of single model on the leaderboard of the VCR benchmark. Code is released at \url{this https URL}.
Design, Assembly, Calibration, and Measurement of an Augmented Reality Haploscope

A haploscope is an optical system which produces a carefully controlled virtual image. Since the development of Wheatstone's original stereoscope in 1838, haploscopes have been used to measure perceptual properties of human stereoscopic vision. This paper presents an augmented reality (AR) haploscope, which allows the viewing of virtual objects superimposed against the real world. Our lab has used generations of this device to make a careful series of perceptual measurements of AR phenomena, which have been described in publications over the previous 8 years. This paper systematically describes the design, assembly, calibration, and measurement of our AR haploscope. These methods have been developed and improved in our lab over the past 10 years. Despite the fact that 180 years have elapsed since the original report of Wheatstone's stereoscope, we have not previously found a paper that describes these kinds of details.
The tidal evolution of dark matter substructure – I. subhalo density profiles

Accurately predicting the abundance and structural evolution of dark matter subhaloes is crucial for understanding galaxy formation, modelling galaxy clustering, and constraining the nature of dark matter. Due to the non-linear nature of subhalo evolution, cosmological N-body simulations remain its primary method of investigation. However, it has recently been demonstrated that such simulations are still heavily impacted by artificial disruption, diminishing the information content on small scales, and reducing the reliability of all simulation-calibrated semi-analytical models. In this paper, we utilize the recently released DASH library of high-resolution, idealized simulations of the tidal evolution of subhaloes, which are unhindered by numerical overmerging due to discreteness noise or force softening, to calibrate an improved, more accurate model of the evolution of the density profiles of subhaloes that undergo tidal heating and stripping within their host halo. By testing previous findings that the structural evolution of a tidally truncated subhalo depends solely on the fraction of mass stripped, independent of the details of the stripping, we identify an additional dependence on the initial subhalo concentration. We provide significantly improved fitting functions for the subhalo density profiles and structural parameters (Vmax and rmax) that are unimpeded by numerical systematics and applicable to a wide range of parameter space. This model will be an integral component of a future semi-analytical treatment of substructure evolution, which can be used to predict key quantities, such as the evolved subhalo mass function and annihilation boost factors, and validate such calculations performed with cosmological simulations.
On the black hole content and initial mass function of 47 Tuc

The globular cluster (GC) 47 Tuc has recently been proposed to host an intermediate-mass black hole (IMBH) or a population of stellar mass black holes (BHs). To shed light on its dark content, we present an application of self-consistent multimass models with a varying mass function and content of stellar remnants, which we fit to various observational constraints. Our best-fitting model successfully matches the observables and correctly predicts the radial distribution of millisecond pulsars and their gravitational accelerations inferred from long-term timing observations. The data favours a population of BHs with a total mass of $430^{+386}_{-301}$ M⊙, but the most likely model has very few BHs. Since our models do not include a central IMBH and accurately reproduce the observations, we conclude that there is currently no need to invoke the presence of an IMBH in 47 Tuc. The global present-day mass function inferred is significantly depleted in low-mass stars (power-law slope $\alpha =-0.52^{+0.17}_{-0.16}$). Given the orbit and predicted mass-loss history of this massive GC, the dearth of low-mass stars is difficult to explain with a standard initial mass function (IMF) followed by long-term preferential escape of low-mass stars driven by two-body relaxation, and instead suggests that 47 Tuc may have formed with a bottom-light IMF. We discuss alternative evolutionary origins for the flat mass function and ways to reconcile this with the low BH retention fraction. Finally, by capturing the effect of dark remnants, our method offers a new way to probe the IMF in a GC above the current main-sequence turn-off mass, for which we find a slope of −2.49 ± 0.08.
Structure of Brightest Cluster Galaxies and Intracluster Light

Observations of 170 local (z ≲ 0.08) galaxy clusters in the northern hemisphere have been obtained with the Wendelstein Telescope Wide Field Imager (WWFI). We correct for systematic effects such as point-spread function broadening, foreground star contamination, relative bias offsets, and charge persistence. Background inhomogeneities induced by scattered light are reduced down to ΔSB > 31 g′ mag arcsec−2 by large dithering and subtraction of night-sky flats. Residual background inhomogeneities brighter than SBσ < 27.6 g′ mag arcsec−2 caused by galactic cirrus are detected in front of 23% of the clusters. However, the large field of view allows discrimination between accretion signatures and galactic cirrus. We detect accretion signatures in the form of tidal streams in 22%, shells in 9.4%, and multiple nuclei in 47% of the brightest cluster galaxies (BCGs) and find two BCGs in 7% of the clusters. We measure semimajor-axis surface brightness profiles of the BCGs and their surrounding intracluster light (ICL) down to a limiting surface brightness of SB = 30 g′ mag arcsec−2. The spatial resolution in the inner regions is increased by combining the WWFI light profiles with those that we measured from archival Hubble Space Telescope images or deconvolved WWFI images. We find that 71% of the BCG+ICL systems have surface brightness (SB) profiles that are well described by a single Sérsic function, whereas 29% require a double Sérsic function to obtain a good fit. We find that BCGs have scaling relations that differ markedly from those of normal ellipticals, likely due to their indistinguishable embedding in the ICL.
Evolution of neutral oxygen during the epoch of reionization and its use in estimating the neutral hydrogen fraction

We use synthetic sightlines drawn through snapshots of the Technicolour Dawn simulations to explore how the statistics of neutral oxygen (${\mathrm{O}\,{\small I}}$) absorbers respond to hydrogen reionization. The ionization state of the circumgalactic medium (CGM) initially roughly tracks that of the intergalactic medium, but beginning at z = 8 the CGM grows systematically more neutral owing to self-shielding. Weak absorbers trace diffuse gas that lies farther from haloes, hence they are ionized first, whereas stronger systems are less sensitive to reionization. The overall ${\mathrm{O}\,{\small I}}$ covering fraction decreases slowly with time owing to competition between ongoing enrichment and gradual encroachment of ionization fronts into increasingly overdense gas. While the declining covering fraction is partially offset by continued formation of new haloes, the ionization of the diffuse gas causes the predicted line-of-sight incidence rate of ${\mathrm{O}\,{\small I}}$ absorbers to decline abruptly at the overlap epoch, in qualitative agreement with observations. In comparison to the recently observed equivalent width (EW) distribution at z ≈ 6, the simulations underproduce systems with $\mathrm{EW} \ge 0.1 \mathring{\rm A}$, although they reproduce weaker systems with $\mathrm{EW} \ge 0.05 \mathring{\rm A}$. By z ≈ 5, the incidence of $\mathrm{EW} \lt 0.1 \mathring{\rm A}$ systems are overproduced, consistent with previous indications that the simulated ionizing background is too weak at z < 6. The summed column densities of $\mathrm{Si}\,{\small II}$ and $\mathrm{Si}\,{\small IV}$ trace the total oxygen column, and hence the ratio of the ${\mathrm{O}\,{\small I}}$ and $\mathrm{Si}\,{\small II}+ \mathrm{Si}\,{\small IV}$ comoving mass densities traces the progress of reionization. This probe may prove particularly useful in the regime where $x_{\mathrm{H}\,{\small I}} \gt 10{{\ \rm per\ cent}}$.
Intent term selection and refinement in e-commerce queries

In e-commerce, a user tends to search for the desired product by issuing a query to the search engine and examining the retrieved results. If the search engine was successful in correctly understanding the user's query, it will return results that correspond to the products whose attributes match the terms in the query that are representative of the query's product intent. However, the search engine may fail to retrieve results that satisfy the query's product intent and thus degrading user experience due to different issues in query processing: (i) when multiple terms are present in a query it may fail to determine the relevant terms that are representative of the query's product intent, and (ii) it may suffer from vocabulary gap between the terms in the query and the product's description, i.e., terms used in the query are semantically similar but different from the terms in the product description. Hence, identifying the terms that describe the query's product intent and predicting additional terms that describe the query's product intent better than the existing query terms to the search engine is an essential task in e-commerce search. In this paper, we leverage the historical query reformulation logs of a major e-commerce retailer to develop distant-supervised approaches to solve both these problems. Our approaches exploit the fact that the significance of a term is dependent upon the context (other terms in the neighborhood) in which it is used in order to learn the importance of the term towards the query's product intent. We show that identifying and emphasizing the terms that define the query's product intent leads to a 3% improvement in ranking. Moreover, for the tasks of identifying the important terms in a query and for predicting the additional terms that represent product intent, experiments illustrate that our approaches outperform the non-contextual baselines.
From Community to Role-based Graph Embeddings

Roles are sets of structurally similar nodes that are more similar to nodes inside the set than outside, whereas communities are sets of nodes with more connections inside the set than outside (based on proximity/closeness, density). Roles and communities are fundamentally different but important complementary notions. Recently, the notion of roles has become increasingly important and has gained a lot of attention due to the proliferation of work on learning representations (node/edge embeddings) from graphs that preserve the notion of roles. Unfortunately, recent work has sometimes confused the notion of roles and communities leading to misleading or incorrect claims about the capabilities of network embedding methods. As such, this manuscript seeks to clarify the differences between roles and communities, and formalize the general mechanisms (e.g., random walks, feature diffusion) that give rise to community or role-based embeddings. We show mathematically why embedding methods based on these identified mechanisms are either community or role-based. These mechanisms are typically easy to identify and can help researchers quickly determine whether a method is more prone to learn community or role-based embeddings. Furthermore, they also serve as a basis for developing new and better methods for community or role-based embeddings. Finally, we analyze and discuss the applications and data characteristics where community or role-based embeddings are most appropriate.
RNNs Evolving on an Equilibrium Manifold: A Panacea for Vanishing and Exploding Gradients?

Recurrent neural networks (RNNs) are particularly well-suited for modeling long-term dependencies in sequential data, but are notoriously hard to train because the error backpropagated in time either vanishes or explodes at an exponential rate. While a number of works attempt to mitigate this effect through gated recurrent units, well-chosen parametric constraints, and skip-connections, we develop a novel perspective that seeks to evolve the hidden state on the equilibrium manifold of an ordinary differential equation (ODE). We propose a family of novel RNNs, namely {\em Equilibriated Recurrent Neural Networks} (ERNNs) that overcome the gradient decay or explosion effect and lead to recurrent models that evolve on the equilibrium manifold. We show that equilibrium points are stable, leading to fast convergence of the discretized ODE to fixed points. Furthermore, ERNNs account for long-term dependencies, and can efficiently recall informative aspects of data from the distant past. We show that ERNNs achieve state-of-the-art accuracy on many challenging data sets with 3-10x speedups, 1.5-3x model size reduction, and with similar prediction cost relative to vanilla RNNs.
Mobility-Aware Content Preference Learning in Decentralized Caching Networks

Due to the drastic increase of mobile traffic, wireless caching is proposed to serve repeated requests for content download. To determine the caching scheme for decentralized caching networks, the content preference learning problem based on mobility prediction is studied. We first formulate preference prediction as a decentralized regularized multi-task learning (DRMTL) problem without considering the mobility of mobile terminals (MTs). The problem is solved by a hybrid Jacobian and Gauss-Seidel proximal multi-block alternating direction method (ADMM) based algorithm, which is proven to conditionally converge to the optimal solution with a rate ${O}$ (1/ ${k}$ ). Then we use the tool of Markov renewal process to predict the moving path and sojourn time for MTs, and integrate the mobility pattern with the DRMTL model by reweighting the training samples and introducing a transfer penalty in the objective. We solve the problem and prove that the developed algorithm has the same convergence property but with different conditions. Through simulation we show the convergence analysis on proposed algorithms. Our real trace driven experiments illustrate that the mobility-aware DRMTL model can provide a more accurate prediction on geography preference than DRMTL model. Besides, the hit ratio achieved by most popular proactive caching (MPC) policy with preference predicted by mobility-aware DRMTL outperforms the MPC with preference from DRMTL and random caching (RC) schemes.
DC3 -- A Diagnostic Case Challenge Collection for Clinical Decision Support

In clinical care, obtaining a correct diagnosis is the first step towards successful treatment and, ultimately, recovery. Depending on the complexity of the case, the diagnostic phase can be lengthy and ridden with errors and delays. Such errors have a high likelihood to cause patients severe harm or even lead to their death and are estimated to cost the U.S. healthcare system several hundred billion dollars each year. To avoid diagnostic errors, physicians increasingly rely on diagnostic decision support systems drawing from heuristics, historic cases, textbooks, clinical guidelines and scholarly biomedical literature. The evaluation of such systems, however, is often conducted in an ad-hoc fashion, using non-transparent methodology, and proprietary data. This paper presents DC3, a collection of 31 extremely difficult diagnostic case challenges, manually compiled and solved by clinical experts. For each case, we present a number of temporally ordered physician-generated observations alongside the eventually confirmed true diagnosis. We additionally provide inferred dense relevance judgments for these cases among the PubMed collection of 27 million scholarly biomedical articles.
Feedbackward Decoding for Semantic Segmentation

We propose a novel approach for semantic segmentation that uses an encoder in the reverse direction to decode. Many semantic segmentation networks adopt a feedforward encoder-decoder architecture. Typically, an input is first downsampled by the encoder to extract high-level semantic features and continues to be fed forward through the decoder module to recover low-level spatial clues. Our method works in an alternative direction that lets information flow backward from the last layer of the encoder towards the first. The encoder performs encoding in the forward pass and the same network performs decoding in the backward pass. Therefore, the encoder itself is also the decoder. Compared to conventional encoder-decoder architectures, ours doesn't require additional layers for decoding and further reuses the encoder weights thereby reducing the total number of parameters required for processing. We show by using only the 13 convolutional layers from VGG-16 plus one tiny classification layer, our model significantly outperforms other frequently cited models that are also adapted from VGG-16. On the Cityscapes semantic segmentation benchmark, our model uses 50.0% less parameters than SegNet and achieves an 18.1% higher "IoU class" score; it uses 28.3% less parameters than DeepLab LargeFOV and the achieved "IoU class" score is 3.9% higher; it uses 89.1% fewer parameters than FCN-8s and the achieved "IoU class" score is 3.1% higher. Our code will be publicly available on Github later.
Revealing the Dark Secrets of BERT

BERT-based architectures currently give state-of-the-art performance on many NLP tasks, but little is known about the exact mechanisms that contribute to its success. In the current work, we focus on the interpretation of self-attention, which is one of the fundamental underlying components of BERT. Using a subset of GLUE tasks and a set of handcrafted features-of-interest, we propose the methodology and carry out a qualitative and quantitative analysis of the information encoded by the individual BERT’s heads. Our findings suggest that there is a limited set of attention patterns that are repeated across different heads, indicating the overall model overparametrization. While different heads consistently use the same attention patterns, they have varying impact on performance across different tasks. We show that manually disabling attention in certain heads leads to a performance improvement over the regular fine-tuned BERT models.
Training Optimus Prime, M.D.: Generating Medical Certification Items by Fine-Tuning OpenAI's gpt2 Transformer Model

This article describes new results of an application using transformer-based language models to automated item generation (AIG), an area of ongoing interest in the domain of certification testing as well as in educational measurement and psychological testing. OpenAI's gpt2 pre-trained 345M parameter language model was retrained using the public domain text mining set of PubMed articles and subsequently used to generate item stems (case vignettes) as well as distractor proposals for multiple-choice items. This case study shows promise and produces draft text that can be used by human item writers as input for authoring. Future experiments with more recent transformer models (such as Grover, TransformerXL) using existing item pools are expected to improve results further and to facilitate the development of assessment materials.
Online Causal Inference for Advertising in Real-Time Bidding Auctions

Real-time bidding (RTB) systems, which utilize auctions to allocate user impressions to competing advertisers, continue to enjoy success in digital advertising. Assessing the effectiveness of such advertising remains a challenge in research and practice. This paper proposes a new approach to perform causal inference on advertising bought through such mechanisms. Leveraging the economic structure of ﬁrst- and second-price auctions, we ﬁrst show that the effects of advertising are identiﬁed by the optimal bids. Hence, since these optimal bids are the only objects that need to be recov-ered, we introduce an adapted Thompson sampling algorithm to solve a multi-armed bandit problem that succeeds in recovering such bids and, consequently, the effects of advertising while minimizing the costs of experimentation. We derive a regret bound for our algorithm which is order optimal and use data from RTB auctions to show that it outperforms commonly used methods that estimate the effects of advertising.
Security Analysis Methods on Ethereum Smart Contract Vulnerabilities: A Survey

Smart contracts are software programs featuring both traditional applications and distributed data storage on blockchains. Ethereum is a prominent blockchain platform with the support of smart contracts. The smart contracts act as autonomous agents in critical decentralized applications and hold a significant amount of cryptocurrency to perform trusted transactions and agreements. Millions of dollars as part of the assets held by the smart contracts were stolen or frozen through the notorious attacks just between 2016 and 2018, such as the DAO attack, Parity Multi-Sig Wallet attack, and the integer underflow/overflow attacks. These attacks were caused by a combination of technical flaws in designing and implementing software codes. However, many more vulnerabilities of less severity are to be discovered because of the scripting natures of the Solidity language and the non-updateable feature of blockchains. Hence, we surveyed 16 security vulnerabilities in smart contract programs, and some vulnerabilities do not have a proper solution. This survey aims to identify the key vulnerabilities in smart contracts on Ethereum in the perspectives of their internal mechanisms and software security vulnerabilities. By correlating 16 Ethereum vulnerabilities and 19 software security issues, we predict that many attacks are yet to be exploited. And we have explored many software tools to detect the security vulnerabilities of smart contracts in terms of static analysis, dynamic analysis, and formal verification. This survey presents the security problems in smart contracts together with the available analysis tools and the detection methods. We also investigated the limitations of the tools or analysis methods with respect to the identified security vulnerabilities of the smart contracts.