nils-herrmann/clean_scirepeval huggingface dataset:
To Tune or Not To Tune? How About the Best of Both Worlds?

The introduction of pre-trained language models has revolutionized natural language research communities. However, researchers still know relatively little regarding their theoretical and empirical properties. In this regard, Peters et al. perform several experiments which demonstrate that it is better to adapt BERT with a light-weight task-specific head, rather than building a complex one on top of the pre-trained language model, and freeze the parameters in the said language model. However, there is another option to adopt. In this paper, we propose a new adaptation method which we first train the task model with the BERT parameters frozen and then fine-tune the entire model together. Our experimental results show that our model adaptation method can achieve 4.7% accuracy improvement in semantic similarity task, 0.99% accuracy improvement in sequence labeling task and 0.72% accuracy improvement in the text classification task.
ReCoSa: Detecting the Relevant Contexts with Self-Attention for Multi-turn Dialogue Generation

In multi-turn dialogue generation, response is usually related with only a few contexts. Therefore, an ideal model should be able to detect these relevant contexts and produce a suitable response accordingly. However, the widely used hierarchical recurrent encoder-decoder models just treat all the contexts indiscriminately, which may hurt the following response generation process. Some researchers try to use the cosine similarity or the traditional attention mechanism to find the relevant contexts, but they suffer from either insufficient relevance assumption or position bias problem. In this paper, we propose a new model, named ReCoSa, to tackle this problem. Firstly, a word level LSTM encoder is conducted to obtain the initial representation of each context. Then, the self-attention mechanism is utilized to update both the context and masked response representation. Finally, the attention weights between each context and response representations are computed and used in the further decoding process. Experimental results on both Chinese customer services dataset and English Ubuntu dialogue dataset show that ReCoSa significantly outperforms baseline models, in terms of both metric-based and human evaluations. Further analysis on attention shows that the detected relevant contexts by ReCoSa are highly coherent with human’s understanding, validating the correctness and interpretability of ReCoSa.
Semantic Parsing with Dual Learning

Semantic parsing converts natural language queries into structured logical forms. The lack of training data is still one of the most serious problems in this area. In this work, we develop a semantic parsing framework with the dual learning algorithm, which enables a semantic parser to make full use of data (labeled and even unlabeled) through a dual-learning game. This game between a primal model (semantic parsing) and a dual model (logical form to query) forces them to regularize each other, and can achieve feedback signals from some prior-knowledge. By utilizing the prior-knowledge of logical form structures, we propose a novel reward signal at the surface and semantic levels which tends to generate complete and reasonable logical forms. Experimental results show that our approach achieves new state-of-the-art performance on ATIS dataset and gets competitive performance on OVERNIGHT dataset.
Dawn of the dark: unified dark sectors and the EDGES Cosmic Dawn 21-cm signal

While the origin and composition of dark matter and dark energy remains unknown, it is possible that they might represent two manifestations of a single entity, as occurring in unified dark sector models. On the other hand, advances in our understanding of the dark sector of the Universe might arise from Cosmic Dawn, the epoch when the first stars formed. In particular, the first detection of the global 21-cm absorption signal at Cosmic Dawn from the EDGES experiment opens up a new arena wherein to test models of dark matter and dark energy. Here, we consider generalized and modified Chaplygin gas models as candidate unified dark sector models. We first constrain these models against Cosmic Microwave Background data from the Planck satellite, before exploring how the inclusion of the global 21-cm signal measured by EDGES can improve limits on the model parameters, finding that the uncertainties on the parameters of the Chaplygin gas models can be reduced by a factor between 1.5 and 10. We also find that within the generalized Chaplygin gas model, the tension between the CMB and local determinations of the Hubble constant H0 is reduced from ≈ 4σ to ≈ 1.3σ. In conclusion, we find that the global 21-cm signal at Cosmic Dawn can provide an extraordinary window onto the physics of unified dark sectors.
Gravitational Waves from Black Holes in Merging Ultra-Dwarf Galaxies

The origin of the black-hole:black-hole mergers discovered through gravitational waves with for example the LIGO/Virgo collaboration are a mystery. We investigate the idea that some of these black holes originate from the centers of extremely low-mass ultra-dwarf galaxies that have merged together in the distant past at $z>1$. Extrapolating the central black hole to stellar mass ratio suggests that the black holes in these mergers could arise from galaxies of masses $\sim 10^{5} - 10^{6}$ M$_{\odot}\,$. We investigate whether these galaxies merge enough, or too much, to be consistent with the observed GW rate of $\sim 9.7-101$ Gpc$^{-3}$ yr$^{-1}$ using the latest LIGO/Virgo results. We show that in the nearby universe the merger rate and number densities of ultra-dwarf galaxies are too low, by an order or magnitude, to produce these black hole mergers. However, by considering that the merger fraction, merger-time scales, and the number densities of low-mass galaxies all conspire at $z>1-1.5$ to increase the merger rate for these galaxies at higher redshifts we argue that it is possible that some of the observed GW events arise from BHs in the centers of low-mass galaxies. The major uncertainty in this calculation is the dynamical time-scales for black holes in low-mass galaxies. Our results however suggest a very long BH merger time-scale of 4-7 Gyr, consistent with an extended black hole merger history. Further simulations are needed to verify this possibility, however our theory can be tested by searching for host galaxies of gravitational wave events. Results from these searches would will put limits on dwarf galaxy mergers and/or the presence and formation mechanisms of black holes through PopIII stars in the lowest mass galaxies.
Warfarin dose estimation on multiple datasets with automated hyperparameter optimisation and a novel software framework

Warfarin is an effective preventative treatment for arterial and venous thromboembolism, but requires individualised dosing due to its narrow therapeutic range and high individual variation. A plethora of statistical and machine learning techniques have been demonstrated in this domain. This study evaluated the accuracy of the most promising algorithms on the International Warfarin Pharmacogenetics Consortium dataset and a novel clinical dataset of South African patients. Support vectors and linear regression were consistently amongst the top performers in both datasets and performed comparably to recent ensemble approaches. We also evaluated the use of genetic programming to design and optimise learning models without human guidance, finding that performance matched that of models hand-crafted by human experts. Finally, we present a novel software framework (warfit-learn) for standardising future research by leveraging the most successful techniques in preprocessing, imputation, and evaluation, with the goal of making results more reproducible in this domain.
Online Inference and Detection of Curbs in Partially Occluded Scenes with Sparse LIDAR

Road boundaries, or curbs, provide autonomous vehicles with essential information when interpreting road scenes and generating behaviour plans. Although curbs convey important information, they are difficult to detect in complex urban environments (in particular in comparison to other elements of the road such as traffic signs and road markings). These difficulties arise from occlusions by other traffic participants as well as changing lighting and/or weather conditions. Moreover, road boundaries have various shapes, colours and structures while motion planning algorithms require accurate and precise metric information in real-time to generate their plans.In this paper, we present a real-time LIDAR-based approach for accurate curb detection around the vehicle (360 degree). Our approach deals with both occlusions from traffic and changing environmental conditions. To this end, we project 3D LIDAR pointcloud data into 2D bird’s-eye view images (akin to Inverse Perspective Mapping). These images are then processed by trained deep networks to infer both visible and occluded road boundaries. Finally, a post-processing step filters detected curb segments and tracks them over time. Experimental results demonstrate the effectiveness of the proposed approach on real-world driving data. Hence, we believe that our LIDAR-based approach provides an efficient and effective way to detect visible and occluded curbs around the vehicles in challenging driving scenarios.
Method of moments for 3D single particle ab initio modeling with non-uniform distribution of viewing angles

Single-particle reconstruction in cryo-electron microscopy (cryo-EM) is an increasingly popular technique for determining the 3D structure of a molecule from several noisy 2D projections images taken at unknown viewing angles. Most reconstruction algorithms require a low-resolution initialization for the 3D structure, which is the goal of ab initio modeling. Suggested by Zvi Kam in 1980, the method of moments (MoM) offers one approach, wherein low-order statistics of the 2D images are computed and a 3D structure is estimated by solving a system of polynomial equations. Unfortunately, Kam’s method suffers from restrictive assumptions, most notably that viewing angles should be distributed uniformly. Often unrealistic, uniformity entails the computation of higher-order correlations, as in this case first and second moments fail to determine the 3D structure. In the present paper, we remove this hypothesis, by permitting an unknown, non-uniform distribution of viewing angles in MoM. Perhaps surprisingly, we show that this case is statistically easier than the uniform case, as now first and second moments generically suffice to determine low-resolution expansions of the molecule. In the idealized setting of a known, non-uniform distribution, we find an efficient provable algorithm inverting first and second moments. For unknown, non-uniform distributions, we use non-convex optimization methods to solve for both the molecule and distribution.
Optimal destabilization of K-unstable Fano varieties via stability thresholds

We show that for a K-unstable Fano variety, any divisorial valuation computing its stability threshold induces a non-trivial special test configuration preserving the stability threshold. When such a divisorial valuation exists, we show that the Fano variety degenerates to a uniquely determined twisted K-polystable Fano variety. We also show that the stability threshold can be approximated by divisorial valuations induced by special test configurations. As an application of the above results and the analytic work of Datar, Szekelyhidi, and Ross, we deduce that greatest Ricci lower bounds of Fano manifolds of fixed dimension form a finite set of rational numbers. As a key step in the proofs, we adapt the process of Li and Xu producing special test configurations to twisted K-stability in the sense of Dervan.
Incrementalizing RASA's Open-Source Natural Language Understanding Pipeline

As spoken dialogue systems and chatbots are gaining more widespread adoption, commercial and open-sourced services for natural language understanding are emerging. In this paper, we explain how we altered the open-source RASA natural language understanding pipeline to process incrementally (i.e., word-by-word), following the incremental unit framework proposed by Schlangen and Skantze. To do so, we altered existing RASA components to process incrementally, and added an update-incremental intent recognition model as a component to RASA. Our evaluations on the Snips dataset show that our changes allow RASA to function as an effective incremental natural language understanding service.
Using Chinese Characters To Generate Text-Based Passwords For Information Security

Graphical passwords (GPWs) are in many areas of the current world. Topological graphic passwords (Topsnut-gpws) are a new type of cryptography, and they differ from the existing GPWs. A Topsnut-gpw consists of two parts: one is a topological structure (graph), and one is a set of discrete elements (a graph labelling, or coloring), the topological structure connects these discrete elements together to form an interesting "story". Our idea is to transform Chinese characters into computer and electronic equipments with touch screen by speaking, writing and keyboard for forming Hanzi-graphs and Hanzi-gpws. We will use Hanzigpws to produce text-based passwords (TB-paws). We will introduce flawed graph labellings on disconnected Hanzi-graphs.
On the Prospect of Using the Maximum Circular Velocity of Halos to Encapsulate Assembly Bias in the Galaxy–Halo Connection

We investigate a conceptual modification of the halo occupation distribution approach, using the halos’ present-day maximal circular velocity, Vmax, as an alternative to halo mass. In particular, using a semianalytic galaxy formation model applied to the Millennium WMAP7 simulation, we explore the extent that switching to Vmax as the primary halo property incorporates the effects of assembly bias into the formalism. We consider fixed number density galaxy samples ranked by stellar mass and examine the variations in the halo occupation functions with either halo concentration or formation time. We find that using Vmax results in a significant reduction in the occupancy variation of the central galaxies, particularly for concentration. The satellites’ occupancy variation on the other hand increases in all cases. We find effectively no change in the halo clustering dependence on concentration, for fixed bins of Vmax compared to fixed halo mass. Most crucially, we calculate the impact of assembly bias on galaxy clustering by comparing the amplitude of clustering to that of a shuffled galaxy sample, finding that the level of galaxy assembly bias remains largely unchanged. Our results suggest that while using Vmax as a proxy for halo mass diminishes some of the occupancy variations exhibited in the galaxy–halo relation, it is not able to encapsulate the effects of assembly bias potentially present in galaxy clustering. The use of other more complex halo properties, such as Vpeak, the peak value of Vmax over the assembly history, provides some improvement and warrants further investigation.
Cosmological constraints on invisible neutrino decays revisited

Invisible neutrino decay modes are difficult to target at laboratory experiments, and current bounds on such decays from solar neutrino and neutrino oscillation experiments are somewhat weak. It has been known for some time that Cosmology can serve as a powerful probe of invisible neutrino decays. In this work, we show that in order for Big Bang Nucleosynthesis to be successful, the invisible neutrino decay lifetime is bounded to be $\tau_\nu > 10^{-3}\,\text{s}$ at 95\% CL. We revisit Cosmic Microwave Background constraints on invisible neutrino decays, and by using Planck2018 observations we find the following bound on the neutrino lifetime: $\tau_\nu > (1.3-0.3)\times 10^{9}\,\text{s} \, \left({m_\nu}/{ 0.05\,\text{eV} }\right)^3$ at $95\%$ CL. We show that this bound is robust to modifications of the cosmological model, in particular that it is independent of the presence of dark radiation. We find that lifetimes relevant for Supernova observations ($\tau_\nu \sim 10^{5}\,\text{s}\, \left({m_\nu}/{ 0.05\,\text{eV} }\right)^3$) are disfavoured at more than $5\,\sigma$ with respect to $\Lambda$CDM given the latest Planck CMB observations. Finally, we show that when including high-$\ell$ Planck polarization data, neutrino lifetimes $\tau_\nu = (2-16)\times 10^{9}\,\text{s} \, \left({m_\nu}/{ 0.05\,\text{eV} }\right)^3$ are mildly preferred -- with a 1-2 $\sigma$ significance -- over neutrinos being stable.
Imitation-Projected Policy Gradient for Programmatic Reinforcement Learning

We present Imitation-Projected Policy Gradient (IPPG), an algorithmic framework for learning policies that are parsimoniously represented in a structured programming language. Such programmatic policies can be more interpretable, generalizable, and amenable to formal verification than neural policies; however, designing rigorous learning approaches for programmatic policies remains a challenge. IPPG, our response to this challenge, is based on three insights. First, we view our learning task as optimization in policy space, modulo the constraint that the desired policy has a programmatic representation, and solve this optimization problem using a "lift-and-project" perspective that takes a gradient step into the unconstrained policy space and then projects back onto the constrained space. Second, we view the unconstrained policy space as mixing neural and programmatic representations, which enables employing state-of-the-art deep policy gradient approaches. Third, we cast the projection step as program synthesis via imitation learning, and exploit contemporary combinatorial methods for this task. We present theoretical convergence results for IPPG, as well as an empirical evaluation in three continuous control domains. The experiments show that IPPG can significantly outperform state-of-the-art approaches for learning programmatic policies.
Predicting engagement in online social networks: Challenges and opportunities

Since the introduction of social media, user participation or engagement has received little research attention. In this survey article, we establish the notion of participation in social media and main challenges that researchers may face while exploring this phenomenon. We surveyed a handful of research articles that had been done in this area, and tried to extract, analyze and summarize the techniques performed by the researchers. We classified these works based on our task definitions, and explored the machine learning models that have been used for any kind of participation prediction. We also explored the vast amount of features that have been proven useful, and classified them into categories for better understanding and ease of re-implementation. We have found that the success of a technique mostly depends on the type of the network that has been researched on, and there is no universal machine learning algorithm or feature sets that works reasonably well in all types of social media. There is a lack of attempts in implementing state-of-the-art machine learning techniques like neural networks, and the possibility of transfer learning and domain adaptation has not been explored.
Learning Key-Value Store Design

We introduce the concept of design continuums for the data layout of key-value stores. A design continuum unifies major distinct data structure designs under the same model. The critical insight and potential long-term impact is that such unifying models 1) render what we consider up to now as fundamentally different data structures to be seen as views of the very same overall design space, and 2) allow seeing new data structure designs with performance properties that are not feasible by existing designs. The core intuition behind the construction of design continuums is that all data structures arise from the very same set of fundamental design principles, i.e., a small set of data layout design concepts out of which we can synthesize any design that exists in the literature as well as new ones. We show how to construct, evaluate, and expand, design continuums and we also present the first continuum that unifies major data structure designs, i.e., B+tree, B-epsilon-tree, LSM-tree, and LSH-table. 
The practical benefit of a design continuum is that it creates a fast inference engine for the design of data structures. For example, we can predict near instantly how a specific design change in the underlying storage of a data system would affect performance, or reversely what would be the optimal data structure (from a given set of designs) given workload characteristics and a memory budget. In turn, these properties allow us to envision a new class of self-designing key-value stores with a substantially improved ability to adapt to workload and hardware changes by transitioning between drastically different data structure designs to assume a diverse set of performance properties at will.
On the Optimality of Trees Generated by ID3

Since its inception in the 1980s, ID3 has become one of the most successful and widely used algorithms for learning decision trees. However, its theoretical properties remain poorly understood. In this work, we analyze the heuristic of growing a decision tree with ID3 for a limited number of iterations $t$ and given that nodes are split as in the case of exact information gain and probability computations. In several settings, we provide theoretical and empirical evidence that the TopDown variant of ID3, introduced by Kearns and Mansour (1996), produces trees with optimal or near-optimal test error among all trees with $t$ internal nodes. We prove optimality in the case of learning conjunctions under product distributions and learning read-once DNFs with 2 terms under the uniform distribition. Using efficient dynamic programming algorithms, we empirically show that TopDown generates trees that are near-optimal ($\sim \%1$ difference from optimal test error) in a large number of settings for learning read-once DNFs under product distributions.
Grounding Value Alignment with Ethical Principles

An important step in the development of value alignment (VA) systems in AI is understanding how values can interrelate with facts. Designers of future VA systems will need to utilize a hybrid approach in which ethical reasoning and empirical observation interrelate successfully in machine behavior. In this article we identify two problems about this interrelation that have been overlooked by AI discussants and designers. The first problem is that many AI designers commit inadvertently a version of what has been called by moral philosophers the "naturalistic fallacy," that is, they attempt to derive an "ought" from an "is." We illustrate when and why this occurs. The second problem is that AI designers adopt training routines that fail fully to simulate human ethical reasoning in the integration of ethical principles and facts. Using concepts of quantified modal logic, we proceed to offer an approach that promises to simulate ethical reasoning in humans by connecting ethical principles on the one hand and propositions about states of affairs on the other.
Geometry of Scheduling on Multiple Machines

textabstractWe consider the following general scheduling problem: there are m identical machines and n jobs all released at time 0. Each job j has a processing time pj, and an arbitrary non-decreasing function fj that specifies the cost incurred for j, for each possible completion time. The goal is to find a preemptive migratory schedule of minimum cost. This models several natural objectives such as weighted norm of completion time, weighted tardiness and much more. 
We give the first O(1) approximation algorithm for this problem, improving upon the O(loglognP) bound due to Moseley (2019). To do this, we first view the job-cover inequalities of Moseley geometri