nils-herrmann/clean_scirepeval huggingface dataset:
Serious Educational Reinforcement Game in Preschool

In Education, is constant the searching of techniques to strengthen and extend the educational strategies in order to achieve that students get knowledge received in classroom, training personal courses and workshops. Serious games are part of that set of educational strategies to reinforce or extend the knowledge using a set of rules and policies in an interactive and amuse way. Nowadays, using technology, it is possible to add images and sounds into serious games under a digital platform. In this paper, we present a solution based on serious games to reinforce the mathematical knowledge in the preschool education following the study plan of the Bolivarian Republic of Venezuela. Thus, we offer a tool with expectation of improving skills and knowledge of children on a specific area, also allowing follow the student track getting information of the activities performed. This compilation is made performing reports which will be available for teachers and facilitators. Reports allow adjust orientation and organization of topics imparted in classroom, and to know the general overview of a preschool educational course. Test performed determine the computational performance of solution, also the impact on children and teachers. The wide attention of children was verified, accomplishing the educational tasks in an indirect way.
AHA! an 'Artificial Hippocampal Algorithm' for Episodic Machine Learning

The majority of ML research concerns slow, statistical learning of i.i.d. samples from large, labelled datasets. Animals do not learn this way. An enviable characteristic of animal learning is 'episodic' learning - the ability to rapidly memorize a specific experience as a composition of existing concepts, without provided labels. The new knowledge can then be used to distinguish between similar experiences, to generalize between classes, and to selectively consolidate to long-term memory. The Hippocampus is known to be vital to these abilities. AHA is a biologically-plausible computational model of the Hippocampus. Unlike most machine learning models, AHA is trained without any external labels and uses only local and immediate credit assignment. We demonstrate AHA in a superset of the Omniglot classification benchmark. The extended benchmark covers a wider range of known Hippocampal functions by testing pattern separation, completion, and reconstruction of original input. These functions are all performed within a single configuration of the computational model. Despite these constraints, results are comparable to state-of-the-art deep convolutional ANNs. In addition to the demonstrated high degree of functional overlap with the Hippocampal region, AHA is remarkably aligned to current macro-scale biological models and uses biologically plausible micro-scale learning rules.
TinyBERT: Distilling BERT for Natural Language Understanding

Language model pre-training, such as BERT, has significantly improved the performances of many natural language processing tasks. However, pre-trained language models are usually computationally expensive, so it is difficult to efficiently execute them on resource-restricted devices. To accelerate inference and reduce model size while maintaining accuracy, we first propose a novel Transformer distillation method that is specially designed for knowledge distillation (KD) of the Transformer-based models. By leveraging this new KD method, the plenty of knowledge encoded in a large “teacher” BERT can be effectively transferred to a small “student” TinyBERT. Then, we introduce a new two-stage learning framework for TinyBERT, which performs Transformer distillation at both the pre-training and task-specific learning stages. This framework ensures that TinyBERT can capture the general-domain as well as the task-specific knowledge in BERT. TinyBERT4 with 4 layers is empirically effective and achieves more than 96.8% the performance of its teacher BERT-Base on GLUE benchmark, while being 7.5x smaller and 9.4x faster on inference. TinyBERT4 is also significantly better than 4-layer state-of-the-art baselines on BERT distillation, with only ~28% parameters and ~31% inference time of them. Moreover, TinyBERT6 with 6 layers performs on-par with its teacher BERT-Base.
Class-dependent Compression of Deep Neural Networks

Today's deep neural networks require substantial computation resources for their training, storage, and inference, which limits their effective use on resource-constrained devices. Many recent research activities explore different options for compressing and optimizing deep models. On the one hand, in many real-world applications, we face the data imbalance challenge, i.e. when the number of labeled instances of one class considerably outweighs the number of labeled instances of the other class. On the other hand, applications may pose a class imbalance problem, i.e. higher number of false positives produced when training a model and optimizing its performance may be tolerable, yet the number of false negatives must stay low. The problem originates from the fact that some classes are more important for the application than others, e.g. detection problems in medical and surveillance domains. Motivated by the success of the lottery ticket hypothesis, in this paper we propose an iterative deep model compression technique, which keeps the number of false negatives of the compressed model close to the one of the original model at the price of increasing the number of false positives if necessary. Our experimental evaluation using two benchmark data sets shows that the resulting compressed sub-networks 1) achieve up to 35% lower number of false negatives than the compressed model without class optimization, 2) provide an overall higher AUC_ROC measure, and 3) use up to 99% fewer parameters compared to the original network.
A Consolidated System for Robust Multi-Document Entity Risk Extraction and Taxonomy Augmentation

We introduce a hybrid human-automated system that provides scalable entity-risk relation extractions across large data sets. Given an expert-defined keyword taxonomy, entities, and data sources, the system returns text extractions based on bidirectional token distances between entities and keywords and expands taxonomy coverage with word vector encodings. Our system represents a more simplified architecture compared to alerting focused systems - motivated by high coverage use cases in the risk mining space such as due diligence activities and intelligence gathering. We provide an overview of the system and expert evaluations for a range of token distances. We demonstrate that single and multi-sentence distance groups significantly outperform baseline extractions with shorter, single sentences being preferred by analysts. As the taxonomy expands, the amount of relevant information increases and multi-sentence extractions become more preferred, but this is tempered against entity-risk relations become more indirect. We discuss the implications of these observations on users, management of ambiguity and taxonomy expansion, and future system modifications.
Specificity-Based Sentence Ordering for Multi-Document Extractive Risk Summarization

Risk mining technologies seek to find relevant textual extractions that capture entity-risk relationships. However, when high volume data sets are processed, a multitude of relevant extractions can be returned, shifting the focus to how best to present the results. We provide the details of a risk mining multi-document extractive summarization system that produces high quality output by modeling shifts in specificity that are characteristic of well-formed discourses. In particular, we propose a novel selection algorithm that alternates between extracts based on human curated or expanded autoencoded key terms, which exhibit greater specificity or generality as it relates to an entity-risk relationship. Through this extract ordering, and without the need for more complex discourse-aware NLP, we induce felicitous shifts in specificity in the alternating summaries that outperform non-alternating summaries on automatic ROUGE and BLEU scores, and manual understandability and preferences evaluations - achieving no statistically significant difference when compared to human authored summaries.
NLVR2 Visual Bias Analysis

NLVR2 (Suhr et al., 2019) was designed to be robust for language bias through a data collection process that resulted in each natural language sentence appearing with both true and false labels. The process did not provide a similar measure of control for visual bias. This technical report analyzes the potential for visual bias in NLVR2. We show that some amount of visual bias likely exists. Finally, we identify a subset of the test data that allows to test for model performance in a way that is robust to such potential biases. We show that the performance of existing models (Li et al., 2019; Tan and Bansal 2019) is relatively robust to this potential bias. We propose to add the evaluation on this subset of the data to the NLVR2 evaluation protocol, and update the official release to include it. A notebook including an implementation of the code used to replicate this analysis is available at this http URL.
Mechanics of good trade execution in the framework of linear temporary market impact

We define the concept of good trade execution and we construct explicit adapted good trade execution strategies in the framework of linear temporary market impact. Good trade execution strategies are dynamic, in the sense that they react to the actual realisation of the traded asset price path over the trading period; this is paramount in volatile regimes, where price trajectories can considerably deviate from their expected value. Remarkably, however, the implementation of our strategies does not require the full specification of an SDE evolution for the traded asset price, making them robust across different models. Moreover, rather than minimising the expected trading cost, good trade execution strategies minimise trading costs in a pathwise sense, a point of view not yet considered in the literature. The mathematical apparatus for such a pathwise minimisation hinges on certain random Young differential equations that correspond to the Euler–Lagrange equations of the classical Calculus of Variations. These Young differential equations characterise our good trade execution strategies in terms of an initial value problem that allows for easy implementations.
Improving Generative Visual Dialog by Answering Diverse Questions

Prior work on training generative Visual Dialog models with reinforcement learning ((Das et al., ICCV 2017) has explored a Q-Bot-A-Bot image-guessing game and shown that this ‘self-talk’ approach can lead to improved performance at the downstream dialog-conditioned image-guessing task. However, this improvement saturates and starts degrading after a few rounds of interaction, and does not lead to a better Visual Dialog model. We find that this is due in part to repeated interactions between Q-Bot and A-BOT during self-talk, which are not informative with respect to the image. To improve this, we devise a simple auxiliary objective that incentivizes Q-Bot to ask diverse questions, thus reducing repetitions and in turn enabling A-Bot to explore a larger state space during RL i.e. be exposed to more visual concepts to talk about, and varied questions to answer. We evaluate our approach via a host of automatic metrics and human studies, and demonstrate that it leads to better dialog, i.e. dialog that is more diverse (i.e. less repetitive), consistent (i.e. has fewer conflicting exchanges), fluent (i.e., more human-like), and detailed, while still being comparably image-relevant as prior work and ablations.
Community Detection and Improved Detectability in Multiplex Networks

We investigate the widely encountered problem of detecting communities in multiplex networks, such as social networks, with an unknown arbitrary heterogeneous structure. To improve detectability, we propose a generative model that leverages the multiplicity of a single community in multiple layers, with no prior assumption on the relation of communities among different layers. Our model relies on a novel idea of incorporating a large set of generic localized community label constraints across the layers, in conjunction with the celebrated Stochastic Block Model (SBM) in each layer. Accordingly, we build a probabilistic graphical model over the entire multiplex network by treating the constraints as Bayesian priors. We mathematically prove that these constraints/priors promote existence of identical communities across layers without introducing further correlation between individual communities. The constraints are further tailored to render a sparse graphical model and the numerically efficient Belief Propagation algorithm is subsequently employed. We further demonstrate by numerical experiments that in the presence of consistent communities between different layers, consistent communities are matched, and the detectability is improved over a single layer. We compare our model with a “correlated model” which exploits the prior knowledge of community correlation between layers. Similar detectability improvement is obtained under such a correlation, even though our model relies on much milder assumptions than the correlated model. Our model even shows a better detection performance over a certain correlation and signal to noise ratio (SNR) range. In the absence of community correlation, the correlation model naturally fails, while ours maintains its performance.
Cross-Lingual Natural Language Generation via Pre-Training

In this work we focus on transferring supervision signals of natural language generation (NLG) tasks between multiple languages. We propose to pretrain the encoder and the decoder of a sequence-to-sequence model under both monolingual and cross-lingual settings. The pre-training objective encourages the model to represent different languages in the shared space, so that we can conduct zero-shot cross-lingual transfer. After the pre-training procedure, we use monolingual data to fine-tune the pre-trained model on downstream NLG tasks. Then the sequence-to-sequence model trained in a single language can be directly evaluated beyond that language (i.e., accepting multi-lingual input and producing multi-lingual output). Experimental results on question generation and abstractive summarization show that our model outperforms the machine-translation-based pipeline methods for zero-shot cross-lingual generation. Moreover, cross-lingual transfer improves NLG performance of low-resource languages by leveraging rich-resource language data. Our implementation and data are available at this https URL.
Superconductivity up to 243 K in yttrium hydrides under high pressure

The discovery of high-temperature conventional superconductivity in H3S with a critical temperature of Tc=203 K was followed by the recent record of Tc ~250 K in the face-centered cubic (fcc) lanthanum hydride LaH10 compound. It was realized in a new class of hydrogen-dominated compounds having a clathrate-like crystal structure in which hydrogen atoms form a 3D framework and surround a host atom of rare earth elements. Yttrium hydrides are predicted to have even higher Tc exceeding room temperature. In this paper, we synthesized and refined the crystal structure of new hydrides: YH4, YH6, and YH9 at pressures up to 237 GPa finding that YH4 crystalizes in the I4/mmm lattice, YH6 in Im-3m lattice and YH9 in P63/mmc lattice in excellent agreement with the calculations. The observed very high-temperature superconductivity is comparable to that found in fcc-LaH10: the pressure dependence of Tc for YH9 also displays a "dome like shape" with the highest Tc of 243 K at 201 GPa. We also observed a Tc of 227 K at 237 GPa for the YH6 phase. However, the measured Tcs are notably lower by ~30 K than predicted. Evidence for superconductivity includes the observation of zero electrical resistance, a decrease of Tc under an external magnetic field and an isotope effect. The theoretically predicted fcc YH10 with the promising highest Tc>300 K was not stabilized in our experiments under pressures up to 237 GPa.
Formalism for Supporting the Development of Verifiably Safe Medical Guidelines with Statecharts

Improving the effectiveness and safety of patient care is the ultimate objective for medical cyber-physical systems. Many medical best practice guidelines exist, but most of the existing guidelines in handbooks are difficult for medical staff to remember and apply clinically. Furthermore, although the guidelines have gone through clinical validations, validations by medical professionals alone do not provide guarantees for the safety of medical cyber-physical systems. Hence, formal verification is also needed. The paper presents the formal semantics for a framework that we developed to support the development of verifiably safe medical guidelines. 
The framework allows computer scientists to work together with medical professionals to transform medical best practice guidelines into executable statechart models, Yakindu in particular, so that medical functionalities and properties can be quickly prototyped and validated. Existing formal verification technologies, UPPAAL timed automata in particular, is integrated into the framework to provide formal verification capabilities to verify safety properties. However, some components used/built into the framework, such as the open-source Yakindu statecharts as well as the transformation rules from statecharts to timed automata, do not have built-in semantics. The ambiguity becomes unavoidable unless formal semantics is defined for the framework, which is what the paper is to present.
Computations with greater Quantum depth are strictly more powerful (relative to an oracle)

A conjecture of Jozsa () states that any polynomial-time quantum computation can be simulated by polylogarithmic-depth quantum computation interleaved with polynomial-depth classical computation. Separately, Aaronson conjectured that there exists an oracle O such that BQP O ≠ (BPPBQNC) O . These conjectures are intriguing allusions to the unresolved potential of combining classical and low-depth quantum computation. In this work we show that the Welded Tree Problem, which is an oracle problem that can be solved in quantum polynomial time as shown by Childs et al. , cannot be solved in BPPBQNC, nor can it be solved in the class that Jozsa describes. This proves Aaronson’s oracle separation conjecture and provides a counterpoint to Jozsa’s conjecture relative to the Welded Tree oracle problem. More precisely, we define two complexity classes, HQC and JC whose languages are decided by two different families of interleaved quantum-classical circuits. HQC contains BPPBQNC and is therefore relevant to Aaronson’s conjecture, while JC captures the model of computation that Jozsa considers. We show that the Welded Tree Problem gives an oracle separation between either of {JC, HQC} and BQP. Therefore, even when interleaved with arbitrary polynomial-time classical computation, greater ”quantum depth” leads to strictly greater computational ability in this relativized setting.
Learning Dense Representations for Entity Retrieval

We show that it is feasible to perform entity linking by training a dual encoder (two-tower) model that encodes mentions and entities in the same dense vector space, where candidate entities are retrieved by approximate nearest neighbor search. Unlike prior work, this setup does not rely on an alias table followed by a re-ranker, and is thus the first fully learned entity retrieval model. We show that our dual encoder, trained using only anchor-text links in Wikipedia, outperforms discrete alias table and BM25 baselines, and is competitive with the best comparable results on the standard TACKBP-2010 dataset. In addition, it can retrieve candidates extremely fast, and generalizes well to a new dataset derived from Wikinews. On the modeling side, we demonstrate the dramatic value of an unsupervised negative mining algorithm for this task.
Quantum Inflation: A General Approach to Quantum Causal Compatibility

Causality is a seminal concept in science: any research discipline, from sociology and medicine to physics and chemistry, aims at understanding the causes that could explain the correlations observed among some measured variables. While several methods exist to characterize classical causal models, no general construction is known for the quantum case. In this work we present quantum inflation, a systematic technique to falsify if a given quantum causal model is compatible with some observed correlations. We demonstrate the power of the technique by reproducing known results and solving open problems for some paradigmatic examples of causal networks. Our results may find an application in many fields: from the characterization of correlations in quantum networks to the study of quantum effects in thermodynamic and biological processes.
Simulations of the star-forming molecular gas in an interacting M51-like galaxy: cloud population statistics

To investigate how molecular clouds react to different environmental conditions at a galactic scale, we present a catalogue of giant molecular clouds (GMCs) resolved down to masses of ∼10 M⊙ from a simulation of the entire disc of an interacting M51-like galaxy and a comparable isolated galaxy. Our model includes time-dependent gas chemistry, sink particles for star formation, and supernova feedback, meaning we are not reliant on star formation recipes based on threshold densities and can follow the physics of the cold molecular phase. We extract GMCs from the simulations and analyse their properties. In the disc of our simulated galaxies, spiral arms seem to act merely as snowplows, gathering gas, and clouds without dramatically affecting their properties. In the centre of the galaxy, on the other hand, environmental conditions lead to larger, more massive clouds. While the galaxy interaction has little effect on cloud masses and sizes, it does promote the formation of counter-rotating clouds. We find that the identified clouds seem to be largely gravitationally unbound at first glance, but a closer analysis of the hierarchical structure of the molecular interstellar medium shows that there is a large range of virial parameters with a smooth transition from unbound to mostly bound for the densest structures. The common observation that clouds appear to be virialized entities may therefore be due to CO bright emission highlighting a specific level in this hierarchical binding sequence. The small fraction of gravitationally bound structures found suggests that low galactic star formation efficiencies may be set by the process of cloud formation and initial collapse.