nils-herrmann/clean_scirepeval huggingface dataset:
Trading off Complexity for Expressiveness in Programming Languages: Visions and Preliminary Experiences

When programming resource-scarce embedded smart devices, the designer often requires both the low-level system programming features of a language such as C and higher level capability typical of a language like Java. The choice of a particular language typically implies trade offs between conflicting design goals such as performance, costs, and overheads. The large variety of languages, virtual machines, and translators provides the designer with a dense trade off space, ranging from minimalistic to rich full-fledged approaches, but once a choice is made it is often difficult for the designer to revise it. In this work we propose a system of light-weighted and modular extensions as a method to flexibly reshape the target programming language as needed, adding only those application layer features that match the current design goals. In so doing complexity is made transparent, but not hidden: While the programmer can benefit of higher level constructs, the designer can deal with modular building blocks each characterized by a certain algorithmic complexity and therefore each accountable for a given share of the overhead. As a result the designer is given a finer control on the amount of resources that are consumed by the run-time executive of the chosen programming language.
High-Dimensional Multivariate Forecasting with Low-Rank Gaussian Copula Processes

Predicting the dependencies between observations from multiple time series is critical for applications such as anomaly detection, financial risk management, causal analysis, or demand forecasting. However, the computational and numerical difficulties of estimating time-varying and high-dimensional covariance matrices often limits existing methods to handling at most a few hundred dimensions or requires making strong assumptions on the dependence between series. We propose to combine an RNN-based time series model with a Gaussian copula process output model with a low-rank covariance structure to reduce the computational complexity and handle non-Gaussian marginal distributions. This permits to drastically reduce the number of parameters and consequently allows the modeling of time-varying correlations of thousands of time series. We show on several real-world datasets that our method provides significant accuracy improvements over state-of-the-art baselines and perform an ablation study analyzing the contributions of the different components of our model.
Improving Neural Machine Translation Robustness via Data Augmentation: Beyond Back-Translation

Neural Machine Translation (NMT) models have been proved strong when translating clean texts, but they are very sensitive to noise in the input. Improving NMT models robustness can be seen as a form of “domain” adaption to noise. The recently created Machine Translation on Noisy Text task corpus provides noisy-clean parallel data for a few language pairs, but this data is very limited in size and diversity. The state-of-the-art approaches are heavily dependent on large volumes of back-translated data. This paper has two main contributions: Firstly, we propose new data augmentation methods to extend limited noisy data and further improve NMT robustness to noise while keeping the models small. Secondly, we explore the effect of utilizing noise from external data in the form of speech transcripts and show that it could help robustness.
Coherent quantum enhancement of pair production in the null domain

We present an exactly solvable example of coherent quantum interference effects in the creation of electron-positron pairs from the collision of a photon with ultra-short laser pulses. Being characterised entirely by null, or lightlike, directions, this setup realises an all-optical double-slit in the "null" domain, and exhibits features both in common with, and distinct from, a time domain double-slit (Ramsey interferometer). We show that by tailoring the order and amplitude of the pulses one can control signatures of both quantum and classical physics in the produced positron spectrum.
Hyperspectral holography and spectroscopy: computational features of inverse discrete cosine transform

Broadband hyperspectral digital holography and Fourier transform spectroscopy are important instruments in various science and application fields. In the digital hyperspectral holography and spectroscopy the variable of interest are obtained as inverse discrete cosine transforms of observed diffractive intensity patterns. In these notes, we provide a variety of algorithms for the inverse cosine transform with the proofs of perfect spectrum reconstruction, as well as we discuss and illustrate some nontrivial features of these algorithms.
Artificial Intelligence: Powering Human Exploration of the Moon and Mars

Over the past decade, the NASA Autonomous Systems and Operations (ASO) project has developed and demonstrated numerous autonomy enabling technologies employing AI techniques. Our work has employed AI in three distinct ways to enable autonomous mission operations capabilities. Crew Autonomy gives astronauts tools to assist in the performance of each of these mission operations functions. Vehicle System Management uses AI techniques to turn the astronaut's spacecraft into a robot, allowing it to operate when astronauts are not present, or to reduce astronaut workload. AI technology also enables Autonomous Robots as crew assistants or proxies when the crew are not present. We first describe human spaceflight mission operations capabilities. We then describe the ASO project, and the development and demonstration performed by ASO since 2011. We will describe the AI techniques behind each of these demonstrations, which include a variety of symbolic automated reasoning and machine learning based approaches. Finally, we conclude with an assessment of future development needs for AI to enable NASA's future Exploration missions.
Is a Good Representation Sufficient for Sample Efficient Reinforcement Learning?

Modern deep learning methods provide effective means to learn good representations. However, is a good representation itself sufficient for sample efficient reinforcement learning? This question has largely been studied only with respect to (worst-case) approximation error, in the more classical approximate dynamic programming literature. With regards to the statistical viewpoint, this question is largely unexplored, and the extant body of literature mainly focuses on conditions which permit sample efficient reinforcement learning with little understanding of what are necessary conditions for efficient reinforcement learning. 
This work shows that, from the statistical viewpoint, the situation is far subtler than suggested by the more traditional approximation viewpoint, where the requirements on the representation that suffice for sample efficient RL are even more stringent. Our main results provide sharp thresholds for reinforcement learning methods, showing that there are hard limitations on what constitutes good function approximation (in terms of the dimensionality of the representation), where we focus on natural representational conditions relevant to value-based, model-based, and policy-based learning. These lower bounds highlight that having a good (value-based, model-based, or policy-based) representation in and of itself is insufficient for efficient reinforcement learning, unless the quality of this approximation passes certain hard thresholds. Furthermore, our lower bounds also imply exponential separations on the sample complexity between 1) value-based learning with perfect representation and value-based learning with a good-but-not-perfect representation, 2) value-based learning and policy-based learning, 3) policy-based learning and supervised learning and 4) reinforcement learning and imitation learning.
Perspectives on the formation of peakons in the stochastic Camassa–Holm equation

A famous feature of the Camassa–Holm equation is its admission of peaked soliton solutions known as peakons. We investigate this equation under the influence of stochastic transport. Noting that peakons are weak solutions of the equation, we present a finite-element discretization for it, which we use to explore the formation of peakons. Our simulations using this discretization reveal that peakons can still form in the presence of stochastic perturbations. Peakons can emerge both through wave breaking, as the slope turns vertical, and without wave breaking as the inflection points of the velocity profile rise to reach the summit.
Flood Detection On Low Cost Orbital Hardware

Satellite imaging is a critical technology for monitoring and responding to natural disasters such as flooding. Despite the capabilities of modern satellites, there is still much to be desired from the perspective of first response organisations like UNICEF. Two main challenges are rapid access to data, and the ability to automatically identify flooded regions in images. We describe a prototypical flood segmentation system, identifying cloud, water and land, that could be deployed on a constellation of small satellites, performing processing on board to reduce downlink bandwidth by 2 orders of magnitude. We target PhiSat-1, part of the FSSCAT mission, which is planned to be launched by the European Space Agency (ESA) near the start of 2020 as a proof of concept for this new technology.
Bregman-divergence-guided Legendre exponential dispersion model with finite cumulants (K-LED)

Exponential dispersion model is a useful framework in machine learning and statistics. Primarily, thanks to the additive structure of the model, it can be achieved without difficulty to estimate parameters including mean. However, tight conditions on cumulant function, such as analyticity, strict convexity, and steepness, reduce the class of exponential dispersion model. In this work, we present relaxed exponential dispersion model K-LED (Legendre exponential dispersion model with K cumulants). The cumulant function of the proposed model is a convex function of Legendre type having continuous partial derivatives of K-th order on the interior of a convex domain. Most of the K-LED models are developed via Bregman-divergence-guided log-concave density function with coercivity shape constraints. The main advantage of the proposed model is that the first cumulant (or the mean parameter space) of the 1-LED model is easily computed through the extended global optimum property of Bregman divergence. An extended normal distribution is introduced as an example of 1-LED based on Tweedie distribution. On top of that, we present 2-LED satisfying mean-variance relation of quasi-likelihood function. There is an equivalence between a subclass of quasi-likelihood function and a regular 2-LED model, of which the canonical parameter space is open. A typical example is a regular 2-LED model with power variance function, i.e., a variance is in proportion to the power of the mean of observations. This model is equivalent to a subclass of beta-divergence (or a subclass of quasi-likelihood function with power variance function). Furthermore, a new parameterized K-LED model, the cumulant function of which is the convex extended logistic loss function, is proposed. This model includes Bernoulli distribution and Poisson distribution.
Synthesizing credit card transactions

As noted by Turing Laureates Geoffrey Hinton and Yan LeCun , two elements have been essential to AI's recent boom: (1) deep neural nets and the theory and practice behind them; and (2) cloud computing with its abundant labeled data and large computing resources. Abundant labeled data is available for key domains such as images, speech, natural language processing, and recommendation engines. However, in many other domains such data is not available, or access is highly restricted for privacy reasons, as with health and financial data. Even when abundant data is available, it is often not labeled. Doing such labeling is labor-intensive and non-scalable. To get around these data problems there have been many proposals to generate synthetic data [20, 24, 29, 30, 35, 39]. However, to the best of our knowledge, key domains still lack labeled data or have at most toy data; or the synthetic data must have access to real data from which it can mimic new data. Looking to some of the challenges outlined in  at ICAIF'2020, this paper outlines work to generate realistic synthetic data without those restrictions and for an important domain: credit card transactions - including both normal and fraudulent transactions. At first glance it may appear simple to generate such transactions - just formalize a few items of the nature, "Sally sold slacks to Sue on Sunday." However, there are many patterns and correlations in real purchases. And there are millions of merchants and innumerable locations. And those merchants offer a wide variety of goods. Determining who shops where and when becomes daunting. Challenging also is the question of how much people pay. Inserting fraudulent transactions in the mix and doing all of these things with no real seed data provide final challenges. Generating good data to overcome these obstacles benefits from a mixture of technical approaches and domain knowledge. Those domains of knowledge include mechanics of credit card processing as well as a broad set of consumer domains, from electronics to clothing to hair styling to home improvement and many more. We also find that creation of a virtual world depicting people's commercial lives facilitates generation of high-quality, realistic data. This paper outlines some of our key techniques and provides evidence that the data generated is indeed realistic via comparisons to Federal Reserve data, recent data from a major card issuer, and more. At the end of the paper we also provide a link to a public sample of our data . Although beyond the scope of this paper, our synthetic credit-card data also facilitates development and training of models to predict fraud. Those models coupled with the synthetic dataset also provide foundations for designing acceleration hardware, just as GPUs, TPUs [10, 19] and other devices have been used for domains such as image classification, object detection, natural language processing, etc.
Measuring Attack Surface Reduction in the Presence of Code (Re-)Randomization

Just-in-time return-oriented programming (JIT-ROP) technique allows one to dynamically discover instruction pages and launch code reuse attacks, effectively bypassing most fine-grained address space layout randomization (ASLR) protection. However, in-depth questions regarding the impact of code (re-)randomization on code reuse attacks have not been studied. For example, how do starting pointers in JIT-ROP impact gadget availability?; how would one compute the re-randomization interval effectively to defeat JIT-ROP attacks? what impact do fine-grained randomization and re-randomization have on the Turing completeness of JIT-ROP payloads? We conduct a comprehensive measurement study on the effectiveness of fine-grained code randomization and re-randomization, with 5 tools, 13 applications, and 19 dynamic libraries. We provide methodologies to measure JIT-ROP gadget availability, quality, and their Turing completeness, as well as to empirically determine the upper bound of re-randomization intervals in re-randomization schemes. Experiments show that instruction reordering is the only fine-grained single-round randomization approach that thwarts current gadget finding techniques under the JIT-ROP threat model. Our results also show that the locations of leaked pointers used in JIT-ROP attacks have no impacts on gadget availability, suggesting high pointer-based connectivity among code pages.
Gunrock: A Social Bot for Complex and Engaging Long Conversations

Gunrock is the winner of the 2018 Amazon Alexa Prize, as evaluated by coherence and engagement from both real users and Amazon-selected expert conversationalists. We focus on understanding complex sentences and having in-depth conversations in open domains. In this paper, we introduce some innovative system designs and related validation analysis. Overall, we found that users produce longer sentences to Gunrock, which are directly related to users’ engagement (e.g., ratings, number of turns). Additionally, users’ backstory queries about Gunrock are positively correlated to user satisfaction. Finally, we found dialog flows that interleave facts and personal opinions and stories lead to better user satisfaction.
Graph Few-shot Learning via Knowledge Transfer

Towards the challenging problem of semi-supervised node classification, there have been extensive studies. As a frontier, Graph Neural Networks (GNNs) have aroused great interest recently, which update the representation of each node by aggregating information of its neighbors. However, most GNNs have shallow layers with a limited receptive field and may not achieve satisfactory performance especially when the number of labeled nodes is quite small. To address this challenge, we innovatively propose a graph few-shot learning (GFL) algorithm that incorporates prior knowledge learned from auxiliary graphs to improve classification accuracy on the target graph. Specifically, a transferable metric space characterized by a node embedding and a graph-specific prototype embedding function is shared between auxiliary graphs and the target, facilitating the transfer of structural knowledge. Extensive experiments and ablation studies on four real-world graph datasets demonstrate the effectiveness of our proposed model and the contribution of each component.
A Bank Salvage Model by Impulse Stochastic Controls

The present paper is devoted to the study of a bank salvage model with a finite time horizon that is subjected to stochastic impulse controls. In our model, the bank’s default time is a completely inaccessible random quantity generating its own filtration, then reflecting the unpredictability of the event itself. In this framework the main goal is to minimize the total cost of the central controller, which can inject capitals to save the bank from default. We address the latter task, showing that the corresponding quasi-variational inequality (QVI) admits a unique viscosity solution—Lipschitz continuous in space and Hölder continuous in time. Furthermore, under mild assumptions on the dynamics the smooth-fit W l o c ( 1 , 2 ) , p property is achieved for any 1 < p < + ∞ .
Decentralized Multi-Agent Actor-Critic with Generative Inference

Recent multi-agent actor-critic methods have utilized centralized training with decentralized execution to address the non-stationarity of co-adapting agents. This training paradigm constrains learning to the centralized phase such that only pre-learned policies may be used during the decentralized phase, which performs poorly when agent communications are delayed, noisy, or disrupted. In this work, we propose a new system that can gracefully handle partially-observable information due to communication disruptions during decentralized execution. Our approach augments the multi-agent actor-critic method's centralized training phase with generative modeling so that agents may infer other agents' observations when provided with locally available context. Our method is evaluated on three tasks that require agents to combine local and remote observations communicated by other agents. We evaluate our approach by introducing both partial observability during decentralized execution, and show that decentralized training on inferred observations performs as well or better than existing actor-critic methods.
Impact of Inference Accelerators on hardware selection

As opportunities for AI-assisted healthcare grow steadily, model deployment faces challenges due to the specific characteristics of the industry. The configuration choice for a production device can impact model performance while influencing operational costs. Moreover, in healthcare some situations might require fast, but not real time, inference. We study different configurations and conduct a cost-performance analysis to determine the optimized hardware for the deployment of a model subject to healthcare domain constraints. We observe that a naive performance comparison may not lead to an optimal configuration selection. In fact, given realistic domain constraints, CPU execution might be preferable to GPU accelerators. Hence, defining beforehand precise expectations for model deployment is crucial.
Make Up Your Mind! Adversarial Generation of Inconsistent Natural Language Explanations

To increase trust in artificial intelligence systems, a promising research direction consists of designing neural models capable of generating natural language explanations for their predictions. In this work, we show that such models are nonetheless prone to generating mutually inconsistent explanations, such as ”Because there is a dog in the image.” and ”Because there is no dog in the [same] image.”, exposing flaws in either the decision-making process of the model or in the generation of the explanations. We introduce a simple yet effective adversarial framework for sanity checking models against the generation of inconsistent natural language explanations. Moreover, as part of the framework, we address the problem of adversarial attacks with full target sequences, a scenario that was not previously addressed in sequence-to-sequence attacks. Finally, we apply our framework on a state-of-the-art neural natural language inference model that provides natural language explanations for its predictions. Our framework shows that this model is capable of generating a significant number of inconsistent explanations.
Do Speed Bumps Curb Low-Latency Investment? Evidence From a Laboratory Market

Exchanges implement intentional trade delays to limit the harmful impact of low-latency trading. Do such “speed bumps” curb investment in fast trading technology? Data are scarce since trading technologies are proprietary. We build an experimental trading platform where participants face speed bumps and can invest in fast trading technology. We find that asymmetric speed bumps, on average, reduce investment in speed by only 20%. Increasing the magnitude of the speed bump by one standard deviation further reduces low-latency investment by 8.33%. Finally, introducing a symmetric speed bump leads to the same investment level as no speed bump at all.
Abrupt phase transition of epidemic spreading in simplicial complexes

Recent studies on network geometry, a way of describing network structures as geometrical objects, are revolutionizing our way to understand dynamical processes on networked systems. Here, we cope with the problem of epidemic spreading, using the Susceptible-Infected-Susceptible (SIS) model, in simplicial complexes. In particular, we analyze the dynamics of SIS in complex networks characterized by pairwise interactions (links), and three-body interactions (filled triangles, also known as 2-simplices). This higher-order description of the epidemic spreading is analytically formulated using a microscopic Markov chain approximation. The analysis of the fixed point solutions of the model, reveal an interesting phase transition that becomes abrupt with the infectivity parameter of the 2-simplices. Our results pave the way for network theorists to advance in our physical understanding of epidemic spreading in real scenarios where diseases are transmitted among groups as well as among pairs, and to better understand the behaviour of dynamical processes in simplicial complexes.