nils-herrmann/clean_scirepeval huggingface dataset:
Reorderability of node-filtered order complexes.

Growing graphs describe a multitude of developing processes from maturing brains to expanding vocabularies to burgeoning public transit systems. Each of these growing processes likely adheres to proliferation rules that establish an effective order of node and connection emergence. When followed, such proliferation rules allow the system to properly develop along a predetermined trajectory. But rules are rarely followed. Here we ask what topological changes in the growing graph trajectories might occur after the specific but basic perturbation of permuting the node emergence order. Specifically, we harness applied topological methods to determine which of six growing graph models exhibit topology that is robust to randomizing node order, termed global reorderability, and robust to temporally local node swaps, termed local reorderability. We find that the six graph models fall upon a spectrum of both local and global reorderability, and furthermore we provide theoretical connections between robustness to node pair ordering and robustness to arbitrary node orderings. Finally, we discuss real-world applications of reorderability analyses and suggest possibilities for designing reorderable networks.
MASS: Masked Sequence to Sequence Pre-training for Language Generation

Pre-training and fine-tuning, e.g., BERT, have achieved great success in language understanding by transferring knowledge from rich-resource pre-training task to the low/zero-resource downstream tasks. Inspired by the success of BERT, we propose MAsked Sequence to Sequence pre-training (MASS) for the encoder-decoder based language generation tasks. MASS adopts the encoder-decoder framework to reconstruct a sentence fragment given the remaining part of the sentence: its encoder takes a sentence with randomly masked fragment (several consecutive tokens) as input, and its decoder tries to predict this masked fragment. In this way, MASS can jointly train the encoder and decoder to develop the capability of representation extraction and language modeling. By further fine-tuning on a variety of zero/low-resource language generation tasks, including neural machine translation, text summarization and conversational response generation (3 tasks and totally 8 datasets), MASS achieves significant improvements over the baselines without pre-training or with other pre-training methods. Specially, we achieve the state-of-the-art accuracy (37.5 in terms of BLEU score) on the unsupervised English-French translation, even beating the early attention-based supervised model.
Gravitational Waves from Mirror World

In this paper we consider the properties of the 10 confirmed by the LIGO (Laser Interferometer Gravitational-Wave Observatory) Collaboration gravitational wave signals from the black hole mergers. We want to explain non-observation of electromagnetic counterpart and higher then expected merging rates of these events, assuming the existence of their sources in the hidden mirror universe. Mirror matter, which interacts with our world only through gravity, is a candidate of dark matter and its density can exceed ordinary matter density five times. Since mirror world is considered to be colder, star formation there started earlier and mirror black holes had more time to pick up the mass and to create more binary systems within the LIGO reachable zone. In total, we estimate factor of 15 amplification of black holes merging rate in mirror world with respect to our world, which is consistent with the LIGO observations.
Neural 3D Morphable Models: Spiral Convolutional Networks for 3D Shape Representation Learning and Generation

Generative models for 3D geometric data arise in many important applications in 3D computer vision and graphics. In this paper, we focus on 3D deformable shapes that share a common topological structure, such as human faces and bodies. Morphable Models and their variants, despite their linear formulation, have been widely used for shape representation, while most of the recently proposed nonlinear approaches resort to intermediate representations, such as 3D voxel grids or 2D views. In this work, we introduce a novel graph convolutional operator, acting directly on the 3D mesh, that explicitly models the inductive bias of the fixed underlying graph. This is achieved by enforcing consistent local orderings of the vertices of the graph, through the spiral operator, thus breaking the permutation invariance property that is adopted by all the prior work on Graph Neural Networks. Our operator comes by construction with desirable properties (anisotropic, topology-aware, lightweight, easy-to-optimise), and by using it as a building block for traditional deep generative architectures, we demonstrate state-of-the-art results on a variety of 3D shape datasets compared to the linear Morphable Model and other graph convolutional operators.
Deep Flow-Guided Video Inpainting

Video inpainting, which aims at filling in missing regions in a video, remains challenging due to the difficulty of preserving the precise spatial and temporal coherence of video contents. In this work we propose a novel flow-guided video inpainting approach. Rather than filling in the RGB pixels of each frame directly, we consider the video inpainting as a pixel propagation problem. We first synthesize a spatially and temporally coherent optical flow field across video frames using a newly designed Deep Flow Completion network, then use the synthesized flow fields to guide the propagation of pixels to fill up the missing regions in the video. Specifically, the Deep Flow Competion network follows a coarse-to-fine refinement strategy to complete the flow fields, while their quality is further improved by hard flow example mining. Following the guide of the completed flow fields, the missing video regions can be filled up precisely. Our method is evaluated on DAVIS and YouTubeVOS datasets qualitatively and quantitatively, achieving the state-of-the-art performance in terms of inpainting quality and speed.
Meta-learning of Sequential Strategies

In this report we review memory-based meta-learning as a tool for building sample-efficient strategies that learn from past experience to adapt to any task within a target class. Our goal is to equip the reader with the conceptual foundations of this tool for building new, scalable agents that operate on broad domains. To do so, we present basic algorithmic templates for building near-optimal predictors and reinforcement learners which behave as if they had a probabilistic model that allowed them to efficiently exploit task structure. Furthermore, we recast memory-based meta-learning within a Bayesian framework, showing that the meta-learned strategies are near-optimal because they amortize Bayes-filtered data, where the adaptation is implemented in the memory dynamics as a state-machine of sufficient statistics. Essentially, memory-based meta-learning translates the hard problem of probabilistic sequential inference into a regression problem.
RWTH ASR Systems for LibriSpeech: Hybrid vs Attention - w/o Data Augmentation

We present state-of-the-art automatic speech recognition (ASR) systems employing a standard hybrid DNN/HMM architecture compared to an attention-based encoder-decoder design for the LibriSpeech task. Detailed descriptions of the system development, including model design, pretraining schemes, training schedules, and optimization approaches are provided for both system architectures. Both hybrid DNN/HMM and attention-based systems employ bi-directional LSTMs for acoustic modeling/encoding. For language modeling, we employ both LSTM and Transformer based architectures. All our systems are built using RWTHs open-source toolkits RASR and RETURNN. To the best knowledge of the authors, the results obtained when training on the full LibriSpeech training set, are the best published currently, both for the hybrid DNN/HMM and the attention-based systems. Our single hybrid system even outperforms previous results obtained from combining eight single systems. Our comparison shows that on the LibriSpeech 960h task, the hybrid DNN/HMM system outperforms the attention-based system by 15% relative on the clean and 40% relative on the other test sets in terms of word error rate. Moreover, experiments on a reduced 100h-subset of the LibriSpeech training corpus even show a more pronounced margin between the hybrid DNN/HMM and attention-based architectures.
Unified Language Model Pre-training for Natural Language Understanding and Generation

This paper presents a new Unified pre-trained Language Model (UniLM) that can be fine-tuned for both natural language understanding and generation tasks. The model is pre-trained using three types of language modeling tasks: unidirectional, bidirectional, and sequence-to-sequence prediction. The unified modeling is achieved by employing a shared Transformer network and utilizing specific self-attention masks to control what context the prediction conditions on. UniLM compares favorably with BERT on the GLUE benchmark, and the SQuAD 2.0 and CoQA question answering tasks. Moreover, UniLM achieves new state-of-the-art results on five natural language generation datasets, including improving the CNN/DailyMail abstractive summarization ROUGE-L to 40.51 (2.04 absolute improvement), the Gigaword abstractive summarization ROUGE-L to 35.75 (0.86 absolute improvement), the CoQA generative question answering F1 score to 82.5 (37.1 absolute improvement), the SQuAD question generation BLEU-4 to 22.12 (3.75 absolute improvement), and the DSTC7 document-grounded dialog response generation NIST-4 to 2.67 (human performance is 2.65). The code and pre-trained models are available at this https URL.
Entrofy Your Cohort: A Data Science Approach to Candidate Selection

Selecting a cohort from a set of candidates is a common task within and beyond academia. Admitting students, awarding grants, choosing speakers for a conference are situations where human biases may affect the make-up of the final cohort. We propose a new algorithm, Entrofy, designed to be part of a larger decision making strategy aimed at making cohort selection as just, quantitative, transparent, and accountable as possible. We suggest this algorithm be embedded in a two-step selection procedure. First, all application materials are stripped of markers of identity that could induce conscious or sub-conscious bias. During blind review, the committee selects all applicants, submissions, or other entities that meet their merit-based criteria. This often yields a cohort larger than the admissible number. In the second stage, the target cohort can be chosen from this meritorious pool via a new algorithm and software tool. Entrofy optimizes differences across an assignable set of categories selected by the human committee. Criteria could include gender, academic discipline, experience with certain technologies, or other quantifiable characteristics. The Entrofy algorithm yields the computational maximization of diversity by solving the tie-breaking problem with provable performance guarantees. We show how Entrofy selects cohorts according to pre-determined characteristics in simulated sets of applications and demonstrate its use in a case study. This cohort selection process allows human judgment to prevail when assessing merit, but assigns the assessment of diversity to a computational process less likely to be beset by human bias. Importantly, the stage at which diversity assessments occur is fully transparent and auditable with Entrofy. Splitting merit and diversity considerations into their own assessment stages makes it easier to explain why a given candidate was selected or rejected.
Fast Uniform Generation of Random Graphs with Given Degree Sequences

In this paper we provide an algorithm that generates a graph with given degree sequence uniformly at random. Provided that Δ^4=O(m), where Δ is the maximal degree and m is the number of edges, the algorithm runs in expected time O(m). Our algorithm significantly improves the previously most efficient uniform sampler, which runs in expected time O(m^2Δ^2) for the same family of degree sequences. Our method uses a novel ingredient which progressively relaxes restrictions on an object being generated uniformly at random, and we use this to give fast algorithms for uniform sampling of graphs with other degree sequences as well. Using the same method, we also obtain algorithms with expected run time which is (i) linear for power-law degree sequences in cases where the previous best was O(n^4.081), and (ii) O(nd+d^4) for d-regular graphs when d=o(√ n), where the previous best was O(nd^3).
Simulating Problem Difficulty in Arithmetic Cognition Through Dynamic Connectionist Models

The present study aims to investigate similarities between how humans and connectionist models experience difficulty in arithmetic problems. Problem difficulty was operationalized by the number of carries involved in solving a given problem. Problem difficulty was measured in humans by response time, and in models by computational steps. The present study found that both humans and connectionist models experience difficulty similarly when solving binary addition and subtraction. Specifically, both agents found difficulty to be strictly increasing with respect to the number of carries. Another notable similarity is that problem difficulty increases more steeply in subtraction than in addition, for both humans and connectionist models. Further investigation on two model hyperparameters --- confidence threshold and hidden dimension --- shows higher confidence thresholds cause the model to take more computational steps to arrive at the correct answer. Likewise, larger hidden dimensions cause the model to take more computational steps to correctly answer arithmetic problems; however, this effect by hidden dimensions is negligible.
Mappa Mundi: An Interactive Artistic Mind Map Generator with Artificial Imagination

We present a novel real-time, collaborative, and interactive AI painting system, Mappa Mundi, for artistic Mind Map creation. The system consists  of a voice-based input interface, an automatic topic expansion module, and an image projection module. The key innovation is to inject Artificial Imagination into painting creation by considering lexical and phonological similarities of language, learning and inheriting
artist’s original painting style, and applying the principles of Dadaism and impossibility of improvisation. Our system indicates that AI and artist can collaborate seamlessly to create imaginative artistic painting and Mappa Mundi has been applied in art exhibition in UCCA, Beijing.
Improving Discrete Latent Representations With Differentiable Approximation Bridges

Modern neural network training relies on piece-wise (sub-)differentiable functions in order to use backpropagation to update model parameters. In this work, we introduce a novel method to allow non-differentiable functions at intermediary layers of deep neural networks. We do so by training with a differentiable approximation bridge (DAB) neural network which approximates the non-differentiable forward function and provides gradient updates during backpropagation. We present empirical results from more than 600 experiments in four different domains: unsupervised (image) representation learning, variational (image) density estimation, image classification, and sequence sorting to demonstrate that our proposed method improves state of the art performance. We demonstrate that training with DAB aided discrete non-differentiable functions improves image reconstruction quality and posterior linear separability by 10% against the Gumbel-Softmax relaxed estimator [26],  as well as providing a 9% improvement in the test variational lower bound in comparison to the state of the art RELAX [16] discrete estimator. We also observe an accuracy improvement of 77% in neural sequence sorting and a 25% improvement against the straight-through estimator [5] in an image classification setting. The DAB network is not used for inference and expands the class of functions that are usable in neural networks. Our code is available at https://github.com/apple/ml-dab.
Testing Emergent Gravity with Optical, X-Ray, and Weak Lensing Measurements in Massive, Relaxed Galaxy Clusters

We test the predictions of emergent gravity (EG) using matter densities of relaxed, massive clusters of galaxies observed from optical and X-ray wavebands. We improve upon previous work in this area by including the baryon mass contribution of the brightest cluster galaxy (BCG) in each system, in addition to total mass profiles from gravitational lensing and mass profiles of the X-ray emitting gas from Chandra. We use this data in the context of EG to predict the “apparent” dark matter (DM) distribution from the observed baryon distribution, and vice versa. We find that although the inclusion of the BCG in the analysis improves the agreement with observations in the inner regions of the clusters ( kpc), at larger radii ( kpc) the EG predictions for mass profiles and baryon mass fractions are not in agreement with observations by a factor of up to ∼2−6, though the agreement improves at radii near r500. At least in its current form, EG does not appear to reproduce the observed characteristics of relaxed galaxy clusters as well as cold DM models.
Neural-Guided RANSAC: Learning Where to Sample Model Hypotheses

We present Neural-Guided RANSAC (NG-RANSAC), an extension to the classic RANSAC algorithm from robust optimization. NG-RANSAC uses prior information to improve model hypothesis search, increasing the chance of finding outlier-free minimal sets. Previous works use heuristic side-information like hand-crafted descriptor distance to guide hypothesis search. In contrast, we learn hypothesis search in a principled fashion that lets us optimize an arbitrary task loss during training, leading to large improvements on classic computer vision tasks. We present two further extensions to NG-RANSAC. Firstly, using the inlier count itself as training signal allows us to train neural guidance in a self-supervised fashion. Secondly, we combine neural guidance with differentiable RANSAC to build neural networks which focus on certain parts of the input data and make the output predictions as good as possible. We evaluate NG-RANSAC on a wide array of computer vision tasks, namely estimation of epipolar geometry, horizon line estimation and camera re-localization. We achieve superior or competitive results compared to state-of-the-art robust estimators, including very recent, learned ones.
DeepVCP: An End-to-End Deep Neural Network for Point Cloud Registration

We present DeepVCP - a novel end-to-end learning-based 3D point cloud registration framework that achieves comparable registration accuracy to prior state-of-the-art geometric methods. Different from other keypoint based methods where a RANSAC procedure is usually needed, we implement the use of various deep neural network structures to establish an end-to-end trainable network. Our keypoint detector is trained through this end-to-end structure and enables the system to avoid the interference of dynamic objects, leverages the help of sufficiently salient features on stationary objects, and as a result, achieves high robustness. Rather than searching the corresponding points among existing points, the key contribution is that we innovatively generate them based on learned matching probabilities among a group of candidates, which can boost the registration accuracy. We comprehensively validate the effectiveness of our approach using both the KITTI dataset and the Apollo-SouthBay dataset. Results demonstrate that our method achieves comparable registration accuracy and runtime efficiency to the state-of-the-art geometry-based methods, but with higher robustness to inaccurate initial poses. Detailed ablation and visualization analysis are included to further illustrate the behavior and insights of our network. The low registration error and high robustness of our method make it attractive to the substantial applications relying on the point cloud registration task.
Complete basis set extrapolation of electronic correlation energies using the Riemann zeta function.

In this communication we demonstrate the effectiveness of the method of complete basis set (CBS) extrapolation of correlation energies based on the application of the Riemann zeta function. Instead of fitting the results obtained with a systematic sequence of one-electron bases with a certain functional form, an analytic re-summation of the missing contributions coming from higher angular momenta, $l$ is performed. The assumption that these contributions vanish asymptotically as an inverse power of $l$ leads to an expression for the CBS limit given in terms of the zeta function. This result is turned into an extrapolation method that is very easy to use and requires no "empirical" parameters to be optimized. The performance of the method is assessed by comparing the results with very accurate reference data obtained with explicitly correlated theories and with results obtained with standard extrapolation schemes. On average, the errors of the zeta-function extrapolation are several times smaller compared with the conventional schemes employing the same sequence of bases. A recipe for the estimation of the residual extrapolation error is also proposed.
Unique Multimessenger Signal of QCD Axion Dark Matter.

We propose a multimessenger probe of QCD axion dark matter based on observations of black hole-neutron star binary inspirals. It is suggested that a dense dark matter spike may grow around intermediate mass black holes (10^{3}-10^{5}  M_{⊙}). The presence of such a spike produces two unique effects: a distinct phase shift in the gravitational wave strain during the inspiral and an enhancement of the radio emission due to the resonant axion-photon conversion occurring in the neutron star magnetosphere throughout the inspiral and merger. Remarkably, the observation of the gravitational wave signal can be used to infer the dark matter density and, consequently, to predict the radio emission. We study the projected reach of the LISA interferometer and next-generation radio telescopes such as the Square Kilometre Array. Given a sufficiently nearby system, such observations will potentially allow for the detection of QCD axion dark matter in the mass range 10^{-7}  eV to 10^{-5}  eV.
Governance by Glass-Box: Implementing Transparent Moral Bounds for AI Behaviour

Artificial Intelligence (AI) applications are being used to predict and assess behaviour in multiple domains which directly affect human well-being. However, if AI is to improve people’s lives, then
people must be able to trust it, by being able to understand what the system is doing and why. Although transparency is often seen as the requirement in this case, realistically it might not always
be possible, whereas the need to ensure that the system operates within set moral bounds remains. 
In this paper, we present an approach to evaluate the moral bounds of an AI system based on the monitoring of its inputs and outputs. We place a ‘Glass-Box’ around the system by mapping moral values into explicit verifiable norms that constrain inputs and outputs, in such a way that if these remain within the box we can guarantee that the system adheres to the value. The focus on inputs and outputs allows for the verification and comparison of vastly different intelligent systems; from deep neural networks to agent-based systems.
The explicit transformation of abstract moral values into concrete norms brings great benefits in terms of explainability; stakeholders know exactly how the system is interpreting and employing relevant abstract moral human values and calibrate their trust accordingly. Moreover, by operating at a higher level we can check the compliance of the system with different interpretations of the same value.