nils-herrmann/clean_scirepeval huggingface dataset:
Rethinking Text Attribute Transfer: A Lexical Analysis

Text attribute transfer is modifying certain linguistic attributes (e.g. sentiment, style, author-ship, etc.) of a sentence and transforming them from one type to another. In this paper, we aim to analyze and interpret what is changed during the transfer process. We start from the observation that in many existing models and datasets, certain words within a sentence play important roles in determining the sentence attribute class. These words are referred as the Pivot Words. Based on these pivot words, we propose a lexical analysis framework, the Pivot Analysis, to quantitatively analyze the effects of these words in text attribute classification and transfer. We apply this framework to existing datasets and models and show that: (1) the pivot words are strong features for the classification of sentence attributes; (2) to change the attribute of a sentence, many datasets only requires to change certain pivot words; (3) consequently, many transfer models only perform the lexical-level modification,while leaving higher-level sentence structures unchanged. Our work provides an in-depth understanding of linguistic attribute transfer and further identifies the future requirements and challenges of this task
Hardware Design and Analysis of the ACE and WAGE Ciphers

This paper presents the hardware design and analysis of ACE and WAGE, two candidate ciphers for the NIST Lightweight Cryptography standardization. Both ciphers use sLiSCP's unified sponge duplex mode. ACE has an internal state of 320 bits, uses three 64 bit Simeck boxes, and implements both authenticated encryption and hashing. WAGE is based on the Welch-Gong stream cipher and provides authenticated encryption. WAGE has 259 bits of state, two 7 bit Welch-Gong permutations, and four lightweight 7 bit S-boxes. ACE and WAGE have the same external interface and follow the same I/O protocol to transition between phases. The paper illustrates how a hardware perspective influenced key aspects of the ACE and WAGE algorithms. The paper reports area, power, and energy results for both serial and parallel (unrolled) implementations using four different ASIC libraries: two 65 nm libraries, a 90 nm library, and a 130 nm library. ACE implementations range from a throughput of 0.5 bits-per-clock cycle (bpc) and an area of 4210 GE (averaged across the four ASIC libraries) up to 4 bpc and 7260 GE. WAGE results range from 0.57 bpc with 2920 GE to 4.57 bpc with 11080 GE.
At Stability's Edge: How to Adjust Hyperparameters to Preserve Minima Selection in Asynchronous Training of Neural Networks?

Recent developments have made it possible to accelerate neural networks training significantly using large batch sizes and data parallelism. Training in an asynchronous fashion, where delay occurs, can make training even more scalable. However, asynchronous training has its pitfalls, mainly a degradation in generalization, even after convergence of the algorithm. This gap remains not well understood, as theoretical analysis so far mainly focused on the convergence rate of asynchronous methods.  We examine asynchronous training from the perspective of dynamical stability. We find that the degree of delay interacts with the learning rate, to change the set of minima accessible by an asynchronous stochastic gradient descent algorithm. We derive closed-form rules on how the learning rate could be changed, while keeping the accessible set the same. Specifically, for high delay values, we find that the learning rate should be kept inversely proportional to the delay. We then extend this analysis to include momentum. We find momentum should be either turned off, or modified to improve training stability. We provide empirical experiments to validate our theoretical findings.
Compressed Sensing Microscopy with Scanning Line Probes

In applications of scanning probe microscopy, images are acquired by raster scanning a point probe across a sample. Viewed from the perspective of compressed sensing (CS), this pointwise sampling scheme is inefficient, especially when the target image is structured. While replacing point measurements with delocalized, incoherent measurements has the potential to yield order-of-magnitude improvements in scan time, implementing the delocalized measurements of CS theory is challenging. In this paper we study a partially delocalized probe construction, in which the point probe is replaced with a continuous line, creating a sensor which essentially acquires line integrals of the target image. We show through simulations, rudimentary theoretical analysis, and experiments, that these line measurements can image sparse samples far more efficiently than traditional point measurements, provided the local features in the sample are enough separated. Despite this promise, practical reconstruction from line measurements poses additional difficulties: the measurements are partially coherent, and real measurements exhibit nonidealities. We show how to overcome these limitations using natural strategies (reweighting to cope with coherence, blind calibration for nonidealities), culminating in an end-to-end demonstration.
Thermodynamics and reentrant phase transition for logarithmic nonlinear charged black holes in massive gravity

We investigate a new class of  see text]-dimensional topological black hole solutions in the context of massive gravity and in the presence of logarithmic nonlinear electrodynamics. Exploring higher-dimensional solutions in massive gravity coupled to nonlinear electrodynamics is motivated by holographic hypothesis as well as string theory. We first construct exact solutions of the field equations and then explore the behavior of the metric functions for different values of the model parameters. We observe that our black holes admit the multi-horizons caused by a quantum effect called anti-evaporation. Next, by calculating the conserved and thermodynamic quantities, we obtain a generalized Smarr formula. We find that the first law of black holes thermodynamics is satisfied on the black hole horizon. We study thermal stability of the obtained solutions in both canonical and grand canonical ensembles. We reveal that depending on the model parameters, our solutions exhibit a rich variety of phase structures. Finally, we explore, for the first time without extending thermodynamics phase space, the critical behavior and reentrant phase transition for black hole solutions in massive gravity theory. We realize that there is a zeroth-order phase transition for a specified range of charge value and the system experiences a large/small/large reentrant phase transition due to the presence of nonlinear electrodynamics.
Interactive Particle Systems on Hypergraphs, Drift Analysis and the WalkSAT algorithm

We analyze the expected running time of WalkSAT, a well-known local search procedure for satisfiability solving, on satisfiable instances of the k-XOR SAT problem. We obtain estimates of this expected running time by reducing the problem to a setting amenable to classical techniques from drift analysis. 
A crucial ingredient of this reduction is the definition of (new, explosive) hypergraph versions of interacting particle systems, notably of coalescing and annihilating random walks as well as the voter model. The use of these tools allows to show that the expected running time of WalkSAT depends on structural parameter (we call odd Cheeger drift) of the dual of the formula hypergraph.
Realtime Simulation of Thin-Shell Deformable Materials Using CNN-Based Mesh Embedding

We address the problem of accelerating thin-shell deformable object simulations by dimension reduction. We present a new algorithm to embed a high-dimensional configuration space of deformable objects in a low-dimensional feature space, where the configurations of objects and feature points have approximate one-to-one mapping. Our key technique is a graph-based convolutional neural network (CNN) defined on meshes with arbitrary topologies and a new mesh embedding approach based on physics-inspired loss term. We have applied our approach to accelerate high-resolution thin shell simulations corresponding to cloth-like materials, where the configuration space has tens of thousands of degrees of freedom. We show that our physics-inspired embedding approach leads to higher accuracy compared with prior mesh embedding methods. Finally, we show that the temporal evolution of the mesh in the feature space can also be learned using a recurrent neural network (RNN) leading to fully learnable physics simulators. After training our learned simulator runs 500–10000× faster and the accuracy is high enough for robot manipulation tasks.
Radio constraints on dark matter annihilation in Canes Venatici I with LOFAR†

Dwarf galaxies are dark matter (DM) dominated and therefore promising targets for the search for weakly interacting massive particles (WIMPs), which are well-known candidates for DM. The annihilation of WIMPs produces ultrarelativistic cosmic ray electrons and positrons that emit synchrotron radiation in the presence of magnetic fields. For typical magnetic field strengths (few μG) and $\mathcal {O}$(GeV–TeV) WIMP masses, this emission peaks at hundreds of MHz. Here, we use the non-detection of 150-MHz radio continuum emission from the dwarf spheroidal galaxy Canes Venatici I with the Low-Frequency Array to derive constraints on the annihilation cross-section of WIMPs into primary electron–positron and other fundamental particle–antiparticle pairs. Our main underlying assumption is that the transport of the cosmic rays can be described by the diffusion approximation, thus requiring a non-zero magnetic field strength with small-scale structure. In particular, by adopting magnetic field strengths of $\mathcal {O}(1\, \mu$G) and diffusion coefficients $\sim \!10^{27}~\rm cm^2\, s^{-1}$, we obtain limits that are comparable with those set by the Fermi Large Area Telescope using gamma-ray observations of this particular galaxy. Assuming s-wave annihilation and WIMPs making up 100 per cent of the DM density, our benchmark limits exclude several thermal WIMP realizations in the [2, 20]-GeV mass range. We caution, however, that our limits for the cross-section are subject to enormous uncertainties that we also quantitatively assess. In particular, variations on the propagation parameters or on the DM halo can shift our limits up by several orders of magnitude (in the pessimistic scenario).
Can We Trust You? On Calibration of a Probabilistic Object Detector for Autonomous Driving

Reliable uncertainty estimation is crucial for perception systems in safe autonomous driving. Recently, many methods have been proposed to model uncertainties in deep learning based object detectors. However, the estimated probabilities are often uncalibrated, which may lead to severe problems in safety critical scenarios. In this work, we identify such uncertainty miscalibration problems in a probabilistic LiDAR 3D object detection network, and propose three practical methods to significantly reduce errors in uncertainty calibration. Extensive experiments on several datasets show that our methods produce well-calibrated uncertainties, and generalize well between different datasets.
Overparameterized neural networks implement associative memory

Development of computational models of memory is a subject of long-standing interest at the intersection of machine learning and neuroscience. Our main finding is that overparameterized neural networks trained using standard optimization methods provide a simple mechanism for implementing associative memory. Remarkably, this mechanism allows for the storage and retrieval of sequences of examples. This finding also sheds light on inductive biases in overparameterized networks: while there are many functions that can achieve zero training loss in the overparameterized regime, our result shows that increasing depth and width in neural networks leads to maps that are more contractive around training examples, thereby allowing for storage and retrieval of more training examples. Identifying computational mechanisms for memorization and retrieval of data is a long-standing problem at the intersection of machine learning and neuroscience. Our main finding is that standard overparameterized deep neural networks trained using standard optimization methods implement such a mechanism for real-valued data. We provide empirical evidence that 1) overparameterized autoencoders store training samples as attractors and thus iterating the learned map leads to sample recovery, and that 2) the same mechanism allows for encoding sequences of examples and serves as an even more efficient mechanism for memory than autoencoding. Theoretically, we prove that when trained on a single example, autoencoders store the example as an attractor. Lastly, by treating a sequence encoder as a composition of maps, we prove that sequence encoding provides a more efficient mechanism for memory than autoencoding.
RL-LIM: Reinforcement Learning-based Locally Interpretable Modeling

Understanding black-box machine learning models is important towards their widespread adoption. However, developing globally interpretable models that explain the behavior of the entire model is challenging. An alternative approach is to explain black-box models through explaining individual prediction using a locally interpretable model. In this paper, we propose a novel method for locally interpretable modeling - Reinforcement Learning-based Locally Interpretable Modeling (RL-LIM). RL-LIM employs reinforcement learning to select a small number of samples and distill the black-box model prediction into a low-capacity locally interpretable model. Training is guided with a reward that is obtained directly by measuring agreement of the predictions from the locally interpretable model with the black-box model. RL-LIM near-matches the overall prediction performance of black-box models while yielding human-like interpretability, and significantly outperforms state of the art locally interpretable models in terms of overall prediction performance and fidelity.
The SAMI Galaxy Survey: first detection of a transition in spin orientation with respect to cosmic filaments in the stellar kinematics of galaxies

We present the first detection of mass-dependent galactic spin alignments with local cosmic filaments with >2σ confidence using IFS kinematics. The 3D network of cosmic filaments is reconstructed on Mpc scales across GAlaxy and Mass Assembly fields using the cosmic web extractor DisPerSe. We assign field galaxies from the SAMI survey to their nearest filament segment in 3D and estimate the degree of alignment between SAMI galaxies’ kinematic spin axis and their nearest filament in projection. Low-mass galaxies align their spin with their nearest filament while higher mass counterparts are more likely to display an orthogonal orientation. The stellar transition mass from the first trend to the second is bracketed between $10^{10.4}$ and $10^{10.9}\, \mathrm{ M}_{\odot }$, with hints of an increase with filament scale. Consistent signals are found in the Horizon-AGN cosmological hydrodynamic simulation. This supports a scenario of early angular momentum build-up in vorticity rich quadrants around filaments at low stellar mass followed by progressive flip of spins orthogonal to the cosmic filaments through mergers at high stellar mass. Conversely, we show that dark matter only simulations post-processed with a semi-analytical model treatment of galaxy formation struggles to reproduce this alignment signal. This suggests that gas physics is key in enhancing the galaxy-filament alignment.
A Survey of Machine Learning Applied to Computer Architecture Design

Machine learning has enabled significant benefits in diverse fields, but, with a few exceptions, has had limited impact on computer architecture. Recent work, however, has explored broader applicability for design, optimization, and simulation. Notably, machine learning based strategies often surpass prior state-of-the-art analytical, heuristic, and human-expert approaches. This paper reviews machine learning applied system-wide to simulation and run-time optimization, and in many individual components, including memory systems, branch predictors, networks-on-chip, and GPUs. The paper further analyzes current practice to highlight useful design strategies and identify areas for future work, based on optimized implementation strategies, opportune extensions to existing work, and ambitious long term possibilities. Taken together, these strategies and techniques present a promising future for increasingly automated architectural design.
Graph-Preserving Grid Layout: A Simple Graph Drawing Method for Graph Classification using CNNs

Graph convolutional networks (GCNs) suffer from the irregularity of graphs, while more widely-used convolutional neural networks (CNNs) benefit from regular grids. To bridge the gap between GCN and CNN, in contrast to previous works on generalizing the basic operations in CNNs to graph data, in this paper we address the problem of how to project undirected graphs onto the grid in a {\em principled} way where CNNs can be used as backbone for geometric deep learning. To this end, inspired by the literature of graph drawing we propose a novel graph-preserving grid layout (GPGL), an integer programming that minimizes the topological loss on the grid. Technically we propose solving GPGL approximately using a {\em regularized} Kamada-Kawai algorithm, a well-known nonconvex optimization technique in graph drawing, with a vertex separation penalty that improves the rounding performance on top of the solutions from relaxation. Using GPGL we can easily conduct data augmentation as every local minimum will lead to a grid layout for the same graph. Together with the help of multi-scale maxout CNNs, we demonstrate the empirical success of our method for graph classification.
CS Sparse K-means: An Algorithm for Cluster-Specific Feature Selection in High-Dimensional Clustering

Feature selection is an important and challenging task in high dimensional clustering. For example, in genomics, there may only be a small number of genes that are differentially expressed, which are informative to the overall clustering structure. Existing feature selection methods, such as Sparse K-means, rarely tackle the problem of accounting features that can only separate a subset of clusters. In genomics, it is highly likely that a gene can only define one subtype against all the other subtypes or distinguish a pair of subtypes but not others. In this paper, we propose a K-means based clustering algorithm that discovers informative features as well as which cluster pairs are separable by each selected features. The method is essentially an EM algorithm, in which we introduce lasso-type constraints on each cluster pair in the M step, and make the E step possible by maximizing the raw cross-cluster distance instead of minimizing the intra-cluster distance. The results were demonstrated on simulated data and a leukemia gene expression dataset.
A Quest for Structure: Jointly Learning the Graph Structure and Semi-Supervised Classification

Semi-supervised learning (SSL) is effectively used for numerous classification problems, thanks to its ability to make use of abundant unlabeled data. The main assumption of various SSL algorithms is that the nearby points on the data manifold are likely to share a label. Graph-based SSL constructs a graph from point-cloud data as an approximation to the underlying manifold, followed by label inference. It is no surprise that the quality of the constructed graph in capturing the essential structure of the data is critical to the accuracy of the subsequent inference step . How should one construct a graph from the input point-cloud data for graph-based SSL? In this work we introduce a new, parallel graph learning framework (called PG-learn) for the graph construction step of SSL. Our solution has two main ingredients: (1) a gradient-based optimization of the edge weights (more specifically, different kernel bandwidths in each dimension) based on a validation loss function, and (2) a parallel hyperparameter search algorithm with an adaptive resource allocation scheme. In essence, (1) allows us to search around a (random) initial hyperparameter configuration for a better one with lower validation loss. Since the search space of hyperparameters is huge for high-dimensional problems, (2) empowers our gradient-based search to go through as many different initial configurations as possible, where runs for relatively unpromising starting configurations are terminated early to allocate the time for others. As such, PG-learn is a carefully-designed hybrid of random and adaptive search. Through experiments on multi-class classification problems, we show that PG-learn significantly outperforms a variety of existing graph construction schemes in accuracy (per fixed time budget for hyperparameter tuning), and scales more effectively to high dimensional problems.
Machine Learning Reveals the Seismic Signature of Eruptive Behavior at Piton de la Fournaise Volcano

Volcanic tremor is key to our understanding of active magmatic systems, but due to its complexity, there is still a debate concerning its origins and how it can be used to characterize eruptive dynamics. In this study we leverage machine learning techniques using 6 years of continuous seismic data from the Piton de la Fournaise volcano (La Réunion island) to describe specific patterns of seismic signals recorded during eruptions. These results unveil what we interpret as signals associated with various eruptive dynamics of the volcano, including the effusion of a large volume of lava during the August–October 2015 eruption as well as the closing of the eruptive vent during the September–November 2018 eruption. The machine learning workflow we describe can easily be applied to other active volcanoes, potentially leading to an enhanced understanding of the temporal and spatial evolution of volcanic eruptions.
CAQL: Continuous Action Q-Learning

Value-based reinforcement learning (RL) methods like Q-learning have shown success in a variety of domains. One challenge in applying Q-learning to continuous-action RL problems, however, is the continuous action maximization (max-Q) required for optimal Bellman backup. In this work, we develop CAQL, a (class of) algorithm(s) for continuous-action Q-learning that can use several plug-and-play optimizers for the max-Q problem. Leveraging recent optimization results for deep neural networks, we show that max-Q can be solved optimally using mixed-integer programming (MIP). When the Q-function representation has sufficient power, MIP-based optimization gives rise to better policies and is more robust than approximate methods (e.g., gradient ascent, cross-entropy search). We further develop several techniques to accelerate inference in CAQL, which despite their approximate nature, perform well. We compare CAQL with state-of-the-art RL algorithms on benchmark continuous-control problems that have different degrees of action constraints and show that CAQL outperforms policy-based methods in heavily constrained environments, often dramatically.
Optimizing Speech Recognition For The Edge

While most deployed speech recognition systems today still run on servers, we are in the midst of a transition towards deployments on edge devices. This leap to the edge is powered by the progression from traditional speech recognition pipelines to end-to-end (E2E) neural architectures, and the parallel development of more efficient neural network topologies and optimization techniques. Thus, we are now able to create highly accurate speech recognizers that are both small and fast enough to execute on typical mobile devices. In this paper, we begin with a baseline RNN-Transducer architecture comprised of Long Short-Term Memory (LSTM) layers. We then experiment with a variety of more computationally efficient layer types, as well as apply optimization techniques like neural connection pruning and parameter quantization to construct a small, high quality, on-device speech recognizer that is an order of magnitude smaller than the baseline system without any optimizations.