nils-herrmann/clean_scirepeval huggingface dataset:
Dependent Mat\'ern Processes for Multivariate Time Series

For the challenging task of modeling multivariate time series, we propose a new class of models that use dependent Matérn processes to capture the underlying structure of data, explain their interdependencies, and predict their unknown values. Although similar models have been proposed in the econometric, statistics, and machine learning literature, our approach has several advantages that distinguish it from existing methods: 1) it is flexible to provide high prediction accuracy, yet its complexity is controlled to avoid overfitting; 2) its interpretability separates it from black-box methods; 3) finally, its computational efficiency makes it scalable for high-dimensional time series. In this paper, we use several simulated and real data sets to illustrate these advantages. We will also briefly discuss some extensions of our model.
Stochastic and Adversarial Combinatorial Bandits

This paper investigates stochastic and adversarial combinatorial multi-armed bandit problems. In the stochastic setting, we first derive problem-specific regret lower bounds, and analyze how these bounds scale with the dimension of the decision space. We then propose COMBUCB, algorithms that efficiently exploit the combinatorial structure of the problem, and derive finite-time upper bound on their regrets. These bounds improve over regret upper bounds of existing algorithms, and we show numerically thatCOMBUCB significantly outperforms any other algorithm. In the adversarial setting, we propose two simple algorithms, namely COMBEXP-1 and COMBEXP-2 for semi-bandit and bandit feedback, respectively. Their regrets have similar scaling as state-of-the-art algorithms, in spite of the simplicity of their implementation
Gradient-based Hyperparameter Optimization through Reversible Learning

Tuning hyperparameters of learning algorithms is hard because gradients are usually unavailable. We compute exact gradients of cross-validation performance with respect to all hyperparameters by chaining derivatives backwards through the entire training procedure. These gradients allow us to optimize thousands of hyperparameters, including step-size and momentum schedules, weight initialization distributions, richly parameterized regularization schemes, and neural network architectures. We compute hyperparameter gradients by exactly reversing the dynamics of stochastic gradient descent with momentum.
Novel variational approach for analysis of photonic crystal slabs

We propose a new method, based on variational principle, for the analysis of photonic crystal (PC) slabs. Most of the methods used today treat PC slabs as three-dimensional (3D) crystal, and this makes these methods very time and/or memory consuming. In our proposed method, we use the Bloch theorem to expand the field on infinite plane waves, whose amplitudes depend on the component perpendicular to the slab surface. By approximating these amplitudes with appropriate functions, we can find modes of PC slabs almost as fast as we can find modes of two-dimensional crystals. In addition to this advantage, we can also calculate radiation modes with this method, which is not feasible with the 3D plane wave expansion method.
Spectral Sparsification of Random-Walk Matrix Polynomials

We consider a fundamental algorithmic question in spectral graph theory: Compute a spectral sparsifier of random-walk matrix-polynomial $$L_\alpha(G)=D-\sum_{r=1}^d\alpha_rD(D^{-1}A)^r$$ where $A$ is the adjacency matrix of a weighted, undirected graph, $D$ is the diagonal matrix of weighted degrees, and $\alpha=(\alpha_1...\alpha_d)$ are nonnegative coefficients with $\sum_{r=1}^d\alpha_r=1$. Recall that $D^{-1}A$ is the transition matrix of random walks on the graph. The sparsification of $L_\alpha(G)$ appears to be algorithmically challenging as the matrix power $(D^{-1}A)^r$ is defined by all paths of length $r$, whose precise calculation would be prohibitively expensive. 
In this paper, we develop the first nearly linear time algorithm for this sparsification problem: For any $G$ with $n$ vertices and $m$ edges, $d$ coefficients $\alpha$, and $\epsilon > 0$, our algorithm runs in time $O(d^2m\log^2n/\epsilon^{2})$ to construct a Laplacian matrix $\tilde{L}=D-\tilde{A}$ with $O(n\log n/\epsilon^{2})$ non-zeros such that $\tilde{L}\approx_{\epsilon}L_\alpha(G)$. 
Matrix polynomials arise in mathematical analysis of matrix functions as well as numerical solutions of matrix equations. Our work is particularly motivated by the algorithmic problems for speeding up the classic Newton's method in applications such as computing the inverse square-root of the precision matrix of a Gaussian random field, as well as computing the $q$th-root transition (for $q\geq1$) in a time-reversible Markov model. The key algorithmic step for both applications is the construction of a spectral sparsifier of a constant degree random-walk matrix-polynomials introduced by Newton's method. Our algorithm can also be used to build efficient data structures for effective resistances for multi-step time-reversible Markov models, and we anticipate that it could be useful for other tasks in network analysis.
Single Digit Representations of Natural Numbers

In this work, we established symmetric representation of numbers where one can use any of 9 digits giving the same number. The representations of natural numbers from 0 to 1000 are given using only single digit in all the nine cases, i.e., 1, 2, 3, 4, 5, 6, 7, 8 and 9. This is done only using basic operations: addition, subtraction, multiplication, potentiation and division.
MADE: Masked Autoencoder for Distribution Estimation

There has been a lot of recent interest in designing neural network models to estimate a distribution from a set of examples. We introduce a simple modification for autoencoder neural networks that yields powerful generative models. Our method masks the autoencoder's parameters to respect autoregressive constraints: each input is reconstructed only from previous inputs in a given ordering. Constrained this way, the autoencoder outputs can be interpreted as a set of conditional probabilities, and their product, the full joint probability. We can also train a single network that can decompose the joint probability in multiple different orderings. Our simple framework can be applied to multiple architectures, including deep ones. Vectorized implementations, such as on GPUs, are simple and fast. Experiments demonstrate that this approach is competitive with state-of-the-art tractable distribution estimators. At test time, the method is significantly faster and scales better than other autoregressive estimators.
Random Walks on Context Spaces: Towards an Explanation of the Mysteries of Semantic Word Embeddings

The papers of Mikolov et al. 2013 as well as subsequent works have led to dramatic progress in solving word analogy tasks using semantic word embeddings. This leverages linear structure that is often found in the word embeddings, which is surprising since the training method is usually nonlinear. There were attempts ---notably by Levy and Goldberg and Pennington et al.--- to explain how this linear structure arises. The current paper points out the gaps in these explanations and provides a more complete explanation using a loglinear generative model for the corpus that directly models the latent semantic structure in words. The novel methodological twist is that instead of trying to fit the best model parameters to the data, a rigorous mathematical analysis is performed using the model priors to arrive at a simple closed form expression that approximately relates co-occurrence statistics and word embeddings. This expression closely corresponds to ---and a bit simpler than--- the existing training methods, and leads to good solutions to analogy tasks. Empirical support is provided also for the validity of the modeling assumptions. 
This methodology of letting some mathematical analysis substitute for some of the computational difficulty may be useful in other settings with generative models.
Speeding up Permutation Testing in Neuroimaging

Multiple hypothesis testing is a significant problem in nearly all neuroimaging studies. In order to correct for this phenomena, we require a reliable estimate of the Family-Wise Error Rate (FWER). The well known Bonferroni correction method, while simple to implement, is quite conservative, and can substantially under-power a study because it ignores dependencies between test statistics. Permutation testing, on the other hand, is an exact, non-parametric method of estimating the FWER for a given α-threshold, but for acceptably low thresholds the computational burden can be prohibitive. In this paper, we show that permutation testing in fact amounts to populating the columns of a very large matrix P. By analyzing the spectrum of this matrix, under certain conditions, we see that P has a low-rank plus a low-variance residual decomposition which makes it suitable for highly sub-sampled - on the order of 0.5% - matrix completion methods. Based on this observation, we propose a novel permutation testing methodology which offers a large speedup, without sacrificing the fidelity of the estimated FWER. Our evaluations on four different neuroimaging datasets show that a computational speedup factor of roughly 50× can be achieved while recovering the FWER distribution up to very high accuracy. Further, we show that the estimated α-threshold is also recovered faithfully, and is stable.
Quantum teleportation between moving detectors

We consider the quantum teleportation of continuous variables modeled by Unruh-DeWitt detectors coupled to a common quantum field initially in the Minkowski vacuum. An unknown coherent state of an Unruh-DeWitt detector is teleported from one inertial agent (Alice) to an almost uniformly accelerated agent (Rob, for relativistic motion), using a detector pair initially entangled and shared by these two agents. The averaged physical fidelity of quantum teleportation, which is independent of the observer's frame, always drops below the best fidelity value from classical teleportation before the detector pair becomes disentangled with the measure of entanglement evaluated around the future lightcone of the joint measurement event by Alice. The distortion of the quantum state of the entangled detector pair from the initial state can suppress the fidelity significantly even when the detectors are still strongly entangled around the lightcone. We point out that the dynamics of entanglement of the detector pair observed in Minkowski frame or in quasi-Rindler frame are not directly related to the physical fidelity of quantum teleportation in our setup. These results are useful as a guide to making judicious choices of states and parameter ranges and estimation of the efficiency of quantum teleportation in relativistic quantum systems under environmental influences.
Weighted SGD for $\ell_p$ Regression with Randomized Preconditioning

In recent years, stochastic gradient descent (SGD) methods and randomized linear algebra (RLA) algorithms have been applied to many large-scale problems in machine learning and data analysis. We aim to bridge the gap between these two methods in solving constrained overdetermined linear regression problems---e.g., $\ell_2$ and $\ell_1$ regression problems. We propose a hybrid algorithm named pwSGD that uses RLA techniques for preconditioning and constructing an importance sampling distribution, and then performs an SGD-like iterative process with weighted sampling on the preconditioned system. We prove that pwSGD inherits faster convergence rates that only depend on the lower dimension of the linear system, while maintaining low computation complexity. Particularly, when solving $\ell_1$ regression with size $n$ by $d$, pwSGD returns an approximate solution with $\epsilon$ relative error in the objective value in $\mathcal{O}(\log n \cdot \text{nnz}(A) + \text{poly}(d)/\epsilon^2)$ time. This complexity is uniformly better than that of RLA methods in terms of both $\epsilon$ and $d$ when the problem is unconstrained. For $\ell_2$ regression, pwSGD returns an approximate solution with $\epsilon$ relative error in the objective value and the solution vector measured in prediction norm in $\mathcal{O}(\log n \cdot \text{nnz}(A) + \text{poly}(d) \log(1/\epsilon) /\epsilon)$ time. We also provide lower bounds on the coreset complexity for more general regression problems, indicating that still new ideas will be needed to extend similar RLA preconditioning ideas to weighted SGD algorithms for more general regression problems. Finally, the effectiveness of such algorithms is illustrated numerically on both synthetic and real datasets.
Ordering-Sensitive and Semantic-Aware Topic Modeling

Topic modeling of textual corpora is an important and challenging problem. In most previous work, the “bag-of-words” assumption is usually made which ignores the ordering of words. This assumption simplifies the computation, but it unrealistically loses the ordering information and the semantic of words in the context. In this paper, we present a Gaussian Mixture Neural Topic Model (GMNTM) which incorporates both the ordering of words and the semantic meaning of sentences into topic modeling. Specifically, we represent each topic as a cluster of multi-dimensional vectors and embed the corpus into a collection of vectors generated by the Gaussian mixture model. Each word is affected not only by its topic, but also by the embedding vector of its surrounding words and the context. The Gaussian mixture components and the topic of documents, sentences and words can be learnt jointly. Extensive experiments show that our model can learn better topics and more accurate word distributions for each topic. Quantitatively, comparing to state-of-the-art topic modeling approaches, GMNTM obtains significantly better performance in terms of perplexity, retrieval accuracy and classification accuracy.
Optics in a nonlinear gravitational plane wave

Gravitational waves can act like gravitational lenses, affecting the observed positions, brightnesses, and redshifts of distant objects. Exact expressions for such effects are derived here in general relativity, allowing for arbitrarily-moving sources and observers in the presence of plane-symmetric gravitational waves. At least for freely falling sources and observers, it is shown that the commonly-used predictions of linear perturbation theory can be generically overshadowed by nonlinear effects; even for very weak gravitational waves, higher-order perturbative corrections involve secularly-growing terms which cannot necessarily be neglected when considering observations of sufficiently distant sources. Even on more moderate scales where linear effects remain at least marginally dominant, nonlinear corrections are qualitatively different from their linear counterparts. There is a sense in which they can, for example, mimic the existence of a third type of gravitational wave polarization.
Computing Rational Decisions In Extensive Games With Limited Foresight

We introduce a class of extensive form games whereplayers might not be able to foresee the possible consequences of their decisions and form a model of theiropponents which they exploit to achieve a more profitable outcome. We improve upon existing models ofgames with limited foresight, endowing players with theability of higher order reasoning and proposing a novelsolution concept to address intuitions coming from realgame play. We analyse the resulting equilibria, devisingan effective procedure to compute them.
Correcting Illumina sequencing errors for human data

We present a new tool to correct sequencing errors in Illumina data produced from high-coverage whole-genome shotgun resequencing. It uses a non-greedy algorithm and shows comparable performance and higher accuracy in an evaluation on real human data. This evaluation has the most complete collection of high-performance error correctors so far. 
Availability and implementation: this https URL 
Contact: hengli@broadinstitute.org
Gravitational lensing by spinning black holes in astrophysics, and in the movie Interstellar

Interstellar is the first Hollywood movie to attempt depicting a black hole as it would actually be seen by somebody nearby. For this, our team at Double Negative Visual Effects, in collaboration with physicist Kip Thorne, developed a code called Double Negative Gravitational Renderer (DNGR) to solve the equations for ray-bundle (light-beam) propagation through the curved spacetime of a spinning (Kerr) black hole, and to render IMAX-quality, rapidly changing images. Our ray-bundle techniques were crucial for achieving IMAX-quality smoothness without flickering; and they differ from physicists’ image-generation techniques (which generally rely on individual light rays rather than ray bundles), and also differ from techniques previously used in the film industry’s CGI community. This paper has four purposes: (i) to describe DNGR for physicists and CGI practitioners, who may find interesting and useful some of our unconventional techniques. (ii) To present the equations we use, when the camera is in arbitrary motion at an arbitrary location near a Kerr black hole, for mapping light sources to camera images via elliptical ray bundles. (iii) To describe new insights, from DNGR, into gravitational lensing when the camera is near the spinning black hole, rather than far away as in almost all prior studies; we focus on the shapes, sizes and influence of caustics and critical curves, the creation and annihilation of stellar images, the pattern of multiple images, and the influence of almost-trapped light rays, and we find similar results to the more familiar case of a camera far from the hole. (iv) To describe how the images of the black hole Gargantua and its accretion disk, in the movie Interstellar, were generated with DNGR—including, especially, the influences of (a) colour changes due to doppler and gravitational frequency shifts, (b) intensity changes due to the frequency shifts, (c) simulated camera lens flare, and (d) decisions that the film makers made about these influences and about the Gargantua’s spin, with the goal of producing images understandable for a mass audience. There are no new astrophysical insights in this accretion-disk section of the paper, but disk novices may find it pedagogically interesting, and movie buffs may find its discussions of Interstellar interesting.
Visualizing Interstellar's Wormhole

Christopher Nolan's science fiction movie Interstellar offers a variety of opportunities for students in elementary courses on general relativity theory. This paper describes such opportunities, including: (i) At the motivational level, the manner in which elementary relativity concepts underlie the wormhole visualizations seen in the movie; (ii) At the briefest computational level, instructive calculations with simple but intriguing wormhole metrics, including, e.g., constructing embedding diagrams for the three-parameter wormhole that was used by our visual effects team and Christopher Nolan in scoping out possible wormhole geometries for the movie; (iii) Combining the proper reference frame of a camera with solutions of the geodesic equation, to construct a light-ray-tracing map backward in time from a camera's local sky to a wormhole's two celestial spheres; (iv) Implementing this map, for example,  (v) With the student's implementation, exploring how the wormhole's three parameters influence what the camera sees—which is precisely how Christopher Nolan, using our implementation, chose the parameters for Interstellar's wormhole; (vi) Using the student's implementation, exploring the wormhole's Einstein ring and particularly the peculiar motions of star images near the ring, and exploring what it looks like to travel through a wormhole.
Is language evolution grinding to a halt?: Exploring the life and death of words in English fiction

The Google Books corpus, derived from millions of books in a range of major languages, would seem to offer many possibilities for research into cultural, social, and linguistic evolution. In a previous work, we found that the 2009 and 2012 versions of the unfiltered English data set as well as the 2009 version of the English Fiction data set are all heavily saturated with scientific and medical literature, rendering them unsuitable for rigorous analysis. By contrast, the 2012 version of English Fiction appeared to be uncompromised, and we use this data set to explore language dynamics for English from 1820--2000. We critique a previous method for measuring birth and death rates of words, and provide a robust, principled to examining the volume of word flux across various relative frequency usage thresholds. We use the contributions to the Jensen-Shannon divergence of words crossing thresholds between consecutive decades to illuminate the major driving factors behind the flux. We find that while individual word usage may vary greatly, the overall statistical structure of the language appears to remain fairly stable. We also find indications that scholarly works about fiction are strongly represented in the 2012 English Fiction corpus, and suggest that a future revision of the corpus should attempt to separate critical works from fiction itself.
Dynamic Move Chains and Move Tables in Computer Chess

The idea of dynamic move chains has been described in a preceding paper . It allows the search tree to be forward-pruned, which is known to be dangerous, because it can remove important moves that would only be evaluated through a more exhaustive search process. This paper has added to the forward-pruning technique, through the use of 'Move Tables' that can act in the same way as Transposition Tables, but for moves not positions. They use an efficient memory structure and have put the design into the context of short or long-term memories. The forward-pruning technique can also be fortified, to help to remove some of the error. For example, a minimum beam width can guarantee x dynamic moves are searched, making the result more reliable. Or a minimum window for the evaluation difference can determine if a full search is subsequently required. Therefore, with some configuration, dynamic move chains can be reliably used and relatively independently of the position. This has advanced some of the future work theory of the earlier paper as well and made more explicit where logical plans or more knowledge-based approaches might be applied.
Socializing the Semantic Gap

Where previous reviews on content-based image retrieval emphasize what can be seen in an image to bridge the semantic gap, this survey considers what people tag about an image. A comprehensive treatise of three closely linked problems (i.e., image tag assignment, refinement, and tag-based image retrieval) is presented. While existing works vary in terms of their targeted tasks and methodology, they rely on the key functionality of tag relevance, that is, estimating the relevance of a specific tag with respect to the visual content of a given image and its social context. By analyzing what information a specific method exploits to construct its tag relevance function and how such information is exploited, this article introduces a two-dimensional taxonomy to structure the growing literature, understand the ingredients of the main works, clarify their connections and difference, and recognize their merits and limitations. For a head-to-head comparison with the state of the art, a new experimental protocol is presented, with training sets containing 10,000, 100,000, and 1 million images, and an evaluation on three test sets, contributed by various research groups. Eleven representative works are implemented and evaluated. Putting all this together, the survey aims to provide an overview of the past and foster progress for the near future.