nils-herrmann/clean_scirepeval huggingface dataset:
Sequential Neural Processes

Neural Processes combine the strengths of neural networks and Gaussian processes to achieve both flexible learning and fast prediction in stochastic processes. However, a large class of problems comprises underlying temporal dependency structures in a sequence of stochastic processes that Neural Processes (NP) do not explicitly consider. In this paper, we propose Sequential Neural Processes (SNP) which incorporates a temporal state-transition model of stochastic processes and thus extends its modeling capabilities to dynamic stochastic processes. In applying SNP to dynamic 3D scene modeling, we introduce the Temporal Generative Query Networks. To our knowledge, this is the first 4D model that can deal with the temporal dynamics of 3D scenes. In experiments, we evaluate the proposed methods in dynamic (non-stationary) regression and 4D scene inference and rendering.
Efficient Multi-Domain Network Learning by Covariance Normalization

The problem of multi-domain learning of deep networks is considered. An adaptive layer is induced per target domain and a novel procedure, denoted covariance normalization (CovNorm), proposed to reduce its parameters. CovNorm is a data driven method of fairly simple implementation, requiring two principal component analyzes (PCA) and fine-tuning of a mini-adaptation layer. Nevertheless, it is shown, both theoretically and experimentally, to have several advantages over previous approaches, such as batch normalization or geometric matrix approximations. Furthermore, CovNorm can be deployed both when target datasets are available sequentially or simultaneously. Experiments show that, in both cases, it has performance comparable to a fully fine-tuned network, using as few as 0.13% of the corresponding parameters per target domain.
Serif or Sans: Visual Font Analytics on Book Covers and Online Advertisements

In this paper, we conduct a large-scale study of font statistics in book covers and online advertisements. Through the statistical study, we try to understand how graphic designers relate fonts and content genres and identify the relationship between font styles, colors, and genres. We propose an automatic approach to extract font information from graphic designs by applying a sequence of character detection, style classification, and clustering techniques to the graphic designs. The extracted font information is accumulated together with genre information, such as romance or business, for further trend analysis. Through our unique empirical study, we show that the collected font statistics reveal interesting trends in terms of how typographic design represents the impression and the atmosphere of the content genres.
Neural Proximal/Trust Region Policy Optimization Attains Globally Optimal Policy

Proximal policy optimization and trust region policy optimization (PPO and TRPO) with actor and critic parametrized by neural networks achieve significant empirical success in deep reinforcement learning. However, due to nonconvexity, the global convergence of PPO and TRPO remains less understood, which separates theory from practice. In this paper, we prove that a variant of PPO and TRPO equipped with overparametrized neural networks converges to the globally optimal policy at a sublinear rate. The key to our analysis is the global convergence of infinite-dimensional mirror descent under a notion of one-point monotonicity, where the gradient and iterate are instantiated by neural networks. In particular, the desirable representation power and optimization geometry induced by the overparametrization of such neural networks allow them to accurately approximate the infinite-dimensional gradient and iterate.
Against the Norm: Modeling Daily Stock Returns with the Laplace Distribution

Modeling stock returns is not a new task for mathematicians, investors, and portfolio managers, but it remains a difficult objective due to the ebb and flow of stock markets. One common solution is to approximate the distribution of stock returns with a normal distribution. However, normal distributions place infinitesimal probabilities on extreme outliers, but these outliers are of particular importance in the practice of investing. In this paper, we investigate the normality of the distribution of daily returns of major stock market indices. We find that the normal distribution is not a good model for stock returns, even over several years' worth of data. Moreover, we propose using the Laplace distribution as a model for daily stock returns.
Exploring Self-Supervised Regularization for Supervised and Semi-Supervised Learning

Recent advances in semi-supervised learning have shown tremendous potential in overcoming a major barrier to the success of modern machine learning algorithms: access to vast amounts of human-labeled training data. Previous algorithms based on consistency regularization can harness the abundance of unlabeled data to produce impressive results on a number of semi-supervised benchmarks, approaching the performance of strong supervised baselines using only a fraction of the available labeled data. In this work, we challenge the long-standing success of consistency regularization by introducing self-supervised regularization as the basis for combining semantic feature representations from unlabeled data. We perform extensive comparative experiments to demonstrate the effectiveness of self-supervised regularization for supervised and semi-supervised image classification on SVHN, CIFAR-10, and CIFAR-100 benchmark datasets. We present two main results: (1) models augmented with self-supervised regularization significantly improve upon traditional supervised classifiers without the need for unlabeled data; (2) together with unlabeled data, our models yield semi-supervised performance competitive with, and in many cases exceeding, prior state-of-the-art consistency baselines. Lastly, our models have the practical utility of being efficiently trained end-to-end and require no additional hyper-parameters to tune for optimal performance beyond the standard set for training neural networks. Reference code and data are available at this https URL
Dynamic time series clustering via volatility change-points

This note outlines a method for clustering time series based on a statistical model in which volatility shifts at unobserved change-points. The model accommodates some classical stylized features of returns and its relation to GARCH is discussed. Clustering is performed using a probability metric evaluated between posterior distributions of the most recent change-point associated with each series. This implies series are grouped together at a given time if there is evidence the most recent shifts in their respective volatilities were coincident or closely timed. The clustering method is dynamic, in that groupings may be updated in an online manner as data arrive. Numerical results are given analyzing daily returns of constituents of the S&P 500.
On the accuracy of message-passing approaches to percolation in complex networks

The Message-Passing Approach (MPA) is the state-of-the-art technique to obtain quasi-analytical predictions for percolation on real complex networks. Besides being intuitive and straightforward, it has the advantage of being mathematically principled: it is exact on trees, while yielding generally good predictions on networks containing cycles as do most real complex networks. Here we show that the MPA does not perform its calculations on some ill-defined tree-like approximation of the network, as its formulation leads to believe, but rather considers a random network ensemble in which the original network is cloned and shuffled an infinite number of times. We conclude that the fact that the MPA is exact on trees does not imply that it is nearly exact on tree-like networks. In fact we find that the closer a non-tree network is to a tree, the worse the MPA accuracy becomes.
Learning Features with Differentiable Closed-Form Solver for Tracking

We present a novel and easy-to-implement training framework for visual tracking. Our approach mainly focuses on learning feature embeddings in an end-to-end way, which can generalize well to the trackers based on online discriminatively trained ridge regression model. This goal is efficiently achieved by taking advantage of the following two important theories. 1) Ridge regression problem has closedform solution and is implicit differentiation under the optimality condition. Therefore, its solver can be embedded as a layer with efficient forward and backward processes in training deep convolutional neural networks. 2) Woodbury identity can be utilized to ensure efficient solution of ridge regression problem when the high-dimensional feature embeddings are employed. Moreover, in order to address the extreme foreground-background class imbalance during training, we modify the origin shrinkage loss and then employ it as the loss function for efficient and effective training. It is worth mentioning that the above core parts of our proposed training framework are easy to be implemented with several lines of code under the current popular deep learning frameworks, thus our approach is easy to be followed. Extensive experiments on six public benchmarks, OTB2015, NFS, TrackingNet, GOT10k, VOT2018, and VOT2019, show that the proposed tracker achieves state-of-the-art performance, while running at over 30 FPS. Code will be made available.
Learning Causal State Representations of Partially Observable Environments

Intelligent agents can cope with sensory-rich environments by learning task-agnostic state abstractions. In this paper, we propose mechanisms to approximate causal states, which optimally compress the joint history of actions and observations in partially-observable Markov decision processes. Our proposed algorithm extracts causal state representations from RNNs that are trained to predict subsequent observations given the history. We demonstrate that these learned task-agnostic state abstractions can be used to efficiently learn policies for reinforcement learning problems with rich observation spaces. We evaluate agents using multiple partially observable navigation tasks with both discrete (GridWorld) and continuous (VizDoom, ALE) observation processes that cannot be solved by traditional memory-limited methods. Our experiments demonstrate systematic improvement of the DQN and tabular models using approximate causal state representations with respect to recurrent-DQN baselines trained with raw inputs.
From IP ID to Device ID and KASLR Bypass (Extended Version)

IP headers include a 16-bit ID field. Our work examines the generation of this field in Windows (versions 8 and higher), Linux and Android, and shows that the IP ID field enables remote servers to assign a unique ID to each device and thus be able to identify subsequent transmissions sent from that device. This identification works across all browsers and over network changes. In modern Linux and Android versions, this field leaks a kernel address, thus we also break KASLR. 
Our work includes reverse-engineering of the Windows IP ID generation code, and a cryptanalysis of this code and of the Linux kernel IP ID generation code. It provides practical techniques to partially extract the key used by each of these algorithms, overcoming different implementation issues, and observing that this key can identify individual devices. We deployed a demo (for Windows) showing that key extraction and machine fingerprinting works in the wild, and tested it from networks around the world.
Beneficial perturbation network for continual learning

Sequential learning of multiple tasks in artificial neural networks using gradient descent leads to catastrophic forgetting, whereby previously learned knowledge is erased during learning of new, disjoint knowledge. Here, we propose a fundamentally new type of method - Beneficial Perturbation Network (BPN). We add task-dependent memory (biasing) units to allow the network to operate in different regimes for different tasks. We compute the most beneficial directions to train these units, in a manner inspired by recent work on adversarial examples. At test time, beneficial perturbations for a given task bias the network toward that task to overcome catastrophic forgetting. BPN is not only more parameter-efficient than network expansion methods, but also does not need to store any data from previous tasks, in contrast with episodic memory methods. Experiments on variants of the MNIST, CIFAR-10, CIFAR-100 datasets demonstrate strong performance of BPN when compared to the state-of-the-art.
An AGI with Time-Inconsistent Preferences

This paper reveals a trap for artificial general intelligence (AGI) theorists who use economists' standard method of discounting. This trap is implicitly and falsely assuming that a rational AGI would have time-consistent preferences. An agent with time-inconsistent preferences knows that its future self will disagree with its current self concerning intertemporal decision making. Such an agent cannot automatically trust its future self to carry out plans that its current self considers optimal.
A World Unto Itself: Human Communication as Active Inference

Recent theoretical work in developmental psychology suggests that humans are predisposed to align their mental states with those of other individuals. One way this manifests is in cooperative communication; that is, intentional communication aimed at aligning individuals’ mental states with respect to events in their shared environment. This idea has received strong empirical support. The purpose of this paper is to extend this account by proposing an integrative model of the biobehavioral dynamics of cooperative communication. Our formulation is based on active inference. Active inference suggests that action-perception cycles operate to minimize uncertainty and optimize an individual’s internal model of the world. We propose that humans are characterized by an evolved adaptive prior belief that their mental states are aligned with, or similar to, those of conspecifics (i.e., that ‘we are the same sort of creature, inhabiting the same sort of niche’). The use of cooperative communication emerges as the principal means to gather evidence for this belief, allowing for the development of a shared narrative that is used to disambiguate interactants’ (hidden and inferred) mental states. Thus, by using cooperative communication, individuals effectively attune to a hermeneutic niche composed, in part, of others’ mental states; and, reciprocally, attune the niche to their own ends via epistemic niche construction. This means that niche construction enables features of the niche to encode precise, reliable cues about the deontic or shared value of certain action policies (e.g., the utility of using communicative constructions to disambiguate mental states, given expectations about shared prior beliefs). In turn, the alignment of mental states (prior beliefs) enables the emergence of a novel, contextualizing scale of cultural dynamics that encompasses the actions and mental states of the ensemble of interactants and their shared environment. The dynamics of this contextualizing layer of cultural organization feedback, across scales, to constrain the variability of the prior expectations of the individuals who constitute it. Our theory additionally builds upon the active inference literature by introducing a new set of neurobiologically plausible computational hypotheses for cooperative communication. We conclude with directions for future research.
Minimal Length Effect on Thermodynamics and Weak Cosmic Censorship Conjecture in Anti-de Sitter Black Holes via Charged Particle Absorption

In this paper, we investigate minimal length effects on the thermodynamics and weak cosmic censorship conjecture in a RN-AdS black bole via charged particle absorption. We first use the generalized uncertainty principle (GUP) to investigate the minimal length effect on the Hamilton-Jacobi equation. After the deformed Hamilton-Jacobi equation is derived, we use it to study the variations of the thermodynamic quantities of a RN-Ads black hole via absorbing a charged practice. Furthermore, we check the second law of thermodynamics and the weak cosmic censorship conjecture in two phase spaces. In the normal phase space, the second law of thermodynamics and the weak cosmic censorship conjecture are satisfied in the usual and GUP deformed cases, and the minimal length effect makes the increase of entropy faster than the usual case. After the charge particle absorption, the extremal RN-AdS black hole becomes non-extremal. In the extended phase space, the black hole entropy can either increase or decrease. When $T>2Pr_{+}$, the second law is satisfied. When $T<2Pr_{+}$, the second law of thermodynamics is violated for the extremal or near-extremal black hole. Finally, we find that the weak cosmic censorship conjecture is legal for extremal and near-extremal RN-Ads black holes in the GUP deformed case.
Gauge theory and twins paradox of disentangled representations

Achieving disentangled representations of information is one of the key goals of deep network based machine learning system. Recently there are more discussions on this issue. In this paper, by comparing the geometric structure of disentangled representation and the geometry of the evolution of mixed states in quantum mechanics, we give a fibre bundle based geometric picture of disentangled representation which can be regarded as a kind of gauge theory. From this perspective we can build a connection between the disentangled representations and the twins paradox in relativity. This can help to clarify some problems about disentangled representation.
Integer Representations and Trajectories of the 3x+1 Problem

This paper studies certain trajectories of the Collatz function. I show that if for each odd number $n$, $n\sim 3n+2$ then every positive integer $n \in \mathbb{N}\setminus 2^{\mathbb{N}}$ has the representation $$n=\left(2^{a_{k+1}}-\sum_{i=0}^{k}{2^{a_i}3^{k-i}}\right)/ 3^{k+1}$$ where $0\le a_0 \le a_1 \le \cdot \cdot \cdot \le a_{k+1}$. As a consequence, in order to prove Collatz Conjecture I illustrate that it is sufficient to prove $n\sim 3n+2$ for any odd $n\in \mathbb{N}\setminus 2^{\mathbb{N}} $. This is the main result of the paper.
A Self-supervised Approach to Hierarchical Forecasting with Applications to Groupwise Synthetic Controls

When forecasting time series with a hierarchical structure, the existing state of the art is to forecast each time series independently, and, in a post-treatment step, to reconcile the time series in a way that respects the hierarchy (Hyndman et al., 2011; Wickramasuriya et al., 2018). We propose a new loss function that can be incorporated into any maximum likelihood objective with hierarchical data, resulting in reconciled estimates with confidence intervals that correctly account for additional uncertainty due to imperfect reconciliation. We evaluate our method using a non-linear model and synthetic data on a counterfactual forecasting problem, where we have access to the ground truth and contemporaneous covariates, and show that we largely improve over the existing state-of-the-art method.
Newswire versus Social Media for Disaster Response and Recovery

In a disaster situation, first responders need to quickly acquire situational awareness and prioritize response based on the need, resources available and impact. Can they do this based on digital media such as Twitter alone, or newswire alone, or some combination of the two? We examine this question in the context of the 2015 Nepal earthquakes. We propose a method to link tweets and newswire, so that we can compare their key characteristics: timeliness, whether tweets appear earlier than their corresponding newswires, and content. Whenever possible, we present both quantitative and qualitative evaluations. One of our main findings is that tweets and newswire articles provide complementary perspectives that form a holistic view of the disaster situation. Another finding is that during the Nepal earthquake, significant information of earthquake-related news appeared on Twitter before newswire. Our results also show that 9.08% of the tweets published before their matched newswire contain actionable information regarding the Nepal earthquake.
On Capital Allocation under Information Constraints

Attempts to allocate capital across a selection of different investments are often hampered by the fact that investors' decisions are made under limited information (no historical return data) and during an extremely limited timeframe. Nevertheless, in some cases, rational investors with a certain level of experience are able to ordinally rank investment alternatives through relative assessments of the probabilities that investments will be successful. However, to apply traditional portfolio optimization models, analysts must use historical (or simulated/expected) return data as the basis for their calculations. This paper develops an alternative portfolio optimization framework that is able to handle this kind of information (given by an ordinal ranking of investment alternatives) and to calculate an optimal capital allocation based on a Cobb-Douglas function, which we call the Sorted Weighted Portfolio (SWP). Considering risk-neutral investors, we show that the results of this portfolio optimization model usually outperform the output generated by the (intuitive) Equally Weighted Portfolio (EWP) of different investment alternatives, which is the result of optimization when one is unable to incorporate additional data (the ordinal ranking of the alternatives). To further extend this work, we show that our model can also address risk-averse investors to capture correlation effects.
Circulant networks of identical Kuramoto oscillators: Seeking dense networks that do not globally synchronize and sparse ones that do

There is a critical connectivity $0<\mu_c<1$ for systems of identical Kuramoto oscillators. Any network of size $n$ in which each oscillator interacts with at least $\mu(n-1)$ others is globally synchronizing if $\mu\geq\mu_c$; otherwise, it may not be. The best known bounds on $\mu_c$ are $15/22 \approx 0.6818 \leq \mu_c\leq (3-\sqrt{2})/2 \approx 0.7929$. Focusing on circulant networks leads us to conjecture that $\mu_c = 0.75$. At the sparse end of the connectivity spectrum, we find that a ring of $n$ oscillators can be turned into a globally synchronizing network by adding just $\mathcal{O}(n \log_2 n)$ edges.
Monte Carlo Gradient Estimation in Machine Learning

This paper is a broad and accessible survey of the methods we have at our disposal for Monte Carlo gradient estimation in machine learning and across the statistical sciences: the problem of computing the gradient of an expectation of a function with respect to parameters defining the distribution that is integrated; the problem of sensitivity analysis. In machine learning research, this gradient problem lies at the core of many learning problems, in supervised, unsupervised and reinforcement learning. We will generally seek to rewrite such gradients in a form that allows for Monte Carlo estimation, allowing them to be easily and efficiently used and analysed. We explore three strategies--the pathwise, score function, and measure-valued gradient estimators--exploring their historical developments, derivation, and underlying assumptions. We describe their use in other fields, show how they are related and can be combined, and expand on their possible generalisations. Wherever Monte Carlo gradient estimators have been derived and deployed in the past, important advances have followed. A deeper and more widely-held understanding of this problem will lead to further advances, and it is these advances that we wish to support.