nils-herrmann/clean_scirepeval huggingface dataset:
High Dimensional Robust Estimation of Sparse Models via Trimmed Hard Thresholding

We study the problem of sparsity constrained M -estimation with arbitrary corruptions to both explanatory and response variables in the high-dimensional regime, where the number of variables d is larger than the sample size n. Our main contribution is a highly efficient gradient-based optimization algorithm that we call Trimmed Hard Thresholding – a robust variant of Iterative Hard Thresholding (IHT) by using trimmed mean in gradient computations. Our algorithm can deal with a wide class of sparsity constrained M -estimation problems, and we can tolerate a nearly dimension independent fraction of arbitrarily corrupted samples. More specifically, when the corrupted fraction satisfies . 1/ (√ k log(nd) ) , where k is the sparsity of the parameter, we obtain accurate estimation and model selection guarantees with optimal sample complexity. Furthermore, we extend our algorithm to sparse Gaussian graphical model (precision matrix) estimation via a neighborhood selection approach. We demonstrate the effectiveness of robust estimation in sparse linear, logistic regression, and sparse precision matrix estimation on synthetic and real-world US equities data.
Visualizing Topographic Independent Component Analysis with Movies

Independent component analysis (ICA) has often been used as a tool to model natural image statistics by separating multivariate signals in the image into components that are assumed to be independent. However, these estimated components oftentimes have higher order dependencies, such as co-activation of components, that are not accounted for in the model. Topographic independent component analysis(TICA), a modification of ICA, takes into account higher order dependencies and orders components topographically as a function of dependence. Here, we aim to visualize the time course of TICA basis activations to movie stimuli. We find that the activity of TICA bases are often clustered and move continuously, potentially resembling activity of topographically organized cells in the visual cortex.
Unsupervised Image-to-Image Translation with Self-Attention Networks

Unsupervised image translation aims to learn the transformation from a source domain to a target domain given unpaired training data. Several state-of-the-art works have yielded impressive results in the GANs-based unsupervised image-to-image translation. It fails to capture strong geometric changes between domains, or it produces unsatisfactory results for complex scenes, compared to local texture mapping tasks such as style transfer. Recently, SAGAN (Han Zhang, 2018) showed that the self-attention network produces better results than the convolution-based GAN. However, the effectiveness of the self-attention network in unsupervised image-to-image translation tasks have not been verified. In this paper, we propose an unsupervised image-to-image translation with self-attention networks, in which long range dependency helps to not only capture strong geometric change but also generate details using cues from all feature locations. In experiments, we qualitatively and quantitatively show superiority of the proposed method compared to existing state-of-the-art unsupervised image-to-image translation task.
Classical and quantum analysis of 3D electromagnetic pp-wave spacetime

The general classical solution of the 3D electromagnetic pp-wave spacetime has been obtained. The relevant line element contains an arbitrary essential function providing an infinite number of in-equivalent geometries as solutions. A classification is presented based on the symmetry group. To the best of our knowledge, the solution corresponding to only one of the Classes is known. The dynamics of some of the Classes was also derived from a minisuperspace Lagrangian which has been constructed. This Lagrangian contains a degree of freedom (the lapse) which can be considered either as dynamical or non-dynamical (indicating a singular or a regular Lagrangian correspondingly). Surprisingly enough, on the space of classical solutions, an equivalence of these two points of view can be established. The canonical quantization is then used in order to quantize the system for both the singular and regular Hamiltonian. A subsequent interpretation of quantum states is based on a Bohm-like analysis. The semi-classical trajectories deviate from the classical only for the regular Hamiltonian and in particular for a superposition of eigenstates (a Gaussian initial state has been used). Thus, the above mentioned equivalence is broken at the quantum level. It is noteworthy that the semi-classical trajectories tend to the classical ones in the limit where the initial wavepacket is widely spread. Hence, even with this simple superposition state, the classical solutions are acquired as a limit of the semi-classical.
Measurements of Three-Level Hierarchical Structure in the Outliers in the Spectrum of Deepnet Hessians

We consider deep classifying neural networks. We expose a structure in the derivative of the logits with respect to the parameters of the model, which is used to explain the existence of outliers in the spectrum of the Hessian. Previous works decomposed the Hessian into two components, attributing the outliers to one of them, the so-called Covariance of gradients. We show this term is not a Covariance but a second moment matrix, i.e., it is influenced by means of gradients. These means possess an additive two-way structure that is the source of the outliers in the spectrum. This structure can be used to approximate the principal subspace of the Hessian using certain "averaging" operations, avoiding the need for high-dimensional eigenanalysis. We corroborate this claim across different datasets, architectures and sample sizes.
Deep Learning Algorithms for Bearing Fault Diagnosticsx—A Comprehensive Review

In this survey paper, we systematically summarize existing literature on bearing fault diagnostics with deep learning (DL) algorithms. While conventional machine learning (ML) methods, including artificial neural network, principal component analysis, support vector machines, etc., have been successfully applied to the detection and categorization of bearing faults for decades, recent developments in DL algorithms in the last five years have sparked renewed interest in both industry and academia for intelligent machine health monitoring. In this paper, we first provide a brief review of conventional ML methods, before taking a deep dive into the state-of-the-art DL algorithms for bearing fault applications. Specifically, the superiority of DL based methods are analyzed in terms of fault feature extraction and classification performances; many new functionalities enabled by DL techniques are also summarized. In addition, to obtain a more intuitive insight, a comparative study is conducted on the classification accuracy of different algorithms utilizing the open source Case Western Reserve University (CWRU) bearing dataset. Finally, to facilitate the transition on applying various DL algorithms to bearing fault diagnostics, detailed recommendations and suggestions are provided for specific application conditions. Future research directions to further enhance the performance of DL algorithms on health monitoring are also discussed.
FANDA: A Novel Approach to Perform Follow-up Query Analysis

Recent work on Natural Language Interfaces to Databases (NLIDB) has attracted considerable attention. NLIDB allow users to search databases using natural language instead of SQL-like query languages. While saving the users from having to learn query languages, multi-turn interaction with NLIDB usually involves multiple queries where contextual information is vital to understand the users’ query intents. In this paper, we address a typical contextual understanding problem, termed as follow-up query analysis. In spite of its ubiquity, follow-up query analysis has not been well studied due to two primary obstacles: the multifarious nature of follow-up query scenarios and the lack of high-quality datasets. Our work summarizes typical follow-up query scenarios and provides a new FollowUp dataset with 1000 query triples on 120 tables. Moreover, we propose a novel approach FANDA, which takes into account the structures of queries and employs a ranking model with weakly supervised max-margin learning. The experimental results on FollowUp demonstrate the superiority of FANDA over multiple baselines across multiple metrics.
Perturbation of elliptic operators in 1-sided NTA domains satisfying the capacity density condition

Let $\Omega\subset\mathbb{R}^{n+1}$, $n\ge 2$, be a 1-sided non-tangentially accessible domain (aka uniform domain), that is, a set which satisfies the interior Corkscrew and Harnack chain conditions, respectively scale-invariant/quantitative versions of openness and path-connectedness. Assume that $\Omega$ satisfies the so-called capacity density condition. Let $L_0u=-{\rm div\,}(A_0\nabla u)$, $Lu=-{\rm div\,}(A\nabla u)$ be two real (non-necessarily symmetric) uniformly elliptic operators, and write $\omega_{L_0}$, $\omega_L$ for the associated elliptic measures. The goal of this paper is to find sufficient conditions guaranteeing that $\omega_L$ satisfies an $A_\infty$-condition or a $RH_q$-condition with respect to $\omega_{L_0}$. We show that if the discrepancy of the two matrices satisfies a natural Carleson measure condition with respect to $\omega_{L_0}$ then $\omega_L\in A_\infty(\omega_{L_0})$. Moreover, $\omega_L\in RH_q(\omega_{L_0})$ for any given $1<q<\infty$ if the Carleson measure condition is assumed to hold with a sufficiently small constant. This extends previous work of Fefferman-Kenig-Pipher and Milakis-Pipher-Toro who considered Lipschitz and chord-arc domains. Here we go beyond as the capacity density condition is much weaker than the existence of exterior Corkscrew balls. The "large constant" case, where the discrepancy satisfies a Carleson measure condition, is new even for nice domains such as the unit ball, the upper half-space, or Lipschitz domains. Our results do not have a nice surface measure: all the analysis is done with the underlying measure $\omega_{L_0}$. When particularized to Lipschitz, chord-arc, or 1-sided chord-arc domains, we recover previous results and extend some of them. Of independent interest is the extension of the Dahlberg-Jerison-Kenig result concerning square functions and non-tangential maximal functions to our general setting.
QGAN: Quantized Generative Adversarial Networks

The intensive computation and memory requirements of generative adversarial neural networks (GANs) hinder its real-world deployment on edge devices such as smartphones. Despite the success in model reduction of CNNs, neural network quantization methods have not yet been studied on GANs, which are mainly faced with the issues of both the effectiveness of quantization algorithms and the instability of training GAN models. In this paper, we start with an extensive study on applying existing successful methods to quantize GANs. Our observation reveals that none of them generates samples with reasonable quality because of the underrepresentation of quantized values in model weights, and the generator and discriminator networks show different sensitivities upon quantization methods. Motivated by these observations, we develop a novel quantization method for GANs based on EM algorithms, named as QGAN. We also propose a multi-precision algorithm to help find the optimal number of bits of quantized GAN models in conjunction with corresponding result qualities. Experiments on CIFAR-10 and CelebA show that QGAN can quantize GANs to even 1-bit or 2-bit representations with results of quality comparable to original models.
Th\'eorie de la r\'eduction pour les groupes p-divisibles

Starting from our work on Harder-Narasimhan filtrations of finite flat group schemes over a $p$-adic field, we developp a theory of Harder-Narasimhan filtrations for $p$-divisible groups. We apply this to the study of the geometry of period morphisms for Rapoport-Zink spaces and to the $p$-adic geometry of Shimura varieties. We define and study in particular some fundamental domains for the action of Hecke correspondences.
Federated Deep Reinforcement Learning

In deep reinforcement learning, building policies of high-quality is challenging when the feature space of states is small and the training data is limited. Despite the success of previous transfer learning approaches in deep reinforcement learning, directly transferring data or models from an agent to another agent is often not allowed due to the privacy of data and/or models in many privacy-aware applications. In this paper, we propose a novel deep reinforcement learning framework to federatively build models of high-quality for agents with consideration of their privacies, namely Federated deep Reinforcement Learning (FedRL). To protect the privacy of data and models, we exploit Gausian differentials on the information shared with each other when updating their local models. In the experiment, we evaluate our FedRL framework in two diverse domains, Grid-world and Text2Action domains, by comparing to various baselines.
Heavy-Tailed Universality Predicts Trends in Test Accuracies for Very Large Pre-Trained Deep Neural Networks

Given two or more Deep Neural Networks (DNNs) with the same or similar architectures, and trained on the same dataset, but trained with different solvers, parameters, hyper-parameters, regularization, etc., can we predict which DNN will have the best test accuracy, and can we do so without peeking at the test data? In this paper, we show how to use a new Theory of Heavy-Tailed Self-Regularization (HT-SR) to answer this. HT-SR suggests, among other things, that modern DNNs exhibit what we call Heavy-Tailed Mechanistic Universality (HT-MU), meaning that the correlations in the layer weight matrices can be fit to a power law (PL) with exponents that lie in common Universality classes from Heavy-Tailed Random Matrix Theory (HT-RMT). From this, we develop a Universal capacity control metric that is a weighted average of PL exponents. Rather than considering small toy NNs, we examine over 50 different, large-scale pre-trained DNNs, ranging over 15 different architectures, trained on ImagetNet, each of which has been reported to have different test accuracies. We show that this new capacity metric correlates very well with the reported test accuracies of these DNNs, looking across each architecture (VGG16/.../VGG19, ResNet10/.../ResNet152, etc.). We also show how to approximate the metric by the more familiar Product Norm capacity measure, as the average of the log Frobenius norm of the layer weight matrices. Our approach requires no changes to the underlying DNN or its loss function, it does not require us to train a model (although it could be used to monitor training), and it does not even require access to the ImageNet data.
SAM: a modular framework for self-adapting web menus

This paper presents SAM, a modular and extensible JavaScript framework for self-adapting menus on webpages. SAM allows control of two elementary aspects for adapting web menus: (1) the target policy, which assigns scores to menu items for adaptation, and (2) the adaptation style, which specifies how they are adapted on display. By decoupling them, SAM enables the exploration of different combinations independently. Several policies from literature are readily implemented, and paired with adaptation styles such as reordering and highlighting. The process---including user data logging---is local, offering privacy benefits and eliminating the need for server-side modifications. Researchers can use SAM to experiment adaptation policies and styles, and benchmark techniques in an ecological setting with real webpages. Practitioners can make websites self-adapting, and end-users can dynamically personalise typically static web menus.
Anomaly Detection in Road Traffic Using Visual Surveillance

Computer vision has evolved in the last decade as a key technology for numerous applications replacing human supervision. Timely detection of traffic violations and abnormal behavior of pedestrians at public places through computer vision and visual surveillance can be highly effective for maintaining traffic order in cities. However, despite a handful of computer vision–based techniques proposed in recent times to understand the traffic violations or other types of on-road anomalies, no methodological survey is available that provides a detailed insight into the classification techniques, learning methods, datasets, and application contexts. Thus, this study aims to investigate the recent visual surveillance–related research on anomaly detection in public places, particularly on road. The study analyzes various vision-guided anomaly detection techniques using a generic framework such that the key technical components can be easily understood. Our survey includes definitions of related terminologies and concepts, judicious classifications of the vision-guided anomaly detection approaches, detailed analysis of anomaly detection methods including deep learning–based methods, descriptions of the relevant datasets with environmental conditions, and types of anomalies. The study also reveals vital gaps in the available datasets and anomaly detection capability in various contexts, and thus gives future directions to the computer vision–guided anomaly detection research. As anomaly detection is an important step in automatic road traffic surveillance, this survey can be a useful resource for interested researchers working on solving various issues of Intelligent Transportation Systems (ITS).
Deep Learning on Attributed Graphs: A Journey from Graphs to Their Embeddings and Back

A graph is a powerful concept for representation of relations between pairs of entities. Data with underlying graph structure can be found across many disciplines and there is a natural desire for understanding such data better. Deep learning (DL) has achieved significant breakthroughs in a variety of machine learning tasks in recent years, especially where data is structured on a grid, such as in text, speech, or image understanding. However, surprisingly little has been done to explore the applicability of DL on arbitrary graph-structured data directly. 
The goal of this thesis is to investigate architectures for DL on graphs and study how to transfer, adapt or generalize concepts that work well on sequential and image data to this domain. We concentrate on two important primitives: embedding graphs or their nodes into a continuous vector space representation (encoding) and, conversely, generating graphs from such vectors back (decoding). To that end, we make the following contributions. 
First, we introduce Edge-Conditioned Convolutions (ECC), a convolution-like operation on graphs performed in the spatial domain where filters are dynamically generated based on edge attributes. The method is used to encode graphs with arbitrary and varying structure. 
Second, we propose SuperPoint Graph, an intermediate point cloud representation with rich edge attributes encoding the contextual relationship between object parts. Based on this representation, ECC is employed to segment large-scale point clouds without major sacrifice in fine details. 
Third, we present GraphVAE, a graph generator allowing us to decode graphs with variable but upper-bounded number of nodes making use of approximate graph matching for aligning the predictions of an autoencoder with its inputs. The method is applied to the task of molecule generation.
Reentrant phase transitions in threshold driven contagion on multiplex networks

Models of threshold driven contagion explain the cascading spread of information, behavior, systemic risk, and epidemics on social, financial, and biological networks. At odds with empirical observations, these models predict that single-layer unweighted networks become resistant to global cascades after reaching sufficient connectivity. We investigate threshold driven contagion on weight heterogeneous multiplex networks and show that they can remain susceptible to global cascades at any level of connectivity, and with increasing edge density pass through alternating phases of stability and instability in the form of reentrant phase transitions of contagion. Our results provide a theoretical explanation for the observation of large-scale contagion in highly connected but heterogeneous networks.
eV-Scale Sterile Neutrinos

We address the phenomenology of light sterile neutrinos, with an emphasis on short-baseline neutrino oscillations. After reviewing the observed short-baseline neutrino oscillation anomalies, we discuss the global fit of the data and the current appearance–disappearance tension. We also review briefly the effects of light sterile neutrinos in β decay, neutrinoless double-β decay, and cosmology. Finally, we discuss future perspectives of the search for the effects of eV-scale sterile neutrinos.
Nonlinear variational problems with lack of compactness.

In this thesis we deal with two different classes of variational problems: 1) the problem of closed curves with prescribed curvature, or $H$-loop problem; 2) the study of the nodal solutions of the fractional Brezis-Nirenberg problem. In both cases we deal with nonlinear equations (an ODE system for problem 1, and an elliptic equation for problem 2) which admit a variational structure. Nevertheless, in both cases, a lack of compactness occurs and this constitutes a strong obstruction for the application of the direct methods of the calculus of variations or even standard variational methods. Therefore, we are lead to use more refined techniques or also to follow different approaches, like the Lyapunov-Schmidt method and blow-up analysis. This will allow us not only to obtain existence and multiplicity results, but also qualitative properties of the solutions.
Deep Reasoning with Multi-scale Context for Salient Object Detection

To detect salient objects accurately, existing methods usually design complex backbone network architectures to learn and fuse powerful features. However, the saliency inference module that performs saliency prediction from the fused features receives much less attention on its architecture design and typically adopts only a few fully convolutional layers. In this paper, we find the limited capacity of the saliency inference module indeed makes a fundamental performance bottleneck, and enhancing its capacity is critical for obtaining better saliency prediction. Correspondingly, we propose a deep yet light-weight saliency inference module that adopts a multi-dilated depth-wise convolution architecture. Such a deep inference module, though with simple architecture, can directly perform reasoning about salient objects from the multi-scale convolutional features fast, and give superior salient object detection performance with less computational cost. To our best knowledge, we are the first to reveal the importance of the inference module for salient object detection, and present a novel architecture design with attractive efficiency and accuracy. Extensive experimental evaluations demonstrate that our simple framework performs favorably compared with the state-of-the-art methods with complex backbone design.