nils-herrmann/clean_scirepeval huggingface dataset:
Putting An End to End-to-End: Gradient-Isolated Learning of Representations

We propose a novel deep learning method for local self-supervised representation learning that does not require labels nor end-to-end backpropagation but exploits the natural order in data instead. Inspired by the observation that biological neural networks appear to learn without backpropagating a global error signal, we split a deep neural network into a stack of gradient-isolated modules. Each module is trained to maximally preserve the information of its inputs using the InfoNCE bound from Oord et al. [2018]. Despite this greedy training, we demonstrate that each module improves upon the output of its predecessor, and that the representations created by the top module yield highly competitive results on downstream classification tasks in the audio and visual domain. The proposal enables optimizing modules asynchronously, allowing large-scale distributed training of very deep neural networks on unlabelled datasets.
Black hole surrounded by a dark matter halo in the M87 galactic center and its identification with shadow images

In this paper we present a new black hole solution surrounded by dark matter halo in the galactic center using the mass model of M87 and that coming from the Universal Rotation Curve (URC) dark matter profile representing family of spiral galaxies. In both cases the DM halo density is cored with a size $r_0$ and a central density $\rho_0$: $\rho(r)= \rho_0/(1+r/r_0)(1+(r/r_0)^2)$. Since $r_0\rho_0=120 M_{\odot}$/pc$^2$ [Donato et al., MNRAS, 397, 1169, 2009], then by varying the central density one can reproduce the DM profile in any spiral. Using the Newman-Jains method we extend our solution to obtain a rotating black hole surrounded by dark matter halo. We find that, the apparent shape of the shadow beside the black hole spin $a$, it also depends on the central density of the surrounded dark matter $\rho_0$. As a specific example we consider the galaxy M87, with a central density $\rho_0=6.9\,\times 10^{6} M_{\odot}$/kpc$^3$ and a core radius $r_0=91.2$ kpc. In the case of M87, our analyses show that the effect of dark matter on the size of the black hole shadow is almost negligible compared to the shadow size of the Kerr vacuum solution hence the angular diameter $ 42$ $\mu$as remains almost unaltered when the dark matter is considered. For a small totally dark matter dominated spiral such as UGC 7232, we find similar effect of dark matter on the shadow images compared to the M87. However, in specific conditions having a core radius comparable to the black hole mass and dark matter with very high density, we show that the shadow images decreases compared to the Kerr vacuum black hole. The effect of dark matter on the apparent shadow shape can shed some light in future observations as an indirect way to detect dark matter using the shadow images.
Prediction of Construction Cost for Field Canals Improvement Projects in Egypt

Field canals improvement projects (FCIPs) are one of the ambitious projects constructed to save fresh water. To finance this project, Conceptual cost models are important to accurately predict preliminary costs at the early stages of the project. The first step is to develop a conceptual cost model to identify key cost drivers affecting the project. Therefore, input variables selection remains an important part of model development, as the poor variables selection can decrease model precision. The study discovered the most important drivers of FCIPs based on a qualitative approach and a quantitative approach. Subsequently, the study has developed a parametric cost model based on machine learning methods such as regression methods, artificial neural networks, fuzzy model and case-based reasoning.
An Incremental Turn-Taking Model For Task-Oriented Dialog Systems

In a human-machine dialog scenario, deciding the appropriate time for the machine to take the turn is an open research problem. In contrast, humans engaged in conversations are able to timely decide when to interrupt the speaker for competitive or non-competitive reasons. In state-of-the-art turn-by-turn dialog systems the decision on the next dialog action is taken at the end of the utterance. In this paper, we propose a token-by-token prediction of the dialog state from incremental transcriptions of the user utterance. To identify the point of maximal understanding in an ongoing utterance, we a) implement an incremental Dialog State Tracker which is updated on a token basis (iDST) b) re-label the Dialog State Tracking Challenge 2 (DSTC2) dataset and c) adapt it to the incremental turn-taking experimental scenario. The re-labeling consists of assigning a binary value to each token in the user utterance that allows to identify the appropriate point for taking the turn. Finally, we implement an incremental Turn Taking Decider (iTTD) that is trained on these new labels for the turn-taking decision. We show that the proposed model can achieve a better performance compared to a deterministic handcrafted turn-taking algorithm.
Artificial Consciousness and Security

This paper describes a possible way to improve computer security by implementing a program which implements the following three features related to a weak notion of artificial consciousness: (partial) self-monitoring, ability to compute the truth of quantifier-free propositions and the ability to communicate with the user. The integrity of the program could be enhanced by using a trusted computing approach, that is to say a hardware module that is at the root of a chain of trust. This paper outlines a possible approach but does not refer to an implementation (which would need further work), but the author believes that an implementation using current processors, a debugger, a monitoring program and a trusted processing module is currently possible.
Shredder: Learning Noise to Protect Privacy with Partial DNN Inference on the Edge

A wide variety of DNN applications increasingly rely on the cloud to perform their huge computation. This heavy trend toward cloud-hosted inference services raises serious privacy concerns. This model requires the sending of private and privileged data over the network to remote servers, exposing it to the service provider. Even if the provider is trusted, the data can still be vulnerable over communication channels or via side-channel attacks  at the provider. To that end, this paper aims to reduce the information content of the communicated data without compromising the cloud service's ability to provide a DNN inference with acceptably high accuracy. This paper presents an end-to-end framework, called Shredder, that, without altering the topology or the weights of a pre-trained network, learns an additive noise distribution that significantly reduces the information content of communicated data while maintaining the inference accuracy. Shredder learns the additive noise by casting it as a tensor of trainable parameters enabling us to devise a loss functions that strikes a balance between accuracy and information degradation. The loss function exposes a knob for a disciplined and controlled asymmetric trade-off between privacy and accuracy. While keeping the DNN intact, Shredder enables inference on noisy data without the need to update the model or the cloud. Experimentation with real-world DNNs shows that Shredder reduces the mutual information between the input and the communicated data to the cloud by 70.2% compared to the original execution while only sacrificing 1.46% loss in accuracy.
A Parameterized Perspective on Protecting Elections

We study the parameterized complexity of the optimal defense and optimal attack problems in voting. In both the problems, the input is a set of voter groups (every voter group is a set of votes) and two integers k_a and k_d corresponding to respectively the number of voter groups the attacker can attack and the number of voter groups the defender can defend. A voter group gets removed from the election if it is attacked but not defended. In the optimal defense problem, we want to know if it is possible for the defender to commit to a strategy of defending at most k_d voter groups such that, no matter which k_a voter groups the attacker attacks, the out-come of the election does not change. In the optimal attack problem, we want to know if it is possible for the attacker to commit to a strategy of attacking k_a voter groups such that, no matter which k_d voter groups the defender defends, the outcome of the election is always different from the original (without any attack) one. We show that both the optimal defense problem and the optimal attack problem are computationally intractable for every scoring rule and the Condorcet voting rule even when we have only3candidates. We also show that the optimal defense problem for every scoring rule and the Condorcet voting rule is W[2]-hard for both the parameters k_a and k_d, while it admits a fixed parameter tractable algorithm parameterized by the combined parameter (ka, kd). The optimal attack problem for every scoring rule and the Condorcet voting rule turns out to be much harder – it is W[1]-hard even for the combined parameter (ka, kd). We propose two greedy algorithms for the OPTIMAL DEFENSE problem and empirically show that they perform effectively on reasonable voting profiles.
On Dropout and Nuclear Norm Regularization

We give a formal and complete characterization of the explicit regularizer induced by dropout in deep linear networks with squared loss. We show that (a) the explicit regularizer is composed of an $\ell_2$-path regularizer and other terms that are also re-scaling invariant, (b) the convex envelope of the induced regularizer is the squared nuclear norm of the network map, and (c) for a sufficiently large dropout rate, we characterize the global optima of the dropout objective. We validate our theoretical findings with empirical results.
BREIZHCROPS: A TIME SERIES DATASET FOR CROP TYPE MAPPING

We present BreizhCrops, a novel benchmark dataset for the supervised classification of field crops from satellite time series. We aggregated label data and Sentinel-2 top-of-atmosphere as well as bottom-of-atmosphere time series in the region of Brittany (Breizh in local language), north-east France. We compare seven recently proposed deep neural networks along with a Random Forest baseline. The dataset, model (re-)implementations and pre-trained model weights are available at the associated GitHub repository (https://github.com/dl4sits/breizhcrops) that has been designed with applicability for practitioners in mind. We plan to maintain the repository with additional data and welcome contributions of novel methods to build a state-of-the-art benchmark on methods for crop type mapping.
Monetary Stabilization in Cryptocurrencies – Design Approaches and Open Questions

The price volatility of cryptocurrencies is often cited as a major hindrance to their wide-scale adoption. Consequently, during the last two years, multiple so called stablecoins have surfaced – cryptocurrencies focused on maintaining stable exchange rates. In this paper, we systematically explore and analyze the stablecoin landscape. Based on a survey of 24 specific stablecoin projects, we go beyond individual coins for extracting general concepts and approaches. We combine our findings with learnings from classical monetary policy, resulting in a comprehensive taxonomy of cryptocurrency stabilization. We use our taxonomy to highlight the current state of development from different perspectives and show blank spots. For instance, while over 91% of projects promote 1-to-1 stabilization targets to external assets, monetary policy literature suggests that the smoothing of short term volatility is often a more sustainable alternative. Our taxonomy bridges computer science and economics, fostering the transfer of expertise. For example, we find that 38% of the reviewed projects use a combination of exchange rate targeting and specific stabilization techniques that can render them vulnerable to speculative economic attacks – an avoidable design flaw.
Dusty disk winds at the sublimation rim of the highly inclined, low mass young stellar object SU Aurigae

T Tauri stars are low-mass young stars whose disks provide the setting for planet formation. Despite this, their structure is poorly understood. We present new infrared interferometric observations of the SU Aurigae circumstellar environment that offer resolution that is three times higher and a better baseline position angle coverage than previous observations.
Aims. We aim to investigate the characteristics of the circumstellar material around SU Aur, constrain the disk geometry, composition and inner dust rim structure.
Methods. The CHARA array offers unique opportunities for long baseline observations, with baselines up to 331 m. Using the CLIMB three-telescope combiner in the K-band allows us to measure visibilities as well as closure phase. We undertook image reconstruction for model-independent analysis, and fitted geometric models such as Gaussian and ring distributions. Additionally, the fitting of radiative transfer models constrain the physical parameters of the disk. For the first time, a dusty disk wind is introduced to the radiative transfer code TORUS to model protoplanetary disks. Our implementation is motivated by theoretical models of dusty disk winds, where magnetic field lines drive dust above the disk plane close to the sublimation zone.
Results. Image reconstruction reveals an inclined disk with slight asymmetry along its minor-axis, likely due to inclination effects obscuring the inner disk rim through absorption of incident star light on the near-side and thermal re-emission and scattering of the far-side. Geometric modelling of a skewed ring finds the inner rim at 0.17 ± 0.02 au with an inclination of 50.9 ± 1.0° and minor axis position angle 60.8 ± 1.2°. Radiative transfer modelling shows a flared disk with an inner radius at 0.18 au which implies a grain size of 0.4 μm assuming astronomical silicates and a scale height of 15.0 at 100 au. Among the tested radiative transfer models, only the dusty disk wind successfully accounts for the K-band excess by introducing dust above the mid-plane.
A Cross-Domain Transferable Neural Coherence Model

Coherence is an important aspect of text quality and is crucial for ensuring its readability. One important limitation of existing coherence models is that training on one domain does not easily generalize to unseen categories of text. Previous work advocates for generative models for cross-domain generalization, because for discriminative models, the space of incoherent sentence orderings to discriminate against during training is prohibitively large. In this work, we propose a local discriminative neural model with a much smaller negative sampling space that can efficiently learn against incorrect orderings. The proposed coherence model is simple in structure, yet it significantly outperforms previous state-of-art methods on a standard benchmark dataset on the Wall Street Journal corpus, as well as in multiple new challenging settings of transfer to unseen categories of discourse on Wikipedia articles.
Network Deconvolution

Convolution is a central operation in Convolutional Neural Networks (CNNs), which applies a kernel to overlapping regions shifted across the image. However, because of the strong correlations in real-world image data, convolutional kernels are in effect re-learning redundant data. In this work, we show that this redundancy has made neural network training challenging, and propose network deconvolution, a procedure which optimally removes pixel-wise and channel-wise correlations before the data is fed into each layer. Network deconvolution can be efficiently calculated at a fraction of the computational cost of a convolution layer. We also show that the deconvolution filters in the first layer of the network resemble the center-surround structure found in biological neurons in the visual regions of the brain. Filtering with such kernels results in a sparse representation, a desired property that has been missing in the training of neural networks. Learning from the sparse representation promotes faster convergence and superior results without the use of batch normalization. We apply our network deconvolution operation to 10 modern neural network models by replacing batch normalization within each. Extensive experiments show that the network deconvolution operation is able to deliver performance improvement in all cases on the CIFAR-10, CIFAR-100, MNIST, Fashion-MNIST, Cityscapes, and ImageNet datasets.
SignalTrain: Profiling Audio Compressors with Deep Neural Networks

In this work we present a data-driven approach for predicting the behavior of (i.e., profiling) a given non-linear audio signal processing effect (henceforth "audio effect"). Our objective is to learn a mapping function that maps the unprocessed audio to the processed by the audio effect to be profiled, using time-domain samples. To that aim, we employ a deep auto-encoder model that is conditioned on both time-domain samples and the control parameters of the target audio effect. As a test-case study, we focus on the offline profiling of two dynamic range compression audio effects, one software-based and the other analog. Compressors were chosen because they are a widely used and important set of effects and because their parameterized nonlinear time-dependent nature makes them a challenging problem for a system aiming to profile "general" audio effects. Results from our experimental procedure show that the primary functional and auditory characteristics of the compressors can be captured, however there is still sufficient audible noise to merit further investigation before such methods are applied to real-world audio processing workflows.
Cerberus: A Multi-headed Derenderer

To generalize to novel visual scenes with new viewpoints and new object poses, a visual system needs representations of the shapes of the parts of an object that are invariant to changes in viewpoint or pose. 3D graphics representations disentangle visual factors such as viewpoints and lighting from object structure in a natural way. It is possible to learn to invert the process that converts 3D graphics representations into 2D images, provided the 3D graphics representations are available as labels. When only the unlabeled images are available, however, learning to derender is much harder. We consider a simple model which is just a set of free floating parts. Each part has its own relation to the camera and its own triangular mesh which can be deformed to model the shape of the part. At test time, a neural network looks at a single image and extracts the shapes of the parts and their relations to the camera. Each part can be viewed as one head of a multi-headed derenderer. During training, the extracted parts are used as input to a differentiable 3D renderer and the reconstruction error is backpropagated to train the neural net. We make the learning task easier by encouraging the deformations of the part meshes to be invariant to changes in viewpoint and invariant to the changes in the relative positions of the parts that occur when the pose of an articulated body changes. Cerberus, our multi-headed derenderer, outperforms previous methods for extracting 3D parts from single images without part annotations, and it does quite well at extracting natural parts of human figures.
EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks

Convolutional Neural Networks (ConvNets) are commonly developed at a fixed resource budget, and then scaled up for better accuracy if more resources are available. In this paper, we systematically study model scaling and identify that carefully balancing network depth, width, and resolution can lead to better performance. Based on this observation, we propose a new scaling method that uniformly scales all dimensions of depth/width/resolution using a simple yet highly effective compound coefficient. We demonstrate the effectiveness of this method on scaling up MobileNets and ResNet. 
To go even further, we use neural architecture search to design a new baseline network and scale it up to obtain a family of models, called EfficientNets, which achieve much better accuracy and efficiency than previous ConvNets. In particular, our EfficientNet-B7 achieves state-of-the-art 84.4% top-1 / 97.1% top-5 accuracy on ImageNet, while being 8.4x smaller and 6.1x faster on inference than the best existing ConvNet. Our EfficientNets also transfer well and achieve state-of-the-art accuracy on CIFAR-100 (91.7%), Flowers (98.8%), and 3 other transfer learning datasets, with an order of magnitude fewer parameters. Source code is at this https URL.
Unsupervised Learning From Video With Deep Neural Embeddings

Because of the rich dynamical structure of videos andtheir ubiquity in everyday life, it is a natural idea that video data could serve as a powerful unsupervised learning signal for visual representations. However, instantiating this idea, especially at large scale, has remained a significant artificial intelligence challenge. Here we present the Video Instance Embedding (VIE) framework, which trains deep nonlinear embeddings on video sequence inputs. By learning embedding dimensions that identify and group similar videos together, while pushing inherently different videos apart in the embedding space, VIE captures the strong statistical structure inherent in videos, without the need for external annotation labels. We find that, when trained on a large-scale video dataset, VIE yields powerful representations both for action recognition and single-frame object categorization, showing substantially improving on the state of the art wherever direct comparisons are possible. We show that a two-pathway model with both static and dynamic processingpathways is optimal, provide analyses indicating how the model works, and perform ablation studies showing the importance of key architecture and loss function choices. Our results suggest that deep neural embeddings are a promising approach to unsupervised video learning fora wide variety of task domains.
Chasing Convex Bodies Optimally

In the chasing convex bodies problem, an online player receives a request sequence of $N$ convex sets $K_1,\dots, K_N$ contained in a normed space $\mathbb R^d$. The player starts at $x_0\in \mathbb R^d$, and after observing each $K_n$ picks a new point $x_n\in K_n$. At each step the player pays a movement cost of $||x_n-x_{n-1}||$. The player aims to maintain a constant competitive ratio against the minimum cost possible in hindsight, i.e. knowing all requests in advance. The existence of a finite competitive ratio for convex body chasing was first conjectured in 1991 by Friedman and Linial. This conjecture was recently resolved with an exponential $2^{O(d)}$ upper bound on the competitive ratio. 
In this paper, we drastically improve the exponential upper bound. We give an algorithm achieving competitive ratio $d$ for arbitrary normed spaces, which is exactly tight for $\ell^{\infty}$. In Euclidean space, our algorithm achieves nearly optimal competitive ratio $O(\sqrt{d\log N})$, compared to a lower bound of $\sqrt{d}$. Our approach extends another recent work which chases nested convex bodies using the classical Steiner point of a convex body. We define the functional Steiner point of a convex function and apply it to the work function to obtain our algorithm.