nils-herrmann/clean_scirepeval huggingface dataset:
The Benefits of Over-parameterization at Initialization in Deep ReLU Networks

It has been noted in existing literature that over-parameterization in ReLU networks generally improves performance. While there could be several factors involved behind this, we prove some desirable theoretical properties at initialization which may be enjoyed by ReLU networks. Specifically, it is known that He initialization in deep ReLU networks asymptotically preserves variance of activations in the forward pass and variance of gradients in the backward pass for infinitely wide networks, thus preserving the flow of information in both directions. Our paper goes beyond these results and shows novel properties that hold under He initialization: i) the norm of hidden activation of each layer is equal to the norm of the input, and, ii) the norm of weight gradient of each layer is equal to the product of norm of the input vector and the error at output layer. These results are derived using the PAC analysis framework, and hold true for finitely sized datasets such that the width of the ReLU network only needs to be larger than a certain finite lower bound. As we show, this lower bound depends on the depth of the network and the number of samples, and by the virtue of being a lower bound, over-parameterized ReLU networks are endowed with these desirable properties. For the aforementioned hidden activation norm property under He initialization, we further extend our theory and show that this property holds for a finite width network even when the number of data samples is infinite. Thus we overcome several limitations of existing papers, and show new properties of deep ReLU networks at initialization.
Dependency Weighted Aggregation on Factorized Databases

We study a new class of aggregation problems, called dependency weighted aggregation. The underlying idea is to aggregate the answer tuples of a query while accounting for dependencies between them, where two tuples are considered dependent when they have the same value on some attribute. The main problem we are interested in is to compute the dependency weighted count of a conjunctive query. This aggregate can be seen as a form of weighted counting, where the weights of the answer tuples are computed by solving a linear program. This linear program enforces that dependent tuples are not over represented in the final weighted count. The dependency weighted count can be used to compute the s-measure, a measure that is used in data mining to estimate the frequency of a pattern in a graph database. Computing the dependency weighted count of a conjunctive query is NP-hard in general. In this paper, we show that this problem is actually tractable for a large class of structurally restricted conjunctive queries such as acyclic or bounded hypertree width queries. Our algorithm works on a factorized representation of the answer set, in order to avoid enumerating it exhaustively. Our technique produces a succinct representation of the weighting of the answers. It can be used to solve other dependency weighted aggregation tasks, such as computing the (dependency) weighted average of the value of an attribute in the answers set.
Variation Network: Learning High-level Attributes for Controlled Input Manipulation

This paper presents the Variation Network (VarNet), a generative model providing means to manipulate the high-level attributes of a given input. The originality of our approach is that VarNet is not only capable of handling pre-defined attributes but can also learn the relevant attributes of the dataset by itself. These two settings can be easily combined which makes VarNet applicable for a wide variety of tasks. Further, VarNet has a sound probabilistic interpretation which grants us with a novel way to navigate in the latent spaces as well as means to control how the attributes are learned. We demonstrate experimentally that this model is capable of performing interesting input manipulation and that the learned attributes are relevant and interpretable.
A General Optimization-based Framework for Local Odometry Estimation with Multiple Sensors

Accurate state estimation is a fundamental problem for autonomous robots. To achieve locally accurate and globally drift-free state estimation, multiple sensors with complementary properties are usually fused together. Local sensors (camera, IMU, LiDAR, etc) provide precise pose within a small region, while global sensors (GPS, magnetometer, barometer, etc) supply noisy but globally drift-free localization in a large-scale environment. In this paper, we propose a sensor fusion framework to fuse local states with global sensors, which achieves locally accurate and globally drift-free pose estimation. Local estimations, produced by existing VO/VIO approaches, are fused with global sensors in a pose graph optimization. Within the graph optimization, local estimations are aligned into a global coordinate. Meanwhile, the accumulated drifts are eliminated. We evaluate the performance of our system on public datasets and with real-world experiments. Results are compared against other state-of-the-art algorithms. We highlight that our system is a general framework, which can easily fuse various global sensors in a unified pose graph optimization. Our implementations are open source\footnote{this https URL}.
Short Steps in Noncommutative Geometry

Noncommutative geometry (NCG) is a branch of mathematics concerned with a geometric approach to noncommutative algebras, and with the construction of spaces that are locally presented by noncommutative algebras of functions (possibly in some generalized sense). A noncommutative algebra is an associative algebra in which the multiplication is not commutative, that is, for which $xy$ does not always equal $yx$; or more generally an algebraic structure in which one of the principal binary operations is not commutative; one also allows additional structures, e.g. topology or norm, to be possibly carried by the noncommutative algebra of functions. These notes just to start understand what we need to study Noncommutative Geometry.
A General Optimization-based Framework for Global Pose Estimation with Multiple Sensors

Accurate state estimation is a fundamental problem for autonomous robots. To achieve locally accurate and globally drift-free state estimation, multiple sensors with complementary properties are usually fused together. Local sensors (camera, IMU, LiDAR, etc) provide precise pose within a small region, while global sensors (GPS, magnetometer, barometer, etc) supply noisy but globally drift-free localization in a largescale environment. In this paper, we propose a sensor fusion framework to fuse local states with global sensors, which achieves locally accurate and globally drift-free pose estimation. Local estimations, produced by existing VO/VIO approaches, are fused with global sensors in a pose graph optimization. Within the graph optimization, local estimations are aligned into a global coordinate. Meanwhile, the accumulated drifts are eliminated. We evaluate the performance of our system on public datasets and with real-world experiments. Results are compared against other state-of-the-art algorithms. We highlight that our system is a general framework, which can easily fuse various global sensors in a unified pose graph optimization. Our implementations are open source.
ParaBank: Monolingual Bitext Generation and Sentential Paraphrasing via Lexically-constrained Neural Machine Translation

We present PARABANK, a large-scale English paraphrase dataset that surpasses prior work in both quantity and quality. Following the approach of PARANMT (Wieting and Gimpel, 2018), we train a Czech-English neural machine translation (NMT) system to generate novel paraphrases of English reference sentences. By adding lexical constraints to the NMT decoding procedure, however, we are able to produce multiple high-quality sentential paraphrases per source sentence, yielding an English paraphrase resource with more than 4 billion generated tokens and exhibiting greater lexical diversity. Using human judgments, we also demonstrate that PARABANK’s paraphrases improve over PARANMT on both semantic similarity and fluency. Finally, we use PARABANK to train a monolingual NMT model with the same support for lexically-constrained decoding for sentence rewriting tasks.
An extended model of wine fermentation including aromas and acids

The art of viticulture and the quest for making wines has a long tradition and it just started recently that mathematicians entered this field with their main contribution of modelling alcoholic fermentation. These models consist of systems of ordinary differential equations that describe the kinetics of the bio-chemical reactions occurring in the fermentation process. The aim of this paper is to present a new model of wine fermentation that accurately describes the yeast dying component, the presence of glucose transporters, and the formation of aromas and acids. Therefore the new model could become a valuable tool to predict the taste of the wine and provide the starting point for an emerging control technology that aims at improving the quality of the wine by steering a well-behaved fermentation process that is also energetically more efficient. Results of numerical simulations are presented that successfully confirm the validity of the proposed model by comparison with real data.
Weightless Neural Network with Transfer Learning to Detect Distress in Asphalt

The present paper shows a solution to the problem of automatic distress detection, more precisely the detection of holes in paved roads. To do so, the proposed solution uses a weightless neural network known as Wisard to decide whether an image of a road has any kind of cracks. In addition, the proposed architecture also shows how the use of transfer learning was able to improve the overall accuracy of the decision system. As a verification step of the research, an experiment was carried out using images from the streets at the Federal University of Tocantins, Brazil. The architecture of the developed solution presents a result of 85.71% accuracy in the dataset, proving to be superior to approaches of the state-of-the-art.
An Optical Frontend for a Convolutional Neural Network

The parallelism of optics and the miniaturization of optical components using nanophotonic structures, such as metasurfaces, present a compelling alternative to electronic implementations of convolutional neural networks. The lack of a low-power optical nonlinearity, however, requires slow and energy-inefficient conversions between the electronic and optical domains. Here, we design an architecture that utilizes a single electrical to optical conversion by designing a free-space optical frontend unit that implements the linear operations of the first layer with the subsequent layers realized electronically. Speed and power analysis of the architecture indicates that the hybrid photonic-electronic architecture outperforms a fully electronic architecture for large image sizes and kernels. Benchmarking of the photonic-electronic architecture on a modified version of AlexNet achieves high classification accuracies on images from the Kaggle's Cats and Dogs challenge and MNIST databases.
Individual Common Dolphin Identification Via Metric Embedding Learning

Photo-identification (photo-id) of dolphin individuals is a commonly used technique in ecological sciences to monitor state and health of individuals, as well as to study the social structure and distribution of a population. Traditional photo-id involves a laborious manual process of matching each dolphin fin photograph captured in the field to a catalogue of known individuals. We examine this problem in the context of open-set recognition and utilise a triplet loss function to learn a compact representation of fin images in a Euclidean embedding, where the Euclidean distance metric represents fin similarity. We show that this compact representation can be successfully learnt from a fairly small (in deep learning context) training set and still generalise well to out-of-sample identities (completely new dolphin individuals), with top-1 and top-5 test set (37 individuals) accuracy of 90.5 ± 2 and 93.6 ± 1 percent. In the presence of 1200 distractors, top-1 accuracy dropped by 12%; however, top-5 accuracy saw only a 2.8% drop.
On the Global Convergence of Imitation Learning: A Case for Linear Quadratic Regulator

We study the global convergence of generative adversarial imitation learning for linear quadratic regulators, which is posed as minimax optimization. To address the challenges arising from non-convex-concave geometry, we analyze the alternating gradient algorithm and establish its Q-linear rate of convergence to a unique saddle point, which simultaneously recovers the globally optimal policy and reward function. We hope our results may serve as a small step towards understanding and taming the instability in imitation learning as well as in more general non-convex-concave alternating minimax optimization that arises from reinforcement learning and generative adversarial learning.
Estimating influenza incidence using search query deceptiveness and generalized ridge regression

Seasonal influenza is a sometimes surprisingly impactful disease, causing thousands of deaths per year along with much additional morbidity. Timely knowledge of the outbreak state is valuable for managing an effective response. The current state of the art is to gather this knowledge using in-person patient contact. While accurate, this is time-consuming and expensive. This has motivated inquiry into new approaches using internet activity traces, based on the theory that lay observations of health status lead to informative features in internet data. These approaches risk being deceived by activity traces having a coincidental, rather than informative, relationship to disease incidence; to our knowledge, this risk has not yet been quantitatively explored. We evaluated both simulated and real activity traces of varying deceptiveness for influenza incidence estimation using linear regression. We found that deceptiveness knowledge does reduce error in such estimates, that it may help automatically-selected features perform as well or better than features that require human curation, and that a semantic distance measure derived from the Wikipedia article category tree serves as a useful proxy for deceptiveness. This suggests that disease incidence estimation models should incorporate not only data about how internet features map to incidence but also additional data to estimate feature deceptiveness. By doing so, we may gain one more step along the path to accurate, reliable disease incidence estimation using internet data. This capability would improve public health by decreasing the cost and increasing the timeliness of such estimates.
Machine Learning Automation Toolbox (MLaut)

In this paper we present MLaut (Machine Learning AUtomation Toolbox) for the python data science ecosystem. MLaut automates large-scale evaluation and benchmarking of machine learning algorithms on a large number of datasets. MLaut provides a high-level workflow interface to machine algorithm algorithms, implements a local back-end to a database of dataset collections, trained algorithms, and experimental results, and provides easy-to-use interfaces to the scikit-learn and keras modelling libraries. Experiments are easy to set up with default settings in a few lines of code, while remaining fully customizable to the level of hyper-parameter tuning, pipeline composition, or deep learning architecture. 
As a principal test case for MLaut, we conducted a large-scale supervised classification study in order to benchmark the performance of a number of machine learning algorithms - to our knowledge also the first larger-scale study on standard supervised learning data sets to include deep learning algorithms. While corroborating a number of previous findings in literature, we found (within the limitations of our study) that deep neural networks do not perform well on basic supervised learning, i.e., outside the more specialized, image-, audio-, or text-based tasks.
Multi-Level Batch Normalization in Deep Networks for Invasive Ductal Carcinoma Cell Discrimination in Histopathology Images

Breast cancer is the most diagnosed cancer and the most predominant cause of death in women worldwide. Imaging techniques such as breast cancer pathology helps in the diagnosis and monitoring of the disease. However identification of malignant cells can be challenging given the high heterogeneity in tissue absorption from staining agents. In this work, we present a novel approach for Invasive Ductal Carcinoma (IDC) cells discrimination in histopathology slides. We propose a model derived from the Inception architecture, proposing a multi-level batch normalization module between each convolutional steps. This module was used as a base block for feature extraction in a CNN architecture. We used the open IDC dataset in which we obtained a balanced accuracy of 0.89 and an F1 score of 0.90, thus surpassing recent state of the art classification algorithms tested on this public dataset.
Benchmarking Substellar Evolutionary Models Using New Age Estimates for HD 4747 B and HD 19467 B

Constraining substellar evolutionary models (SSEMs) is particularly difficult due to a degeneracy between the mass, age, and luminosity of a brown dwarf. In cases where a brown dwarf is found as a directly imaged companion to a star, as in HD 4747 and HD 19467, the mass, age, and luminosity of the brown dwarf are determined independently, making them ideal objects to use to benchmark SSEMs. Using the Center for High Angular Resolution Astronomy Array, we measured the angular diameters and calculated the radii of the host stars HD 4747 A and HD 19467 A. After fitting their parameters to the Dartmouth Stellar Evolution Database, MESA Isochrones and Stellar Tracks, and Yonsei-Yale isochronal models, we adopt age estimates of Gyr for HD 4747 A and Gyr for HD 19467 A. Assuming the brown dwarf companions HD 4747 B and HD 19467 B have the same ages as their host stars, we show that many of the SSEMs under-predict bolometric luminosities by ∼0.75 dex for HD 4747 B and ∼0.5 dex for HD 19467 B. The discrepancies in luminosity correspond to over-predictions of the masses by ∼12% for HD 4747 B and ∼30% for HD 19467 B. We also show that SSEMs that take into account the effect of clouds reduce the under-prediction of luminosity to ∼0.6 dex and the over-prediction of mass to ∼8% for HD 4747 B, an L/T transition object that is cool enough to begin forming clouds. One possible explanation for the remaining discrepancies is missing physics in the models, such as the inclusion of metallicity effects.
Econophysics: still fringe after 30 years?

It is customary to date the initial foray of physicists into economics to the September 1987 Santa Fe conference on “Economics as a complex evolving system", organised by two famous physicists (Phil Anderson & David Pines) and one famous economist (Ken Arrow). When confronted with the theories of Rational Expectations and Efficient Markets, then at the peak of their glory, Phil Anderson famously quipped: Do you guys really believe that? Many inspiring and insightful ideas were discussed during that conference , suggesting a potentially productive cross-fertilisation between economics and the (back then) nascent theory of complex systems.
Report for the Commission of Inquiry Respecting the Muskrat Falls Project

This report was commissioned by the Commission of Inquiry Respecting the Muskrat Falls Project to provide the national and international context in which the Muskrat Falls Project took place. The Commission asked for the report to cover three specific topics of questions: (1) What is the national and international context of the Muskrat Falls Project with regards to cost overrun and schedule overrun? (What are the typical cost and schedule overruns of hydro-electric dam projects? How do hydro-electric dams compare to other capital investment projects? How do Canadian projects compare to other countries?), (2) What are the causes and root causes of cost and schedule overruns? (3) What are recommendations, based on international experience and research into capital investment projects, to prevent cost and schedule overruns in hydro-electric dam projects and other capital investment projects?  Hydroelectric Dams, Megaprojects, Cost Overrun, Schedule Overrun, Optimism Bias, Strategic Misrepresentation, Infrastructure, Capital Investment Projects, Canada, Muskrat Falls
Neumann Networks for Inverse Problems in Imaging

Many challenging image processing tasks can be described by an ill-posed linear inverse problem: deblurring, deconvolution, inpainting, compressed sensing, and superresolution all lie in this framework. Traditional inverse problem solvers minimize a cost function consisting of a data-fit term, which measures how well an image matches the observations, and a regularizer, which reflects prior knowledge and promotes images with desirable properties like smoothness. Recent advances in machine learning and image processing have illustrated that it is often possible to learn a regularizer from training data that can outperform more traditional regularizers. We present an end-to-end, data-driven method of solving inverse problems inspired by the Neumann series, which we call a Neumann network. Rather than unroll an iterative optimization algorithm, we truncate a Neumann series which directly solves the linear inverse problem with a data-driven nonlinear regularizer. The Neumann network architecture outperforms traditional inverse problem solution methods, model-free deep learning approaches, and state-of-the-art unrolled iterative methods on standard datasets. Finally, when the images belong to a union of subspaces and under appropriate assumptions on the forward model, we prove there exists a Neumann network configuration that well-approximates the optimal oracle estimator for the inverse problem and demonstrate empirically that the trained Neumann network has the form predicted by theory.
Compact Galaxies at intermediate redshifts quench faster than normal-sized Galaxies

Massive quiescent compact galaxies have been discovered at high redshifts, associated with rapid compaction and cessation of star formation (SF). In this work we set out to quantify the time-scales in which SF is quenched in compact galaxies at intermediate redshifts. For this, we select a sample of green valley galaxies within the COSMOS field in the midst of quenching their SF at $0.5<z<1.0$ that exhibit varying degrees of compactness. Based on the H$\delta$ absorption line and the 4000 \AA \ break of coadded zCOSMOS spectra for sub-samples of normal-sized and compact galaxies we determine quenching time-scales as a function of compactness. We find that the SF quenching time-scales in green valley compact galaxies are much shorter than in normal-sized ones. In an effort to understand this trend, we use the Illustris simulation to trace the evolution of the SF history, the growth rate of the central super massive black hole (SMBH) {\bf and the AGN-feedback in compact and normal-sized galaxies. We find that the key difference between their SF quenching time-scales is linked to the mode of the AGN-feedback. In the compact galaxies predominates the kinematic-mode, which is highly efficient at quenching the SF by depleting the internal gas. On the normal-sized galaxies, the prevailing thermal-mode injects energy in the circumgalactic gas, impeding the cold gas supply and quenching the SF via the slower strangulation mechanism.} These results are consistent with the violent disk instability and gas-rich mergers scenarios, followed by strong AGN and stellar feedback. Although this kind of event is most expected to occur at $z=2-3$, we find evidences that the formation of compact quiescent galaxies can occur at $z<1$.