nils-herrmann/clean_scirepeval huggingface dataset:
Extraction of Efficient Programs in $I\Sigma_1$-arithmetic.

Clausal Language (CL) is a declarative programming and verifying system used in our teaching of computer science. CL is an implementation of, what we call, $\mathit{PR}{+}I\Sigma_1$ paradigm (primitive recursive functions with $I\Sigma_1$-arithmetic). This paper introduces an extension of $I\Sigma_1$-proofs called extraction proofs where one can extract from the proofs of $\Pi_2$-specifications primitive recursive programs as efficient as the hand-coded ones. This is achieved by having the programming constructs correspond exactly to the proof rules with the computational content.
On the Concavity of Expected Shortfall

It is well known that Expected Shortfall (also called Average Value-at-Risk) is a convex risk measure, i. e. Expected Shortfall of a convex linear combination of arbitrary risk positions is not greater than a convex linear combination with the same weights of Expected Shortfalls of the same risk positions. In this short paper we prove that Expected Shortfall is a concave risk measure with respect to probability distributions, i. e. Expected Shortfall of a finite mixture of arbitrary risk positions is not lower than the linear combination of Expected Shortfalls of the same risk positions (with the same weights as in the mixture).
SlowMo: Improving Communication-Efficient Distributed SGD with Slow Momentum

Distributed optimization is essential for training large models on large datasets. Multiple approaches have been proposed to reduce the communication overhead in distributed training, such as synchronizing only after performing multiple local SGD steps, and decentralized methods (e.g., using gossip algorithms) to decouple communications among workers. Although these methods run faster than AllReduce-based methods, which use blocking communication before every update, the resulting models may be less accurate after the same number of updates. Inspired by the BMUF method of Chen & Huo (2016), we propose a slow momentum (SlowMo) framework, where workers periodically synchronize and perform a momentum update, after multiple iterations of a base optimization algorithm. Experiments on image classification and machine translation tasks demonstrate that SlowMo consistently yields improvements in optimization and generalization performance relative to the base optimizer, even when the additional overhead is amortized over many updates so that the SlowMo runtime is on par with that of the base optimizer. We provide theoretical convergence guarantees showing that SlowMo converges to a stationary point of smooth non-convex losses. Since BMUF can be expressed through the SlowMo framework, our results also correspond to the first theoretical convergence guarantees for BMUF.
Scaling law for the impact of mutant contagion

Contagion, broadly construed, refers to anything that can spread infectiously from peer to peer. Examples include communicable diseases, rumors, misinformation, ideas, innovations, bank failures, and electrical blackouts. Sometimes, as in the 1918 Spanish flu epidemic, a contagion mutates as it propagates. Here, using a simple mathematical model, we quantify the downstream impact of a contagion that mutates exactly once as it travels. Assuming that this mutation occurs at a random node in the contact network, we calculate the distribution of the number of "descendants," $d$, downstream from the initial "Patient Zero" mutant. We find that the tail of the distribution decays as $d^{-2}$ for complete graphs, random graphs, small-world networks and other infinite-dimensional networks. This prediction agrees with the observed statistics of memes propagating and mutating on Facebook, and is expected to hold universally for other effectively infinite-dimensional networks, such as the global human contact network. In a wider context, our approach suggests a possible starting point for a mesoscopic theory of contagion. Such a theory would focus on the paths traced by a spreading contagion, thereby furnishing an intermediate level of description between that of individual nodes and the total infected population. For every discipline concerned with contagion and its prevention, we anticipate that contagion pathways hold valuable lessons, given their role as the conduits through which single mutations, innovations, or failures can sweep through a network as a whole.
Forecasting Chaotic Systems with Very Low Connectivity Reservoir Computers

We explore the hyperparameter space of reservoir computers used for forecasting of the chaotic Lorenz '63 attractor with Bayesian optimization. We use a new measure of reservoir performance, designed to emphasize learning the global climate of the forecasted system rather than short-term prediction. We find that optimizing over this measure more quickly excludes reservoirs that fail to reproduce the climate. The results of optimization are surprising: the optimized parameters often specify a reservoir network with very low connectivity. Inspired by this observation, we explore reservoir designs with even simpler structure and find well-performing reservoirs that have zero spectral radius and no recurrence. These simple reservoirs provide counterexamples to widely used heuristics in the field and may be useful for hardware implementations of reservoir computers.
Enhancing high-content imaging for studying microtubule networks at large-scale

Given the crucial role of microtubules for cell survival, many researchers have found success using microtubule-targeting agents in the search for effective cancer therapeutics. Understanding microtubule responses to targeted interventions requires that the microtubule network within cells can be consistently observed across a large sample of images. However, fluorescence noise sources captured simultaneously with biological signals while using wide-field microscopes can obfuscate fine microtubule structures. Such requirements are particularly challenging for high-throughput imaging, where researchers must make decisions related to the trade-off between imaging quality and speed. Here, we propose a computational framework to enhance the quality of high-throughput imaging data to achieve fast speed and high quality simultaneously. Using CycleGAN, we learn an image model from low-throughput, high-resolution images to enhance features, such as microtubule networks in high-throughput low-resolution images. We show that CycleGAN is effective in identifying microtubules with 0.93+ AUC-ROC and that these results are robust to different kinds of image noise. We further apply CycleGAN to quantify the changes in microtubule density as a result of the application of drug compounds, and show that the quantified responses correspond well with known drug effects
On the ejection of dark matter from globular clusters

We investigate analytically whether in a close encounter with a star, a Dark Matter particle can be accelerated above the escape speed of a Globular Cluster and be ejected. We find that this mechanism is not sufficient to eject a massive, extended Dark Matter halo by the present time. Combined with observations of isolated Globular Clusters that may not have had their halos tidally stripped, these results cast doubt on the scenario in which Globular Clusters formed in Dark Matter halos.
Wasserstein Neural Processes

Neural Processes (NPs) are a class of models that learn a mapping from a context set of input-output pairs to a distribution over functions. They are traditionally trained using maximum likelihood with a KL divergence regularization term. We show that there are desirable classes of problems where NPs, with this loss, fail to learn any reasonable distribution. We also show that this drawback is solved by using approximations of Wasserstein distance which calculates optimal transport distances even for distributions of disjoint support. We give experimental justification for our method and demonstrate performance. These Wasserstein Neural Processes (WNPs) maintain all of the benefits of traditional NPs while being able to approximate a new class of function mappings.
Algebraic structures on graph associahedra

M. Carr and S. Devadoss introduced in  the notion of tubing on a finite simple graph $\Gamma$, in the context of configuration spaces on the Hilbert plane. To any finite simple graph $\Gamma$ they associated a finite partially ordered set, whose elements are the tubings of $\Gamma$ and whose geometric realization is a convex polytope ${\mathcal K}\Gamma$, the graph-associahedron. For the complete graphs they recovered permutahedra, for linear graphs they got Stasheff's associahedra, while for simple graph they obtained the standard simplexes. 
The goal of the present work is to give an \emph{algebraic} description of graph associahedra. We introduce a substitution operation on tubings, which allows us to describe the set of faces of graph-associahedra as a free object, spanned by the set of all connected simple graphs, under operations given via connected subgraphs. The boundary maps of graph-associahedra defines natural derivations in this context. 
Along the way, we introduce a topological interpretation of the graph tubings and our new operations. In the last section, we show that substitution of tubings may be understood in the context of M. Batanin and M. Markl's operadic categories.
Learning to estimate label uncertainty for automatic radiology report parsing

Bootstrapping labels from radiology reports has become the scalable alternative to provide inexpensive ground truth for medical imaging. Because of the domain specific nature, state-of-the-art report labeling tools are predominantly rule-based. These tools, however, typically yield a binary 0 or 1 prediction that indicates the presence or absence of abnormalities. These hard targets are then used as ground truth to train image models in the downstream, forcing models to express high degree of certainty even on cases where specificity is low. This could negatively impact the statistical efficiency of image models. We address such an issue by training a Bidirectional Long-Short Term Memory Network to augment heuristic-based discrete labels of X-ray reports from all body regions and achieve performance comparable or better than domain-specific NLP, but with additional uncertainty estimates which enable finer downstream image model training.
TagSLAM: Robust SLAM with Fiducial Markers

TagSLAM provides a convenient, flexible, and robust way of performing Simultaneous Localization and Mapping (SLAM) with AprilTag fiducial markers. By leveraging a few simple abstractions (bodies, tags, cameras), TagSLAM provides a front end to the GTSAM factor graph optimizer that makes it possible to rapidly design a range of experiments that are based on tags: full SLAM, extrinsic camera calibration with non-overlapping views, visual localization for ground truth, loop closure for odometry, pose estimation etc. We discuss in detail how TagSLAM initializes the factor graph in a robust way, and present loop closure as an application example. TagSLAM is a ROS based open source package and can be found at this https URL.
RITnet: Real-time Semantic Segmentation of the Eye for Gaze Tracking

Accurate eye segmentation can improve eye-gaze estimation and support interactive computing based on visual attention; however, existing eye segmentation methods suffer from issues such as person-dependent accuracy, lack of robustness, and an inability to be run in real-time. Here, we present the RITnet model, which is a deep neural network that combines U-Net and DenseNet. RITnet is under 1 MB and achieves 95.3% accuracy on the 2019 OpenEDS Semantic Segmentation challenge. Using a GeForce GTX 1080 Ti, RITnet tracks at > 300Hz, enabling real-time gaze tracking applications. Pre-trained models and source code are available
Re-balancing Variational Autoencoder Loss for Molecule Sequence Generation

Molecule generation is to design new molecules with specific chemical properties and further to optimize the desired chemical properties. Following previous work, we encode molecules into continuous vectors in the latent space and then decode the embedding vectors into molecules under the variational autoencoder (VAE) framework. We investigate the posterior collapse problem of the current widely-used RNN-based VAEs for the molecule sequence generation. For the first time, we point out that the underestimated reconstruction loss of VAEs leads to the posterior collapse, and we also provide both analytical and experimental evidences to support our findings. To fix the problem and avoid the posterior collapse, we propose an effective and efficient solution in this work. Without bells and whistles, our method achieves the state-of-the-art reconstruction accuracy and competitive validity score on the ZINC 250K dataset. When generating 10,000 unique valid molecule sequences from the random prior sampling, it costs the JT-VAE 1450 seconds while our method only needs 9 seconds on a regular desktop machine.
Decision Automation for Electric Power Network Recovery

Critical infrastructure systems such as electric power networks, water networks, and transportation systems play a major role in the welfare of any community. In the aftermath of disasters, their recovery is of paramount importance; orderly and efficient recovery involves the assignment of limited resources (a combination of human repair workers and machines) to repair damaged infrastructure components. The decision maker must also deal with uncertainty in the outcome of the resource-allocation actions during recovery. The manual assignment of resources seldom is optimal despite the expertise of the decision maker because of the large number of choices and uncertainties in consequences of sequential decisions. This combinatorial assignment problem under uncertainty is known to be \mbox{NP-hard}. We propose a novel decision technique that addresses the massive number of decision choices for large-scale real-world problems; in addition, our method also features an experiential learning component that adaptively determines the utilization of the computational resources based on the performance of a small number of choices. Our framework is closed-loop, and naturally incorporates all the attractive features of such a decision-making system. In contrast to myopic approaches, which do not account for the future effects of the current choices, our methodology has an anticipatory learning component that effectively incorporates \emph{lookahead} into the solutions. To this end, we leverage the theory of regression analysis, Markov decision processes (MDPs), multi-armed bandits, and stochastic models of community damage from natural disasters to develop a method for near-optimal recovery of communities. Our method contributes to the general problem of MDPs with massive action spaces with application to recovery of communities affected by hazards.
IEG: Robust Neural Network Training to Tackle Severe Label Noise

Collecting large-scale data with clean labels for supervised training of neural networks is practically challenging. Although noisy labels are usually cheap to acquire, existing methods suffer severely for training datasets with high noise ratios, making high-cost human labeling a necessity. Here we present a method to train neural networks in a way that is almost invulnerable to severe label noise by utilizing a tiny trusted set. Our method, named IEG, is based on three key insights: (i) Isolation of noisy labels, (ii) Escalation of useful supervision from mislabeled data, and (iii) Guidance from small trusted data. On CIFAR100 with a 40% uniform noise ratio and 10 trusted labeled data per class, our method achieves $80.2{\pm}0.3\%$ classification accuracy, only 1.4% higher error than a neural network trained without label noise. Moreover, increasing the noise ratio to 80%, our method still achieves a high accuracy of $75.5{\pm}0.2\%$, compared to the previous best 47.7%. Finally, our method sets new state of the art on various types of challenging label corruption types and levels and large-scale WebVision benchmarks.
TransGCN: Coupling Transformation Assumptions with Graph Convolutional Networks for Link Prediction

Link prediction is an important and frequently studied task that contributes to an understanding of the structure of knowledge graphs (KGs) in statistical relational learning. Inspired by the success of graph convolutional networks (GCN) in modeling graph data, we propose a unified GCN framework, named TransGCN, to address this task, in which relation and entity embeddings are learned simultaneously. To handle heterogeneous relations in KGs, we introduce a novel way of representing heterogeneous neighborhood by introducing transformation assumptions on the relationship between the subject, the relation, and the object of a triple. Specifically, a relation is treated as a transformation operator transforming a head entity to a tail entity. Both translation assumption in TransE and rotation assumption in RotatE are explored in our framework. Additionally, instead of only learning entity embeddings in the convolution-based encoder while learning relation embeddings in the decoder as done by the state-of-art models, e.g., R-GCN, the TransGCN framework trains relation embeddings and entity embeddings simultaneously during the graph convolution operation, thus having fewer parameters compared with R-GCN. Experiments show that our models outperform the-state-of-arts methods on both FB15K-237 and WN18RR.
Spherical k-Nearest Neighbors Interpolation

Geospatial interpolation is a challenging task due to real world data often being sparse, heterogeneous and inconsistent. For that matter, this work presents SkNNI, a spherical interpolation algorithm capable of working with such challenging geospatial data. This work also presents NDDNISD an accurate and efficient interpolation function for SkNNI which shines due to its spatial awareness in terms of proximity and distribution of observation neighbors. SkNNI's open source implementation is also discussed and illustrated with a simple usage example.
Origin of hydrogen fluoride emission in the Orion Bar

The hydrogen fluoride (HF) molecule is seen in absorption in the interstellar medium (ISM) along many lines of sight. Surprisingly, it is observed in emission toward the Orion Bar, which is an interface between the ionized region around the Orion Trapezium stars and the Orion molecular cloud.
Aims. We aim to understand the origin of HF emission in the Orion Bar by comparing its spatial distribution with other tracers. We examine three mechanisms to explain the HF emission: thermal excitation, radiative dust pumping, and chemical pumping.
Methods. We used a Herschel/HIFI strip map of the HF J = 1 → 0 line, covering 0.5′ by 1.5′ that is oriented perpendicular to the Orion Bar. We used the RADEX non-local thermodynamic equilibrium (non-LTE) code to construct the HF column density map. We use the Meudon PDR code to explain the morphology of HF.
Results. The bulk of the HF emission at 10 km s−1 emerges from the CO-dark molecular gas that separates the ionization front from the molecular gas that is deeper in the Orion Bar. The excitation of HF is caused mainly by collisions with H2 at a density of 105 cm−3 together with a small contribution of electrons in the interclump gas of the Orion Bar. Infrared pumping and chemical pumping are not important.
 We conclude that the HF J = 1 → 0 line traces CO-dark molecular gas. Similarly, bright photodissociation regions associated with massive star formation may be responsible for the HF emission observed toward active galactic nuclei.
Adaptive Continuous Visual Odometry from RGB-D Images

In this paper, we extend the recently developed continuous visual odometry framework for RGB-D cameras to an adaptive framework via online hyperparameter learning. We focus on the case of isotropic kernels with a scalar as the length-scale. In practice and as expected, the length-scale has remarkable impacts on the performance of the original framework. Previously it was handled using a fixed set of conditions within the solver to reduce the length-scale as the algorithm reaches a local minimum. We automate this process by a greedy gradient descent step at each iteration to find the next-best length-scale. Furthermore, to handle failure cases in the gradient descent step where the gradient is not well-behaved, such as the absence of structure or texture in the scene, we use a search interval for the length-scale and guide it gradually toward the smaller values. This latter strategy reverts the adaptive framework to the original setup. The experimental evaluations using publicly available RGB-D benchmarks show the proposed adaptive continuous visual odometry outperforms the original framework and the current state-of-the-art. We also make the software for the developed algorithm publicly available.
Comparing Deep Learning Models for Multi-cell Classification in Liquid-based Cervical Cytology Images

Liquid-based cytology (LBC) is a reliable automated technique for the screening of Papanicolaou (Pap) smear data. It is an effective technique for collecting a majority of the cervical cells and aiding cytopathologists in locating abnormal cells. Most methods published in the research literature rely on accurate cell segmentation as a prior, which remains challenging due to a variety of factors, e.g., stain consistency, presence of clustered cells, etc. We propose a method for automatic classification of cervical slide images through generation of labeled cervical patch data and extracting deep hierarchical features by fine-tuning convolution neural networks, as well as a novel graph-based cell detection approach for cellular level evaluation. The results show that the proposed pipeline can classify images of both single cell and overlapping cells. The VGG-19 model is found to be the best at classifying the cervical cytology patch data with 95 % accuracy under precision-recall curve.
A Pre-defined Sparse Kernel Based Convolution for Deep CNNs

The high demand for computational and storage resources severely impede the deployment of deep convolutional neural networks (CNNs) in limited-resource devices. Recent CNN architectures have proposed reduced complexity versions (e.g. SuffleNet and MobileNet) but at the cost of modest decreases inaccuracy. This paper proposes pSConv, a pre-defined sparse 2D kernel-based convolution, which promises significant improvements in the trade-off between complexity and accuracy for both CNN training and inference. To explore the potential of this approach, we have experimented with two widely accepted datasets, CIFAR-10 and Tiny ImageNet, in sparse variants of both the ResNet18 and VGG16 architectures. Our approach shows a parameter count reduction of up to 4.24x with modest degradation in classification accuracy relative to that of standard CNNs. Our approach outperforms a popular variant of ShuffleNet using a variant of ResNet18 with pSConv having 3x3 kernels with only four of nine elements not fixed at zero. In particular, the parameter count is reduced by 1.7x for CIFAR-10 and 2.29x for Tiny ImageNet with an increased accuracy of ~4%.