nils-herrmann/clean_scirepeval huggingface dataset:
Structured Output Learning with Conditional Generative Flows

Traditional structured prediction models try to learn the conditional likelihood, i.e., p(y|x), to capture the relationship between the structured output y and the input features x. For many models, computing the likelihood is intractable. These models are therefore hard to train, requiring the use of surrogate objectives or variational inference to approximate likelihood. In this paper, we propose conditional Glow (c-Glow), a conditional generative flow for structured output learning. C-Glow benefits from the ability of flow-based models to compute p(y|x) exactly and efficiently. Learning with c-Glow does not require a surrogate objective or performing inference during training. Once trained, we can directly and efficiently generate conditional samples. We develop a sample-based prediction method, which can use this advantage to do efficient and effective inference. In our experiments, we test c-Glow on five different tasks. C-Glow outperforms the state-of-the-art baselines in some tasks and predicts comparable outputs in the other tasks. The results show that c-Glow is versatile and is applicable to many different structured prediction problems.
On the Accuracy of Influence Functions for Measuring Group Effects

Influence functions estimate the effect of removing a training point on a model without the need to retrain. They are based on a first-order Taylor approximation that is guaranteed to be accurate for sufficiently small changes to the model, and so are commonly used to study the effect of individual points in large datasets. However, we often want to study the effects of large groups of training points, e.g., to diagnose batch effects or apportion credit between different data sources. Removing such large groups can result in significant changes to the model. Are influence functions still accurate in this setting? In this paper, we find that across many different types of groups and for a range of real-world datasets, the predicted effect (using influence functions) of a group correlates surprisingly well with its actual effect, even if the absolute and relative errors are large. Our theoretical analysis shows that such strong correlation arises only under certain settings and need not hold in general, indicating that real-world datasets have particular properties that allow the influence approximation to be accurate.
Seeing the Wind: Visual Wind Speed Prediction with a Coupled Convolutional and Recurrent Neural Network

Wind energy resource quantification, air pollution monitoring, and weather forecasting all rely on rapid, accurate measurement of local wind conditions. Visual observations of the effects of wind---the swaying of trees and flapping of flags, for example---encode information regarding local wind conditions that can potentially be leveraged for visual anemometry that is inexpensive and ubiquitous. Here, we demonstrate a coupled convolutional neural network and recurrent neural network architecture that extracts the wind speed encoded in visually recorded flow-structure interactions of a flag and tree in naturally occurring wind. Predictions for wind speeds ranging from 0.75-11 m/s showed agreement with measurements from a cup anemometer on site, with a root-mean-squared error approaching the natural wind speed variability due to atmospheric turbulence. Generalizability of the network was demonstrated by successful prediction of wind speed based on recordings of other flags in the field and in a controlled wind tunnel test. Furthermore, physics-based scaling of the flapping dynamics accurately predicts the dependence of the network performance on the video frame rate and duration.
Counting and Segmenting Sorghum Heads

Phenotyping is the process of measuring an organism's observable traits. Manual phenotyping of crops is a labor-intensive, time-consuming, costly, and error prone process. Accurate, automated, high-throughput phenotyping can relieve a huge burden in the crop breeding pipeline. In this paper, we propose a scalable, high-throughput approach to automatically count and segment panicles (heads), a key phenotype, from aerial sorghum crop imagery. Our counting approach uses the image density map obtained from dot or region annotation as the target with a novel deep convolutional neural network architecture. We also propose a novel instance segmentation algorithm using the estimated density map, to identify the individual panicles in the presence of occlusion. With real Sorghum aerial images, we obtain a mean absolute error (MAE) of 1.06 for counting which is better than using well-known crowd counting approaches such as CCNN, MCNN and CSRNet models. The instance segmentation model also produces respectable results which will be ultimately useful in reducing the manual annotation workload for future data.
A Review of Deep Learning with Special Emphasis on Architectures, Applications and Recent Trends

Deep learning has taken over - both in problems beyond the realm of traditional, hand-crafted machine learning paradigms as well as in capturing the imagination of the practitioner sitting on top of petabytes of data. While the public perception about the efficacy of deep neural architectures in complex pattern recognition tasks grows, sequentially up-to-date primers on the current state of affairs must follow. In this review, we seek to present a refresher of the many different stacked, connectionist networks that make up the deep learning architectures followed by automatic architecture optimization protocols using multi-agent approaches. Further, since guaranteeing system uptime is fast becoming an indispensable asset across multiple industrial modalities, we include an investigative section on testing neural networks for fault detection and subsequent mitigation. This is followed by an exploratory survey of several application areas where deep learning has emerged as a game-changing technology - be it anomalous behavior detection in financial applications or financial time-series forecasting, predictive and prescriptive analytics, medical imaging, natural language processing or power systems research. The thrust of this review is on outlining emerging areas of application-oriented research within the deep learning community as well as to provide a handy reference to researchers seeking to embrace deep learning in their work for what it is: statistical pattern recognizers with unparalleled hierarchical structure learning capacity with the ability to scale with information.
Generative Imaging and Image Processing via Generative Encoder

<p style='text-indent:20px;'>This paper introduces a novel generative encoder (GE) framework for generative imaging and image processing tasks like image reconstruction, compression, denoising, inpainting, deblurring, and super-resolution. GE unifies the generative capacity of GANs and the stability of AEs in an optimization framework instead of stacking GANs and AEs into a single network or combining their loss functions as in existing literature. GE provides a novel approach to visualizing relationships between latent spaces and the data space. The GE framework is made up of a pre-training phase and a solving phase. In the former, a GAN with generator <inline-formula><tex-math id="M1">\begin{document}$ G $\end{document}</tex-math></inline-formula> capturing the data distribution of a given image set, and an AE network with encoder <inline-formula><tex-math id="M2">\begin{document}$ E $\end{document}</tex-math></inline-formula> that compresses images following the estimated distribution by <inline-formula><tex-math id="M3">\begin{document}$ G $\end{document}</tex-math></inline-formula> are trained separately, resulting in two latent representations of the data, denoted as the generative and encoding latent space respectively. In the solving phase, given noisy image <inline-formula><tex-math id="M4">\begin{document}$ x = \mathcal{P}(x^*) $\end{document}</tex-math></inline-formula>, where <inline-formula><tex-math id="M5">\begin{document}$ x^* $\end{document}</tex-math></inline-formula> is the target unknown image, <inline-formula><tex-math id="M6">\begin{document}$ \mathcal{P} $\end{document}</tex-math></inline-formula> is an operator adding an addictive, or multiplicative, or convolutional noise, or equivalently given such an image <inline-formula><tex-math id="M7">\begin{document}$ x $\end{document}</tex-math></inline-formula> in the compressed domain, i.e., given <inline-formula><tex-math id="M8">\begin{document}$ m = E(x) $\end{document}</tex-math></inline-formula>, the two latent spaces are unified via solving the optimization problem</p><p style='text-indent:20px;'><disp-formula> <label/> <tex-math id="FE1"> \begin{document}$ z^* = \underset{z}{\mathrm{argmin}} \|E(G(z))-m\|_2^2+\lambda\|z\|_2^2 $\end{document} </tex-math></disp-formula></p><p style='text-indent:20px;'>and the image <inline-formula><tex-math id="M9">\begin{document}$ x^* $\end{document}</tex-math></inline-formula> is recovered in a generative way via <inline-formula><tex-math id="M10">\begin{document}$ \hat{x}: = G(z^*)\approx x^* $\end{document}</tex-math></inline-formula>, where <inline-formula><tex-math id="M11">\begin{document}$ \lambda>0 $\end{document}</tex-math></inline-formula> is a hyperparameter. The unification of the two spaces allows improved performance against corresponding GAN and AE networks while visualizing interesting properties in each latent space.</p>
Second-Order Multi-Reference Algebraic Diagrammatic Construction Theory for Photoelectron Spectra of Strongly Correlated Systems.

We present a second-order formulation of multi-reference algebraic diagrammatic construction theory [Sokolov, A. Yu.  2018, 149, 204113] for simulating photoelectron spectra of strongly correlated systems (MR-ADC(2)). The MR-ADC(2) method uses second-order multi-reference perturbation theory (MRPT2) to efficiently obtain ionization energies and intensities for many photoelectron transitions in a single computation. In contrast to conventional MRPT2 methods, MR-ADC(2) provides information about ionization of electrons in all orbitals (i.e., core and active) and allows to compute transition intensities in straightforward and efficient way. Although equations of MR-ADC(2) depend on four-particle reduced density matrices, we demonstrate that computation of these large matrices can be completely avoided without introducing any approximations. The resulting MR-ADC(2) implementation has a lower computational scaling compared to conventional MRPT2 methods. We present results of MR-ADC(2) for photoelectron spectra of small molecules, carbon dimer, and equally-spaced hydrogen chains (H10 and H30) and outline directions for future developments.
D$\textbf{S}^3$L: Deep Self-Semi-Supervised Learning for Image Recognition

Despite the recent progress in deep semi-supervised learning (Semi-SL), the amount of labels still plays a dominant role. The success in self-supervised learning (Self-SL) hints a promising direction to exploit the vast unlabeled data by leveraging an additional set of deterministic labels. In this paper, we propose Deep Self-Semi-Supervised learning (D$S^3$L), a flexible multi-task framework with shared parameters that integrates the rotation task in Self-SL with the consistency-based methods in deep Semi-SL. Our method is easy to implement and is complementary to all consistency-based approaches. The experiments demonstrate that our method significantly improves over the published state-of-the-art methods on several standard benchmarks, especially when fewer labels are presented.
Real-time Approximate Bayesian Computation for Scene Understanding

Consider scene understanding problems such as predicting where a person is probably reaching, or inferring the pose of 3D objects from depth images, or inferring the probable street crossings of pedestrians at a busy intersection. This paper shows how to solve these problems using Approximate Bayesian Computation. The underlying generative models are built from realistic simulation software, wrapped in a Bayesian error model for the gap between simulation outputs and real data. The simulators are drawn from off-the-shelf computer graphics, video game, and traffic simulation code. The paper introduces two techniques for speeding up inference that can be used separately or in combination. The first is to train neural surrogates of the simulators, using a simple form of domain randomization to make the surrogates more robust to the gap between the simulation and reality. The second is to adaptively discretize the latent variables using a Tree-pyramid approach adapted from computer graphics. This paper also shows performance and accuracy measurements on real-world problems, establishing that it is feasible to solve these problems in real-time.
Machine Learning Methods for Shark Detection

This essay reviews human observer-based methods employed in shark spotting in Muizenberg Beach. It investigates Machine Learning methods for automated shark detection with the aim of enhancing human observation. A questionnaire and interview were used to collect information about shark spotting, the motivation of the actual Shark Spotter program and its limitations. We have defined a list of desirable properties for our model and chosen the adequate mathematical techniques. The preliminary results of the research show that we can expect to extract useful information from shark images despite the geometric transformations that sharks perform, its features do not change. To conclude, we have partially implemented our model; the remaining implementation requires dataset.
Technical Report of the Video Event Reconstruction and Analysis (VERA) System -- Shooter Localization, Models, Interface, and Beyond

Every minute, hundreds of hours of video are uploaded to social media sites and the Internet from around the world. This material creates a visual record of the experiences of a significant percentage of humanity and can help illuminate how we live in the present moment. When properly analyzed, this video can also help analysts to reconstruct events of interest, including war crimes, human rights violations, and terrorist acts. Machine learning and computer vision can play a crucial role in this process. In this technical report, we describe the Video Event Reconstruction and Analysis (VERA) system. This new tool brings together a variety of capabilities we have developed over the past few years (including video synchronization and geolocation to order unstructured videos lacking metadata over time and space, and sound recognition algorithms) to enable the reconstruction and analysis of events captured on video. Among other uses, VERA enables the localization of a shooter from just a few videos that include the sound of gunshots. To demonstrate the efficacy of this suite of tools, we present the results of estimating the shooter's location of the Las Vegas Shooting in 2017 and show that VERA accurately predicts the shooter's location using only the first few gunshots. We then point out future directions that can help improve the system and further reduce unnecessary human labor in the process. All of the components of VERA run through a web interface that enables human-in-the-loop verification to ensure accurate estimations. All relevant source code, including the web interface and machine learning models, is freely available on Github. We hope that researchers and software developers will be inspired to improve and expand this system moving forward to better meet the needs of human rights and public safety.
Graph Attention Memory for Visual Navigation

Visual navigation in complex environments is inefficient with traditional reactive policy or general-purposed recurrent policy. To address the long-term memory issue, this paper proposes a graph attention memory (GAM) architecture consisting of memory construction module, graph attention module and control module. The memory construction module builds the topological graph based on supervised learning by taking the exploration prior. Then, guided attention features are extracted with the graph attention module. Finally, the deep reinforcement learning based control module makes decisions based on visual observations and guided attention features. Detailed convergence analysis of GAM is presented in this paper. We evaluate GAM-based navigation system in two complex 3D environments. Experimental results show that the GAM-based navigation system significantly improves learning efficiency and outperforms all baselines in average success rate.
MathQA: Towards Interpretable Math Word Problem Solving with Operation-Based Formalisms

We introduce a large-scale dataset of math word problems and an interpretable neural math problem solver by learning to map problems to their operation programs. Due to annotation challenges, current datasets in this domain have been either relatively small in scale or did not offer precise operational annotations over diverse problem types. We introduce a new representation language to model operation programs corresponding to each math problem that aim to improve both the performance and the interpretability of the learned models. Using this representation language, we significantly enhance the AQUA-RAT dataset with fully-specified operational programs. We additionally introduce a neural sequence-to-program model with automatic problem categorization. Our experiments show improvements over competitive baselines in our dataset as well as the AQUA-RAT dataset. The results are still lower than human performance indicating that the dataset poses new challenges for future research. Our dataset is available at https://math-qa.github.io/math-QA/
Combating the Compounding-Error Problem with a Multi-step Model

Model-based reinforcement learning is an appealing framework for creating agents that learn, plan, and act in sequential environments. Model-based algorithms typically involve learning a transition model that takes a state and an action and outputs the next state---a one-step model. This model can be composed with itself to enable predicting multiple steps into the future, but one-step prediction errors can get magnified, leading to unacceptable inaccuracy. This compounding-error problem plagues planning and undermines model-based reinforcement learning. In this paper, we address the compounding-error problem by introducing a multi-step model that directly outputs the outcome of executing a sequence of actions. Novel theoretical and empirical results indicate that the multi-step model is more conducive to efficient value-function estimation, and it yields better action selection compared to the one-step model. These results make a strong case for using multi-step models in the context of model-based reinforcement learning.
Grammar-based Neural Text-to-SQL Generation

The sequence-to-sequence paradigm employed by neural text-to-SQL models typically performs token-level decoding and does not consider generating SQL hierarchically from a grammar. Grammar-based decoding has shown significant improvements for other semantic parsing tasks, but SQL and other general programming languages have complexities not present in logical formalisms that make writing hierarchical grammars difficult. We introduce techniques to handle these complexities, showing how to construct a schema-dependent grammar with minimal over-generation. We analyze these techniques on ATIS and Spider, two challenging text-to-SQL datasets, demonstrating that they yield 14--18\% relative reductions in error.
Constraints on chameleon f(R)-gravity from galaxy rotation curves of the SPARC sample

In chameleon f(R)-gravity, the fifth force will lead to ‘upturns’ in galaxy rotation curves near the screening radius. The location of the upturn depends on the cosmic background value of the scalar field $\bar{f}_\mathrm{R0}$, as well as the mass, size, and environment of the galaxy. We search for this signature of modified gravity in the SPARC sample of measured rotation curves, using an MCMC technique to derive constraints on $\bar{f}_\mathrm{R0}$. Assuming NFW dark matter haloes and with $\bar{f}_\mathrm{R0}$ freely varying for each galaxy, most galaxies prefer f(R) gravity to ΛCDM, but there is a large spread of inferred $\bar{f}_\mathrm{R0}$ values, inconsistent with a single global value. Requiring instead a consistent $\bar{f}_\mathrm{R0}$ value for the whole sample, models with $\log _{10}|\bar{f}_\mathrm{R0}|\gt -6.1$ are excluded. On the other hand, models in the range $-7.5\lt \log _{10}|\bar{f}_\mathrm{R0}|\lt -6.5$ seem to be favoured with respect to ΛCDM, with a significant peak at −7. However, this signal is largely a result of galaxies for which the f(R) signal is degenerate with the core/cusp problem, and when the NFW profile is replaced with a cored halo profile, ΛCDM gives better fits than any given f(R) model. Thus, we find no convincing evidence of f(R) gravity down to the level of $|\bar{f}_\mathrm{R0}|\sim 6 \times 10^{-8}$, with the caveat that if cored halo density profiles cannot ultimately be explained within ΛCDM, a screened modified gravity theory could possibly provide an alternative solution for the core/cusp problem. However, the f(R) models studied here fall short of achieving this.
Discriminative Clustering for Robust Unsupervised Domain Adaptation

Unsupervised domain adaptation seeks to learn an invariant and discriminative representation for an unlabeled target domain by leveraging the information of a labeled source dataset. We propose to improve the discriminative ability of the target domain representation by simultaneously learning tightly clustered target representations while encouraging that each cluster is assigned to a unique and different class from the source. This strategy alleviates the effects of negative transfer when combined with adversarial domain matching between source and target representations. Our approach is robust to differences in the source and target label distributions and thus applicable to both balanced and imbalanced domain adaptation tasks, and with a simple extension, it can also be used for partial domain adaptation. Experiments on several benchmark datasets for domain adaptation demonstrate that our approach can achieve state-of-the-art performance in all three scenarios, namely, balanced, imbalanced and partial domain adaptation.
Identifying Cache-Based Side Channels through Secret-Augmented Abstract Interpretation

Cache-based side channels enable a dedicated attacker to reveal program secrets by measuring the cache access patterns. Practical attacks have been shown against real-world crypto algorithm implementations such as RSA, AES, and ElGamal. By far, identifying information leaks due to cache-based side channels, either in a static or dynamic manner, remains a challenge: the existing approaches fail to offer high precision, full coverage, and good scalability simultaneously, thus impeding their practical use in real-world scenarios. 
In this paper, we propose a novel static analysis method on binaries to detect cache-based side channels. We use abstract interpretation to reason on program states with respect to abstract values at each program point. To make such abstract interpretation scalable to real-world cryptosystems while offering high precision and full coverage, we propose a novel abstract domain called the Secret-Augmented Symbolic domain (SAS). SAS tracks program secrets and dependencies on them for precision, while it tracks only coarse-grained public information for scalability. 
We have implemented the proposed technique into a practical tool named CacheS and evaluated it on the implementations of widely-used cryptographic algorithms in real-world crypto libraries, including Libgcrypt, OpenSSL, and mbedTLS. CacheS successfully confirmed a total of 154 information leaks reported by previous research and 54 leaks that were previously unknown. We have reported our findings to the developers. And they confirmed that many of those unknown information leaks do lead to potential side channels.