nils-herrmann/clean_scirepeval huggingface dataset:
SystemDS: A Declarative Machine Learning System for the End-to-End Data Science Lifecycle

Machine learning (ML) applications become increasingly common in many domains. ML systems to execute these workloads include numerical computing frameworks and libraries, ML algorithm libraries, and specialized systems for deep neural networks and distributed ML. These systems focus primarily on efficient model training and scoring. However, the data science process is exploratory, and deals with underspecified objectives and a wide variety of heterogeneous data sources. Therefore, additional tools are employed for data engineering and debugging, which requires boundary crossing, unnecessary manual effort, and lacks optimization across the lifecycle. In this paper, we introduce SystemDS, an open source ML system for the end-to-end data science lifecycle from data integration, cleaning, and preparation, over local, distributed, and federated ML model training, to debugging and serving. To this end, we aim to provide a stack of declarative languages with R-like syntax for the different lifecycle tasks, and users with different expertise. We describe the overall system architecture, explain major design decisions (motivated by lessons learned from Apache SystemML), and discuss key features and research directions. Finally, we provide preliminary results that show the potential of end-to-end lifecycle optimization.
DRLViz: Understanding Decisions and Memory in Deep Reinforcement Learning

We present DRLViz, a visual analytics interface to interpret the internal memory of an agent (e.g. a robot) trained using deep reinforcement learning. This memory is composed of large temporal vectors updated when the agent moves in an environment and is not trivial to understand due to the number of dimensions, dependencies to past vectors, spatial/temporal correlations, and co‐correlation between dimensions. It is often referred to as a black box as only inputs (images) and outputs (actions) are intelligible for humans. Using DRLViz, experts are assisted to interpret decisions using memory reduction interactions, and to investigate the role of parts of the memory when errors have been made (e.g. wrong direction). We report on DRLViz applied in the context of video games simulators (ViZDoom) for a navigation scenario with item gathering tasks. We also report on experts evaluation using DRLViz, and applicability of DRLViz to other scenarios and navigation problems beyond simulation games, as well as its contribution to black box models interpretability and explain‐ability in the field of visual analytics.
TESS Observations of the WASP-121 b Phase Curve

We study the red-optical photometry of the ultrahot Jupiter WASP-121 b as observed by the Transiting Exoplanet Survey Satellite (TESS) and model its atmosphere through a radiative transfer simulation. Given its short orbital period of ∼1.275 days, inflated state, and bright host star, WASP-121 b is exceptionally favorable for detailed atmospheric characterization. Toward this purpose, we use allesfitter to characterize its full red-optical phase curve, including the planetary phase modulation and secondary eclipse. We measure the day- and nightside brightness temperatures in the TESS passband as and K, respectively, and do not find a statistically significant phase shift between the brightest and substellar points. This is consistent with inefficient heat recirculation on the planet. We then perform an atmospheric retrieval analysis to infer the dayside atmospheric properties of WASP-121 b, such as its bulk composition, albedo, and heat recirculation. We confirm the temperature inversion in the atmosphere and suggest H−, TiO, and VO as potential causes of the inversion, absorbing heat at optical wavelengths at low pressures. Future Hubble Space Telescope and James Webb Space Telescope observations of WASP-121 b will benefit from its first full phase curve measured by TESS.
Fusion rules for permutation extensions of modular tensor categories

We give a construction and algorithmic description of the fusion ring of permutation extensions of an arbitrary modular tensor category using a combinatorial approach inspired by the physics of anyons and symmetry defects in bosonic topological phases of matter. The definition is illustrated with examples, namely bilayer symmetry defects and $S_3$-extensions of small modular tensor categories like the Ising and Fibonacci theories. An implementation of the fusion algorithm is provided in the form of a Mathematica package. We introduce the notions of confinement and deconfinement of anyons and defects, respectively, which develop the tools to generalize our approach to more general fusion rings of $G$-crossed extensions.
Show Your Work: Improved Reporting of Experimental Results

Research in natural language processing proceeds, in part, by demonstrating that new models achieve superior performance (e.g., accuracy) on held-out test data, compared to previous results. In this paper, we demonstrate that test-set performance scores alone are insufficient for drawing accurate conclusions about which model performs best. We argue for reporting additional details, especially performance on validation data obtained during model development. We present a novel technique for doing so: expected validation performance of the best-found model as a function of computation budget (i.e., the number of hyperparameter search trials or the overall training time). Using our approach, we find multiple recent model comparisons where authors would have reached a different conclusion if they had used more (or less) computation. Our approach also allows us to estimate the amount of computation required to obtain a given accuracy; applying it to several recently published results yields massive variation across papers, from hours to weeks. We conclude with a set of best practices for reporting experimental results which allow for robust future comparisons, and provide code to allow researchers to use our technique.
Optical phase curve of the ultra-hot Jupiter WASP-121b

We present the analysis of TESS optical photometry of WASP-121b, which reveals the phase curve of this transiting ultra-hot Jupiter. Its hotspot is located at the sub-stellar point, showing inefficient heat transport from the dayside (2870 ± 50 K) to the nightside (<2500 K at 3σ) at the altitudes probed by TESS. The TESS eclipse depth, measured at the shortest wavelength to date for WASP-121b, confirms the strong deviation from blackbody planetary emission. Our atmospheric retrieval on the complete emission spectrum supports the presence of a temperature inversion, which can be explained by the presence of VO and possibly TiO and FeH. The strong planetary emission at short wavelengths could arise from an H− continuum.
RNN Architecture Learning with Sparse Regularization

Neural models for NLP typically use large numbers of parameters to reach state-of-the-art performance, which can lead to excessive memory usage and increased runtime. We present a structure learning method for learning sparse, parameter-efficient NLP models. Our method applies group lasso to rational RNNs (Peng et al., 2018), a family of models that is closely connected to weighted finite-state automata (WFSAs). We take advantage of rational RNNs’ natural grouping of the weights, so the group lasso penalty directly removes WFSA states, substantially reducing the number of parameters in the model. Our experiments on a number of sentiment analysis datasets, using both GloVe and BERT embeddings, show that our approach learns neural structures which have fewer parameters without sacrificing performance relative to parameter-rich baselines. Our method also highlights the interpretable properties of rational RNNs. We show that sparsifying such models makes them easier to visualize, and we present models that rely exclusively on as few as three WFSAs after pruning more than 90% of the weights. We publicly release our code.
One Explanation Does Not Fit All: A Toolkit and Taxonomy of AI Explainability Techniques

As artificial intelligence and machine learning algorithms make further inroads into society, calls are increasing from multiple stakeholders for these algorithms to explain their outputs. At the same time, these stakeholders, whether they be affected citizens, government regulators, domain experts, or system developers, present different requirements for explanations. Toward addressing these needs, we introduce AI Explainability 360 (this http URL), an open-source software toolkit featuring eight diverse and state-of-the-art explainability methods and two evaluation metrics. Equally important, we provide a taxonomy to help entities requiring explanations to navigate the space of explanation methods, not only those in the toolkit but also in the broader literature on explainability. For data scientists and other users of the toolkit, we have implemented an extensible software architecture that organizes methods according to their place in the AI modeling pipeline. We also discuss enhancements to bring research innovations closer to consumers of explanations, ranging from simplified, more accessible versions of algorithms, to tutorials and an interactive web demo to introduce AI explainability to different audiences and application domains. Together, our toolkit and taxonomy can help identify gaps where more explainability methods are needed and provide a platform to incorporate them as they are developed.
Agora: Towards An Open Ecosystem for Democratizing Data Science & Artificial Intelligence

Data, algorithms, and compute/storage infrastructure are key assets that drive data science and artificial intelligence applications. As providing all these assets requires a huge investment, data science and artificial intelligence technologies are currently dominated by a small number of providers who can afford these investments. This leads to lock-in effects and hinders features that require a flexible exchange of assets among users. In this vision paper, we present Agora, a unified asset ecosystem. The Agora system provides the technical infrastructure that allows for offering and using data and algorithms, as well as physical infrastructure components. Agora is designed as an open ecosystem of asset marketplaces and provides to a broad audience not only data but the entire data value chain (including computational resources and human expertise). Agora (i) leverages a fine-grained exchange of assets, (ii) allows for combining assets to novel applications, and (iii) flexibly executes such applications on available resources. As a result, Agora overcomes lock-in effects and removes entry barriers for new asset providers. In contrast to existing data management systems, Agora operates in a heavily decentralized and dynamic environment: Data, algorithms, and even compute resources are dynamically created, modified, and removed by different stakeholders. Agora presents novel research directions for the data management community as a whole: It requires to combine our traditional expertise in scalable data processing and management with infrastructure provisioning as well as economic and application aspects of data, algorithms, and infrastructure.
Improved Hierarchical Patient Classification with Language Model Pretraining over Clinical Notes

Clinical notes in electronic health records contain highly heterogeneous writing styles, including non-standard terminology or abbreviations. Using these notes in predictive modeling has traditionally required preprocessing (e.g. taking frequent terms or topic modeling) that removes much of the richness of the source data. We propose a pretrained hierarchical recurrent neural network model that parses minimally processed clinical notes in an intuitive fashion, and show that it improves performance for discharge diagnosis classification tasks on the Medical Information Mart for Intensive Care III (MIMIC-III) dataset, compared to models that treat the notes as an unordered collection of terms or that conduct no pretraining. We also apply an attribution technique to examples to identify the words that the model uses to make its prediction, and show the importance of the words' nearby context.
Sequential Convolutional Recurrent Neural Networks for Fast Automatic Modulation Classification

A novel and efficient end-to-end learning model for automatic modulation classification is proposed for wireless spectrum monitoring applications, which automatically learns from the time domain in-phase and quadrature data without requiring the design of hand-crafted expert features. With the intuition of convolutional layers with pooling serving as the role of front-end feature distillation and dimensionality reduction, sequential convolutional recurrent neural networks are developed to take complementary advantage of parallel computing capability of convolutional neural networks and temporal sensitivity of recurrent neural networks. Experimental results demonstrate that the proposed architecture delivers overall superior performance in signal to noise ratio range above −10 dB, and achieves significantly improved classification accuracy from 80% to 92.1% at high signal to noise ratio range, while drastically reduces the average training and prediction time by approximately 74% and 67%, respectively. Response patterns learned by the proposed architecture are visualized to better understand the physics of the model. Furthermore, a comparative study is performed to investigate the impacts of various sequential convolutional recurrent neural network structure settings on classification performance. A representative sequential convolutional recurrent neural network architecture with the two-layer convolutional neural network and subsequent two-layer long short-term memory neural network is developed to suggest the option for fast automatic modulation classification.
On Learning Disentangled Representations for Gait Recognition

Gait, the walking pattern of individuals, is one of the important biometrics modalities. Most of the existing gait recognition methods take silhouettes or articulated body models as gait features. These methods suffer from degraded recognition performance when handling confounding variables, such as clothing, carrying and viewing angle. To remedy this issue, we propose a novel AutoEncoder framework, GaitNet, to explicitly disentangle appearance, canonical and pose features from RGB imagery. The LSTM integrates pose features over time as a dynamic gait feature while canonical features are averaged as a static gait feature. Both of them are utilized as classification features. In addition, we collect a Frontal-View Gait (FVG) dataset to focus on gait recognition from frontal-view walking, which is a challenging problem since it contains minimal gait cues compared to other views. FVG also includes other important variations, e.g., walking speed, carrying, and clothing. With extensive experiments on CASIA-B, USF, and FVG datasets, our method demonstrates superior performance to the SOTA quantitatively, the ability of feature disentanglement qualitatively, and promising computational efficiency. We further compare our GaitNet with state-of-the-art face recognition to demonstrate the advantages of gait biometrics identification under certain scenarios, e.g., long-distance/lower resolutions, cross viewing angles. Source code is available at http://cvlab.cse.msu.edu/project-gaitnet.html.
Calibrating Wayfinding Decisions in Pedestrian Simulation Models: The Entropy Map

This paper presents entropy maps, an approach to describing and visualising uncertainty among alternative potential movement intentions in pedestrian simulation models. In particular, entropy maps show the instantaneous level of randomness in decisions of a pedestrian agent situated in a specific point of the simulated environment with an heatmap approach. Experimental results highlighting the relevance of this tool supporting modelers are provided and discussed.
A spectral approach to numerical simulations of the ADM equations

We present a numerical study of the Einstein equations, according to the Arnowitt-Deser-Misner (ADM) formalism, in order to simulate the dynamics of gravitational fields. We took in consideration the original $3+1$ decomposition of the ADM equations, in vacuum conditions, in simplified geometries. The numerical code is based on spectral methods, making use of filtering (de-aliasing) techniques. The algorithm has been stabilized via an adaptive time-refinement, based on a procedure that checks self-consistently the regularity of the solutions. The accuracy of our numerical model has been validated through a series of standard tests. Finally, we present also a new kind of initial data that can be used for testing numerical codes.
ESA Voyage 2050 white paper: A complete census of the gas phases in and around galaxies, far-UV spectropolarimetry as a prime tool for understanding galaxy evolution and star formation

(abridged) The far-UV wavelength range (912-2000A) provides access to atomic and molecular transitions of many species the interstellar medium (ISM), circumgalactic medium (CGM), and intergalactic medium, within phases spanning a wide range of ionization, density, temperature, and molecular gas fraction. Far-UV space telescopes have enabled detailed studies of the ISM in the Milky Way thanks to absorption features appearing in the UV spectra of hot stars and yielding fundamental insights into the composition and physical characteristics of all phases of the ISM along with the processes that influence them. However, we have yet to design a spectrometer able to observe the full UV domain at resolving power R>10^5 with a signal-to-noise ratio SNR>500. Such a resolution is necessary to resolve lines from both the cold molecular hydrogen and the warm metal ions with a turbulent velocity of about 1 km s-1, and to differentiate distinct velocity components. Future UV spectroscopic studies of the Milky Way ISM must revolutionize our understanding of the ISM as a dynamical, unstable, and magnetized medium, and rise to the challenge brought forward by current theories. Another interesting prospect is to transpose the same level of details that has been reached for the Milky Way to the ISM in external galaxies, in particular in metal-poor galaxies, where the ISM chemical composition, physical conditions, and topology change dramatically, with significant consequences on the star-formation properties. Finally, we need to be able to perform statistical analyses of background quasar lines of sight intersecting the CGM of galaxies at various redshifts and to comprehend the role of gas exchanges and flows for galaxy evolution.
Destination-aware Adaptive Traffic Flow Rule Aggregation in Software-Defined Networks

In this paper, we propose a destination-aware adaptive traffic flow rule aggregation (DATA) mechanism for facilitating traffic flow monitoring in SDN-based networks. This method adapts the number of flow table entries in SDN switches according to the level of detail of traffic flow information that other mechanisms (e.g. for traffic engineering, traffic monitoring, intrusion detection) require. It also prevents performance degradation of the SDN switches by keeping the number of flow table entries well below a critical level. This level is not preset as a hard threshold but learned during operation by using a machine-learning based algorithm. The DATA method is implemented within a RESTful application (DATA App) which monitors and analyzes the ongoing network traffic and provides instructions to the SDN controller to adapt the traffic flow matching strategies accordingly. A thorough performance evaluation of DATA is conducted in an SDN emulation environment. The results show that—compared to the default behavior of common SDN controllers—the proposed DATA approach yields significant SDN switch performance improvements while still providing detailed traffic flow information on demand.
Revisiting 
CP
 violation in 
D→PP
 and 
VP
 decays

Direct CP violation in the hadronic charm decays provides a good testing ground for the Kobayashi-Maskawa mechanism in the Standard Model. Any significant deviations from the expectation would be indirect evidence of physics beyond the Standard Model. In view of improved measurements from LHCb and BESIII experiments, we re-analyze the Cabibbo-favored $D \to P\!P$ and $V\!P$ decays in the topological diagram approach. By assuming certain SU(3)-breaking effects in the tree-type amplitudes, we make predictions for both branching fractions and CP asymmetries of the singly Cabibbo-suppressed decay modes. While the color-allowed and -suppressed amplitudes are preferred to scale by the factor dictated by factorization in the $P\!P$ modes, no such scaling is required in the $V\!P$ modes. The $W$-exchange amplitudes are found to change by 10\% to 50\% and depend on whether $d\overline{d}$ or $s\overline{s}$ pair directly emerges from $W$-exchange. The predictions of branching fractions are generally improved after these SU(3) symmetry breaking effects are taken into account. We show in detail how the tree-type, QCD-penguin, and weak penguin-annihilation diagrams contribute and modify CP asymmetry predictions. Future measurements of sufficiently many direct CP asymmetries will be very useful in removing a discrete ambiguity in the strong phases as well as discriminating among different theory approaches. In particular, we predict $a_{CP}(K^+K^-)-a_{CP}(\pi^+\pi^-) = (-1.14 \pm 0.26) \times 10^{-3}$ or $(-1.25 \pm 0.25) \times 10^{-3}$, consistent with the latest data, and $a_{CP}(K^+K^{*-})-a_{CP}(\pi^+\rho^-) = (-1.52 \pm 0.43) \times 10^{-3}$, an attractive and measurable observable in the near future. Moreover, we observe that such CP asymmetry differences are dominated by long-distance penguin-exchange through final-state rescattering.
“Going on a vacation” takes longer than “Going for a walk”: A Study of Temporal Commonsense Understanding

Understanding time is crucial for understanding events expressed in natural language. Because people rarely say the obvious, it is often necessary to have commonsense knowledge about various temporal aspects of events, such as duration, frequency, and temporal order. However, this important problem has so far received limited attention. This paper systematically studies this temporal commonsense problem. Specifically, we define five classes of temporal commonsense, and use crowdsourcing to develop a new dataset, MCTACO, that serves as a test set for this task. We find that the best current methods used on MCTACO are still far behind human performance, by about 20%, and discuss several directions for improvement. We hope that the new dataset and our study here can foster more future research on this topic.
The Vafa-Witten invariants via surface Deligne-Mumford stacks and S-duality

Motivated by the S-duality conjecture of Vafa-Witten, Tanaka-Thomas have developed a theory of Vafa-Witten invariants for projective surfaces using the moduli space of Higgs sheaves. Their definition and calculation prove the S-duality prediction of Vafa-Witten in many cases in the side of gauge group $SU(r)$. In this survey paper for ICCM-2019 we review the S-duality conjecture in physics by Vafa-Witten and the definition of Vafa-Witten invariants for smooth projective surfaces and surface Deligne-Mumford stacks. We make a prediction that the Vafa-Witten invariants for Deligne-Mumford surfaces may give the generating series for the Langlands dual group $^{L}SU(r)=SU(r)/\zz_r$. We survey a check for the projective plane $\pp^2$.