nils-herrmann/clean_scirepeval huggingface dataset:
Towards Robust Named Entity Recognition for Historic German

In this paper we study the influence of using language model pre-training for named entity recognition for Historic German. We achieve new state-of-the-art results using carefully chosen training data for language models. For a low-resource domain like named entity recognition for Historic German, language model pre-training can be a strong competitor to CRF-only methods. We show that language model pre-training can be more effective than using transfer-learning with labeled datasets. Furthermore, we introduce a new language model pre-training objective, synthetic masked language model pre-training (SMLM), that allows a transfer from one domain (contemporary texts) to another domain (historical texts) by using only the same (character) vocabulary. Results show that using SMLM can achieve comparable results for Historic named entity recognition, even when they are only trained on contemporary texts. Our pre-trained character-based language models improve upon classical CRF-based methods and previous work on Bi-LSTMs by boosting F1 score performance by up to 6%.
Testing goodness of fit for point processes via topological data analysis

We introduce tests for the goodness of fit of point patterns via methods from topological data analysis. More precisely, the persistent Betti numbers give rise to a bivariate functional summary statistic for observed point patterns that is asymptotically Gaussian in large observation windows. We analyze the power of tests derived from this statistic on simulated point patterns and compare its performance with global envelope tests. Finally, we apply the tests to a point pattern from an application context in neuroscience. As the main methodological contribution, we derive sufficient conditions for a functional central limit theorem on bounded persistent Betti numbers of point processes with exponential decay of correlations.
Model Explanations under Calibration

Explaining and interpreting the decisions of recommender systems are becoming extremely relevant both, for improving predictive performance, and providing valid explanations to users. While most of the recent interest has focused on providing local explanations, there has been a much lower emphasis on studying the effects of model dynamics and its impact on explanation. In this paper, we perform a focused study on the impact of model interpretability in the context of calibration. Specifically, we address the challenges of both over-confident and under-confident predictions with interpretability using attention distribution. Our results indicate that the means of using attention distributions for interpretability are highly unstable for un-calibrated models. Our empirical analysis on the stability of attention distribution raises questions on the utility of attention for explainability.
The Mathematical Specification of the Statebox Language

This document defines the mathematical backbone of the Statebox programming language. In the simplest way possible, Statebox can be seen as a clever way to tie together different theoretical structures to maximize their benefits and limit their downsides. Since consistency and correctness are central requisites for our language, it became clear from the beginning that such tying could not be achieved by just hacking together different pieces of code representing implementations of the structures we wanted to leverage: Rigorous mathematics is employed to ensure both conceptual consistency of the language and reliability of the code itself. The mathematics presented here is what guided the implementation process, and we deemed very useful to release it to the public to help people wanting to audit our work to better understand the code itself.
Towards White-box Benchmarks for Algorithm Control

The performance of many algorithms in the fields of hard combinatorial problem solving, machine learning or AI in general depends on tuned hyperparameter configurations. Automated methods have been proposed to alleviate users from the tedious and error-prone task of manually searching for performance-optimized configurations across a set of problem instances. However there is still a lot of untapped potential through adjusting an algorithm's hyperparameters online since different hyperparameters are potentially optimal at different stages of the algorithm. We formulate the problem of adjusting an algorithm's hyperparameters for a given instance on the fly as a contextual MDP, making reinforcement learning (RL) the prime candidate to solve the resulting algorithm control problem in a data-driven way. Furthermore, inspired by applications of algorithm configuration, we introduce new white-box benchmarks suitable to study algorithm control. We show that on short sequences, algorithm configuration is a valid choice, but that with increasing sequence length a black-box view on the problem quickly becomes infeasible and RL performs better.
Better Transfer Learning with Inferred Successor Maps

Humans and animals show remarkable flexibility in adjusting their behaviour when their goals, or rewards in the environment change. While such flexibility is a hallmark of intelligent behaviour, these multi-task scenarios remain an important challenge for machine learning algorithms and neurobiological models alike. We investigated two approaches that could enable this flexibility: factorized representations, which abstract away general aspects of a task from those prone to change, and nonparametric, memory-based approaches, which can provide a principled way of using similarity to past experiences to guide current behaviour. In particular, we combine the successor representation (SR) that factors the value of actions into expected outcomes and corresponding rewards with evaluating task similarity through clustering the space of reward functions. The proposed algorithm inverts a generative model over tasks, and dynamically samples from a flexible number of distinct SR maps while accumulating evidence about the current task context through amortized inference. It improves SR's transfer capabilities and outperforms competing algorithms and baselines in settings with both known and unsignalled rewards changes. Further, as a neurobiological model of spatial coding in the hippocampus, it explains important signatures of this representation, such as the "flickering" behaviour of hippocampal maps, and trajectory-dependent place cells (so-called splitter cells) and their dynamics. We thus provide a novel algorithmic approach for multi-task learning, as well as a common normative framework that links together these different characteristics of the brain's spatial representation.
Generalizing Critical Path Analysis on Mobile Traffic

Critical Path Analysis (CPA) studies the delivery of webpages to identify page resources, their interrelations, as well as their impact on the page loading latency. Despite CPA being a generic methodology, its mechanisms have been applied only to browsers and web traffic, but those do not directly apply to study generic mobile apps. Likewise, web browsing represents only a small fraction of the overall mobile traffic. In this paper, we take a first step towards filling this gap by exploring how CPA can be performed for generic mobile applications. We propose Mobile Critical Path Analysis (MCPA), a methodology based on passive and active network measurements that is applicable to a broad set of apps to expose a fine-grained view of their traffic dynamics. We validate MCPA on popular apps across different categories and usage scenarios. We show that MCPA can identify user interactions with mobile apps only based on traffic monitoring, and the relevant network activities that are bottlenecks. Overall, we observe that apps spend 60% of time and 84% of bytes on critical traffic on average, corresponding to +22% time and +13% bytes than what observed for browsing.
The assembly of the Virgo cluster, traced by its galaxy haloes

Kinematic studies have produced accurate measurements of the total dark matter mass and mean dark matter density within the optical extent of galaxies for large samples of objects. Here we consider theoretical predictions for the latter quantity, $\bar{\rho }_{\rm dm}$, measured within the isophotal radius R23.5, for isolated haloes with universal density profiles. Through a combination of empirical scaling relations, we show that $\bar{\rho }_{\rm dm}$ is expected to depend weakly on halo mass and strongly on redshift. When galaxy haloes fall into larger groups or clusters, they become tidally stripped, reducing their total dark matter mass, but this process is expected to preserve central density until an object is close to disruption. We confirm this with collisonless simulations of cluster formation, finding that subhaloes have values of $\bar{\rho }_{\rm dm}$ close to the analytic predictions. This suggests that $\bar{\rho }_{\rm dm}$ may be a useful indicator of infall redshift on to the cluster. We test this hypothesis with data from the SHIVir survey, which covers a reasonable fraction of the Virgo cluster. We find that galaxies with high $\bar{\rho }_{\rm dm}$ do indeed trace the densest regions of the cluster, with a few notable exceptions. Samples selected by environment have higher densities at a significance of 3.5–4σ, while samples selected by density are more clustered at 3–3.5σ significance. We conclude that halo density can be a powerful tracer of the assembly history of clusters and their member galaxies.
Binary Mergers near a Supermassive Black Hole: Relativistic Effects in Triples

We study the general relativistic (GR) effects induced by a spinning supermassive black hole on the orbital and spin evolution of a merging black hole binary (BHB) in a hierarchical triple system. A sufficiently inclined outer orbit can excite Lidov–Kozai eccentricity oscillations in the BHB and induce its merger. These GR effects generate extra precessions on the BHB orbits and spins, significantly increasing the inclination window for mergers and producing a wide range of spin orientations when the BHB enters LIGO band. This “GR-enhanced” channel may play an important role in BHB mergers.
Constraining the primordial black hole abundance with 21-cm cosmology

The discoveries of a number of binary black hole mergers by LIGO and VIRGO has reinvigorated the interest that primordial black holes (PBHs) of tens of solar masses could contribute non-negligibly to the dark matter energy density. Should even a small population of PBHs with masses $\gtrsim \mathcal{O}(M_\odot)$ exist, they could profoundly impact the properties of the intergalactic medium and provide insight into novel processes at work in the early Universe. We demonstrate here that observations of the 21cm transition in neutral hydrogen during the epochs of reionization and cosmic dawn will likely provide one of the most stringent tests of solar mass PBHs. In the context of 21cm cosmology, PBHs give rise to three distinct observable effects: ${\textit{(i)}}$ the modification to the primordial power spectrum (and thus also the halo mass function) induced by Poisson noise, ${\textit{(ii)}}$ a uniform heating and ionization of the intergalactic medium via X-rays produced during accretion, and ${\textit{(iii)}}$ a local modification to the temperature and density of the ambient medium surrounding isolated PBHs. Using a four-parameter astrophysical model, we show that experiments like SKA and HERA could potentially improve upon existing constraints derived using observations of the cosmic microwave background by more than one order of magnitude.
Constraining Primordial Black Hole Abundance with the Galactic 511 keV Line.

Models in which dark matter consists entirely of primordial black holes (PBHs) with masses around 10^{17}  g are currently unconstrained. However, if PBHs are a component of the Galactic dark matter density, they will inject a large flux of energetic particles into the Galaxy as they radiate. Positrons produced by these black holes will subsequently propagate throughout the Galaxy and annihilate, contributing to the Galactic 511 keV line. Using measurements of this line by the INTEGRAL satellite as a constraint on PBH positron injection, we place new limits on PBH abundance in the mass range 10^{16}-10^{17}  g, ruling out models in which these PBHs constitute the entirety of dark matter.
Naturalness versus stringy naturalness with implications for collider and dark matter searches

The notion of stringy naturalness-- that an observable O_2 is more natural than O_1 if more (phenomenologically acceptable) vacua solutions lead to 
O_2 rather than O_1-- is examined within the context of the Standard Model (SM) and various SUSY extensions: CMSSM/mSUGRA, high-scale SUSY and radiatively-driven natural SUSY (RNS). Rather general arguments from string theory suggest a (possibly mild) statistical draw towards vacua with large soft SUSY breaking terms. These vacua must be tempered by an anthropic veto of non-standard vacua or vacua with too large a value of the weak scale m(weak). We argue that the SM, the CMSSM and the various high-scale SUSY models are all expected to be relatively rare occurances within the string theory landscape of vacua. In contrast, models with TeV-scale soft terms but with m(weak)~100 GeV and consequent light higgsinos (SUSY with radiatively-driven naturalness) should be much more common on the landscape. These latter models have a statistical preference for m_h~ 125 GeV and strongly interacting sparticles beyond current LHC reach. Thus, while conventional naturalness favors sparticles close to the weak scale, stringy naturalness favors sparticles so heavy that electroweak symmetry is barely broken and one is living dangerously close to vacua with charge-or-color breaking minima, no electroweak breaking or pocket universe weak scale values too far from our measured value. Expectations for how landscape SUSY would manifest itself at collider and dark matter search experiments are then modified compared to usual notions.
A Framework for Parallelizing OWL Classification in Description Logic Reasoners

The Web Ontology Language (OWL) is a widely used knowledge representation language for describing knowledge in application domains by using classes, properties, and individuals. Ontology classification is an important and widely used service that computes a taxonomy of all classes occurring in an ontology. It can require significant amounts of runtime, but most OWL reasoners do not support any kind of parallel processing.
This thesis reports on a black-box approach to parallelize existing description logic (DL) reasoners for theWeb Ontology Language. We focus on OWL ontology classification, which is an important inference service and supported by every major OWL/DL reasoner. To the best of our knowledge, we are the first to propose a flexible parallel framework which can be applied to existing OWL reasoners in order to speed up their classification process. There are two versions of our methods discussed: (i) the first version implements a novel thread-level parallel architecture with two parallel strategies to achieve a good speedup factor with an increasing number of threads, but does not rely on locking techniques and thus avoids possible race conditions. (ii) The improved version implements an improved data structure and various parallel computing techniques for precomputing and classification to reduce the overhead of processing ontologies and compete with other DL reasoners based on the wall clock time for classification.
In order to test the performance of both versions of our approaches, we use a real-world repository for choosing the tested ontologies. For the first version of our approach, we evaluated our prototype implementation with a set of selected real-world ontologies. Our experiments demonstrate very good scalability resulting in a speedup that is linear to the number of available cores. For the second version, its performance is evaluated by parallelizing major OWL reasoners for concept classification. Currently, we mainly focus on comparison with two popular DL reasoners: Hermit and JFact. In comparison to the selected black-box reasoners, our results demonstrate that the wall clock time of ontology classification can be improved by one order of the magnitude for most real-world ontologies in the repository.
On the Constrained Least-cost Tour Problem

We introduce the Constrained Least-cost Tour (CLT) problem: given an undirected graph with weight and cost functions on the edges, minimise the total cost of a tour rooted at a start vertex such that the total weight lies within a given range. CLT is related to the family of Travelling Salesman Problems with Profits, but differs by defining the weight function on edges instead of vertices, and by requiring the total weight to be within a range instead of being at least some quota. We prove CLT is $\mathcal{NP}$-hard, even in the simple case when the input graph is a path. We derive an informative lower bound by relaxing the integrality of edges and propose a heuristic motivated by this relaxation. For the case that requires the tour to be a simple cycle, we develop two heuristics which exploit Suurballe's algorithm to find low-cost, weight-feasible cycles. We demonstrate our algorithms by addressing a real-world problem that affects urban populations: finding routes that minimise air pollution exposure for walking, running and cycling in the city of London.
Predicting Evacuation Decisions using Representations of Individuals' Pre-Disaster Web Search Behavior

Predicting the evacuation decisions of individuals before the disaster strikes is crucial for planning first response strategies. In addition to the studies on post-disaster analysis of evacuation behavior, there are various works that attempt to predict the evacuation decisions beforehand. Most of these predictive methods, however, require real time location data for calibration, which are becoming much harder to obtain due to the rising privacy concerns. Meanwhile, web search queries of anonymous users have been collected by web companies. Although such data raise less privacy concerns, they have been under-utilized for various applications. In this study, we investigate whether web search data observed prior to the disaster can be used to predict the evacuation decisions. More specifically, we utilize a session-based query encoder that learns the representations of each user's web search behavior prior to evacuation. Our proposed approach is empirically tested using web search data collected from users affected by a major flood in Japan. Results are validated using location data collected from mobile phones of the same set of users as ground truth. We show that evacuation decisions can be accurately predicted (84%) using only the users' pre-disaster web search data as input. This study proposes an alternative method for evacuation prediction that does not require highly sensitive location data, which can assist local governments to prepare effective first response strategies.
On the interplay between noise and curvature and its effect on optimization and generalization

The speed at which one can minimize an expected loss using stochastic methods depends on two properties: the curvature of the loss and the variance of the gradients. While most previous works focus on one or the other of these properties, we explore how their interaction affects optimization speed. Further, as the ultimate goal is good generalization performance, we clarify how both curvature and noise are relevant to properly estimate the generalization gap. Realizing that the limitations of some existing works stems from a confusion between these matrices, we also clarify the distinction between the Fisher matrix, the Hessian, and the covariance matrix of the gradients.
Evaluating the Performance of Machine Learning Algorithms in Financial Market Forecasting: A Comprehensive Survey

With increasing competition and pace in the financial markets, robust forecasting methods are becoming more and more valuable to investors. While machine learning algorithms offer a proven way of modeling non-linearities in time series, their advantages against common stochastic models in the domain of financial market prediction are largely based on limited empirical results. The same holds true for determining advantages of certain machine learning architectures against others. This study surveys more than 150 related articles on applying machine learning to financial market forecasting. Based on a comprehensive literature review, we build a table across seven main parameters describing the experiments conducted in these studies. Through listing and classifying different algorithms, we also introduce a simple, standardized syntax for textually representing machine learning algorithms. Based on performance metrics gathered from papers included in the survey, we further conduct rank analyses to assess the comparative performance of different algorithm classes. Our analysis shows that machine learning algorithms tend to outperform most traditional stochastic methods in financial market forecasting. We further find evidence that, on average, recurrent neural networks outperform feed forward neural networks as well as support vector machines which implies the existence of exploitable temporal dependencies in financial time series across multiple asset classes and geographies.
Signatures from a merging galaxy cluster and its AGN population: LOFAR observations of Abell 1682

We present LOFAR data from 110–180 MHz of the merging galaxy cluster Abell 1682, alongside archival optical, radio, and X-ray data. Our images of 6 arcsec in resolution at low frequencies reveal new structures associated with numerous radio galaxies in the cluster. At a resolution of 20 arcsec we see diffuse emission throughout the cluster over hundreds of kiloparsecs, indicating particle acceleration mechanisms are in play as a result of the cluster merger event and powerful active galactic nuclei. We show that a significant part of the cluster emission is from an old radio galaxy with very steep spectrum emission (having a spectral index of α <  −2.5). Furthermore, we identify a new region of diffuse steep-spectrum emission (α <  −1.1) as a candidate for a radio halo which is co-spatial with the centre of the cluster merger. We suggest its origin as a population of old and mildly relativistic electrons left over from radio galaxies throughout the cluster which have been re-accelerated to higher energies by shocks and turbulence induced by the cluster merger event. We also note the discovery of six new giant radio galaxies in the vicinity of Abell 1682.
Directed Exploration for Reinforcement Learning

Efficient exploration is necessary to achieve good sample efficiency for reinforcement learning in general. From small, tabular settings such as gridworlds to large, continuous and sparse reward settings such as robotic object manipulation tasks, exploration through adding an uncertainty bonus to the reward function has been shown to be effective when the uncertainty is able to accurately drive exploration towards promising states. However reward bonuses can still be inefficient since they are non-stationary, which means that we must wait for function approximators to catch up and converge again when uncertainties change. We propose the idea of directed exploration, that is learning a goal-conditioned policy where goals are simply other states, and using that to directly try to reach states with large uncertainty. The goal-conditioned policy is independent of uncertainty and is thus stationary. We show in our experiments how directed exploration is more efficient at exploration and more robust to how the uncertainty is computed than adding bonuses to rewards.