nils-herrmann/clean_scirepeval huggingface dataset:
Be Consistent! Improving Procedural Text Comprehension using Label Consistency

Our goal is procedural text comprehension, namely tracking how the properties of entities (e.g., their location) change with time given a procedural text (e.g., a paragraph about photosynthesis, a recipe). This task is challenging as the world is changing throughout the text, and despite recent advances, current systems still struggle with this task. Our approach is to leverage the fact that, for many procedural texts, multiple independent descriptions are readily available, and that predictions from them should be consistent (label consistency). We present a new learning framework that leverages label consistency during training, allowing consistency bias to be built into the model. Evaluation on a standard benchmark dataset for procedural text, ProPara (Dalvi et al., 2018), shows that our approach significantly improves prediction performance (F1) over prior state-of-the-art systems.
Rules of the Road: Predicting Driving Behavior With a Convolutional Model of Semantic Interactions

We focus on the problem of predicting future states of entities in complex, real-world driving scenarios. Previous research has approached this problem via low-level signals to predict short time horizons, and has not addressed how to leverage key assets relied upon heavily by industry self-driving systems: (1) large 3D perception efforts which provide highly accurate 3D states of agents with rich attributes, and (2) detailed and accurate semantic maps of the environment (lanes, traffic lights, crosswalks, etc). We present a unified representation which encodes such high-level semantic information in a spatial grid, allowing the use of deep convolutional models to fuse complex scene context. This enables learning entity-entity and entity-environment interactions with simple, feed-forward computations in each timestep within an overall temporal model of an agent's behavior. We propose different ways of modelling the future as a {\em distribution} over future states using standard supervised learning. We introduce a novel dataset providing industry-grade rich perception and semantic inputs, and empirically show we can effectively learn fundamentals of driving behavior.
Deep Mixture Point Processes: Spatio-temporal Event Prediction with Rich Contextual Information

Predicting when and where events will occur in cities, like taxi pick-ups, crimes, and vehicle collisions, is a challenging and important problem with many applications in fields such as urban planning, transportation optimization and location-based marketing. Though many point processes have been proposed to model events in a continuous spatio-temporal space, none of them allow for the consideration of the rich contextual factors that affect event occurrence, such as weather, social activities, geographical characteristics, and traffic. In this paper, we propose DMPP (Deep Mixture Point Processes), a point process model for predicting spatio-temporal events with the use of rich contextual information; a key advance is its incorporation of the heterogeneous and high-dimensional context available in image and text data. Specifically, we design the intensity of our point process model as a mixture of kernels, where the mixture weights are modeled by a deep neural network. This formulation allows us to automatically learn the complex nonlinear effects of the contextual factors on event occurrence. At the same time, this formulation makes analytical integration over the intensity, which is required for point process estimation, tractable. We use real-world data sets from different domains to demonstrate that DMPP has better predictive performance than existing methods.
Demonstration of a Neural Machine Translation System with Online Learning for Translators

We present a demonstration of our system, which implements online learning for neural machine translation in a production environment. These techniques allow the system to continuously learn from the corrections provided by the translators. We implemented an end-to-end platform integrating our machine translation servers to one of the most common user interfaces for professional translators: SDL Trados Studio. We pretend to save post-editing effort as the machine is continuously learning from its mistakes and adapting the models to a specific domain or user style.
Connectivity-Optimized Representation Learning via Persistent Homology

We study the problem of learning representations with controllable connectivity properties. This is beneficial in situations when the imposed structure can be leveraged upstream. In particular, we control the connectivity of an autoencoder's latent space via a novel type of loss, operating on information from persistent homology. Under mild conditions, this loss is differentiable and we present a theoretical analysis of the properties induced by the loss. We choose one-class learning as our upstream task and demonstrate that the imposed structure enables informed parameter selection for modeling the in-class distribution via kernel density estimators. Evaluated on computer vision data, these one-class models exhibit competitive performance and, in a low sample size regime, outperform other methods by a large margin. Notably, our results indicate that a single autoencoder, trained on auxiliary (unlabeled) data, yields a mapping into latent space that can be reused across datasets for one-class learning.
Backpropagation-Friendly Eigendecomposition

Eigendecomposition (ED) is widely used in deep networks. However, the backpropagation of its results tends to be numerically unstable, whether using ED directly or approximating it with the Power Iteration method, particularly when dealing with large matrices. While this can be mitigated by partitioning the data in small and arbitrary groups, doing so has no theoretical basis and makes its impossible to exploit the power of ED to the full. In this paper, we introduce a numerically stable and differentiable approach to leveraging eigenvectors in deep networks. It can handle large matrices without requiring to split them. We demonstrate the better robustness of our approach over standard ED and PI for ZCA whitening, an alternative to batch normalization, and for PCA denoising, which we introduce as a new normalization strategy for deep networks, aiming to further denoise the network's features.
BERT-based Financial Sentiment Index and LSTM-based Stock Return Predictability

Traditional sentiment construction in finance relies heavily on the dictionary-based approach, with a few exceptions using simple machine learning techniques such as Naive Bayes classifier. While the current literature has not yet invoked the rapid advancement in the natural language processing, we construct in this research a textual-based sentiment index using a novel model BERT recently developed by Google, especially for three actively trading individual stocks in Hong Kong market with hot discussion on Weibo.com. On the one hand, we demonstrate a significant enhancement of applying BERT in sentiment analysis when compared with existing models. On the other hand, by combining with the other two existing methods commonly used on building the sentiment index in the financial literature, i.e., option-implied and market-implied approaches, we propose a more general and comprehensive framework for financial sentiment analysis, and further provide convincing outcomes for the predictability of individual stock return for the above three stocks using LSTM (with a feature of a nonlinear mapping), in contrast to the dominating econometric methods in sentiment influence analysis that are all of a nature of linear regression.
Popularity Prediction on Social Platforms with Coupled Graph Neural Networks

Predicting the popularity of online content on social platforms is an important task for both researchers and practitioners. Previous methods mainly leverage demographics, temporal and structural patterns of early adopters for popularity prediction. However, most existing methods are less effective to precisely capture the cascading effect in information diffusion, in which early adopters try to activate potential users along the underlying network. In this paper, we consider the problem of network-aware popularity prediction, leveraging both early adopters and social networks for popularity prediction. We propose to capture the cascading effect explicitly, modeling the activation state of a target user given the activation state and influence of his/her neighbors. To achieve this goal, we propose a novel method, namely CoupledGNN, which uses two coupled graph neural networks to capture the interplay between node activation states and the spread of influence. By stacking graph neural network layers, our proposed method naturally captures the cascading effect along the network in a successive manner. Experiments conducted on both synthetic and real-world Sina Weibo datasets demonstrate that our method significantly outperforms the state-of-the-art methods for popularity prediction.
Small-time and large-time smile behaviour for the Rough Heston model

We characterize the asymptotic small-time and large-time implied volatility smile for the rough Heston model introduced by El Euch, Jaisson and Rosenbaum. We show that the asymptotic short-maturity smile scales in qualitatively the same way as a general rough stochastic volatility model, and is characterized by the Fenchel-Legendre transform of the solution a Volterra integral equation (VIE). The solution of this VIE satisfies a space-time scaling property which simplifies its computation. We corroborate our results numerically with Monte Carlo simulations. We also compute a power series in the log-moneyness variable for the asymptotic implied volatility, which yields tractable expressions for the vol skew and convexity, thus being useful for calibration purposes. We also derive formal asymptotics for the small-time moderate deviations regime and a formal saddlepoint approximation for call options in the large deviations regime. This goes to higher order than previous works for rough models, and in particular captures the effect of the mean reversion term. In the large maturity case, the limiting asymptotic smile turns out to be the same as for the standard Heston model, for which there is a well known closed-form formula in terms of the SVI parametrization.
Simplex2Vec embeddings for community detection in simplicial complexes

Topological representations are rapidly becoming a popular way to capture and encode higher-order interactions in complex systems. They have found applications in disciplines as different as cancer genomics, brain function, and computational social science, in representing both descriptive features of data and inference models. While intense research has focused on the connectivity and homological features of topological representations, surprisingly scarce attention has been given to the investigation of the community structures of simplicial complexes. To this end, we adopt recent advances in symbolic embeddings to compute and visualize the community structures of simplicial complexes. We first investigate the stability properties of embedding obtained for synthetic simplicial complexes to the presence of higher order interactions. We then focus on complexes arising from social and brain functional data and show how higher order interactions can be leveraged to improve clustering detection and assess the effect of higher order interaction on individual nodes. We conclude delineating limitations and directions for extension of this work.
Inside the Echo Chamber: Disentangling network dynamics from polarization

Echo chambers are defined by the simultaneous presence of opinion polarization with respect to a controversial topic and homophily, i.e. the preference of individuals to interact with like-minded peers. While recent efforts have been devoted to detecting the presence of echo chambers in polarized debates on online social media, the dynamics leading to the emergence of these phenomena remain unclear. Here, we contribute to this endeavor by proposing novel metrics to single out the effect of the network dynamics from the opinion polarization. By using a Twitter data set collected during a controversial political debate in Brazil in 2016, we employ a temporal network approach to gauge the strength of the echo chamber effect over time. We define a measure of opinion coherence in the network showing how the echo chamber becomes weaker across the observed period. The analysis of the hashtags diffusion in the network shows that this is due to the increase of social interactions between users with opposite opinions. Finally, the analysis of the mutual entropy between the opinions expressed and received by the users permits to quantify the social contagion effect. We find empirical evidence that the polarization of the users and the dynamics of their interactions may evolve independently. Our findings may be of interest to the broad array of researchers studying the dynamics of echo chambers and polarization in online social networks.
The Impact of Projection and Backboning on Network Topologies

Bipartite networks are a well known strategy to study a variety of phenomena. The commonly used method to deal with this type of network is to project the bipartite data into a unipartite weighted graph and then using a backboning technique to extract only the meaningful edges. Despite the wide availability of different methods both for projection and backboning, we believe that there has been little attention to the effect that the combination of these two processes has on the data and on the resulting network topology. In this paper we study the effect that the possible combinations of projection and backboning techniques have on a bipartite network. We show that the 12 methods group into two clusters producing unipartite networks with very different topologies. We also show that the resulting level of network centralization is highly affected by the combination of projection and backboning applied.
Community Detection in the Hyperbolic Space

Embedding a network in hyperbolic space can reveal interesting features for the network structure, especially in terms of self-similar characteristics. The hidden metric space, which can be thought of as the underlying structure of the network, is able to preserve some interesting features generally observed in real-world networks such as heterogeneity in the degree distribution, high clustering coefficient, and small-world effect. Moreover, the angular distribution of the nodes in the hyperbolic plane reveals a community structure of the embedded network. It is worth noting that, while a large body of literature compares well-known community detection algorithms, there is still no consensus on what defines an ideal community partition on a network. Moreover, heuristics for communities found on networks embedded in the hyperbolic space have been investigated here for the first time. We compare the partitions found on embedded networks to the partitions obtained before the embedding step, both for a synthetic network and for two real-world networks. The second part of this paper presents the application of our pipeline to a network of retweets in the context of the Italian elections. Our results uncover a community structure reflective of the political spectrum, encouraging further research on the application of community detection heuristics to graphs mapped onto hyperbolic planes.
Evaluating the impact of PrEP on HIV and gonorrhea on a networked population of female sex workers

Sexual contacts are the main spreading route of HIV. This puts sex workers at higher risk of infection even in populations where HIV prevalence is moderate or low. Alongside condom use, Pre-Exposure Prophylaxis (PrEP) is an effective tool for sex workers to reduce their risk of HIV acquisition. However, PrEP provides no direct protection against sexually transmitted infections (STIs) other than HIV, unlike condoms. We use an empirical network of sexual contacts among female sex workers (FSWs) and clients to simulate the spread of HIV and gonorrhea. We then investigate the effect of PrEP adoption and adherence, on both HIV and gonorrhea prevalence. We also study the effect of a potential increase in condomless acts due to lowered risk perception with respect of the no-PrEP scenario (risk compensation). We find that when HIV is the only disease circulating, PrEP is effective in reducing HIV prevalence, even with high risk compensation. Instead, the complex interplay between the two diseases shows that different levels of risk compensation require different intervention strategies. Finally, we find that providing PrEP only to the most active FSWs is less effective than uniform PrEP adoption. Our work shows that the effects emerging from the complex interactions between these diseases and the available prophylactic measures need to be accounted for, to devise effective intervention strategies.
Hybrid Planning for Dynamic Multimodal Stochastic Shortest Paths

Sequential decision problems in applications such as manipulation in warehouses, multi-step meal preparation, and routing in autonomous vehicle networks often involve reasoning about uncertainty, planning over discrete modes as well as continuous states, and reacting to dynamic updates. To formalize such problems generally, we introduce a class of Markov Decision Processes (MDPs) called Dynamic Multimodal Stochastic Shortest Paths (DMSSPs). Much of the work in these domains solves deterministic variants, which can yield poor results when the uncertainty has downstream effects. We develop a Hybrid Stochastic Planning (HSP) algorithm, which uses domain-agnostic abstractions to efficiently unify heuristic search for planning over discrete modes, approximate dynamic programming for stochastic planning over continuous states, and hierarchical interleaved planning and execution. In the domain of autonomous multimodal routing, HSP obtains significantly higher quality solutions than a state-of-the-art Upper Confidence Trees algorithm and a two-level Receding Horizon Control algorithm.
Near-optimal Reinforcement Learning using Bayesian Quantiles

We study model-based reinforcement learning in finite communicating Markov Decision Process. Algorithms in this settings have been developed in two different ways: the first view, which typically provides frequentist performance guarantees, uses optimism in the face of uncertainty as the guiding algorithmic principle. The second view is based on Bayesian reasoning, combined with posterior sampling and Bayesian guarantees. In this paper, we develop a conceptually simple algorithm, Bayes-UCRL that combines the benefits of both approaches to achieve state-of-the-art performance for finite communicating MDP. In particular, we use Bayesian Prior similarly to Posterior Sampling. However, instead of sampling the MDP, we construct an optimistic MDP using the quantiles of the Bayesian prior. We show that this technique enjoys a high probability worst-case regret of order $\tilde{\mathcal{O}}(\sqrt{DSAT})$. Experiments in a diverse set of environments show that our algorithms outperform previous methods.
Categorizing Wireheading in Partially Embedded Agents

$\textit{Embedded agents}$ are not explicitly separated from their environment, lacking clear I/O channels. Such agents can reason about and modify their internal parts, which they are incentivized to shortcut or $\textit{wirehead}$ in order to achieve the maximal reward. In this paper, we provide a taxonomy of ways by which wireheading can occur, followed by a definition of wirehead-vulnerable agents. Starting from the fully dualistic universal agent AIXI, we introduce a spectrum of partially embedded agents and identify wireheading opportunities that such agents can exploit, experimentally demonstrating the results with the GRL simulation platform AIXIjs. We contextualize wireheading in the broader class of all misalignment problems - where the goals of the agent conflict with the goals of the human designer - and conjecture that the only other possible type of misalignment is specification gaming. Motivated by this taxonomy, we define wirehead-vulnerable agents as embedded agents that choose to behave differently from fully dualistic agents lacking access to their internal parts.
A proof of Shelah's eventual categoricity conjecture and an extension to accessible categories with directed colimits

We provide a proof, in ZFC , of Shelah’s eventual categoricity conjecture for abstract elementary classes (AEC’s). Moreover, assuming in addition the Singular Cardinal Hypothesis ( SCH ), we prove a direct generalization to the more general context of accessible categories with directed colimits. If K is such a category, we show that there is a cardinal µ such that if K is λ -categorical for some λ ≥ µ (i.e., it has only one object of internal size λ up to iso-morphism), then K is eventually categorical (i.e., it is λ ′ -categorical for every λ ′ ≥ µ ). When considering cardinalities of models of inﬁnitary theories T of L κ,θ that axiomatize K , the result implies, under SCH , the following inﬁnitary version of Morley’s categoricity theorem: let S be the class of cardinals λ which are of coﬁnality at least θ but are not successors of cardinals of coﬁnality less than θ . Then, if T is a L κ,θ theory whose models have directed colimits and it is λ -categorical for some λ ≥ µ in S , then it is λ ′ -categorical for every λ ′ ≥ µ in S ; moreover, we also exhibit an example that shows that the exceptions in the class S are needed. Along the way we also prove Grossberg conjecture, according to which categoricity in a high enough cardinal implies eventual amalgamation. We establish this result in AEC’s and, assuming in addition SCH , in the more general context of accessible categories whose morphisms are monomorphisms.
Explainable Fact Checking with Probabilistic Answer Set Programming

One challenge in fact checking is the ability to improve the transparency of the decision. We present a fact checking method that uses reference information in knowledge graphs (KGs) to assess claims and explain its decisions. KGs contain a formal representation of knowledge with semantic descriptions of entities and their relationships. We exploit such rich semantics to produce interpretable explanations for the fact checking output. As information in a KG is inevitably incomplete, we rely on logical rule discovery and on Web text mining to gather the evidence to assess a given claim. Uncertain rules and facts are turned into logical programs and the checking task is modeled as an inference problem in a probabilistic extension of answer set programs. Experiments show that the probabilistic inference enables the efficient labeling of claims with interpretable explanations, and the quality of the results is higher than state of the art baselines.