nils-herrmann/clean_scirepeval huggingface dataset:
Talk2Nav: Long-Range Vision-and-Language Navigation in Cities

Autonomous driving models often consider the goal as fixed at the start of the ride. Yet, in practice, passengers will still want to influence the route, e.g. to pick up something along the way. In order to keep such inputs intuitive, we provide automatic way finding in cities based on verbal navigational instructions and street-view images. Our first contribution is the creation of a large-scale dataset with verbal navigation instructions. To this end, we have developed an interactive visual navigation environment based on Google Street View; we further design an annotation method to highlight mined anchor landmarks and local directions between them in order to help annotators formulate typical, human references to those. The annotation task was crowdsourced on the AMT platform, to construct a new Talk2Nav dataset with 10,714 routes. Our second contribution is a new learning method. Inspired by spatial cognition research on the mental conceptualization of navigational instructions, we introduce a soft attention mechanism defined over the segmented language instructions to jointly extract two partial instructions -- one for matching the next upcoming visual landmark and the other for matching the local directions to the next landmark. On the similar lines, we also introduce memory scheme to encode the local directional transitions. Our work takes advantage of the advance in two lines of research: mental formalization of verbal navigational instructions and training neural network agents for automatic way finding. Extensive experiments show that our method significantly outperforms previous navigation methods. For demo video, dataset and code, please refer to our \href{this https URL}{project page}.
Neural Turtle Graphics for Modeling City Road Layouts

We propose Neural Turtle Graphics (NTG), a novel generative model for spatial graphs, and demonstrate its applications in modeling city road layouts. Specifically, we represent the road layout using a graph where nodes in the graph represent control points and edges in the graph represents road segments. NTG is a sequential generative model parameterized by a neural network. It iteratively generates a new node and an edge connecting to an existing node conditioned on the current graph. We train NTG on Open Street Map data and show it outperforms existing approaches using a set of diverse performance metrics. Moreover, our method allows users to control styles of generated road layouts mimicking existing cities as well as to sketch a part of the city road layout to be synthesized. In addition to synthesis, the proposed NTG finds uses in an analytical task of aerial road parsing. Experimental results show that it achieves state-of-the-art performance on the SpaceNet dataset.
Can I Trust the Explainer? Verifying Post-hoc Explanatory Methods

For AI systems to garner widespread public acceptance, we must develop methods capable of explaining the decisions of black-box models such as neural networks. In this work, we identify two issues of current explanatory methods. First, we show that two prevalent perspectives on explanations --- feature-additivity and feature-selection --- lead to fundamentally different instance-wise explanations. In the literature, explainers from different perspectives are currently being directly compared, despite their distinct explanation goals. The second issue is that current post-hoc explainers are either validated under simplistic scenarios (on simple models such as linear regression, or on models trained on syntactic datasets), or, when applied to real-world neural networks, explainers are commonly validated under the assumption that the learned models behave reasonably. However, neural networks often rely on unreasonable correlations, even when producing correct decisions. We introduce a verification framework for explanatory methods under the feature-selection perspective. Our framework is based on a non-trivial neural network architecture trained on a real-world task, and for which we are able to provide guarantees on its inner workings. We validate the efficacy of our evaluation by showing the failure modes of current explainers. We aim for this framework to provide a publicly available, off-the-shelf evaluation when the feature-selection perspective on explanations is needed.
Contrastive Language Adaptation for Cross-Lingual Stance Detection

We study cross-lingual stance detection, which aims to leverage labeled data in one language to identify the relative perspective (or stance) of a given document with respect to a claim in a different target language. In particular, we introduce a novel contrastive language adaptation approach applied to memory networks, which ensures accurate alignment of stances in the source and target languages, and can effectively deal with the challenge of limited labeled data in the target language. The evaluation results on public benchmark datasets and comparison against current state-of-the-art approaches demonstrate the effectiveness of our approach.
"I’m Sorry Dave, I’m Afraid I Can’t Do That" Deep Q-Learning from Forbidden Actions

The use of Reinforcement Learning (RL) is still restricted to simulation or to enhance human-operated systems through recommendations. Real-world environments (e.g. industrial robots or power grids) are generally designed with safety constraints in mind implemented in the shape of valid actions masks or contingency controllers. For example, the range of motion and the angles of the motors of a robot can be limited to physical boundaries. Violating constraints thus results in rejected actions or entering in a safe mode driven by an external controller, making RL agents incapable of learning from their mistakes. In this paper, we propose a simple modification of a state-of-the-art deep RL algorithm (DQN), enabling learning from forbidden actions. To do so, the standard Q-learning update is enhanced with an extra safety loss inspired by structured classification. We empirically show that it reduces the number of hit constraints during the learning phase and accelerates convergence to near-optimal policies compared to using standard DQN. Experiments are done on a Visual Grid World Environment and the TextWorld domain.
Scalar self-force for high spin black holes

We semianalytically investigate the scalar self-force experienced in the final stages of extreme mass ratio inspirals of nonspinning scalar particles into supermassive nearly extremal Kerr black holes. We exploit the near-horizon conformal symmetry to find the self-force for general corotating equatorial geodesics. The angular component of the self-force is shown to be universal at leading order in the high spin limit. We verify that the energy and angular momentum losses of the scalar particle match with the asymptotic fluxes of scalar radiation. In particular, we relate the previously described persistent oscillations in the asymptotic energy and angular momentum fluxes with the local self-force. Such oscillations arise from traveling waves that prevent the near-horizon and the asymptotic region to fully decouple in the extremal limit. Conformal invariance is therefore reduced to discrete scale invariance with associated logarithmic periodicity.
An approach to constraining the Higgs width at the LHC and HL-LHC

Despite the discovery of the Higgs boson decay in five separate channels many parameters of the Higgs boson remain largely unconstrained. In this paper, we present a new approach to constraining the Higgs total width by requiring the Higgs to be resolved as a single high p$_T$ jet and measuring the inclusive Higgs boson cross section. To measure the inclusive Higgs boson cross section, we rely on new approaches from machine learning and a modified jet reconstruction. This approach is found to be complementary to the existing off-shell width measurement and, with the full HL-LHC luminosity, is capable of yielding similar sensitivity to the off-shell projections. We outline the theoretical and experimental limitations and present a path towards making this approach a truly model-independent measurement of the Higgs boson total width.
Detecting the halo heating from AGN feedback with ALMA

The Sunyaev–Zel’dovich (SZ) effect can potentially be used to investigate the heating of the circumgalactic medium and subsequent suppression of cold gas accretion on to the host galaxy caused by quasar feedback. We use a deep ALMA observation of HE0515-4414 in band 4, the most luminous quasar known at the peak of cosmic star formation (z = 1.7), to search for the SZ signal tracing the heating of the galaxy’s halo. ALMA’s sensitivity to a broad range of spatial scales enables us to disentangle emitting compact sources from the negative, extended SZ signal. We obtain a marginal SZ detection (∼3.3σ) on scales of about 300 kpc (30–40 arcsec), at the 0.2 mJy level, 0.5 mJy after applying a correction factor for primary beam attenuation and flux that is resolved out by the array. We show that our result is consistent with a simulated ALMA observation of a similar quasar in the fable cosmological simulations. We emphasize that detecting an SZ signal is more easily achieved in the visibility plane than in the (inferred) images. We also confirm a marginal detection (3.2σ) of a potential SZ dip on smaller scales (<100 kpc) already claimed by other authors, possibly highlighting the complex structure of the halo heating. Finally, we use SZ maps from the fable cosmological simulations, convolved with ALMA simulations, to illustrate that band 3 observations are much more effective in detecting the SZ signal with higher significance, and discuss the optimal observing strategy.
Nearly all Massive Quiescent Disk Galaxies Have a Surprisingly Large Atomic Gas Reservoir

The massive galaxy population above the characteristic Schechter mass M* ≈ 1010.6 contributes to about half of the total stellar mass in the local universe. These massive galaxies usually reside in hot dark matter halos above the critical shock-heating mass ∼1012 , where the external cold gas supply to these galaxies is expected to be suppressed. When galaxies run out of their cold gas reservoir, they become dead and quiescent. Therefore, massive quiescent galaxies living in hot halos are commonly believed to be gas-poor. Based on the data from SDSS, ALFALFA, GASS, and COLD GASS surveys, here we show that the vast majority of the massive, quiescent, central disk galaxies in the nearby universe have a remarkably large amount of cold atomic hydrogen gas, surprisingly similar to star-forming galaxies. Both star-forming and quiescent disk galaxies show identical symmetric double-horn H i spectra, indicating similar regularly rotating H i disks. Relative to their star-forming counterparts, massive quiescent central disk galaxies are quenched because of their significantly reduced molecular gas content, lower dust content, and lower star formation efficiency. Our findings reveal a new picture, which clearly demonstrates the detailed star formation quenching process in massive galaxies and provides a stringent constraint on the physical mechanism of quenching.
Adversarial Examples for Cost-Sensitive Classifiers

Motivated by safety-critical classification problems, we investigate adversarial attacks against cost-sensitive classifiers. We use current state-of-the-art adversarially-resistant neural network classifiers  as the underlying models. Cost-sensitive predictions are then achieved via a final processing step in the feed-forward evaluation of the network. We evaluate the effectiveness of cost-sensitive classifiers against a variety of attacks and we introduce a new cost-sensitive attack which performs better than targeted attacks in some cases. We also explored the measures a defender can take in order to limit their vulnerability to these attacks. This attacker/defender scenario is naturally framed as a two-player zero-sum finite game which we analyze using game theory.
Fused Gromov-Wasserstein Alignment for Hawkes Processes

We propose a novel fused Gromov-Wasserstein alignment method to jointly learn the Hawkes processes in different event spaces, and align their event types. Given two Hawkes processes, we use fused Gromov-Wasserstein discrepancy to measure their dissimilarity, which considers both the Wasserstein discrepancy based on their base intensities and the Gromov-Wasserstein discrepancy based on their infectivity matrices. Accordingly, the learned optimal transport reflects the correspondence between the event types of these two Hawkes processes. The Hawkes processes and their optimal transport are learned jointly via maximum likelihood estimation, with a fused Gromov-Wasserstein regularizer. Experimental results show that the proposed method works well on synthetic and real-world data.
Group-based Fair Learning Leads to Counter-intuitive Predictions

A number of machine learning (ML) methods have been proposed recently to maximize model predictive accuracy while enforcing notions of group parity or fairness across sub-populations. We propose a desirable property for these procedures, slack-consistency: For any individual, the predictions of the model should be monotonic with respect to allowed slack (i.e., maximum allowed group-parity violation). Such monotonicity can be useful for individuals to understand the impact of enforcing fairness on their predictions. Surprisingly, we find that standard ML methods for enforcing fairness violate this basic property. Moreover, this undesirable behavior arises in situations agnostic to the complexity of the underlying model or approximate optimizations, suggesting that the simple act of incorporating a constraint can lead to drastically unintended behavior in ML. We present a simple theoretical method for enforcing slack-consistency, while encouraging further discussions on the unintended behaviors potentially induced when enforcing group-based parity.
Discrete Processes and their Continuous Limits

The possibility that a discrete process can be fruitfully approximated by a continuous one, with the latter involving a differential system, is fascinating. Important theoretical insights, as well as significant computational efficiency gains may lie in store. A great success story in this regard are the Navier-Stokes equations, which model many phenomena in fluid flow rather well. Recent years saw many attempts to formulate more such continuous limits, and thus harvest theoretical and practical advantages, in diverse areas including mathematical biology, image processing, game theory, computational optimization, and machine learning. 
Caution must be applied as well, however. In fact, it is often the case that the given discrete process is richer in possibilities than its continuous differential system limit, and that a further study of the discrete process is practically rewarding. Furthermore, there are situations where the continuous limit process may provide important qualitative, but not quantitative, information about the actual discrete process. This paper considers several case studies of such continuous limits and demonstrates success as well as cause for caution. Consequences are discussed.
Social Learning in Multi Agent Multi Armed Bandits

Motivated by emerging need of learning algorithms for large scale networked and decentralized systems, we introduce a distributed version of the classical stochastic Multi-Arm Bandit (MAB) problem. Our setting consists of a large number of agents n that collaboratively and simultaneously solve the same instance of K armed MAB to minimize the average cumulative regret over all agents. The agents can communicate and collaborate among each other only through a pairwise asynchronous gossip based protocol that exchange a limited number of bits. In our model, agents at each point decide on (i) which arm to play, (ii) whether to, and if so (iii) what and whom to communicate with. Agents in our model are decentralized, namely their actions only depend on their observed history in the past. We develop a novel algorithm in which agents, whenever they choose, communicate only arm-ids and not samples, with another agent chosen uniformly and independently at random. The per-agent regret scaling achieved by our algorithm is $\BigO łeft( \fracłceil\fracK n \rceil+łog(n) Δ łog(T) + \fracłog^3(n) łog łog(n) Δ^2 \right) $. Furthermore, any agent in our algorithm communicates (arm-ids to an uniformly and independently chosen agent) only a total of Θ(łog(T))$ times over a time interval of T. We compare our results to two benchmarks - one where there is no communication among agents and one corresponding to complete interaction, where an agent has access to the entire system history of arms played and rewards obtained of all agents. We show both theoretically and empirically, that our algorithm experiences a significant reduction both in per-agent regret when compared to the case when agents do not collaborate and each agent is playing the standard MAB problem (where regret would scale linearly in K), and in communication complexity when compared to the full interaction setting which requires T communication attempts by an agent over T arm pulls. Our result thus demonstrates that even a minimal level of collaboration among the different agents enables a significant reduction in per-agent regret.
Health Wars and Beyond: The Rapidly Expanding and Efficient Network Insurgency Interlinking Local and Global Online Crowds of Distrust

We present preliminary results on the online war surrounding distrust of expertise in medical science -- specifically, the issue of vaccinations. While distrust and misinformation in politics can damage democratic elections, in the medical context it may also endanger lives through missed vaccinations and DIY cancer cures. We find that this online health war has evolved into a highly efficient network insurgency with direct inter-crowd links across countries, continents and cultures. The online anti-vax crowds (referred to as Red) now appear better positioned to groom new recruits (Green) than those supporting established expertise (Blue). We also present preliminary results from a mathematically-grounded, crowd-based analysis of the war's evolution, which offers an explanation for how Red seems to be turning the tide on Blue.
Developing biomarker combinations in multicenter studies via direct maximization and penalization

Motivated by a study of acute kidney injury, we consider the setting of biomarker studies involving patients at multiple centers where the goal is to develop a biomarker combination for diagnosis, prognosis, or screening. As biomarker studies become larger, this type of data structure will be encountered more frequently. In the presence of multiple centers, one way to assess the predictive capacity of a given combination is to consider the center‐adjusted area under the receiver operating characteristic curve (aAUC), a summary of the ability of the combination to discriminate between cases and controls in each center. Rather than using a general method, such as logistic regression, to construct the biomarker combination, we propose directly maximizing the aAUC. Furthermore, it may be desirable to have a biomarker combination with similar performance across centers. To that end, we allow for penalization of the variability in the center‐specific AUCs. We demonstrate desirable asymptotic properties of the resulting combinations. Simulations provide small‐sample evidence that maximizing the aAUC can lead to combinations with improved performance. We also use simulated data to illustrate the utility of constructing combinations by maximizing the aAUC while penalizing variability. Finally, we apply these methods to data from the study of acute kidney injury.
GENN: Predicting Correlated Drug-drug Interactions with Graph Energy Neural Networks

Gaining more comprehensive knowledge about drug-drug interactions (DDIs) is one of the most important tasks in drug development and medical practice. Recently graph neural networks have achieved great success in this task by modeling drugs as nodes and drug-drug interactions as links and casting DDI predictions as link prediction problems. However, correlations between link labels (e.g., DDI types) were rarely considered in existing works. We propose the graph energy neural network (GENN) to explicitly model link type correlations. We formulate the DDI prediction task as a structure prediction problem and introduce a new energy-based model where the energy function is defined by graph neural networks. Experiments on two real-world DDI datasets demonstrated that GENN is superior to many baselines without consideration of link type correlations and achieved $13.77\%$ and $5.01\%$ PR-AUC improvement on the two datasets, respectively. We also present a case study in which \mname can better capture meaningful DDI correlations compared with baseline models.
Confederated Machine Learning on Horizontally and Vertically Separated Medical Data for Large-Scale Health System Intelligence

Health information is generally fragmented across silos. Though it is technically feasible to unite data for analysis in a manner that underpins a rapid learning healthcare system, privacy concerns and regulatory barriers limit data centralization. Machine learning can be conducted in a federated manner on patient datasets with the same set of variables, but separated across sites of care. But federated learning cannot handle the situation where different data types for a given patient are separated vertically across different organizations and when patient ID matching across different institutions is difficult. We call methods that enable machine learning model training on data separated by two or more degrees confederated machine learning. We proposed and evaluated a confederated learning to training machine learning model to stratify the risk of several diseases among when data are horizontally separated by individual, vertically separated by data type, and separated by identity without patient ID matching.
On the relation between transversal and longitudinal scaling in cities

Does the scaling relationship between population sizes of cities with urban metrics like economic output and infrastructure (transversal scaling) mirror the evolution of individual cities in time (longitudinal scaling)? The answer to this question has important policy implications, but the lack of suitable data has so far hindered rigorous empirical tests. In this paper, we advance the debate by looking at the evolution of two urban variables, GDP and water network length, for over 5500 cities in Brazil. We find that longitudinal scaling exponents are city-specific. However, they are distributed around an average value that approaches the transversal scaling exponent provided that the data is decomposed to eliminate external factors, and only for cities with a sufficiently high growth rate. We also introduce a mathematical framework that connects the microscopic level to global behaviour, finding good agreement between theoretical predictions and empirical evidence in all analyzed cases. Our results add complexity to the idea that the longitudinal dynamics is a micro-scaling version of the transversal dynamics of the entire urban system. The longitudinal analysis can reveal differences in scaling behavior related to population size and nature of urban variables. Our approach also makes room for the role of external factors such as public policies and development, and opens up new possibilities in the research of the effects of scaling and contextual factors.