Miracle-dz/UniFAR-FTUs huggingface dataset:
NeuroCartography: Scalable Automatic Visual Summarization of Concepts in Deep Neural Networks

Existing research on making sense of deep neural networks often focuses on neuron-level interpretation, which may not adequately capture the bigger picture of how concepts are collectively encoded by multiple neurons. We present NEUROCARTOGRAPHY, an interactive system that scalably summarizes and visualizes concepts learned by neural networks. It automatically discovers and groups neurons that detect the same concepts, and describes how such neuron groups interact to form higher-level concepts and the subsequent predictions. NEUROCARTOGRAPHY introduces two scalable summarization techniques: (1) neuron clustering groups neurons based on the semantic similarity of the concepts detected by neurons (e.g., neurons detecting “dog faces” of different breeds are grouped); and (2) neuron embedding encodes the associations between related concepts based on how often they co-occur (e.g., neurons detecting “dog face” and “dog tail” are placed closer in the embedding space). Key to our scalable techniques is the ability to efficiently compute all neuron pairs' relationships, in time linear to the number of neurons instead of quadratic time. NEUROCARTOGRAPHY scales to large data, such as the ImageNet dataset with 1.2M images. The system's tightly coordinated views integrate the scalable techniques to visualize the concepts and their relationships, projecting the concept associations to a 2D space in Neuron Projection View, and summarizing neuron clusters and their relationships in Graph View. Through a large-scale human evaluation, we demonstrate that our technique discovers neuron groups that represent coherent, human-meaningful concepts. And through usage scenarios, we describe how our approaches enable interesting and surprising discoveries, such as concept cascades of related and isolated concepts. The NEUROCARTOGRAPHY visualization runs in modern browsers and is open-sourced.
Artin-Schreier extensions and combinatorial complexity in henselian valued fields

We give explicit formulas witnessing IP, \IPn or TP2 in fields with Artin-Schreier extensions. We use them to control $p$-extensions of mixed characteristic henselian valued fields, allowing us most notably to generalize to the \NIPn context one way of Anscombe-Jahnke's classification of NIP henselian valued fields. As a corollary, we obtain that \NIPn henselian valued fields with NIP residue field are NIP. We also discuss tameness results for NTP2 henselian valued fields.
Tight Guarantees for Static Threshold Policies in the Prophet Secretary Problem

In theprophet secretary problem, n values are drawn independently from known distributions, and presented in a uniformly random order. A decision-maker must accept or reject each value when it is presented, and may accept at most k values in total. The objective is to maximize the expected sum of accepted values. We analyze the performance of static threshold policies, which accept the first k values exceeding a fixed threshold (or all such values, if fewer than k exist). We show that using an appropriately-defined threshold guarantees a γk = 1 - e-k kk/k! fraction of the offline optimal solution, which equals 1-1/e when k=1 and grows asymptotically as 1-Θ(1/√k). This represents thebest-known guarantee for the prophet secretary problem for all k>1, and we also show that it istight for all k if one is restricted to the class of static threshold policies. We also two provide distinct methods for setting the threshold, which make them implementable under different restrictions on what is known about the distributions. Our first method requires computing a threshold such that the expected number of values exceeding the threshold is equal to k, and offers the guarantee of γk if k ≥ 5 (but interestingly, not if k<5). Our second method requires computing a threshold such that k · γk values are accepted in expectation, and offers the guarantee of γk for all k. To establish these guarantees, we prove a new result in Bernoulli optimization about sums of independent Bernoulli random variables, which extends a classical result of Hoeffding (1956) and could be of general interest.
Multiple imputation and test‐wise deletion for causal discovery with incomplete cohort data

Causal discovery algorithms estimate causal graphs from observational data. This can provide a valuable complement to analyses focusing on the causal relation between individual treatment‐outcome pairs. Constraint‐based causal discovery algorithms rely on conditional independence testing when building the graph. Until recently, these algorithms have been unable to handle missing values. In this article, we investigate two alternative solutions: test‐wise deletion and multiple imputation. We establish necessary and sufficient conditions for the recoverability of causal structures under test‐wise deletion, and argue that multiple imputation is more challenging in the context of causal discovery than for estimation. We conduct an extensive comparison by simulating from benchmark causal graphs: as one might expect, we find that test‐wise deletion and multiple imputation both clearly outperform list‐wise deletion and single imputation. Crucially, our results further suggest that multiple imputation is especially useful in settings with a small number of either Gaussian or discrete variables, but when the dataset contains a mix of both neither method is uniformly best. The methods we compare include random forest imputation and a hybrid procedure combining test‐wise deletion and multiple imputation. An application to data from the IDEFICS cohort study on diet‐ and lifestyle‐related diseases in European children serves as an illustrating example.
Automatic Text Evaluation through the Lens of Wasserstein Barycenters

A new metric BaryScore to evaluate text generation based on deep contextualized embeddings (e.g., BERT, Roberta, ELMo) is introduced. This metric is motivated by a new framework relying on optimal transport tools, i.e., Wasserstein distance and barycenter. By modelling the layer output of deep contextualized embeddings as a probability distribution rather than by a vector embedding; this framework provides a natural way to aggregate the different outputs through the Wasserstein space topology. In addition, it provides theoretical grounds to our metric and offers an alternative to available solutions (e.g., MoverScore and BertScore). Numerical evaluation is performed on four different tasks: machine translation, summarization, data2text generation and image captioning. Our results show that BaryScore outperforms other BERT based metrics and exhibits more consistent behaviour in particular for text summarization.
Convergence Rates for Learning Linear Operators from Noisy Data

This paper studies the learning of linear operators between infinite-dimensional Hilbert spaces. The training data comprises pairs of random input vectors in a Hilbert space and their noisy images under an unknown self-adjoint linear operator. Assuming that the operator is diagonalizable in a known basis, this work solves the equivalent inverse problem of estimating the operator's eigenvalues given the data. Adopting a Bayesian approach, the theoretical analysis establishes posterior contraction rates in the infinite data limit with Gaussian priors that are not directly linked to the forward map of the inverse problem. The main results also include learning-theoretic generalization error guarantees for a wide range of distribution shifts. These convergence rates quantify the effects of data smoothness and true eigenvalue decay or growth, for compact or unbounded operators, respectively, on sample complexity. Numerical evidence supports the theory in diagonal and non-diagonal settings.
Whole Brain Vessel Graphs: A Dataset and Benchmark for Graph Learning and Neuroscience (VesselGraph)

Biological neural networks define the brain function and intelligence of humans and other mammals, and form ultra-large, spatial, structured graphs. Their neuronal organization is closely interconnected with the spatial organization of the brain's microvasculature, which supplies oxygen to the neurons and builds a complementary spatial graph. This vasculature (or the vessel structure) plays an important role in neuroscience; for example, the organization of (and changes to) vessel structure can represent early signs of various pathologies, e.g. Alzheimer's disease or stroke. Recently, advances in tissue clearing have enabled whole brain imaging and segmentation of the entirety of the mouse brain's vasculature. Building on these advances in imaging, we are presenting an extendable dataset of whole-brain vessel graphs based on specific imaging protocols. Specifically, we extract vascular graphs using a refined graph extraction scheme leveraging the volume rendering engine Voreen and provide them in an accessible and adaptable form through the OGB and PyTorch Geometric dataloaders. Moreover, we benchmark numerous state-of-the-art graph learning algorithms on the biologically relevant tasks of vessel prediction and vessel classification using the introduced vessel graph dataset. Our work paves a path towards advancing graph learning research into the field of neuroscience. Complementarily, the presented dataset raises challenging graph learning research questions for the machine learning community, in terms of incorporating biological priors into learning algorithms, or in scaling these algorithms to handle sparse,spatial graphs with millions of nodes and edges. All datasets and code are available for download at https://github.com/jocpae/VesselGraph .
Transport-based Counterfactual Models

Counterfactual frameworks have grown popular in machine learning for both explaining algorithmic decisions but also deﬁning individual notions of fairness, more intuitive than typical group fairness conditions. However, state-of-the-art models to compute counterfactuals are either unrealistic or unfeasible. In particular, while Pearl’s causal inference provides appealing rules to calculate counterfactuals, it relies on a model that is unknown and hard to discover in practice. We address the problem of designing realistic and feasible counterfactuals in the absence of a causal model. We deﬁne transport-based counterfactual models as collections of joint probability distributions between observable distributions, and show their connection to causal counterfactuals. More speciﬁcally, we argue that optimal-transport theory deﬁnes relevant transport-based counterfactual models, as they are numerically feasible, statistically-faithful, and can coincide under some assumptions with causal counterfactual models. Finally, these models make counterfactual approaches to fairness feasible, and we illustrate their practicality and eﬃciency on fair learning. With this paper, we aim at laying out the theoretical foundations for a new, implementable approach to counterfactual thinking.
WALNUT: A Benchmark on Weakly Supervised Learning for Natural Language Understanding

Building machine learning models for natural language understanding (NLU) tasks relies heavily on labeled data. Weak supervision has been shown to provide valuable supervision when large amount of labeled data is unavailable or expensive to obtain. Existing works studying weak supervision for NLU either mostly focus on a speciﬁc task or simulate weak supervision signals from ground-truth labels. To date a benchmark for NLU with real world weak supervision signals for a collection of NLU tasks is still not available. In this paper, we propose such a benchmark, named WALNUT 1 , to advocate and facilitate research on weak supervision for NLU. WALNUT consists of NLU tasks with different types, including both document-level prediction tasks and token-level prediction tasks and for each task contains weak labels generated by multiple real-world weak sources. We conduct baseline evaluations on the benchmark to systematically test the value of weak supervision for NLU tasks, with various weak supervision methods and model architectures. We demonstrate the beneﬁts of weak supervision for low-resource NLU tasks and expect WALNUT to stimulate further research on methodologies to best leverage weak supervision. The benchmark and code for baselines will be publicly available at aka.ms/walnut_benchmark .
Graphene aerogel supported platinum nanoparticles for formic acid electro-oxidation

The performance of the direct formic acid fuel cell largely depends on the quality of the electrocatalysts used in the fuel cells. New materials are of great interest in order to improve the activity and stability of the electrocatalyst used in formic acid electro-oxidation for better direct formic acid fuel cell performance. In this study, different support materials including synthesized graphene aerogel (GA), commercial carbon black (C) or their hybrids(GA/C) were used. Platinum (Pt) nanoparticles were decorated on the defect sites of these materials by using microwave irradiation in order to improve the catalytic activity for formic acid electro-oxidation in acidic media. GA/C hybrid supports are synthesized via simple liquid mechanical mixing approach (GA/C(1)) and through hydrothermal process (GA/C(2)). Synthesized catalysts were called Pt/C, Pt/GA and Pt/GA/C(1) and Pt/GA/C(2). The as-prepared catalysts were characterized with SEM, TEM, XRD and ICP-MS. The SEM results show that GA has a 3D macroporous structure. Electrochemical measurements tests showed that either GA or GA/C material is promising support materials for Pt nanoparticles towards formic acid electro-oxidaiton. The electrochemical test results revealed that the support material affect the activity. Synthesized GA supported catalysts gave better activity than synthesized commercial carbon supported catalyst for oxidation of formic acid. Pt/GA and Pt/GA/C(2) catalysts showed higher tolerance against the poisoning effect of CO and direct pathway dominated the electro-oxidation of formic acid.
Unidirectional substitutes and complements

In discrete matching markets, substitutes and complements can be unidirectional between two groups of workers when members of one group are more important or competent than those of the other group for firms. We show that a stable matching exists and can be found by a two-stage Deferred Acceptance mechanism when firms’ preferences satisfy a unidirectional substitutes and complements condition. This result applies to both firm-worker matching and controlled school choice. Under the framework of matching with continuous monetary transfers and quasi-linear utilities, we show that substitutes and complements are bidirectional for a pair of workers.
Hierarchical Reinforcement Learning for Sensor-Based Navigation

—Robotic systems are nowadays capable of solving complex navigation tasks under real-world conditions. However, their capabilities are intrinsically limited to the imagination of the designer and consequently lack generalizability to initially unconsidered situations. This makes deep reinforcement learning especially interesting, as these algorithms promise a self-learning system only relying on feedback from the environment. Having the system itself search for an optimal solution brings the beneﬁt of great generalization or even constant improvement when life-long learning is addressed. In this paper, we address robot navigation in continuous action space using deep hierarchical reinforcement learning without including the target location in the state representation. Our agent self-assigns internal goals and learns to extract reasonable waypoints to reach the desired target position only based on local sensor data. In our experiments we demonstrate that our hierarchical structure improves the performance of the navigation agent in terms of collected reward and success rate in comparison to a ﬂat structure, while not requiring any global or target information.
Characterizing Malicious URL Campaigns

URLs are central to a myriad of cyber-security threats, from phishing to the distribution of malware. Their inherent ease of use and familiarity is continuously abused by attackers to evade defences and deceive end-users. Seemingly dissimilar URLs are being used in an organized way to perform phishing attacks and distribute malware. We refer to such behaviours as campaigns, with the hypothesis being that attacks are often coordinated to maximize success rates and develop evasion tactics. The aim is to gain better insights into campaigns, bolster our grasp of their characteristics, and thus aid the community devise more robust solutions. To this end, we performed extensive research and analysis into 311M records containing 77M unique real-world URLs that were submitted to VirusTotal from Dec 2019 to Jan 2020. From this dataset, 2.6M suspicious campaigns were identified based on their attached metadata, of which 77,810 were doubly verified as malicious. Using the 38.1M records and 9.9M URLs within these malicious campaigns, we provide varied insights such as their targeted victim brands as well as URL sizes and heterogeneity. Some surprising findings were observed, such as detection rates falling to just 13.27% for campaigns that employ more than 100 unique URLs. The paper concludes with several case-studies that illustrate the common malicious techniques employed by attackers to imperil users and circumvent defences.
GASP XXXV: Characteristics of the Diffuse Ionised Gas in Gas-stripped Galaxies

Diffuse ionized gas (DIG) is an important component of the interstellar medium that can provide insights into the different physical processes affecting the gas in galaxies. We utilize optical IFU observations of 71 gas-stripped and control galaxies from the Gas Stripping Phenomena in galaxies (GASP) survey, to analyze the gas properties of dense ionized gas and DIG, such as metallicity, ionization parameter log(q), and the difference between the measured log[O i]/Hα and the value predicted by star-forming models given the measured log[Oiii]/Hβ (Δ log[O i]/Hα). We compare these properties at different spatial scales, among galaxies at different gas-stripping stages, and between disks and tails of the stripped galaxies. The metallicity is similar between the dense gas and DIG at a given galactocentric radius. The log(q) is lower for DIG compared to dense gas. The median values of log(q) correlate best with stellar mass and the most massive galaxies show an increase in log(q) toward their galactic centers. The DIG clearly shows higher Δ log[O i]/Hα values compared to the dense gas, with much of the spaxels having LIER/LINER-like emission. The DIG regions in the tails of highly stripped galaxies show the highest Δ log[O i]/Hα, exhibit high values of log(q), and extend to large projected distances from star-forming areas (up to 10 kpc). We conclude that the DIG in the tails is at least partly ionized by a process other than star formation, probably by mixing, shocks, and accretion of inter-cluster and interstellar medium gas.
Relativistic hybrid stars in light of the NICER PSR 
J0740+6620
 radius measurement

We explore the implications of the recent radius determination of PSR J0740+6620 by the NICER experiment combined with the neutron skin measurement by the PREX-II experiment and the associated inference of the slope of symmetry energy, for the structure of hybrid stars with a strong first-order phase transition from nucleonic to quark matter. We combine a covariant density-functional nucleonic equation of state (EOS) with a constant-speed-of-sound EOS for quark matter. We show that the radius and tidal deformability ranges obtained from GW170817 can be reconciled with the implication of the PREX-II experiment if there is a phase transition to quark matter in the low-mass compact star. In the high-mass segment, the EoS needs to be stiff to comply with the large-radius inference for PSR J0740+6620 and J0030+0451 with masses M ' 2M and M ' 1.4M . We show that twin stars are not excluded, but the mass and radius ranges (with M ≥ M ) are restricted to narrow domains ∆Mtwin . 0.05M and ∆Rtwin ∼ 1.0 km. We also show that the existence of twin configurations is compatible with the light companion in the GW190814 event being a hybrid star in the case of values of the sound-speed square s = 0.6 and s = 1/3.
Variational formulas on spaces of SL(2) Hitchin's spectral covers

Using the developed deformation theory on moduli spaces of quadratic differentials we derive variational formulas for objects associated with generalized SL(2) Hitchin’s spectral covers: Prym matrix, Prym bidifferential, Hodge and Prym tau-functions. The resulting formulas are antisymmetric versions of Donagi-Markman residue formula. The second variation of Prym matrix is a natural analogy to the formula previously derived for the period matrix of the GL(n) spectral cover.
Logarithmic Schrödinger equation and isothermal fluids

We consider the large time behavior in two types of equations, posed on the whole space R^d: the Schr{\"o}dinger equation with a logarithmic nonlinearity on the one hand; compressible, isothermal, Euler, Korteweg and quantum Navier-Stokes equations on the other hand. We explain some connections between the two families of equations, and show how these connections may help having an insight in all cases. We insist on some specific aspects only, and refer to the cited articles for more details, and more complete statements. We try to give a general picture of the results, and present some heuristical arguments that can help the intuition, which are not necessarily found in the mentioned articles.
Demystifying Drug Repurposing Domain Comprehension with Knowledge Graph Embedding

Drug repurposing is more relevant than ever due to drug development's rising costs and the need to respond to emerging diseases quickly. Knowledge graph embedding enables drug repurposing using heterogeneous data sources combined with state-of-the-art machine learning models to predict new drug-disease links in the knowledge graph. As in many machine learning applications, significant work is still required to understand the predictive models' behavior. We propose a structured methodology to understand better machine learning models' results for drug repurposing, suggesting key elements of the knowledge graph to improve predictions while saving computational resources. We reduce the training set of 11.05% and the embedding space by 31.87%, with only a 2% accuracy reduction, and increase accuracy by 60% on the open ogbl-biokg graph adding only 1.53% new triples.
Self-similar geometries within the inertial subrange of scales in boundary layer turbulence

Abstract The inertial subrange of turbulent scales is commonly reflected by a power law signature in ensemble statistics such as the energy spectrum and structure functions – both in theory and from observations. Despite promising findings on the topic of fractal geometries in turbulence, there is no accepted image for the physical flow features corresponding to this statistical signature in the inertial subrange. The present study uses boundary layer turbulence measurements to evaluate the self-similar geometric properties of velocity isosurfaces and investigate their influence on statistics for the velocity signal. The fractal dimension of streamwise velocity isosurfaces, indicating statistical self-similarity in the size of ‘wrinkles’ along each isosurface, is shown to be constant only within the inertial subrange of scales. For the transition between the inertial subrange and production range, it is inferred that the largest wrinkles become increasingly confined by the overall size of large-scale coherent velocity regions such as uniform momentum zones. The self-similarity of isosurfaces yields power-law trends in subsequent one-dimensional statistics. For instance, the theoretical 2/3 power-law exponent for the structure function can be recovered by considering the collective behaviour of numerous isosurface level sets. The results suggest that the physical presence of inertial subrange eddies is manifested in the self-similar wrinkles of isosurfaces.