Miracle-dz/UniFAR-FTUs huggingface dataset:
Equivariant Polynomials for Graph Neural Networks

Graph Neural Networks (GNN) are inherently limited in their expressive power. Recent seminal works (Xu et al., 2019; Morris et al., 2019b) introduced the Weisfeiler-Lehman (WL) hierarchy as a measure of expressive power. Although this hierarchy has propelled significant advances in GNN analysis and architecture developments, it suffers from several significant limitations. These include a complex definition that lacks direct guidance for model improvement and a WL hierarchy that is too coarse to study current GNNs. This paper introduces an alternative expressive power hierarchy based on the ability of GNNs to calculate equivariant polynomials of a certain degree. As a first step, we provide a full characterization of all equivariant graph polynomials by introducing a concrete basis, significantly generalizing previous results. Each basis element corresponds to a specific multi-graph, and its computation over some graph data input corresponds to a tensor contraction problem. Second, we propose algorithmic tools for evaluating the expressiveness of GNNs using tensor contraction sequences, and calculate the expressive power of popular GNNs. Finally, we enhance the expressivity of common GNN architectures by adding polynomial features or additional operations / aggregations inspired by our theory. These enhanced GNNs demonstrate state-of-the-art results in experiments across multiple graph learning benchmarks.
ChameleonControl: Teleoperating Real Human Surrogates through Mixed Reality Gestural Guidance for Remote Hands-on Classrooms

We present ChameleonControl, a real-human teleoperation system for scalable remote instruction in hands-on classrooms. In contrast to existing video or AR/VR-based remote hands-on education, ChameleonControl uses a real human as a surrogate of a remote instructor. Building on existing human-based telepresence approaches, we contribute a novel method to teleoperate a human surrogate through synchronized mixed reality hand gestural navigation and verbal communication. By overlaying the remote instructor’s virtual hands in the local user’s MR view, the remote instructor can guide and control the local user as if they were physically present. This allows the local user/surrogate to synchronize their hand movements and gestures with the remote instructor, effectively teleoperating a real human. We deploy and evaluate our system in classrooms of physiotherapy training, as well as other application domains such as mechanical assembly, sign language and cooking lessons. The study results confirm that our approach can increase engagement and the sense of co-presence, showing potential for the future of remote hands-on classrooms.
Measuring the Vortex−Nucleus Pinning Force from Pulsar Glitch Rates

Superfluid vortex avalanches are one plausible cause of pulsar glitch activity. If they occur according to a state-dependent Poisson process, the measured long-term glitch rate is determined by the spin-down rate of the stellar crust, Ω̇c , and two phenomenological parameters quantifying the vortex−nucleus pinning force: a crust−superfluid angular velocity lag threshold, X cr, and a reference unpinning rate, λ 0. A Bayesian analysis of 541 glitches in 177 pulsars, with N g ≥ 1 events per pulsar, yields Xcr=0.15−0.04+0.09rads−1 , λref=7.6−2.6+3.7×10−8s−1 , and a=−0.27−0.03+0.04 assuming the phenomenological rate law λ 0 = λ ref[τ/(1 yr)] a , where τ denotes the characteristic spin-down age. The results are broadly similar, whether one includes or excludes quasiperiodic glitch activity, giant glitches, or pulsars with N g = 0, up to uncertainties about the completeness of the sample and the total observation time per pulsar. The X cr and λ 0 estimates are consistent with first-principles calculations based on nuclear theory, e.g., in the semiclassical local density approximation.
Simple Analytics of the Government Investment Multiplier

What are the effects of investing in public infrastructure? We answer this question with a New Keynesian model. We recast the model as a Markov chain and develop a general solution method that nests existing ones inside/outside the zero lower bound as special cases. Our framework delivers a simple expression for the contribution of public infrastructure. We show that it provides a unified framework to study the effects of public investment in three scenarios: $(i)$ normal times $(ii)$ short-lived liquidity trap $(iii)$ long-lived liquidity trap. We find that calibrations commonly used lead to multipliers that diverge with the duration of the trap.
Asymptotic Formulas for the Zeros of the Meixner Polynomials

The zeros of the Meixner polynomialmn(x;s,c) are real, distinct, and lie in (0,∞). Let? n,sdenote thesth zero ofmn(n? ;s,c), counted from the right; and let? n,sdenote thesth zero ofmn(n? ;s,c), counted from the left. For each fixeds, asymptotic formulas are obtained for both? n,sand? n,s, asn? ∞.
Inferring HIV transmission patterns from viral deep-sequence data via latent typed point processes.

Viral deep-sequencing data play a crucial role toward understanding disease transmission network flows, providing higher resolution compared to standard Sanger sequencing. To more fully utilize these rich data and account for the uncertainties in outcomes from phylogenetic analyses, we propose a spatial Poisson process model to uncover human immunodeficiency virus (HIV) transmission flow patterns at the population level. We represent pairings of individuals with viral sequence data as typed points, with coordinates representing covariates such as gender and age and point types representing the unobserved transmission statuses (linkage and direction). Points are associated with observed scores on the strength of evidence for each transmission status that are obtained through standard deep-sequence phylogenetic analysis. Our method is able to jointly infer the latent transmission statuses for all pairings and the transmission flow surface on the source-recipient covariate space. In contrast to existing methods, our framework does not require preclassification of the transmission statuses of data points, and instead learns them probabilistically through a fully Bayesian inference scheme. By directly modeling continuous spatial processes with smooth densities, our method enjoys significant computational advantages compared to previous methods that rely on discretization of the covariate space. We demonstrate that our framework can capture age structures in HIV transmission at high resolution, bringing valuable insights in a case study on viral deep-sequencing data from Southern Uganda.
Institutional reforms and the employment effects of spatially targeted investment grants: The case of Germany's GRW

Spatially targeted investment grant schemes are a common tool to support firms in lagging regions. We exploit exogenous variations in Germany's main regional policy instrument (GRW) arriving from institutional reforms to analyse local employment effects of investment grants. Findings for reduced-form and IV regressions point to a significant policy channel running from higher funding rates to increased firm-level investments and newly created jobs. When we contrast effects for regions with high but declining funding rates to those with low but rising rates, we find that GRW reforms led to diminishing employment increases. Especially small firms responded to changing funding conditions.
The Amplification Paradox in Recommender Systems

Automated audits of recommender systems found that blindly following recommendations leads users to increasingly partisan, conspiratorial, or false content. At the same time, studies using real user traces suggest that recommender systems are not the primary driver of attention toward extreme content; on the contrary, such content is mostly reached through other means, e.g., other websites. In this paper, we explain the following apparent paradox: if the recommendation algorithm favors extreme content, why is it not driving its consumption? With a simple agent-based model where users attribute different utilities to items in the recommender system, we show through simulations that the collaborative-filtering nature of recommender systems and the nicheness of extreme content can resolve the apparent paradox: although blindly following recommendations would indeed lead users to niche content, users rarely consume niche content when given the option because it is of low utility to them, which can lead the recommender system to deamplify such content. Our results call for a nuanced interpretation of "algorithmic amplification" and highlight the importance of modeling the utility of content to users when auditing recommender systems. Code available: https://github.com/epfl-dlab/amplification_paradox.
The co-evolution of supermassive black holes and galaxies in luminous AGN over a wide range of redshift

It is well known that supermassive black holes (SMBHs) and their host galaxies co-evolve. A manifestation of this co-evolution is the correlation that has been found between the SMBH mass, M$_{BH}$, and the galaxy bulge or stellar mass, M$_*$. The cosmic evolution of this relation, though, is still a matter of debate. In this work, we examine the M$_{BH}-$M$_*$ relation, using 687 X-ray luminous (median $\rm log\,[L_{X,2-10keV}(ergs^{-1})]=44.3$), broad line AGN, at $\rm 0.2<z<4.0$ (median $\rm z\approx 1.4$) that lie in the XMM-{\it{XXL}} field. Their M$_{BH}$ and M$_*$ range from $\rm 7.5<log\,[M_{BH}\,(M_\odot)]<9.5$ and $\rm 10<log\,[M_*(M_\odot)]<12$, respectively. Most of the AGN live in star-forming galaxies and their Eddington ratios range from 0.01 to 1, with a median value of 0.06. Our results show that M$_{BH}$ and M$_*$ are correlated ($\rm r=0.47\pm0.21$, averaged over different redshift intervals). Our analysis also shows that the mean ratio of the M$_{BH}$ and M$_*$ does not evolve with redshift, at least up to $\rm z=2$ and has a value of $\rm log($M$_{BH}/$M$_*)=-2.44$. The majority of the AGN ($75\%$) are in a SMBH mass growth dominant phase. In these systems, the M$_{BH}-$M$_*$ correlation is weaker and their M$_*$ tends to be lower (for the same M$_{BH}$) compared to systems that are in a galaxy mass growth phase. Our findings suggest that the growth of black hole mass occurs first, while the early stellar mass assembly may not be so efficient.
Stochastic Approximation Beyond Gradient for Signal Processing and Machine Learning

Stochastic Approximation (SA) is a classical algorithm that has had since the early days a huge impact on signal processing, and nowadays on machine learning, due to the necessity to deal with a large amount of data observed with uncertainties. An exemplar special case of SA pertains to the popular stochastic (sub)gradient algorithm which is the working horse behind many important applications. A lesser-known fact is that the SA scheme also extends to non-stochastic-gradient algorithms such as compressed stochastic gradient, stochastic expectation-maximization, and a number of reinforcement learning algorithms. The aim of this article is to overview and introduce the non-stochastic-gradient perspectives of SA to the signal processing and machine learning audiences through presenting a design guideline of SA algorithms backed by theories. Our central theme is to propose a general framework that unifies existing theories of SA, including its non-asymptotic and asymptotic convergence results, and demonstrate their applications on popular non-stochastic-gradient algorithms. We build our analysis framework based on classes of Lyapunov functions that satisfy a variety of mild conditions. We draw connections between non-stochastic-gradient algorithms and scenarios when the Lyapunov function is smooth, convex, or strongly convex. Using the said framework, we illustrate the convergence properties of the non-stochastic-gradient algorithms using concrete examples. Extensions to the emerging variance reduction techniques for improved sample complexity will also be discussed.
Can Learning Be Explained By Local Optimality In Low-rank Matrix Recovery?

We explore the local landscape of low-rank matrix recovery, aiming to reconstruct a $d_1\times d_2$ matrix with rank $r$ from $m$ linear measurements, some potentially noisy. When the true rank is unknown, overestimation is common, yielding an over-parameterized model with rank $k\geq r$. Recent findings suggest that first-order methods with the robust $\ell_1$-loss can recover the true low-rank solution even when the rank is overestimated and measurements are noisy, implying that true solutions might emerge as local or global minima. Our paper challenges this notion, demonstrating that, under mild conditions, true solutions manifest as \textit{strict saddle points}. We study two categories of low-rank matrix recovery, matrix completion and matrix sensing, both with the robust $\ell_1$-loss. For matrix sensing, we uncover two critical transitions. With $m$ in the range of $\max\{d_1,d_2\}r\lesssim m\lesssim \max\{d_1,d_2\}k$, none of the true solutions are local or global minima, but some become strict saddle points. As $m$ surpasses $\max\{d_1,d_2\}k$, all true solutions become unequivocal global minima. In matrix completion, even with slight rank overestimation and mild noise, true solutions either emerge as non-critical or strict saddle points.
Drag reduction by polymer additives in a turbulent pipe flow: numerical and laboratory experiments

In order to study the roles of stress anisotropy and of elasticity in the mechanism of drag reduction by polymer additives we investigate a turbulent pipe flow of a dilute polymer solution. The investigation is carried out by means of direct numerical simulation (DNS) and laser Doppler velocimetry (LDV). In our DNS two different models are used to describe the effects of polymers on the flow. The first is a constitutive equation based on Batchelor's theory of elongated particles suspended in a Newtonian solvent which models the viscous anisotropic effects caused by the polymer orientation. The second is an extension of the first model with an elastic component, and can be interpreted as an anisotropic Maxwell model. The LDV experiments have been carried out in a recirculating pipe flow facility in which we have used a solution of water and 20 w.p.p.m. Superfloc A110. Turbulence statistics up to the fourth moment, as well as power spectra of various velocity components, have been measured. The results of the drag-reduced flow are first compared with those of a standard turbulent pipe flow of water at the same friction velocity at a Reynolds number of Reτ≈1035. Next the results of the numerical simulation and of the measurements are compared in order to elucidate the role of polymers in the phenomenon of drag reduction. For the case of the viscous anisotropic polymer model, almost all turbulence statistics and power spectra calculated agree in a qualitative sense with the measurements. The addition of elastic effects, on the other hand, has an adverse effect on the drag reduction, i.e. the viscoelastic polymer model shows less drag reduction than the anisotropic model without elasticity. Moreover, for the case of the viscoelastic model not all turbulence statistics show the right behaviour. On the basis of these results, we propose that the viscous anisotropic stresses introduced by extended polymers play a key role in the mechanism of drag reduction by polymer additives.
Conformers are All You Need for Visual Speech Recogntion

Visual speech recognition models extract visual features in a hierarchical manner. At the lower level, there is a visual front-end with a limited temporal receptive field that processes the raw pixels depicting the lips or faces. At the higher level, there is an encoder that attends to the embeddings produced by the front-end over a large temporal receptive field. Previous work has focused on improving the visual front-end of the model to extract more useful features for speech recognition. Surprisingly, our work shows that complex visual front-ends are not necessary. Instead of allocating resources to a sophisticated visual front-end, we find that a linear visual front-end paired with a larger Conformer encoder results in lower latency, more efficient memory usage, and improved WER performance. We achieve a new state-of-the-art of 12.8% WER for visual speech recognition on the TED LRS3 dataset, which rivals the performance of audio-only models from just four years ago.
Multilinear $\theta$-type Calderon--Zygmund operators and their commutators on products of weighted amalgam spaces

In this paper, we first introduce several new classes of weighted amalgam spaces. Then we discuss both strong type and weak type estimates for certain multilinear $\theta$-type Calder\'on--Zygmund operators $T_\theta$ recently introduced in the literature on products of these spaces with multiple weights. Furthermore, the strong type and weak end-point estimates for both multilinear commutators and iterated commutators of $T_\theta$ and pointwise multiplication with BMO functions are established as well.
ASSET: Robust Backdoor Data Detection Across a Multiplicity of Deep Learning Paradigms

Backdoor data detection is traditionally studied in an end-to-end supervised learning (SL) setting. However, recent years have seen the proliferating adoption of self-supervised learning (SSL) and transfer learning (TL), due to their lesser need for labeled data. Successful backdoor attacks have also been demonstrated in these new settings. However, we lack a thorough understanding of the applicability of existing detection methods across a variety of learning settings. By evaluating 56 attack settings, we show that the performance of most existing detection methods varies significantly across different attacks and poison ratios, and all fail on the state-of-the-art clean-label attack. In addition, they either become inapplicable or suffer large performance losses when applied to SSL and TL. We propose a new detection method called Active Separation via Offset (ASSET), which actively induces different model behaviors between the backdoor and clean samples to promote their separation. We also provide procedures to adaptively select the number of suspicious points to remove. In the end-to-end SL setting, ASSET is superior to existing methods in terms of consistency of defensive performance across different attacks and robustness to changes in poison ratios; in particular, it is the only method that can detect the state-of-the-art clean-label attack. Moreover, ASSET's average detection rates are higher than the best existing methods in SSL and TL, respectively, by 69.3% and 33.2%, thus providing the first practical backdoor defense for these new DL settings. We open-source the project to drive further development and encourage engagement: https://github.com/ruoxi-jia-group/ASSET.
Characterizing the Conditional Galaxy Property Distribution Using Gaussian Mixture Models

Line-intensity mapping (LIM) is a promising technique to constrain the global distribution of galaxy properties. To combine LIM experiments probing different tracers with traditional galaxy surveys and fully exploit the scientific potential of these observations, it is necessary to have a physically motivated modeling framework. As part of developing such a framework, in this work, we introduce and model the conditional galaxy property distribution (CGPD), i.e., the distribution of galaxy properties conditioned on the host halo mass and redshift. We consider five galaxy properties, including the galaxy stellar mass, molecular gas mass, galaxy radius, gas-phase metallicity, and star formation rate (SFR), which are important for predicting the emission lines of interest. The CGPD represents the full distribution of galaxies in the five-dimensional property space; many important galaxy distribution functions and scaling relations, such as the stellar mass function and SFR main sequence, can be derived from integrating and projecting it. We utilize two different kinds of cosmological galaxy simulations, a semi-analytic model and the IllustrisTNG hydrodynamic simulation, to characterize the CGPD and explore how well it can be represented using a Gaussian mixture model (GMM). We find that with just a few (approximately three) Gaussian components, a GMM can describe the CGPD of the simulated galaxies to high accuracy for both simulations. The CGPD can be mapped to LIM or other observables by constructing the appropriate relationship between galaxy properties and the relevant observable tracers, which will be discussed in future works.
PAD: Towards Principled Adversarial Malware Detection Against Evasion Attacks

Machine Learning (ML) techniques can facilitate the automation of malicious software (malware for short) detection, but suffer from evasion attacks. Many studies counter such attacks in heuristic manners, lacking theoretical guarantees and defense effectiveness. In this article, we propose a new adversarial training framework, termed Principled Adversarial Malware Detection (PAD), which offers convergence guarantees for robust optimization methods. PAD lays on a learnable convex measurement that quantifies distribution-wise discrete perturbations to protect malware detectors from adversaries, whereby for smooth detectors, adversarial training can be performed with theoretical treatments. To promote defense effectiveness, we propose a new mixture of attacks to instantiate PAD to enhance deep neural network-based measurements and malware detectors. Experimental results on two Android malware datasets demonstrate: (i) the proposed method significantly outperforms the state-of-the-art defenses; (ii) it can harden ML-based malware detection against 27 evasion attacks with detection accuracies greater than 83.45%, at the price of suffering an accuracy decrease smaller than 2.16% in the absence of attacks; (iii) it matches or outperforms many anti-malware scanners in VirusTotal against realistic adversarial malware.
Persistent Non-Statistical Dynamics in One-Dimensional Maps

We study a class $\widehat{\mathfrak{F}}$ of one-dimensional full branch maps introduced in [Doubly Intermittent Full Branch Maps with Critical Points and Singularities; D. Coates, S. Luzzatto, M. Mubarak, 2022], admitting two indifferent fixed points as well as critical points and/or singularities with unbounded derivative. We show that $\widehat{\mathfrak{F}}$ can be partitioned into 3 pairwise disjoint subfamilies $$\widehat{\mathfrak{F}} = \mathfrak{F} \cup \mathfrak{F}_\pm \cup \mathfrak{F}_*$$ such that all $g \in \mathfrak{F}$ have a unique physical measure equivalent to Lebesgue, all $g \in \mathfrak{F}_{\pm}$ have a physical measure which is a Dirac-$\delta$ measure on one of the (repelling) fixed points, and all $g \in \mathfrak{F}_{*}$ are non-statistical and in particular have no physical measure. Moreover we show that these subfamilies are intermingled: they can all be approximated by maps in the other subfamilies in natural topologies.