Miracle-dz/UniFAR-FTUs huggingface dataset:
Adaptive Graph Convolutional Recurrent Network for Traffic Forecasting

Modeling complex spatial and temporal correlations in the correlated time series data is indispensable for understanding the traffic dynamics and predicting the future status of an evolving traffic system. Recent works focus on designing complicated graph neural network architectures to capture shared patterns with the help of pre-defined graphs. In this paper, we argue that learning node-specific patterns is essential for traffic forecasting while the pre-defined graph is avoidable. To this end, we propose two adaptive modules for enhancing Graph Convolutional Network (GCN) with new capabilities: 1) a Node Adaptive Parameter Learning (NAPL) module to capture node-specific patterns; 2) a Data Adaptive Graph Generation (DAGG) module to infer the inter-dependencies among different traffic series automatically. We further propose an Adaptive Graph Convolutional Recurrent Network (AGCRN) to capture fine-grained spatial and temporal correlations in traffic series automatically based on the two modules and recurrent networks. Our experiments on two real-world traffic datasets show AGCRN outperforms state-of-the-art by a significant margin without pre-defined graphs about spatial connections.
Explaining Fast Improvement in Online Policy Optimization

Online policy optimization (OPO) views policy optimization for sequential decision making as an online learning problem. In this framework, the algorithm designer defines a sequence of online loss functions such that the regret rate in online learning implies the policy convergence rate and the minimal loss witnessed by the policy class determines the policy performance bias. This reduction technique has been successfully applied to solving various policy optimization problems, including imitation learning, structured prediction, and system identification. Interestingly, the policy improvement speed observed in practice is usually much faster than existing theory suggests. In this work, we provide an explanation of this fast policy improvement phenomenon. Let $\epsilon$ denote the policy class bias and assume the online loss functions are convex, smooth, and non-negative. We prove that, after $N$ rounds of OPO with stochastic feedback, the policy converges in $\tilde{O}(1/N + \sqrt{\epsilon/N})$ in both expectation and high probability. In other words, we show that adopting a sufficiently expressive policy class in OPO has two benefits: both the convergence rate increases and the performance bias decreases, as the policy class becomes reasonably rich. This new theoretical insight is further verified in an online imitation learning experiment.
Regularization Matters: A Nonparametric Perspective on Overparametrized Neural Network

Overparametrized neural networks trained by gradient descent (GD) can provably overfit any training data. However, the generalization guarantee may not hold for noisy data. From a nonparametric perspective, this paper studies how well overparametrized neural networks can recover the true target function in the presence of random noises. We establish a lower bound on the $L_2$ estimation error with respect to the GD iteration, which is away from zero without a delicate choice of early stopping. In turn, through a comprehensive analysis of $\ell_2$-regularized GD trajectories, we prove that for overparametrized one-hidden-layer ReLU neural network with the $\ell_2$ regularization: (1) the output is close to that of the kernel ridge regression with the corresponding neural tangent kernel; (2) minimax {optimal} rate of $L_2$ estimation error is achieved. Numerical experiments confirm our theory and further demonstrate that the $\ell_2$ regularization approach improves the training robustness and works for a wider range of neural networks.
Quantifying the interplay between fine structure and geometry of an individual molecule on a surface

The pathway toward the tailored synthesis of materials starts with precise characterization of the conformational properties and dynamics of individual molecules. Electron spin resonance based scanning tunneling microscopy can potentially address molecular structure with unprecedented resolution. Here, we determine the fine structure and geometry of an individual TiH molecule, utilizing a combination of a newly developed mK ESR-STM in a vector magnetic field and ab initio approaches. We demonstrate a strikingly large anisotropy of the g-tensor unusual for a spin doublet ground state, resulting from a non-trivial orbital angular momentum. We quantify the relationship between the resultant fine structure, hindered rotational modes, and orbital excitations. Our model system provides new avenues to determine the structure and dynamics of individual molecules with unprecedented precision.
Probing Dark Low-mass Halos and Primordial Black Holes with Frequency-dependent Gravitational Lensing Dispersions of Gravitational Waves

We explore the possibility of using amplitude and phase fluctuations of gravitational waves due to gravitational lensing as a probe of the small-scale matter power spectrum. The direct measurement of the small-scale matter power spectrum is made possible by making use of the frequency dependence of such gravitational lensing dispersions originating from the wave optics nature of the propagation of gravitational waves. We first study the small-scale behavior of the matter power spectrum in detail taking the so-called halo model approach, including the effects of baryons and subhalos. We find that the matter power spectrum at the wavenumber k ∼ 106 hMpc−1 is mainly determined by the abundance of dark low-mass halos with mass and is relatively insensitive to baryonic effects. The matter power spectrum at this wavenumber is probed by gravitational lensing dispersions of gravitational waves at frequencies of f ∼ 0.1–1 Hz with predicted signals of . We also find that primordial black holes (PBHs) with MPBH ≳ 0.1 M⊙ can significantly enhance the matter power spectrum at k ≳ 105 hMpc−1 due to both the enhanced halo formation and the shot noise from PBHs. We find that gravitational lensing dispersions at f ∼ 10–100 Hz are particularly sensitive to PBHs and can be enhanced by more than an order of magnitude depending on the mass and abundance of PBHs.
Systematic review and meta-analysis of the association between complementary factor H Y402H polymorphisms and age-related macular degeneration

Age-related macular degeneration (AMD) is the leading cause of blindness in the developed world and complement factor H (CFH) polymorphism has been found to associated with the AMD. We performed a meta-analysis to estimate the magnitude of the gene effect and the possible mode of action. A meta-analysis of eight studies assessing association between the CFH Y402H polymorphism and AMD was performed. Data extraction and study quality assessment were performed in duplicate, and heterogeneity and publication bias were explored. There was strong evidence for association between CFH and AMD, with those having CC and TC genotypes being roughly six and 2.5 times more likely to have AMD than patients with TT genotype, suggesting a co-dominant, multiplicative genetic model. The population attributable risk for the CC/TC genotype is 58.9%, i.e. the CFH polymorphism is involved in over half of all AMD. This meta-analysis summarizes the strong evidence for an association between CFH and AMD and indicates a multiplicative model with each C allele increasing the odds of AMD by 2.5-fold. This result is at least as important at the population level as ApoE4 and Alzheimer’s disease, playing a role in almost 60% of AMD at the population level.
Learning from Failure: Training Debiased Classifier from Biased Classifier

Neural networks often learn to make predictions that overly rely on spurious correlation existing in the dataset, which causes the model to be biased. While previous work tackles this issue with domain-specific knowledge or explicit supervision on the spuriously correlated attributes, we instead tackle a more challenging setting where such information is unavailable. To this end, we first observe that neural networks learn to rely on the spurious correlation only when it is ''easier'' to learn than the desired knowledge, and such reliance is most prominent during the early phase of training. Based on the observations, we propose a failure-based debiasing scheme by training a pair of neural networks simultaneously. Our main idea is twofold; (a) we intentionally train the first network to be biased by repeatedly amplifying its ''prejudice'', and (b) we debias the training of the second network by focusing on samples that go against the prejudice of the biased network in (a). Extensive experiments demonstrate that our method significantly improves the training of network against various types of biases in both synthetic and real-world datasets. Surprisingly, our framework even occasionally outperforms the debiasing methods requiring explicit supervision of the spuriously correlated attributes.
Synchronous linear constraint system games

Synchronous linear constraint system games are nonlocal games that verify whether or not two players share a solution to a given system of equations. Two algebraic objects associated to these games encode information about the existence of perfect strategies. They are called the game algebra and the solution group. Here we show that these objects are essentially the same, i.e., that the game algebra is a suitable quotient of the group algebra of the solution group. We also demonstrate that linear constraint system games are equivalent to graph isomorphism games on a pair of graphs parameterized by the linear system.
Infall of galaxies onto groups

Growth of the structure in the Universe manifest as accretion flows of galaxies onto groups and clusters. Thus, the present day properties of groups and their member galaxies are influenced by the characteristics of this continuous infall pattern. Several works both theoretical, in numerical simulations, and in observations, study this process and provide useful steps for a better understanding of galaxy systems and their evolution. We aim at exploring the streaming flow of galaxies onto groups using observational peculiar velocity data. The effects of distance uncertainties are also analyzed as well as the relation between the infall pattern and group and environment properties.This work deals with analysis of peculiar velocity data and their projection on the direction to group centers, to determine the mean galaxy infall flow. We applied this analysis to the galaxies and groups extracted from the Cosmicflows-3 catalog. We also use mock catalogs derived from numerical simulations to explore the effects of distance uncertainties on the derivation of the galaxy velocity flow onto groups. We determine the infalling velocity field onto galaxy groups with cz < 0.033 using peculiar velocity data. We measure the mean infall velocity onto group samples of different mass range, and also explore the impact of the environment where the group reside. Well beyond the group virial radius, the surrounding large-scale galaxy overdensity may impose additional infalling streaming amplitudes in the range 200 to 400 km s$^{-1}$. Also, we find that groups in samples with a well controlled galaxy density environment show an increasing infalling velocity amplitude with group mass, consistent with the predictions of the linear model. These results from observational data are in excellent agreement with those derived from the mock catalogs.
A Few-Shot Sequential Approach for Object Counting

In this work, we address the problem of few-shot multi-class object counting with point-level annotations. The proposed technique leverages a class agnostic attention mechanism that sequentially attends to objects in the image and extracts their relevant features. This process is employed on an adapted prototypical-based few-shot approach that uses the extracted features to classify each one either as one of the classes present in the support set images or as background. The proposed technique is trained on point-level annotations and uses a novel loss function that disentangles class-dependent and class-agnostic aspects of the model to help with the task of few-shot object counting. We present our results on a variety of object-counting/detection datasets, including FSOD and MS COCO. In addition, we introduce a new dataset that is specifically designed for weakly supervised multi-class object counting/detection and contains considerably different classes and distribution of number of classes/instances per image compared to the existing datasets. We demonstrate the robustness of our approach by testing our system on a totally different distribution of classes from what it has been trained on.
In the Wild: From ML Models to Pragmatic ML Systems

Enabling robust intelligence in the wild entails learning systems that offer uninterrupted inference while affording sustained training, with varying amounts of data & supervision. Such a pragmatic ML system should be able to cope with the openness & flexibility inherent in the real world. The machine learning community has organically broken down this challenging task into manageable sub tasks such as supervised, few-shot, continual, & self-supervised learning; each affording distinctive challenges & leading to a unique set of methods. Notwithstanding this amazing progress, the restricted & isolated nature of these tasks has resulted in methods that excel in one setting, but struggle to extend beyond them. To foster the research required to extend ML models to ML systems, we introduce a unified learning & evaluation framework - iN thE wilD (NED). NED is designed to be a more general paradigm by loosening the restrictive design decisions of past settings (e.g. closed-world assumption) & imposing fewer restrictions on learning algorithms (e.g. predefined train & test phases). The learners can infer the experimental parameters themselves by optimizing for both accuracy & compute. In NED, a learner receives a stream of data & makes sequential predictions while choosing how to update itself, adapt to data from novel categories, & deal with changing data distributions; while optimizing the total amount of compute. We evaluate a large set of existing methods across several sub fields using NED & present surprising yet revealing findings about modern day techniques. For instance, prominent few shot methods break down in NED, achieving dramatic drops of over 40% accuracy relative to simple baselines; & the SOTA self-supervised methods Momentum Contrast obtains 35% lower accuracy than supervised pretraining on novel classes. We also show that a simple baseline outperforms existing methods on NED.
On the coarsening mechanism and deformation of borides under annealing and creep in a polycrystalline superalloy

We have investigated the coarsening mechanism of intergranular Cr-rich M2B borides after creep and annealing at 850C for approximately 3000 hours in a polycrystalline nickel-based superalloy. Borides were found to be coarser after creep, with measured thicknesses in the range of 800-1100nm, compared to borides annealed in the absence of external applied load (400-600nm). The borides had a thickness of 100-200nm before exposure at 850C. Transmission electron microscopy revealed that coarsened borides have either the tetragonal I4/mcm structure, or the orthorhombic Fddd, with those two structures coexisting in a single particle. The presence of a very high density of planar faults is systematically observed within the coarsened borides. The faults were correlated with chemical fluctuations of B and Cr, revealed by atom probe tomography. Our results allow us to suggest that borides coarsen by an epitaxy-like mechanism. In addition, partitioning of Ni and Co was observed at dislocations within the borides after creep providing insights into the deformation of borides. Consequences of coarsened intergranular borides on the creep performance of polycrystalline superalloys are discussed.
TDprop: Does Jacobi Preconditioning Help Temporal Difference Learning?

We investigate whether Jacobi preconditioning, accounting for the bootstrap term in temporal difference (TD) learning, can help boost performance of adaptive optimizers. Our method, TDprop, computes a per parameter learning rate based on the diagonal preconditioning of the TD update rule. We show how this can be used in both $n$-step returns and TD($\lambda$). Our theoretical findings demonstrate that including this additional preconditioning information is, surprisingly, comparable to normal semi-gradient TD if the optimal learning rate is found for both via a hyperparameter search. In Deep RL experiments using Expected SARSA, TDprop meets or exceeds the performance of Adam in all tested games under near-optimal learning rates, but a well-tuned SGD can yield similar improvements -- matching our theory. Our findings suggest that Jacobi preconditioning may improve upon typical adaptive optimization methods in Deep RL, but despite incorporating additional information from the TD bootstrap term, may not always be better than SGD.
Incidence of parasitic worms in stomachs of pelagic and demersal fish of the Rockall Trough, northeastern Atlantic Ocean

The incidence of gyrocotyllid worms in chimaeroids and cestode larvae and nematodes in the stomachs of some 13 600 demersal and pelagic fish from the Rockall Trough, northeastern Atlantic is determined. The observations were made in conjunction with a detailed study of the diets of the fish which belonged to 117 species. No dietary sources of the infections could be identified with certainty. Pelagically caught fish, from between the surface and 2500 m depth, had no helminth infestation of their stomachs. Infestations of demersal fish were more pronounced between 1500 and 2900 m than between 500 and 1250 m depth.
Yield curve and macroeconomy interaction: evidence from the non-parametric functional lagged regression approach

Viewing a yield curve as a sparse collection of measurements on a latent continuous random function allows us to model it statistically as a sparsely observed functional time series. Doing so, we use the state-of-the-art methods in non-parametric statistical inference for sparsely observed functional time series to analyse the lagged regression dependence of the US Treasury yield curve on US macroeconomic variables. Our non-parametric analysis confirms previous findings established under parametric assumptions, namely a strong impact of the federal funds rate on the short end of the yield curve and a moderate effect of the annual inflation on the longer end of the yield curve.
Subgraph probability of random graphs with specified degrees and applications to chromatic number and connectivity

Given a graphical degree sequence d=(d1,…,dn)$$ \mathbf{d}=\left({d}_1,\dots, {d}_n\right) $$ , let 𝒢(n,d) denote a uniformly random graph on vertex set [n]$$ \left[n\right] $$ where vertex i$$ i $$ has degree di$$ {d}_i $$ for every 1≤i≤n$$ 1\le i\le n $$ . We give upper and lower bounds on the joint probability of an arbitrary set of edges in 𝒢(n,d) , and we link these probability estimates to the corresponding probabilities in the configuration model. Then we show that many existing results of 𝒢(n,d) in the literature can be significantly improved with simpler proofs, by applying this new probabilistic tool. One example we give concerns the chromatic number of 𝒢(n,d) . In another application, we use these joint probabilities to study the connectivity of 𝒢(n,d) . Under some rather mild condition on d$$ \mathbf{d} $$ —in particular, if Δ2=o(M)$$ {\Delta}^2=o(M) $$ where Δ$$ \Delta $$ is the maximum component of d$$ \mathbf{d} $$ —we fully characterize the connectivity phase transition of 𝒢(n,d) . We also give sufficient conditions for 𝒢(n,d) being connected when Δ$$ \Delta $$ is unrestricted.
Deep Learning for Anomaly Detection

Anomaly detection, a.k. a. outlier detection or novelty detection, has been a lasting yet active research area in various research communities for several decades. There are still some unique problem complexities and challenges that require advanced approaches. In recent years, deep learning enabled anomaly detection, i.e., deep anomaly detection, has emerged as a critical direction. This article surveys the research of deep anomaly detection with a comprehensive taxonomy, covering advancements in 3 high-level categories and 11 fine-grained categories of the methods. We review their key intuitions, objective functions, underlying assumptions, advantages, and disadvantages and discuss how they address the aforementioned challenges. We further discuss a set of possible future opportunities and new perspectives on addressing the challenges.
Meta-Learning through Hebbian Plasticity in Random Networks

Lifelong learning and adaptability are two defining aspects of biological agents. Modern reinforcement learning (RL) approaches have shown significant progress in solving complex tasks, however once training is concluded, the found solutions are typically static and incapable of adapting to new information or perturbations. While it is still not completely understood how biological brains learn and adapt so efficiently from experience, it is believed that synaptic plasticity plays a prominent role in this process. Inspired by this biological mechanism, we propose a search method that, instead of optimizing the weight parameters of neural networks directly, only searches for synapse-specific Hebbian learning rules that allow the network to continuously self-organize its weights during the lifetime of the agent. We demonstrate our approach on several reinforcement learning tasks with different sensory modalities and more than 450K trainable plasticity parameters. We find that starting from completely random weights, the discovered Hebbian rules enable an agent to navigate a dynamical 2D-pixel environment; likewise they allow a simulated 3D quadrupedal robot to learn how to walk while adapting to different morphological damage in the absence of any explicit reward or error signal.
Towards Semantic Detection of Smells in Cloud Infrastructure Code

Automated deployment and management of Cloud applications relies on descriptions of their deployment topologies, often referred to as Infrastructure Code. As the complexity of applications and their deployment models increases, developers inadvertently introduce software smells to such code specifications, for instance, violations of good coding practices, modular structure, and more. This paper presents a knowledge-driven approach enabling developers to identify the aforementioned smells in deployment descriptions. We detect smells with SPARQL-based rules over pattern-based OWL 2 knowledge graphs capturing deployment models. We show the feasibility of our approach with a prototype and three case studies.
Learning the Prediction Distribution for Semi-Supervised Learning with Normalising Flows

As data volumes continue to grow, the labelling process increasingly becomes a bottleneck, creating demand for methods that leverage information from unlabelled data. Impressive results have been achieved in semi-supervised learning (SSL) for image classification, nearing fully supervised performance, with only a fraction of the data labelled. In this work, we propose a probabilistically principled general approach to SSL that considers the distribution over label predictions, for labels of different complexity, from "one-hot" vectors to binary vectors and images. Our method regularises an underlying supervised model, using a normalising flow that learns the posterior distribution over predictions for labelled data, to serve as a prior over the predictions on unlabelled data. We demonstrate the general applicability of this approach on a range of computer vision tasks with varying output complexity: classification, attribute prediction and image-to-image translation.