Miracle-dz/UniFAR-FTUs huggingface dataset:
ASGNN: Graph Neural Networks with Adaptive Structure

The graph neural network (GNN) models have presented impressive achievements in numerous machine learning tasks. However, many existing GNN models are shown to be vulnerable to adversarial attacks, which creates a stringent need to build robust GNN architectures. In this work, we propose a novel interpretable message passing scheme with adaptive structure (ASMP) to defend against adversarial attacks on graph structure. Layers in ASMP are derived based on optimization steps that minimize an objective function that learns the node feature and the graph structure simultaneously. ASMP is adaptive in the sense that the message passing process in different layers is able to be carried out over dynamically adjusted graphs. Such property allows more fine-grained handling of the noisy (or perturbed) graph structure and hence improves the robustness. Convergence properties of the ASMP scheme are theoretically established. Integrating ASMP with neural networks can lead to a new family of GNN models with adaptive structure (ASGNN). Extensive experiments on semi-supervised node classification tasks demonstrate that the proposed ASGNN outperforms the state-of-the-art GNN architectures in terms of classification performance under various adversarial attacks.
Analysis of Gradient Descent with Varying Step Sizes using Integral Quadratic Constraints

The framework of Integral Quadratic Constraints (IQCs) is used to present a performance analysis for gradient descent with varying step sizes. Two performance metrics are considered: convergence rate and noise amplification. We assume that the step size is produced from a line search and varies in a known interval. We model the algorithm as a linear, parameter-varying (LPV) system and construct a parameterized linear matrix inequality (LMI) condition that certifies algorithm performance. This condition is a generalization of prior results for gradient descent with constant step sizes. The parameterized LMI condition is solved using a gridding approach on the step size interval. Our results provide convergence rate guarantees when the step size lies within a restricted interval. Moreover, we recover existing rate bounds when this interval reduces to a single point, i.e. a constant step size. Finally, we note that the convergence rate depends only on the condition number of the problem. In contrast, the noise amplification performance depends on the individual values of the convexity and smoothness parameters, and varies inversely with them for a fixed condition number.
A high accuracy and low complexity quality control method for image compression

—For large-scale still image coding tasks, the processing platform needs to ensure that the coded images meet the quality requirement. Therefore, the quality control algorithms that generate adaptive QP towards a target quality level for image coding are of significant research value. However, the existing quality control methods are limited by low accuracy, excessive computational cost, or temporal information dependence. In this paper, we propose a concise 𝛌 domain linear distortion model and an accurate model parameters estimation method based on the original data. Since the model parameters are obtained from the original data, the proposed method is decoupled from the RDO process and can be applied to different image encoders. Experiments show that the proposed quality control algorithm achieves the highest control accuracy and the lowest delay in the literature at the same time. The application of Alibaba's e-commerce platform also shows that the proposed algorithm can significantly reduce the overall bitrate while greatly reducing the bad case ratio.
A pinned Mattila-Sj\"{o}lin type theorem for product sets

A BSTRACT . We generalize a result of McDonald and Taylor which concerns the size of the tuples of edge lengths in the set C 1 × C 2 utilizing the notion of thickness. Speciﬁcally, we show that C 1 , C 2 ⊂ R d compact sets with thickness satisfying τ ( C 1 ) τ ( C 2 ) > 1 , then the edge lengths in C 1 × C 2 corresponding to any pinned ﬁnite tree conﬁguration has non-empty interior. Originally proven for Cantor sets on the real line by McDonald and Taylor, we use the notion of thickness introduced by Falconer and Yavicoli which allows us to generalize the result of McDonald and Taylor to compact sets in R d .
Faster Last-iterate Convergence of Policy Optimization in Zero-Sum Markov Games

Multi-Agent Reinforcement Learning (MARL) -- where multiple agents learn to interact in a shared dynamic environment -- permeates across a wide range of critical applications. While there has been substantial progress on understanding the global convergence of policy optimization methods in single-agent RL, designing and analysis of efficient policy optimization algorithms in the MARL setting present significant challenges, which unfortunately, remain highly inadequately addressed by existing theory. In this paper, we focus on the most basic setting of competitive multi-agent RL, namely two-player zero-sum Markov games, and study equilibrium finding algorithms in both the infinite-horizon discounted setting and the finite-horizon episodic setting. We propose a single-loop policy optimization method with symmetric updates from both agents, where the policy is updated via the entropy-regularized optimistic multiplicative weights update (OMWU) method and the value is updated on a slower timescale. We show that, in the full-information tabular setting, the proposed method achieves a finite-time last-iterate linear convergence to the quantal response equilibrium of the regularized problem, which translates to a sublinear last-iterate convergence to the Nash equilibrium by controlling the amount of regularization. Our convergence results improve upon the best known iteration complexities, and lead to a better understanding of policy optimization in competitive Markov games.
Policy Gradients for Probabilistic Constrained Reinforcement Learning

This paper considers the problem of learning safe policies in the context of reinforcement learning (RL). In particular, we consider the notion of probabilistic safety. This is, we aim to design policies that maintain the state of the system in a safe set with high probability. This notion differs from cumulative constraints often considered in the literature. The challenge of working with probabilistic safety is the lack of expressions for their gradients. Indeed, policy optimization algorithms rely on gradients of the objective function and the constraints. To the best of our knowledge, this work is the first one providing such explicit gradient expressions for probabilistic constraints. It is worth noting that the gradient of this family of constraints can be applied to various policy-based algorithms. We demonstrate empirically that it is possible to handle probabilistic constraints in a continuous navigation problem.
TabPFN: A Transformer That Solves Small Tabular Classification Problems in a Second

We present TabPFN, a trained Transformer that can do supervised classification for small tabular datasets in less than a second, needs no hyperparameter tuning and is competitive with state-of-the-art classification methods. TabPFN performs in-context learning (ICL), it learns to make predictions using sequences of labeled examples (x, f(x)) given in the input, without requiring further parameter updates. TabPFN is fully entailed in the weights of our network, which accepts training and test samples as a set-valued input and yields predictions for the entire test set in a single forward pass. TabPFN is a Prior-Data Fitted Network (PFN) and is trained offline once, to approximate Bayesian inference on synthetic datasets drawn from our prior. This prior incorporates ideas from causal reasoning: It entails a large space of structural causal models with a preference for simple structures. On the 18 datasets in the OpenML-CC18 suite that contain up to 1 000 training data points, up to 100 purely numerical features without missing values, and up to 10 classes, we show that our method clearly outperforms boosted trees and performs on par with complex state-of-the-art AutoML systems with up to 230$\times$ speedup. This increases to a 5 700$\times$ speedup when using a GPU. We also validate these results on an additional 67 small numerical datasets from OpenML. We provide all our code, the trained TabPFN, an interactive browser demo and a Colab notebook at https://github.com/automl/TabPFN.
Exact Representation of Sparse Networks with Symmetric Nonnegative Embeddings

Many models for undirected graphs are based on factorizing the graph's adjacency matrix; these models find a vector representation of each node such that the predicted probability of a link between two nodes increases with the similarity (dot product) of their associated vectors. Recent work has shown that these models are unable to capture key structures in real-world graphs, particularly heterophilous structures, wherein links occur between dissimilar nodes. In contrast, a factorization with two vectors per node, based on logistic principal components analysis (LPCA), has been proven not only to represent such structures, but also to provide exact low-rank factorization of any graph with bounded max degree. However, this bound has limited applicability to real-world networks, which often have power law degree distributions with high max degree. Further, the LPCA model lacks interpretability since its asymmetric factorization does not reflect the undirectedness of the graph. We address these issues in two ways. First, we prove a new bound for the LPCA model in terms of arboricity rather than max degree; this greatly increases the bound's applicability to many sparse real-world networks. Second, we propose an alternative graph model whose factorization is symmetric and nonnegative, which allows for link predictions to be interpreted in terms of node clusters. We show that the bounds for exact representation in the LPCA model extend to our new model. On the empirical side, our model is optimized effectively on real-world graphs with gradient descent on a cross-entropy loss. We demonstrate its effectiveness on a variety of foundational tasks, such as community detection and link prediction.
VARIABILITY AND SPECTRAL MODELING OF THE HARD X-RAY EMISSION OF GX 339−4 IN A BRIGHT LOW/HARD STATE

We study the high-energy emission of the Galactic black hole candidate GX 339−4 using INTEGRAL/SPI and simultaneous RXTE/PCA data. By the end of 2007 January, when it reached its peak luminosity in hard X-rays, the source was in a bright hard state. The SPectrometer on INTEGRAL (SPI) data from this period show a good signal-to-noise ratio, allowing a detailed study of the spectral energy distribution up to several hundred keV. As a main result, we report on the detection of a variable hard spectral feature (⩾150 keV) which represents a significant excess with respect to the cutoff power-law shape of the spectrum. The SPI data suggest that the intensity of this feature is positively correlated with the 25–50 keV luminosity of the source and the associated variability timescale is shorter than 7 hr. The simultaneous Proportional Counter Array data, however, show no significant change in the spectral shape, indicating that the source is not undergoing a canonical state transition. We analyzed the broadband spectra in the lights of several physical models, assuming different heating mechanisms and properties of the Comptonizing plasma. For the first time, we performed quantitative model fitting with the new versatile Comptonization code belm, accounting self-consistently for the presence of a magnetic field. We show that a magnetized medium subject to pure non-thermal electron acceleration provides a framework for a physically consistent interpretation of the observed 4–500 keV emission. Moreover, we find that the spectral variability might be triggered by the variations of only one physical parameter, namely the magnetic field strength. Therefore, it appears that the magnetic field is likely to be a key parameter in the production of the Comptonized hard X-ray emission.
Distortion and Tits alternative in smooth mapping class groups

In this article, we study the smooth mapping class group of a surface S relative to a given Cantor set, that is the group of isotopy classes of orientation-preserving smooth diffeomorphisms of S which preserve this Cantor set. When the Cantor set is the standard ternary Cantor set, we prove that the subgroup consisting of diffeomorphisms which are isotopic to the identity on S does not contain any distorted elements. Moreover, we prove a weak Tits alternative for these groups.
The Nature and Function of Syntactic Categories

R. D. Borsley, "Introduction" R. Hudson, "Grammar without Functional Categories" R. Cann, "Functional versus Lexical: A Cognitive Dichotomy" D. Adger, "Feature Checking under Adjacency and VSO Clause Structure" R. D. Borsley and J. Kornfilt, "Mixed Extended Projections" R. Malouf, "Verbal Gerunds as Mixed Categories in HPSG" A. Warner, "English Auxiliaries without Lexical Rules" F. Newmeyer, "The Discrete Nature of Syntactic Categories: Against Prototype-Based Account" R. Kempson, W. Meyer-Viol, and D. Gabbay, "Syntactic Computation as Labeled Deduction: WH-a Case Study" M. -L. Rivero, "Finiteness and Second Position in Long Verb Movement Languages: Breton and Slavic" A. Abeille and D. Godard, "French Word Order and Lexical Weight"
Speed Up the Cold-Start Learning in Two-Sided Bandits with Many Arms

Multi-armed bandit (MAB) algorithms are efficient approaches to reduce the opportunity cost of online experimentation and are used by companies to find the best product from periodically refreshed product catalogs. However, these algorithms face the so-called cold-start at the onset of the experiment due to a lack of knowledge of customer preferences for new products, requiring an initial data collection phase known as the burn-in period. During this period, MAB algorithms operate like randomized experiments, incurring large burn-in costs which scale with the large number of products. We attempt to reduce the burn-in by identifying that many products can be cast into two-sided products, and then naturally model the rewards of the products with a matrix, whose rows and columns represent the two sides respectively. Next, we design two-phase bandit algorithms that first use subsampling and low-rank matrix estimation to obtain a substantially smaller targeted set of products and then apply a UCB procedure on the target products to find the best one. We theoretically show that the proposed algorithms lower costs and expedite the experiment in cases when there is limited experimentation time along with a large product set. Our analysis also reveals three regimes of long, short, and ultra-short horizon experiments, depending on dimensions of the matrix. Empirical evidence from both synthetic data and a real-world dataset on music streaming services validates this superior performance.
HUXt—An open source, computationally efficient reduced-physics solar wind model, written in Python

HUXt is an open source numerical model of the solar wind written in Python. It is based on the solution of the 1D inviscid Burger’s equation. This reduced-physics approach produces solar wind flow simulations that closely emulate the flow produced by 3-D magnetohydrodynamic (MHD) solar wind models at a small fraction of the computational expense. While not intended as a replacement for 3-D MHD, the simplicity and computational efficiency of HUXt offers several key advantages that enable experiments and the use of techniques that would otherwise be cost prohibitive. For example, large ensembles of 102–105 members can easily be run with modest computing resources, which are useful for exploring and quantifying the uncertainty in space weather predictions, as well as for the application of some data assimilation methods. In this article we present the developments in the latest version of HUXt, v4.0, and discuss our plans for future developments and applications of the model. The three key developments in v4.0 are: 1) a restructuring of the models solver to enable fully time-dependent boundary conditions, such that HUXt can in principle be initialised with in-situ observations from any of the fleet of heliospheric monitors; 2) new functionality to trace streaklines through the HUXt flow solutions, which can be used to track features such as the Heliospheric Current Sheet; 3) introduction of a small test-suite so that we can better ensure the reliability and reproducibility of HUXt simulations for all users across future versions. Other more minor developments are discussed in the article. Future applications of HUXt are discussed, including the development of both sequential and variational data assimilation schemes for assimilation of both remote sensing and in-situ plasma measures. Finally, we briefly discuss the progress of transitioning HUXt into an operational model at the UK’s Met Office Space Weather Operations Center as part of the UK governments SWIMMR programme.
On the infinite-depth limit of finite-width neural networks

In this paper, we study the infinite-depth limit of finite-width residual neural networks with random Gaussian weights. With proper scaling, we show that by fixing the width and taking the depth to infinity, the pre-activations converge in distribution to a zero-drift diffusion process. Unlike the infinite-width limit where the pre-activation converge weakly to a Gaussian random variable, we show that the infinite-depth limit yields different distributions depending on the choice of the activation function. We document two cases where these distributions have closed-form (different) expressions. We further show an intriguing change of regime phenomenon of the post-activation norms when the width increases from 3 to 4. Lastly, we study the sequential limit infinite-depth-then-infinite-width and compare it with the more commonly studied infinite-width-then-infinite-depth limit.
Behaviour of elements from lithium to europium in stars with and without planets

We conducted an analysis of the distribution of elements from lithium to europium in 200 dwarfs in the solar neighbourhood (˜20 pc) with temperatures in the range 4800-6200 K and metallicities [Fe/H] higher than -0.5 dex. Determinations of atmospheric parameters and the chemical compositions of the dwarfs were taken from our previous studies. We found that the lithium abundances in the planet-hosting solar-analogue stars of our sample were lower than those in the stars without planetary systems. Our results reveal no significant differences exceeding the determination errors for the abundances of investigated elements, except for aluminium and barium, which are more and less abundant in the planet-hosting stars, respectively. We did not find confident dependences of the lithium, aluminium and barium abundances on the ages of our target stars (which is probably because of the small number of stars). Furthermore, we found no correlation between the abundance differences in [El/Fe] and the condensation temperature (Tcond) for stars in the 16 Cyg binary system, unlike the case for 51 Peg (HD 217014), for which a slight excess of volatile elements and a deficit of refractories were obtained relative to those of solar twins. We found that one of the components of 16 Cyg exhibits a slightly higher average abundance than its counterpart ( = 0.08 ±0.02 dex); however, no significant abundance trend versus Tcond was observed. Owing to the relatively large errors, we cannot provide further constraints for this system.
NeRF: Neural Radiance Field in 3D Vision, A Comprehensive Review

—Neural Radiance Field (NeRF), a new novel view synthesis with implicit scene representation has taken the ﬁeld of Computer Vision by storm. As a novel view synthesis and 3D reconstruction method, NeRF models ﬁnd applications in robotics, urban mapping, autonomous navigation, virtual reality/augmented reality, and more. Since the original paper by Mildenhall et al., more than 250 preprints were published, with more than 100 eventually being accepted in tier one Computer Vision Conferences. Given NeRF popularity and the current interest in this research area, we believe it necessary to compile a comprehensive survey of NeRF papers from the past two years, which we organized into both architecture, and application based taxonomies. We also provide an introduction to the theory of NeRF based novel view synthesis, and a benchmark comparison of the performance and speed of key NeRF models. By creating this survey, we hope to introduce new researchers to NeRF, provide a helpful reference for inﬂuential works in this ﬁeld, as well as motivate future research directions with our discussion section.
Visual Prompt Tuning for Generative Transfer Learning

Learning generative image models from various domains efficiently needs transferring knowledge from an image synthesis model trained on a large dataset. We present a recipe for learning vision transformers by generative knowledge transfer. We base our framework on generative vision transformers representing an image as a sequence of visual tokens with the autoregressive or non-autoregressive transformers. To adapt to a new domain, we employ prompt tuning, which prepends learnable tokens called prompts to the image token sequence and introduces a new prompt design for our task. We study on a variety of visual domains with varying amounts of training images. We show the effectiveness of knowledge transfer and a significantly better image generation quality.11https://github.com/google-research/generative_transfer
Many-Body Localization and Thermalization in Quantum Statistical Mechanics

We review some recent developments in the statistical mechanics of isolated quantum systems. We provide a brief introduction to quantum thermalization, paying particular attention to the eigenstate thermalization hypothesis (ETH) and the resulting single-eigenstate statistical mechanics. We then focus on a class of systems that fail to quantum thermalize and whose eigenstates violate the ETH: These are the many-body Anderson-localized systems; their long-time properties are not captured by the conventional ensembles of quantum statistical mechanics. These systems can forever locally remember information about their local initial conditions and are thus of interest for possibilities of storing quantum information. We discuss key features of many-body localization (MBL) and review a phenomenology of the MBL phase. Single-eigenstate statistical mechanics within the MBL phase reveal dynamically stable ordered phases, and phase transitions among them, that are invisible to equilibrium statistical mechanics and.
Block-wise Training of Residual Networks via the Minimizing Movement Scheme

End-to-end backpropagation has a few shortcomings: it requires loading the entire model during training, which can be impossible in constrained settings, and suffers from three locking problems (forward locking, update locking and backward locking), which prohibit training the layers in parallel. Solving layer-wise optimization problems can address these problems and has been used in on-device training of neural networks. We develop a layer-wise training method, particularly welladapted to ResNets, inspired by the minimizing movement scheme for gradient flows in distribution space. The method amounts to a kinetic energy regularization of each block that makes the blocks optimal transport maps and endows them with regularity. It works by alleviating the stagnation problem observed in layer-wise training, whereby greedily-trained early layers overfit and deeper layers stop increasing test accuracy after a certain depth. We show on classification tasks that the test accuracy of block-wise trained ResNets is improved when using our method, whether the blocks are trained sequentially or in parallel.