Miracle-dz/UniFAR-FTUs huggingface dataset:
Spectral Augmentations for Graph Contrastive Learning

Contrastive learning has emerged as a premier method for learning representations with or without supervision. Recent studies have shown its utility in graph representation learning for pre-training. Despite successes, the understanding of how to design effective graph augmentations that can capture structural properties common to many different types of downstream graphs remains incomplete. We propose a set of well-motivated graph transformation operations derived via graph spectral analysis to provide a bank of candidates when constructing augmentations for a graph contrastive objective, enabling contrastive learning to capture useful structural representation from pre-training graph datasets. We first present a spectral graph cropping augmentation that involves filtering nodes by applying thresholds to the eigenvalues of the leading Laplacian eigenvectors. Our second novel augmentation reorders the graph frequency components in a structural Laplacian-derived position graph embedding. Further, we introduce a method that leads to improved views of local subgraphs by performing alignment via global random walk embeddings. Our experimental results indicate consistent improvements in out-of-domain graph data transfer compared to state-of-the-art graph contrastive learning methods, shedding light on how to design a graph learner that is able to learn structural properties common to diverse graph types.
On the applicability of the Landau-Zener formula to axion-photon conversion

Axions and photons resonantly interconvert in regions where the plasma frequency approximately equals the axion mass. This process is directly analogous to an avoided level crossing in quantum mechanics, for which the celebrated Landau-Zener (LZ) formula provides a simple, non-perturbative solution for the conversion probability. The LZ formula is commonly used in studies of axion-photon conversion; however, in this context, it relies on the assumption that the magnetic field variation is small compared to variations of the plasma frequency, which is frequently not the case in real plasmas. We derive a generalised version of the LZ formula by allowing the boundaries to be located at a scale that is similar to the inhomogeneities. We find that the LZ formula fails when the oscillation range is small compared to the resonance region. This failure is more severe in the adiabatic limit, when the plasma frequency varies slowly, resulting in a conversion probability that is not maximal as opposed to a naive application of the LZ formula. Moreover, we consider circumstances where the generalised LZ formula does not apply and present an alternative semi-classical approximation with complementary regime of validity.
Instantaneous normal modes in liquids: a heterogeneous-elastic-medium approach

The concept of vibrational density of states in glasses has been mirrored in liquids by the instantaneous-normal-mode spectrum. While in glasses instantaneous configurations correspond to minima of the potential-energy hypersurface and all eigenvalues of the associated Hessian matrix are therefore positive, in liquids this is no longer true, and modes corresponding to both positive and negative eigenvalues exist. The instantaneous-normal-mode spectrum has been numerically investigated in the past, and it has been demonstrated to bring important information on the liquid dynamics. A systematic deeper theoretical understanding is now needed. Heterogeneous-elasticity theory has proven to be successful in explaining many details of the low-frequency excitations in glasses, ranging from the thoroughly studied boson peak, down to the more elusive non-phononic excitations observed in numerical simulations at the lowest frequencies. Here we present an extension of heterogeneous-elasticity theory to the liquid state, and show that the outcome of the theory agrees well to the results of extensive molecular-dynamics simulations of a model liquid at different temperatures. We show that the spectral shape strongly depends on temperature, being symmetric at high temperatures and becoming rather asymmetric at low temperatures, close to the dynamical critical temperature. Most importantly, we demonstrate that the theory naturally reproduces a surprising phenomenon, a zero-energy spectral singularity with a cusp-like character developing in the vibrational spectra upon cooling. This feature, known from a few previous numerical studies, has been generally overlooked in the past due to a misleading representation of the data. We provide a thorough analysis of this issue, based on both very accurate predictions of our theory, and computational studies of model liquid systems with extended size.
Never Skip Leg Day Again: Training the Lower Body with Vertical Jumps in a Virtual Reality Exergame

Virtual Reality (VR) exergames can increase engagement in and motivation for physical activities. Most VR exergames focus on the upper body because many VR setups only track the users’ heads and hands. To become a serious alternative to existing exercise programs, VR exergames must provide a balanced workout and train the lower limbs, too. To address this issue, we built a VR exergame focused on vertical jump training to explore full-body exercise applications. To create a safe and effective training, nine domain experts participated in our prototype design. Our mixed-methods study confirms that the jump-centered exercises provided a worthy challenge and positive player experience, indicating long-term retention. Based on our findings, we present five design implications to guide future work: avoid an unintended forward drift, consider technical constraints, address safety concerns in full-body VR exergames, incorporate rhythmic elements with fluent movement patterns, adapt difficulty to players’ fitness progression status.
Construction Grammar Provides Unique Insight into Neural Language Models

Construction Grammar (CxG) has recently been used as the basis for probing studies that have investigated the performance of large pretrained language models (PLMs) with respect to the structure and meaning of constructions. In this position paper, we make suggestions for the continuation and augmentation of this line of research. We look at probing methodology that was not designed with CxG in mind, as well as probing methodology that was designed for specific constructions. We analyse selected previous work in detail, and provide our view of the most important challenges and research questions that this promising new field faces.
Efficient Online Reinforcement Learning with Offline Data

Sample efficiency and exploration remain major challenges in online reinforcement learning (RL). A powerful approach that can be applied to address these issues is the inclusion of offline data, such as prior trajectories from a human expert or a sub-optimal exploration policy. Previous methods have relied on extensive modifications and additional complexity to ensure the effective use of this data. Instead, we ask: can we simply apply existing off-policy methods to leverage offline data when learning online? In this work, we demonstrate that the answer is yes; however, a set of minimal but important changes to existing off-policy RL algorithms are required to achieve reliable performance. We extensively ablate these design choices, demonstrating the key factors that most affect performance, and arrive at a set of recommendations that practitioners can readily apply, whether their data comprise a small number of expert demonstrations or large volumes of sub-optimal trajectories. We see that correct application of these simple recommendations can provide a $\mathbf{2.5\times}$ improvement over existing approaches across a diverse set of competitive benchmarks, with no additional computational overhead. We have released our code at https://github.com/ikostrikov/rlpd.
Holistic Deep-Reinforcement-Learning-based Training of Autonomous Navigation Systems

In recent years, Deep Reinforcement Learning emerged as a promising approach for autonomous navigation of ground vehicles and has been utilized in various areas of navigation such as cruise control, lane changing, or obstacle avoidance. However, most research works either focus on providing an end-to-end solution training the whole system using Deep Reinforcement Learning or focus on one specific aspect such as local motion planning. This however, comes along with a number of problems such as catastrophic forgetfulness, inefficient navigation behavior, and non-optimal synchronization between different entities of the navigation stack. In this paper, we propose a holistic Deep Reinforcement Learning training approach in which the training procedure is involving all entities of the navigation stack. This should enhance the synchronization between- and understanding of all entities of the navigation stack and as a result, improve navigational performance. We trained several agents with a number of different observation spaces to study the impact of different input on the navigation behavior of the agent. In profound evaluations against multiple learning-based and classic model-based navigation approaches, our proposed agent could outperform the baselines in terms of efficiency and safety attaining shorter path lengths, less roundabout paths, and less collisions.
Efficient optimization of deep neural quantum states toward machine precision

Neural quantum states (NQSs) have emerged as a novel promising numerical method to solve the quantum many-body problem. However, it has remained a central challenge to train modern large-scale deep network architectures to desired quantum state accuracy, which would be vital in utilizing the full power of NQSs and making them competitive or superior to conventional numerical approaches. Here, we propose a minimum-step stochastic reconfiguration (MinSR) method that reduces the optimization complexity by orders of magnitude while keeping similar accuracy as compared to conventional stochastic reconfiguration. MinSR allows for accurate training on unprecedentedly deep NQS with up to 64 layers and more than $10^5$ parameters in the spin-1/2 Heisenberg $J_1$-$J_2$ models on the square lattice. We find that this approach yields better variational energies as compared to existing numerical results and we further observe that the accuracy of our ground state calculations approaches different levels of machine precision on modern GPU and TPU hardware. The MinSR method opens up the potential to make NQS superior as compared to conventional computational methods with the capability to address yet inaccessible regimes for two-dimensional quantum matter in the future.
RLSbench: Domain Adaptation Under Relaxed Label Shift

Despite the emergence of principled methods for domain adaptation under label shift, their sensitivity to shifts in class conditional distributions is precariously under explored. Meanwhile, popular deep domain adaptation heuristics tend to falter when faced with label proportions shifts. While several papers modify these heuristics in attempts to handle label proportions shifts, inconsistencies in evaluation standards, datasets, and baselines make it difficult to gauge the current best practices. In this paper, we introduce RLSbench, a large-scale benchmark for relaxed label shift, consisting of $>$500 distribution shift pairs spanning vision, tabular, and language modalities, with varying label proportions. Unlike existing benchmarks, which primarily focus on shifts in class-conditional $p(x|y)$, our benchmark also focuses on label marginal shifts. First, we assess 13 popular domain adaptation methods, demonstrating more widespread failures under label proportion shifts than were previously known. Next, we develop an effective two-step meta-algorithm that is compatible with most domain adaptation heuristics: (i) pseudo-balance the data at each epoch; and (ii) adjust the final classifier with target label distribution estimate. The meta-algorithm improves existing domain adaptation heuristics under large label proportion shifts, often by 2--10\% accuracy points, while conferring minimal effect ($<$0.5\%) when label proportions do not shift. We hope that these findings and the availability of RLSbench will encourage researchers to rigorously evaluate proposed methods in relaxed label shift settings. Code is publicly available at https://github.com/acmi-lab/RLSbench.
Offline Learning in Markov Games with General Function Approximation

We study offline multi-agent reinforcement learning (RL) in Markov games, where the goal is to learn an approximate equilibrium -- such as Nash equilibrium and (Coarse) Correlated Equilibrium -- from an offline dataset pre-collected from the game. Existing works consider relatively restricted tabular or linear models and handle each equilibria separately. In this work, we provide the first framework for sample-efficient offline learning in Markov games under general function approximation, handling all 3 equilibria in a unified manner. By using Bellman-consistent pessimism, we obtain interval estimation for policies' returns, and use both the upper and the lower bounds to obtain a relaxation on the gap of a candidate policy, which becomes our optimization objective. Our results generalize prior works and provide several additional insights. Importantly, we require a data coverage condition that improves over the recently proposed"unilateral concentrability". Our condition allows selective coverage of deviation policies that optimally trade-off between their greediness (as approximate best responses) and coverage, and we show scenarios where this leads to significantly better guarantees. As a new connection, we also show how our algorithmic framework can subsume seemingly different solution concepts designed for the special case of two-player zero-sum games.
Hatemongers ride on echo chambers to escalate hate speech diffusion

Abstract Recent years have witnessed a swelling rise of hateful and abusive content over online social networks. While detection and moderation of hate speech have been the early go-to countermeasures, the solution requires a deeper exploration of the dynamics of hate generation and propagation. We analyze more than 32 million posts from over 6.8 million users across three popular online social networks to investigate the interrelations between hateful behavior, information dissemination, and polarized organization mediated by echo chambers. We find that hatemongers play a more crucial role in governing the spread of information compared to singled-out hateful content. This observation holds for both the growth of information cascades as well as the conglomeration of hateful actors. Dissection of the core-wise distribution of these networks points towards the fact that hateful users acquire a more well-connected position in the social network and often flock together to build up information cascades. We observe that this cohesion is far from mere organized behavior; instead, in these networks, hatemongers dominate the echo chambers—groups of users actively align themselves to specific ideological positions. The observed dominance of hateful users to inflate information cascades is primarily via user interactions amplified within these echo chambers. We conclude our study with a cautionary note that popularity-based recommendation of content is susceptible to be exploited by hatemongers given their potential to escalate content popularity via echo-chambered interactions.
Experimental Studies of Supersonic Flow past a Circular Cylinder

Experimental studies using shock tubes are presented of the mixed Type flow around a circular cylinder placed normal to the general flow of Mach number 1.35, 1.8, 2.7, 3.5, 4.0, and 6.0. With the Mach-Zehnder interferometer and schlieren method the followings were measured: a) shape of detached shock wave, b) distance d of shock wave from the cylinder nose, c) pressure coefficient C p on the surface of the cylinder, d) density distribution behind the detached shock, e) streamlines in subsonic region.
Hybrid Genetic Optimisation for Quantum Feature Map Design

Kernel methods are an important class of techniques in machine learning. To be effective, good feature maps are crucial for mapping non-linearly separable input data into a higher dimensional (feature) space, thus allowing the data to be linearly separable in feature space. Previous work has shown that quantum feature map design can be automated for a given dataset using NSGA-II, a genetic algorithm, while both minimizing circuit size and maximizing classification accuracy. However, the evaluation of the accuracy achieved by a candidate feature map is costly. In this work, we demonstrate the suitability of kernel-target alignment as a substitute for accuracy in genetic algorithm-based quantum feature map design. Kernel-target alignment is faster to evaluate than accuracy and doesn't require some data points to be reserved for its evaluation. To further accelerate the evaluation of genetic fitness, we provide a method to approximate kernel-target alignment. To improve kernel-target alignment and root mean squared error, the final trainable parameters of the generated circuits are further trained using COBYLA to determine whether a hybrid approach applying conventional circuit parameter training can easily complement the genetic structure optimization approach. A total of eight new approaches are compared to the original across nine varied binary classification problems from the UCI machine learning repository, showing that kernel-target alignment and its approximation produce feature map circuits enabling comparable accuracy to the previous work but with larger margins on training data (in excess of 20\% larger) that improve further with circuit parameter training.
AMD-HookNet for Glacier Front Segmentation

Knowledge on changes in glacier calving front positions is important for assessing the status of glaciers. Remote sensing imagery provides the ideal database for monitoring calving front positions; however, it is not feasible to perform this task manually for all calving glaciers globally due to time constraints. Deep-learning-based methods have shown great potential for glacier calving front delineation from optical and radar satellite imagery. The calving front is represented as a single thin line between the ocean and the glacier, which makes the task vulnerable to inaccurate predictions. The limited availability of annotated glacier imagery leads to a lack of data diversity (not all possible combinations of different weather conditions, terminus shapes, sensors, etc. are present in the data), which exacerbates the difficulty of accurate segmentation. In this article, we propose attention-multihooking-deep-supervision HookNet (AMD-HookNet), a novel glacier calving front segmentation framework for synthetic aperture radar (SAR) images. The proposed method aims to enhance the feature representation capability through multiple information interactions between low-resolution and high-resolution inputs based on a two-branch U-Net. The attention mechanism, integrated into the two branch U-Net, aims to interact between the corresponding coarse and fine-grained feature maps. This allows the network to automatically adjust feature relationships, resulting in accurate pixel classification predictions. Extensive experiments and comparisons on the challenging glacier segmentation benchmark dataset CaFFe show that our AMD-HookNet achieves a mean distance error (MDE) of 438 m to the ground truth outperforming the current state of the art by 42%, which validates its effectiveness.
Contrast with Reconstruct: Contrastive 3D Representation Learning Guided by Generative Pretraining

Mainstream 3D representation learning approaches are built upon contrastive or generative modeling pretext tasks, where great improvements in performance on various downstream tasks have been achieved. However, we find these two paradigms have different characteristics: (i) contrastive models are data-hungry that suffer from a representation over-fitting issue; (ii) generative models have a data filling issue that shows inferior data scaling capacity compared to contrastive models. This motivates us to learn 3D representations by sharing the merits of both paradigms, which is non-trivial due to the pattern difference between the two paradigms. In this paper, we propose Contrast with Reconstruct (ReCon) that unifies these two paradigms. ReCon is trained to learn from both generative modeling teachers and single/cross-modal contrastive teachers through ensemble distillation, where the generative student guides the contrastive student. An encoder-decoder style ReCon-block is proposed that transfers knowledge through cross attention with stop-gradient, which avoids pretraining over-fitting and pattern difference issues. ReCon achieves a new state-of-the-art in 3D representation learning, e.g., 91.26% accuracy on ScanObjectNN. Codes have been released at https://github.com/qizekun/ReCon.
On the Role of Contrastive Representation Learning in Adversarial Robustness: An Empirical Study

—Self-supervised contrastive learning has solved one of the signiﬁcant obstacles in deep learning by alleviating the annotation cost. This advantage comes with the price of false negative-pair selection without any label information. Supervised contrastive learning has emerged as an extension of contrastive learning to eliminate this issue. However, aside from accuracy, there is a lack of understanding about the impacts of adversarial training on the representations learned by these learning schemes. In this work, we utilize supervised learning as a baseline to comprehensively study the robustness of contrastive and supervised contrastive learning under different adversarial training scenarios. Then, we begin by looking at how adversarial training affects the learned representations in hidden layers, discovering more redundant representations between layers of the model. Our results on CIFAR-10 and CIFAR-100 image classiﬁcation benchmarks demonstrate that this redundancy is highly reduced by adversarial ﬁne-tuning applied to the contrastive learning scheme, leading to more robust representations. However, adversarial ﬁne-tuning is not very effective for supervised contrastive learning and supervised learning schemes. Our code is released at https://github.com/softsys4ai/CL-Robustness.
The effect of nonnormality on some multivariate tests and robustness to nonnormality in the linear model

SUMMARY The effect of nonnormality on multivariate regression tests, on thle one-way multivariate analysis of variance and on tests of equality of covariance matrices is studied following the approach of Box & Watson (1962). In the nonnormal case, an approximation to the distribution of a generalized Mahalanobis distance type of statistic for the multivariate regression problem is derived. It is shown that sensitivity to nonnormality in the multivariate observations is determined by the extent of nonnormality of the regressors. The randomization distribution of the generalized Mahalanobis distance is deduced. The multivariate analysis of variance is found to be robust to nonnormality whereas the tests for equality of covariance matrices are found to be sensitive to nonnormality. An explanation for this varying degree of sensitivity to nonnormality is given.