Miracle-dz/UniFAR-FTUs huggingface dataset:
On the Last-iterate Convergence in Time-varying Zero-sum Games: Extra Gradient Succeeds where Optimism Fails

Last-iterate convergence has received extensive study in two player zero-sum games starting from bilinear, convex-concave up to settings that satisfy the MVI condition. Typical methods that exhibit last-iterate convergence for the aforementioned games include extra-gradient (EG) and optimistic gradient descent ascent (OGDA). However, all the established last-iterate convergence results hold for the restrictive setting where the underlying repeated game does not change over time. Recently, a line of research has focused on regret analysis of OGDA in time-varying games, i.e., games where payoffs evolve with time; the last-iterate behavior of OGDA and EG in time-varying environments remains unclear though. In this paper, we study the last-iterate behavior of various algorithms in two types of unconstrained, time-varying, bilinear zero-sum games: periodic and convergent perturbed games. These models expand upon the usual repeated game formulation and incorporate external environmental factors, such as the seasonal effects on species competition and vanishing external noise. In periodic games, we prove that EG will converge while OGDA and momentum method will diverge. This is quite surprising, as to the best of our knowledge, it is the first result that indicates EG and OGDA have qualitatively different last-iterate behaviors and do not exhibit similar behavior. In convergent perturbed games, we prove all these algorithms converge as long as the game itself stabilizes with a faster rate than $1/t$.
Towards an Interpretable Representation of Speaker Identity via Perceptual Voice Qualities

Unlike other data modalities such as text and vision, speech does not lend itself to easy interpretation. While lay people can understand how to describe an image or sentence via perception, non-expert descriptions of speech often end at high-level demographic information, such as gender or age. In this paper, we propose a possible interpretable representation of speaker identity based on perceptual voice qualities (PQs). By adding gendered PQs to the pathology-focused Consensus Auditory-Perceptual Evaluation of Voice (CAPE-V) protocol, our PQ-based approach provides a perceptual latent space of the character of adult voices that is an intermediary of abstraction between high-level demographics and low-level acoustic, physical, or learned representations. Contrary to prior belief, we demonstrate that these PQs are hearable by ensembles of non-experts, and further demonstrate that the information encoded in a PQ-based representation is predictable by various speech representations.
Harmonic Control Lyapunov Barrier Functions for Constrained Optimal Control with Reach-Avoid Specifications

This paper introduces harmonic control Lyapunov barrier functions (harmonic CLBF) that aid in constrained control problems such as reach-avoid problems. Harmonic CLBFs exploit the maximum principle that harmonic functions satisfy to encode the properties of control Lyapunov barrier functions (CLBFs). As a result, they can be initiated at the start of an experiment rather than trained based on sample trajectories. The control inputs are selected to maximize the inner product of the system dynamics with the steepest descent direction of the harmonic CLBF. Numerical results are presented with four different systems under different reach-avoid environments. Harmonic CLBFs show a significantly low risk of entering unsafe regions and a high probability of entering the goal region.
High‐T, low‐P metamorphism in the Palaeoproterozoic Halls Creek Orogen, northern Australia: the middle crustal response to a mantle‐related transient thermal pulse

High‐T, low‐P metamorphic rocks of the Palaeoproterozoic central Halls Creek Orogen in northern Australia are characterised by low radiogenic heat production, high upper crustal thermal gradients (locally exceeding 40 °C km−1) sustained for over 30 Myr, and a large number of layered mafic‐ultramafic intrusions with mantle‐related geochemical signatures. In order to account for this combination of geological and thermal characteristics, we model the middle crustal response to a transient mantle‐related heat pulse resulting from a temporary reduction in the thickness of the mantle lithosphere. This mechanism has the potential to raise mid‐crustal temperatures by 150–400 °C within 10–20 Myr following initiation of the mantle temperature anomaly, via conductive dissipation through the crust. The magnitude and timing of maximum temperatures attained depend strongly on the proximity, duration and lateral extent of the thermal anomaly in the mantle lithosphere, and decrease sharply in response to anomalies that are seated deeper than 50–60 km, maintained for <5 Myr in duration and/or have half‐widths <100 km. Maximum temperatures are also intimately linked to the thermal properties of the model crust, primarily due to their influence on the steady‐state (background) thermal gradient. The amplitudes of temperature increases in the crust are principally a function of depth, and are broadly independent of crustal thermal parameters.
Global mesoscale variability from the Geosat Exact Repeat Mission - Correlation with ocean depth

A simple derivative filter technique for extracting mesoscale variability from satellite altimeter data having large-amplitude, long-wavelength errors, with no data adjustments necessary, is presented. The method is applied to the first year of Geosat Exact Repeat Mission data. The main advantage over previous filter methods is that the derivative filter is only two points long, so that it produces no edge effects. Short profiles can thus be included in the analysis. The derivative filter also transforms a dynamic height profile into a dynamic slope profile. The dynamic slope of the sea surface is a parameter that can be measured independently. Moreover, if the ocean current is in geostrophic balance, then the dynamic gradient of the sea surface is proportional to the current velocity.
Experimental petrology of monotonous intermediate magmas

Abstract Large-volume, high-crystallinity, chemically homogeneous ignimbrites, dubbed ‘monotonous intermediates’, provide a unique opportunity to investigate the evolution of crustal magmatic reservoirs. We present the results of hydrothermal experiments on a dacite from Fish Canyon Tuff (FCT) in Colorado (USA), a classic example of a monotonous intermediate deposit, in order to characterize the variations in chemical and physical properties of hydrous dacite magmas as a function of temperature. The experiments (200 MPa, 720–1100°C) span the inferred pre-eruptive conditions of FCT magmas, and are shown to provide the best match to the chemical and physical properties of the erupted magmas at 790±10°C under conditions at or close to water-saturation. The results show the important effect of water content in controlling the chemical and physical evolution of magma, and the contrasted behaviour of water-saturated v. water-undersaturated magmas. In both cases, however, there is a broad interval of temperature (200°C) over which crystal fraction changes little. By recasting this behaviour in terms of enthalpy, rather than temperature, as the independent variable we show that this interval corresponds to a minimum in the change in crystallinity per unit of energy added or subtracted from the system, such that small perturbations to the heat content of the system (e.g. by cooling or new magma injections) results in very little change in magma properties. The crystal content in this interval is 55–65 wt%, which is close to the phenocryst content (40–55 wt%) of monotonous intermediates. We propose that crystal-rich magmas tend to settle in this ‘petrological trap’, changing little in physical and chemical properties over time as the system grows. Petrological trapping enables very large volumes of intermediate magma to accumulate in the shallow crust until such time as the net buoyancy force of these crystal-rich magma is sufficient to overcome the strength of the roof rocks, leading to a potentially very large eruption.
OneAdapt: Fast Adaptation for Deep Learning Applications via Backpropagation

Deep learning inference on streaming media data, such as object detection in video or LiDAR feeds and text extraction from audio waves, is now ubiquitous. To achieve high inference accuracy, these applications typically require significant network bandwidth to gather high-fidelity data and extensive GPU resources to run deep neural networks (DNNs). While the high demand for network bandwidth and GPU resources could be substantially reduced by optimally adapting the configuration knobs, such as video resolution and frame rate, current adaptation techniques fail to meet three requirements simultaneously: adapt configurations (i) with minimum extra GPU or bandwidth overhead (ii) to reach near-optimal decisions based on how the data affects the final DNN's accuracy, and (iii) do so for a range of configuration knobs. This paper presents OneAdapt, which meets these requirements by leveraging a gradient-ascent strategy to adapt configuration knobs. The key idea is to embrace DNNs' differentiability to quickly estimate the accuracy's gradient to each configuration knob, called AccGrad. Specifically, OneAdapt estimates AccGrad by multiplying two gradients: InputGrad (i.e., how each configuration knob affects the input to the DNN) and DNNGrad (i.e., how the DNN input affects the DNN inference output). We evaluate OneAdapt across five types of configurations, four analytic tasks, and five types of input data. Compared to state-of-the-art adaptation schemes, OneAdapt cuts bandwidth usage and GPU usage by 15-59% while maintaining comparable accuracy or improves accuracy by 1-5% while using equal or fewer resources.
Learning to Reach Goals via Diffusion

We present a novel perspective on goal-conditioned reinforcement learning by framing it within the context of denoising diffusion models. Analogous to the diffusion process, where Gaussian noise is used to create random trajectories that walk away from the data manifold, we construct trajectories that move away from potential goal states. We then learn a goal-conditioned policy to reverse these deviations, analogously to the score function. This approach, which we call Merlin, can reach specified goals from an arbitrary initial state without learning a separate value function. In contrast to recent works utilizing diffusion models in offline RL, Merlin stands out as the first method to perform diffusion in the state space, requiring only one ``denoising"iteration per environment step. We experimentally validate our approach in various offline goal-reaching tasks, demonstrating substantial performance enhancements compared to state-of-the-art methods while improving computational efficiency over other diffusion-based RL methods by an order of magnitude. Our results suggest that this perspective on diffusion for RL is a simple, scalable, and practical direction for sequential decision making.
Emergence of flat bands in the quasicrystal limit of boron nitride twisted bilayers

We investigate the electronic structure and the optical absorption onset of close-to-30\degree twisted hexagonal boron nitride bilayers. Our study is carried out with a purposely developed tight-binding model validated against DFT simulations. We demonstrate that approaching 30\degree (quasicrystal limit), all bilayers sharing the same moir\'e supercell develop identical band structures, irrespective of their stacking sequence. This band structure features a bundle of flat bands laying slightly above the bottom conduction state which is responsible for an intense peak at the onset of the absorption spectrum. These results suggest the presence of strong, stable and stacking-independent excitons in boron nitride 30\degree-twisted bilayers. By carefully analyzing the electronic structure and its spatial distribution, we elucidate the origin of these states as moir\'e-induced K-valley scattering due to interlayer B$-$B coupling. We take advantage of the the physical transparency of the tight-binding parameters to derive a simple triangular model based on the B sublattice that accurately describes the emergence of the bundle. Being our conclusions very general, we predict that a similar bundle should emerge in other close-to-30{\degree} bilayers, like transition metal dichalcogenides, shedding new light on the unique potential of 2D materials.
Out-of-Distribution Detection by Leveraging Between-Layer Transformation Smoothness

Effective out-of-distribution (OOD) detection is crucial for reliable machine learning models, yet most current methods are limited in practical use due to requirements like access to training data or intervention in training. We present a novel method for detecting OOD data in Transformers based on transformation smoothness between intermediate layers of a network (BLOOD), which is applicable to pre-trained models without access to training data. BLOOD utilizes the tendency of between-layer representation transformations of in-distribution (ID) data to be smoother than the corresponding transformations of OOD data, a property that we also demonstrate empirically. We evaluate BLOOD on several text classification tasks with Transformer networks and demonstrate that it outperforms methods with comparable resource requirements. Our analysis also suggests that when learning simpler tasks, OOD data transformations maintain their original sharpness, whereas sharpness increases with more complex tasks.
Automatic Multi-Path Web Story Creation from a Structural Article

Web articles such as Wikipedia serve as one of the major sources of knowledge dissemination and online learning. However, their in-depth information--often in a dense text format--may not be suitable for mobile browsing, even in a responsive UI. We propose an automatic approach that converts a structural article of any length into a set of interactive Web Stories that are ideal for mobile experiences. We focused on Wikipedia articles and developed Wiki2Story, a pipeline based on language and layout models, to demonstrate the concept. Wiki2Story dynamically slices an article and plans one to multiple Story paths according to the document hierarchy. For each slice, it generates a multi-page summary Story composed of text and image pairs in visually-appealing layouts. We derived design principles from an analysis of manually-created Story practices. We executed our pipeline on 500 Wikipedia documents and conducted user studies to review selected outputs. Results showed that Wiki2Story effectively captured and presented salient content from the original articles and sparked interest in viewers.
Low-Resource Languages Jailbreak GPT-4

AI safety training and red-teaming of large language models (LLMs) are measures to mitigate the generation of unsafe content. Our work exposes the inherent cross-lingual vulnerability of these safety mechanisms, resulting from the linguistic inequality of safety training data, by successfully circumventing GPT-4's safeguard through translating unsafe English inputs into low-resource languages. On the AdvBenchmark, GPT-4 engages with the unsafe translated inputs and provides actionable items that can get the users towards their harmful goals 79% of the time, which is on par with or even surpassing state-of-the-art jailbreaking attacks. Other high-/mid-resource languages have significantly lower attack success rate, which suggests that the cross-lingual vulnerability mainly applies to low-resource languages. Previously, limited training on low-resource languages primarily affects speakers of those languages, causing technological disparities. However, our work highlights a crucial shift: this deficiency now poses a risk to all LLMs users. Publicly available translation APIs enable anyone to exploit LLMs' safety vulnerabilities. Therefore, our work calls for a more holistic red-teaming efforts to develop robust multilingual safeguards with wide language coverage.
Hire When You Need to: Gradual Participant Recruitment for Auction-based Federated Learning

The success of Federated Learning (FL) depends on the quantity and quality of the data owners (DOs) as well as their motivation to join FL model training. Reputation-based FL participant selection methods have been proposed. However, they still face the challenges of the cold start problem and potential selection bias towards highly reputable DOs. Such a bias can result in lower reputation DOs being prematurely excluded from future FL training rounds, thereby reducing the diversity of training data and the generalizability of the resulting models. To address these challenges, we propose the Gradual Participant Selection scheme for Auction-based Federated Learning (GPS-AFL). Unlike existing AFL incentive mechanisms which generally assume that all DOs required for an FL task must be selected in one go, GPS-AFL gradually selects the required DOs over multiple rounds of training as more information is revealed through repeated interactions. It is designed to strike a balance between cost saving and performance enhancement, while mitigating the drawbacks of selection bias in reputation-based FL. Extensive experiments based on real-world datasets demonstrate the significant advantages of GPS-AFL, which reduces costs by 33.65% and improved total utility by 2.91%, on average compared to the best-performing state-of-the-art approach.
MuSe-GNN: Learning Unified Gene Representation From Multimodal Biological Graph Data

Discovering genes with similar functions across diverse biomedical contexts poses a significant challenge in gene representation learning due to data heterogeneity. In this study, we resolve this problem by introducing a novel model called Multimodal Similarity Learning Graph Neural Network, which combines Multimodal Machine Learning and Deep Graph Neural Networks to learn gene representations from single-cell sequencing and spatial transcriptomic data. Leveraging 82 training datasets from 10 tissues, three sequencing techniques, and three species, we create informative graph structures for model training and gene representations generation, while incorporating regularization with weighted similarity learning and contrastive learning to learn cross-data gene-gene relationships. This novel design ensures that we can offer gene representations containing functional similarity across different contexts in a joint space. Comprehensive benchmarking analysis shows our model's capacity to effectively capture gene function similarity across multiple modalities, outperforming state-of-the-art methods in gene representation learning by up to 97.5%. Moreover, we employ bioinformatics tools in conjunction with gene representations to uncover pathway enrichment, regulation causal networks, and functions of disease-associated or dosage-sensitive genes. Therefore, our model efficiently produces unified gene representations for the analysis of gene functions, tissue functions, diseases, and species evolution.
Consistent Dimer Models on Surfaces with Boundary

A dimer model is a quiver with faces embedded in a surface. We define and investigate notions of consistency for dimer models on general surfaces with boundary which restrict to well-studied consistency conditions in the disk and torus case. We define weak consistency in terms of the associated dimer algebra and show that it is equivalent to the absence of bad configurations on the strand diagram. In the disk and torus case, weakly consistent models are nondegenerate, meaning that every arrow is contained in a perfect matching; this is not true for general surfaces. Strong consistency is defined to require weak consistency as well as nondegeneracy. We prove that the completed dimer algebra of a weakly consistent dimer model as well as the noncompleted dimer algebra of a strongly consistent dimer model are bimodule internally 3-Calabi-Yau with respect to their boundary idempotents. As a consequence, the Gorenstein-projective module category of the completed boundary algebra of suitable dimer models categorifies the cluster algebra given by their underlying quiver. We provide additional consequences of weak and strong consistency, including that one may reduce a strongly consistent dimer model by removing digon-faces and that consistency behaves well under taking dimer submodels.
Ophiuchus: Scalable Modeling of Protein Structures through Hierarchical Coarse-graining SO(3)-Equivariant Autoencoders

Three-dimensional native states of natural proteins display recurring and hierarchical patterns. Yet, traditional graph-based modeling of protein structures is often limited to operate within a single fine-grained resolution, and lacks hourglass neural architectures to learn those high-level building blocks. We narrow this gap by introducing Ophiuchus, an SO(3)-equivariant coarse-graining model that efficiently operates on all-atom protein structures. Our model departs from current approaches that employ graph modeling, instead focusing on local convolutional coarsening to model sequence-motif interactions with efficient time complexity in protein length. We measure the reconstruction capabilities of Ophiuchus across different compression rates, and compare it to existing models. We examine the learned latent space and demonstrate its utility through conformational interpolation. Finally, we leverage denoising diffusion probabilistic models (DDPM) in the latent space to efficiently sample protein structures. Our experiments demonstrate Ophiuchus to be a scalable basis for efficient protein modeling and generation.
ViT-ReciproCAM: Gradient and Attention-Free Visual Explanations for Vision Transformer

This paper presents a novel approach to address the challenges of understanding the prediction process and debugging prediction errors in Vision Transformers (ViT), which have demonstrated superior performance in various computer vision tasks such as image classification and object detection. While several visual explainability techniques, such as CAM, Grad-CAM, Score-CAM, and Recipro-CAM, have been extensively researched for Convolutional Neural Networks (CNNs), limited research has been conducted on ViT. Current state-of-the-art solutions for ViT rely on class agnostic Attention-Rollout and Relevance techniques. In this work, we propose a new gradient-free visual explanation method for ViT, called ViT-ReciproCAM, which does not require attention matrix and gradient information. ViT-ReciproCAM utilizes token masking and generated new layer outputs from the target layer's input to exploit the correlation between activated tokens and network predictions for target classes. Our proposed method outperforms the state-of-the-art Relevance method in the Average Drop-Coherence-Complexity (ADCC) metric by $4.58\%$ to $5.80\%$ and generates more localized saliency maps. Our experiments demonstrate the effectiveness of ViT-ReciproCAM and showcase its potential for understanding and debugging ViT models. Our proposed method provides an efficient and easy-to-implement alternative for generating visual explanations, without requiring attention and gradient information, which can be beneficial for various applications in the field of computer vision.
Experimental analysis of catamaran forms with bulbous bows operating in shallow water

In recent years fast ferries which are capable of speeds in excess of 40 knots have been operational in water of depth of less than 10 m. As high-speed operations near sensitive shorelines increase, complaints from the public on extensive wave wake or wake wash from these fast vessels have increased. Although the leading waves in the wash are very small in terms of wave amplitude compared to storm waves, they have a very long period and build in height rapidly in shallow water at the shoreline thereby causing substantial surges on beaches as well as breaching sea walls at high tide. This wake wash is likely to have environmental effects such as shoreline erosion as well as endangering swimmers and small boats.