Miracle-dz/UniFAR-FTUs huggingface dataset:
Understanding and Mitigating Overfitting in Prompt Tuning for Vision-Language Models

Pretrained vision-language models (VLMs) such as CLIP have shown impressive generalization capability in downstream vision tasks with appropriate text prompts. Instead of designing prompts manually, Context Optimization (CoOp) has been recently proposed to learn continuous prompts using task-specific training data. Despite the performance improvements on downstream tasks, several studies have reported that CoOp suffers from the overfitting issue in two aspects: (i) the test accuracy on base classes first improves and then worsens during training; (ii) the test accuracy on novel classes keeps decreasing. However, none of the existing studies can understand and mitigate such overfitting problems. In this study, we first explore the cause of overfitting by analyzing the gradient flow. Comparative experiments reveal that CoOp favors generalizable and spurious features in the early and later training stages, respectively, leading to the non-overfitting and overfitting phenomena. Given those observations, we propose Subspace Prompt Tuning (Sub PT) to project the gradients in back-propagation onto the low-rank subspace spanned by the early-stage gradient flow eigenvectors during the entire training process and successfully eliminate the overfitting problem. In addition, we equip CoOp with a Novel Feature Learner (NFL) to enhance the generalization ability of the learned prompts onto novel categories beyond the training set, needless of image training data. Extensive experiments on 11 classification datasets demonstrate that Sub PT+NFL consistently boost the performance of CoOp and outperform the state-of-the-art CoCoOp approach. Experiments on more challenging vision downstream tasks, including open-vocabulary object detection and zero-shot semantic segmentation, also verify the effectiveness of the proposed method. Codes can be found at https://tinyurl.com/mpe64f89.
Roots of unity and higher ramification in iterated extensions

Given a field $K$, a rational function $\phi \in K(x)$, and a point $b \in \mathbb{P}^1(K)$, we study the extension $K(\phi^{-\infty}(b))$ generated by the union over $n$ of all solutions to $\phi^n(x) = b$, where $\phi^n$ is the $n$th iterate of $\phi$. We ask when a finite extension of $K(\phi^{-\infty}(b))$ can contain all $m$-power roots of unity for some $m \geq 2$, and prove that several families of rational functions do so. A motivating application is to understand the higher ramification filtration when $K$ is a finite extension of $\mathbb{Q}_p$ and $p$ divides the degree of $\phi$, especially when $\phi$ is post-critically finite (PCF). We show that all higher ramification groups are infinite for new families of iterated extensions, for example those given by bicritical rational functions with periodic critical points. We also give new examples of iterated extensions with subextensions satisfying an even stronger ramification-theoretic condition called arithmetic profiniteness. We conjecture that every iterated extension arising from a PCF map should have a subextension with this stronger property, which would give a dynamical analogue of Sen's theorem for PCF maps.
On the upper tail large deviation rate function for chemical distance in supercritical percolation

We consider the supercritical bond percolation on $\mathbb Z^d$ and study the graph distance on the percolation graph called the chemical distance. It is well-known that there exists a deterministic constant $\mu(x)$ such that the chemical distance $\mathcal D(0,nx)$ between two connected points $0$ and $nx$ grows like $n\mu(x)$. Garet and Marchand (Ann. Prob., 2007) proved that the probability of the upper tail large deviation event $\left\{n\mu(x)(1+\varepsilon)<\mathcal D(0,nx)<\infty\right\} $ decays exponentially with respect to $n$. In this paper, we prove the existence of the rate function for upper tail large deviation when $d\ge 3$ and $\varepsilon>0$ is small enough. Moreover, we show that for any $\varepsilon>0$, the upper tail large deviation event is created by space-time cut-points (points that all paths from $0$ to $nx$ must cross after a given time) that force the geodesics to consume more time by going in a non-optimal direction or by wiggling considerably. This enables us to express the rate function in regards to space-time cut-points.
Reservoir Computing via Quantum Recurrent Neural Networks

Recent developments in quantum computing and machine learning have propelled the interdisciplinary study of quantum machine learning. Sequential modeling is an important task with high scientific and commercial value. Existing VQC or QNN-based methods require significant computational resources to perform the gradient-based optimization of a larger number of quantum circuit parameters. The major drawback is that such quantum gradient calculation requires a large amount of circuit evaluation, posing challenges in current near-term quantum hardware and simulation software. In this work, we approach sequential modeling by applying a reservoir computing (RC) framework to quantum recurrent neural networks (QRNN-RC) that are based on classical RNN, LSTM and GRU. The main idea to this RC approach is that the QRNN with randomly initialized weights is treated as a dynamical system and only the final classical linear layer is trained. Our numerical simulations show that the QRNN-RC can reach results comparable to fully trained QRNN models for several function approximation and time series prediction tasks. Since the QRNN training complexity is significantly reduced, the proposed model trains notably faster. In this work we also compare to corresponding classical RNN-based RC implementations and show that the quantum version learns faster by requiring fewer training epochs in most cases. Our results demonstrate a new possibility to utilize quantum neural network for sequential modeling with greater quantum hardware efficiency, an important design consideration for noisy intermediate-scale quantum (NISQ) computers.
THE NATURE OF THE SECOND PARAMETER IN THE IRX–β RELATION FOR LOCAL GALAXIES

We present an analysis of 98 galaxies of low-dust content, selected from the Spitzer Local Volume Legacy survey, aimed at examining the relation between the ultraviolet (UV) color and dust attenuation in normal star-forming galaxies. The IRX–β diagram relates the total dust attenuation in a galaxy, traced by the far-IR (FIR) to UV ratio, to the observed UV color, indicated by β. Previous research has indicated that while starburst galaxies exhibit a relatively tight IRX–β relation, normal star-forming galaxies do not, and have a much larger spread in the total-IR to far-UV (FUV) luminosity for a fixed UV color. We examine the role that the age of the stellar population plays as the “second parameter” responsible for the observed deviation and spread of star-forming galaxies from the starburst relation. We model the FUV to FIR spectral energy distribution of each galaxy according to two broad bins of star formation history (SFH): constant and instantaneous burst. We find clear trends between stellar population mean age estimators (extinction-corrected FUV/NIR, U − B, and EW(Hα)) and the UV color β; the trends are mostly driven by the galaxies best-described by instantaneous burst populations. We also find a significant correlation between β and the mean age directly determined from the best-fit instantaneous models. As already indicated by other authors, the UV attenuation in star-forming galaxies may not be recovered with the UV color alone and is highly influenced by the stellar population's mean age and SFH. Overall, the scatter in the IRX–β diagram is better correlated with β than with the perpendicular distance, dp.
Discretized sum-product type problems: Energy variants and Applications

In this paper, we provide estimates for the additive discretized energy of \[\sum_{c\in C} |\{(a_1, a_2, b_1, b_2)\in A^2\times B^2: |(a_1 +cb_1) - (a_2 + cb_2)|\le \delta\}|_{\delta},\] that depend on non-concentration conditions of the sets. Our proof follows the Guth-Katz-Zahl approach (2021) with appropriate changes along the way clarifying and optimizing many of the steps. Several applications will also be discussed.
Evidence for Multiple Shocks from the γ-Ray Emission of RS Ophiuchi

In 2021 August, the Fermi Large Area Telescope, H.E. S.S. , and MAGIC detected GeV and TeV γ-ray emission from an outburst of recurrent nova RS Ophiuchi. This detection represents the first very high-energy γ-rays observed from a nova, and it opens a new window to study particle acceleration. Both H.E. S.S. and MAGIC described the observed γ-rays as arising from a single, external shock. In this paper, we perform detailed, multi-zone modeling of RS Ophiuchi’s 2021 outburst, including a self-consistent prescription for particle acceleration and magnetic field amplification. We demonstrate that, contrary to previous work, a single shock cannot simultaneously explain RS Ophiuchi’s GeV and TeV emission, in particular the spectral shape and distinct light-curve peaks. Instead, we put forward a model involving multiple shocks that reproduces the observed γ-ray spectrum and temporal evolution. The simultaneous appearance of multiple distinct velocity components in the nova optical spectrum over the first several days of the outburst supports the presence of distinct shocks, which may arise either from the strong latitudinal dependence of the density of the external circumbinary medium (e.g., in the binary equatorial plane versus the poles) or due to internal collisions within the white dwarf ejecta (which power the γ-ray emission in classical novae).
A Knowledge Distillation Framework for Enhancing Ear-EEG Based Sleep Staging with Scalp-EEG Data

Sleep plays a crucial role in the well-being of human lives. Traditional sleep studies using Polysomnography are associated with discomfort and often lower sleep quality caused by the acquisition setup. Previous works have focused on developing less obtrusive methods to conduct high-quality sleep studies, and ear- EEG is among popular alternatives. However, the performance of sleep staging based on ear-EEG is still inferior to scalp- EEG based sleep staging. In order to address the performance gap between scalp-EEG and ear- EEG based sleep staging, we propose a cross-modal knowledge distillation strategy1†https://github.com/Mithunjha/EarEEG_KnowledgeDistillation, which is a domain adaptation approach. We employ model architectures from the transformer and convolutional neural network families to demonstrate the model-agnostic nature of the method. Our experiments and analysis validate the effectiveness of the proposed approach with existing architectures, where it enhances the accuracy of the ear-EEG based sleep staging by 3.46% and Cohen's kappa coefficient by a margin of 0.038. Furthermore, our findings indicate that our approach is not limited to a specific model architecture and can be applied to a wide range of deep learning models.
Scaling Multimodal Pre-Training via Cross-Modality Gradient Harmonization

Self-supervised pre-training recently demonstrates success on large-scale multimodal data, and state-of-the-art contrastive learning methods often enforce the feature consistency from cross-modality inputs, such as video/audio or video/text pairs. Despite its convenience to formulate and leverage in practice, such cross-modality alignment (CMA) is only a weak and noisy supervision, since two modalities can be semantically misaligned even they are temporally aligned. For example, even in the commonly adopted instructional videos, a speaker can sometimes refer to something that is not visually present in the current frame; and the semantic misalignment would only be more unpredictable for the raw videos from the internet. We conjecture that might cause conflicts and biases among modalities, and may hence prohibit CMA from scaling up to training with larger and more heterogeneous data. This paper first verifies our conjecture by observing that, even in the latest VATT pre-training using only instructional videos, there exist strong gradient conflicts between different CMA losses within the same video, audio, text triplet, indicating them as the noisy source of supervision. We then propose to harmonize such gradients, via two techniques: (i) cross-modality gradient realignment: modifying different CMA loss gradients for each sample triplet, so that their gradient directions are more aligned; and (ii) gradient-based curriculum learning: leveraging the gradient conflict information on an indicator of sample noisiness, to develop a curriculum learning strategy to prioritize training on less noisy sample triplets. Applying those techniques to pre-training VATT on the HowTo100M dataset, we consistently improve its performance on different downstream tasks. Moreover, we are able to scale VATT pre-training to more complicated non-narrative Youtube8M dataset to further improve the state-of-the-arts.
Weisfeiler and Leman go Hyperbolic: Learning Distance Preserving Node Representations

In recent years, graph neural networks (GNNs) have emerged as a promising tool for solving machine learning problems on graphs. Most GNNs are members of the family of message passing neural networks (MPNNs). There is a close connection between these models and the Weisfeiler-Leman (WL) test of isomorphism, an algorithm that can successfully test isomorphism for a broad class of graphs. Recently, much research has focused on measuring the expressive power of GNNs. For instance, it has been shown that standard MPNNs are at most as powerful as WL in terms of distinguishing non-isomorphic graphs. However, these studies have largely ignored the distances between the representations of nodes/graphs which are of paramount importance for learning tasks. In this paper, we define a distance function between nodes which is based on the hierarchy produced by the WL algorithm, and propose a model that learns representations which preserve those distances between nodes. Since the emerging hierarchy corresponds to a tree, to learn these representations, we capitalize on recent advances in the field of hyperbolic neural networks. We empirically evaluate the proposed model on standard node and graph classification datasets where it achieves competitive performance with state-of-the-art models.
Canonical nilpotent structure under bounded ricci curvature and reifenberg local covering geometry over regular limits

It is known that a closed collapsed Riemannian $n$-manifold $(M,g)$ of bounded Ricci curvature and Reifenberg local covering geometry admits a nilpotent structure in the sense of Cheeger-Fukaya-Gromov with respect to a smoothed metric $g(t)$. We prove that a canonical nilpotent structure over a regular limit space that describes the collapsing of original metric $g$ can be defined and uniquely determined up to a conjugation, and prove that the nilpotent structures arising from nearby metrics $g_\epsilon$ with respect to $g_\epsilon$'s sectional curvature bound are equivalent to the canonical one.
Isospectral graphs with identical nodal counts

According to a recent conjecture, isospectral objects have different nodal count sequences (Gnutzmann et al 2005 J. Phys. A: Math. Gen. 38 8921–33). We study generalized Laplacians on discrete graphs, and use them to construct the first non-trivial counterexamples to this conjecture. In addition, these examples demonstrate a surprising connection between isospectral discrete and quantum graphs.
YANGIANS IN INTEGRABLE FIELD THEORIES, SPIN CHAINS AND GAUGE-STRING DUALITIES

In the following paper, which is based on the author's PhD thesis submitted to Imperial College London, we explore the applicability of Yangian symmetry to various integrable models, in particular, in relation with S-matrices. One of the main themes in this work is that, after a careful study of the mathematics of the symmetry algebras one finds that in an integrable model, one can directly reconstruct S-matrices just from the algebra. It has been known for a long time that S-matrices in integrable models are fixed by symmetry. However, Lie algebra symmetry, the Yang–Baxter equation, crossing and unitarity, which constrain the S-matrix in integrable models, are often taken to be separate, independent properties of the S-matrix. Here, we construct scattering matrices purely from the Yangian, showing that the Yangian is the right algebraic object to unify all required symmetries of many integrable models. In particular, we reconstruct the S-matrix of the principal chiral field, and, up to a CDD factor, of other integrable field theories with 𝔰𝔲(n) symmetry. Furthermore, we study the AdS/CFT correspondence, which is also believed to be integrable in the planar limit. We reconstruct the S-matrices at weak and at strong coupling from the Yangian or its classical limit. We give a pedagogical introduction into the subject, presenting a unified perspective of Yangians and their applications in physics. This paper should hence be accessible to mathematicians who would like to explore the application of algebraic objects to physics as well as to physicists interested in a deeper understanding of the mathematical origin of physical quantities.
A numerical study of vortex nucleation in 2D rotating Bose-Einstein condensates

This article introduces a new numerical method for the minimization under constraints of a discrete energy modeling multicomponents rotating Bose-Einstein condensates in the regime of strong confinement and with rotation. Moreover, we consider both segregation and coexistence regimes between the components. The method includes a discretization of a continuous energy in space dimension 2 and a gradient algorithm with adaptive time step and projection for the minimization. It is well known that, depending on the regime, the minimizers may display different structures, sometimes with vorticity (from singly quantized vortices, to vortex sheets and giant holes). In order to study numerically the structures of the minimizers, we introduce in this paper a numerical algorithm for the computation of the indices of the vortices, as well as an algorithm for the computation of the indices of vortex sheets. Several computations are carried out, to illustrate the efficiency of the method, to cover different physical cases, to validate recent theoretical results as well as to support conjectures. Moreover, we compare this method with an alternative method from the literature.
Pangu-Weather: A 3D High-Resolution Model for Fast and Accurate Global Weather Forecast

In this paper, we present Pangu-Weather, a deep learning based system for fast and accurate global weather forecast. For this purpose, we establish a data-driven environment by downloading $43$ years of hourly global weather data from the 5th generation of ECMWF reanalysis (ERA5) data and train a few deep neural networks with about $256$ million parameters in total. The spatial resolution of forecast is $0.25^\circ\times0.25^\circ$, comparable to the ECMWF Integrated Forecast Systems (IFS). More importantly, for the first time, an AI-based method outperforms state-of-the-art numerical weather prediction (NWP) methods in terms of accuracy (latitude-weighted RMSE and ACC) of all factors (e.g., geopotential, specific humidity, wind speed, temperature, etc.) and in all time ranges (from one hour to one week). There are two key strategies to improve the prediction accuracy: (i) designing a 3D Earth Specific Transformer (3DEST) architecture that formulates the height (pressure level) information into cubic data, and (ii) applying a hierarchical temporal aggregation algorithm to alleviate cumulative forecast errors. In deterministic forecast, Pangu-Weather shows great advantages for short to medium-range forecast (i.e., forecast time ranges from one hour to one week). Pangu-Weather supports a wide range of downstream forecast scenarios, including extreme weather forecast (e.g., tropical cyclone tracking) and large-member ensemble forecast in real-time. Pangu-Weather not only ends the debate on whether AI-based methods can surpass conventional NWP methods, but also reveals novel directions for improving deep learning weather forecast systems.
A General Purpose Neural Architecture for Geospatial Systems

Geospatial Information Systems are used by researchers and Humanitarian Assistance and Disaster Response (HADR) practitioners to support a wide variety of important applications. However, collaboration between these actors is difficult due to the heterogeneous nature of geospatial data modalities (e.g., multi-spectral images of various resolutions, timeseries, weather data) and diversity of tasks (e.g., regression of human activity indicators or detecting forest fires). In this work, we present a roadmap towards the construction of a general-purpose neural architecture (GPNA) with a geospatial inductive bias, pre-trained on large amounts of unlabelled earth observation data in a self-supervised manner. We envision how such a model may facilitate cooperation between members of the community. We show preliminary results on the first step of the roadmap, where we instantiate an architecture that can process a wide variety of geospatial data modalities and demonstrate that it can achieve competitive performance with domain-specific architectures on tasks relating to the U.N.'s Sustainable Development Goals.
On the coherence conjecture of Pappas and Rapoport

We prove the (generalized) coherence conjecture of Pappas and Rapoport. As a corollary, one theorem of Pappas an Rapoport, which describes the geometry of the special fibers of the local models for ramified unitary groups, holds unconditionally. Our proof is based on the study of the geometry (in particular certain line bundles and $\ell$-adic sheaves) of the global Schubert varieties, which are the (generalized) equal characteristic counterparts of the local models.
Weight-based Channel-model Matrix Framework provides a reasonable solution for EEG-based cross-dataset emotion recognition

Cross-dataset emotion recognition as an extremely challenging task in the field of EEG-based affective computing is influenced by many factors, which makes the universal models yield unsatisfactory results. Facing the situation that lacks EEG information decoding research, we first analyzed the impact of different EEG information(individual, session, emotion and trial) for emotion recognition by sample space visualization, sample aggregation phenomena quantification, and energy pattern analysis on five public datasets. Based on these phenomena and patterns, we provided the processing methods and interpretable work of various EEG differences. Through the analysis of emotional feature distribution patterns, the Individual Emotional Feature Distribution Difference(IEFDD) was found, which was also considered as the main factor of the stability for emotion recognition. After analyzing the limitations of traditional modeling approach suffering from IEFDD, the Weight-based Channel-model Matrix Framework(WCMF) was proposed. To reasonably characterize emotional feature distribution patterns, four weight extraction methods were designed, and the optimal was the correction T-test(CT) weight extraction method. Finally, the performance of WCMF was validated on cross-dataset tasks in two kinds of experiments that simulated different practical scenarios, and the results showed that WCMF had more stable and better emotion recognition ability.
Transversality of isotropic projections, unrectifiability and Heisenberg groups

We show that the family of $m$-dimensional isotropic projections in $\R^{2n}$ is transversal. As an application we show that the Besicovitch-Federer projection theorem holds for isotropic projections. We also use transversality to obtain almost sure estimates on the Hausdorff dimension of isotropic projections of subsets $E \subset \R^{2n}$. These results may also be applied to gain information on the horizontal projections of the Heisenberg group $\H^n$.