Miracle-dz/UniFAR-FTUs huggingface dataset:
MathNAS: If Blocks Have a Role in Mathematical Architecture Design

Neural Architecture Search (NAS) has emerged as a favoured method for unearthing effective neural architectures. Recent development of large models has intensified the demand for faster search speeds and more accurate search results. However, designing large models by NAS is challenging due to the dramatical increase of search space and the associated huge performance evaluation cost. Consider a typical modular search space widely used in NAS, in which a neural architecture consists of $m$ block nodes and a block node has $n$ alternative blocks. Facing the space containing $n^m$ candidate networks, existing NAS methods attempt to find the best one by searching and evaluating candidate networks directly.Different from the general strategy that takes architecture search as a whole problem, we propose a novel divide-and-conquer strategy by making use of the modular nature of the search space.Here, we introduce MathNAS, a general NAS framework based on mathematical programming.In MathNAS, the performances of the $m*n$ possible building blocks in the search space are calculated first, and then the performance of a network is directly predicted based on the performances of its building blocks. Although estimating block performances involves network training, just as what happens for network performance evaluation in existing NAS methods, predicting network performance is completely training-free and thus extremely fast. In contrast to the $n^m$ candidate networks to evaluate in existing NAS methods, which require training and a formidable computational burden, there are only $m*n$ possible blocks to handle in MathNAS. Therefore, our approach effectively reduces the complexity of network performance evaluation.Our code is available at https://github.com/wangqinsi1/MathNAS.
Removing RLHF Protections in GPT-4 via Fine-Tuning

As large language models (LLMs) have increased in their capabilities, so does their potential for dual use. To reduce harmful outputs, produces and vendors of LLMs have used reinforcement learning with human feedback (RLHF). In tandem, LLM vendors have been increasingly enabling fine-tuning of their most powerful models. However, concurrent work has shown that fine-tuning can remove RLHF protections. We may expect that the most powerful models currently available (GPT-4) are less susceptible to fine-tuning attacks. In this work, we show the contrary: fine-tuning allows attackers to remove RLHF protections with as few as 340 examples and a 95% success rate. These training examples can be automatically generated with weaker models. We further show that removing RLHF protections does not decrease usefulness on non-censored outputs, providing evidence that our fine-tuning strategy does not decrease usefulness despite using weaker models to generate training data. Our results show the need for further research on protections on LLMs.
Environment Matters: Predicted Differences in the Stellar Mass–Halo Mass Relation and History of Star Formation for Dwarf Galaxies

We are entering an era in which we will be able to detect and characterize hundreds of dwarf galaxies within the Local Volume. It is already known that a strong dichotomy exists in the gas content and star formation properties of field dwarf galaxies versus satellite dwarfs of larger galaxies. In this work, we study the more subtle differences that may be detectable in galaxies as a function of distance from a massive galaxy, such as the Milky Way. We compare smoothed particle hydrodynamic simulations of dwarf galaxies formed in a Local Volume-like environment (several megaparsecs away from a massive galaxy) to those formed nearer to Milky Way–mass halos. We find that the impact of environment on dwarf galaxies extends even beyond the immediate region surrounding Milky Way–mass halos. Even before being accreted as satellites, dwarf galaxies near a Milky Way–mass halo tend to have higher stellar masses for their halo mass than more isolated galaxies. Dwarf galaxies in high-density environments also tend to grow faster and form their stars earlier. We show observational predictions that demonstrate how these trends manifest in lower quenching rates, higher H i fractions, and bluer colors for more isolated dwarf galaxies.
Window Attention is Bugged: How not to Interpolate Position Embeddings

Window attention, position embeddings, and high resolution finetuning are core concepts in the modern transformer era of computer vision. However, we find that naively combining these near ubiquitous components can have a detrimental effect on performance. The issue is simple: interpolating position embeddings while using window attention is wrong. We study two state-of-the-art methods that have these three components, namely Hiera and ViTDet, and find that both do indeed suffer from this bug. To fix it, we introduce a simple absolute window position embedding strategy, which solves the bug outright in Hiera and allows us to increase both speed and performance of the model in ViTDet. We finally combine the two to obtain HieraDet, which achieves 61.7 box mAP on COCO, making it state-of-the-art for models that only use ImageNet-1k pretraining. This all stems from what is essentially a 3 line bug fix, which we name"absolute win".
What is prompt literacy? An exploratory study of language learners' development of new literacy skill using generative AI

In the current study,we propose that, in the era of generative AI, there is now a new form of literacy called"prompt literacy,"which refers to the ability to generate precise prompts as input for AI systems, interpret the outputs, and iteratively refine prompts to achieve desired results. To explore the emergence and development of this literacy skill, the current study examined 30 EFL students' engagement in an AI-powered image creation project, through which they created artworks representing the socio-cultural meanings of English words by iteratively drafting and refining prompts in generative AI tools. By examining AI-generated images and the participants' drafting and revision of their prompts, this study demonstrated the emergence of learners' prompt literacy skills. The survey data further showed the participants' perceived improvement in their vocabulary learning strategies as a result of engaging in the target AI-powered project. In addition, the participants' post-project reflection revealed three benefits of developing prompt literacy: enjoyment from manifesting imagined outcomes; recognition of its importance for communication, problem-solving and career development; and the enhanced understanding of the collaborative nature of human-AI interaction. These findings suggest that prompt literacy is an increasingly crucial literacy for the AI era.
Explicit zero density estimate near unity

We will provide the first explicit zero-density estimate for $\zeta$ of the form $N(\sigma,T)\le \mathcal{C}T^{B(1-\sigma)^{3/2}}(\log T)^C$. In particular, we improve $C$ to $10393/900=11.547\dots. $
Deformation of a N\'eel-type Skyrmion in a Weak Inhomogeneous Magnetic Field: Magnetization Ansatz and Interaction with a Pearl Vortex

In this work, we develop a theory of (meta)stable states of N\'eel-type skyrmions in weak nonuniform magnetic fields. We claim an Ansatz for modeling the non-symmetric magnetization that can be implied for both analytics and numerical simulations. Our theory accounts for changes in the size of skyrmion parameters and also includes deformations from the centrally symmetric shape. The ansatz streamlines the analytic calculation of the skyrmion free energy, enhancing the efficiency of the minimizing process. Performing the minimization in two stages, one can find all the minima, global and local, of the free energy, discovering the stable and metastable states. We apply the developed methodology to investigate the (meta)stable configurations of skyrmions influenced by the stray field of a Pearl vortex. Our study reveals the dependence of skyrmion spatial parameters on the vortex field effective strength and presents a phase diagram identifying regions where metastable configurations are predicted. Corroborated by micromagnetic simulations, our findings offer a detailed perspective on the interaction between magnetic skyrmions and superconducting vortices.
Extending Regev's factoring algorithm to compute discrete logarithms

Regev recently introduced a quantum factoring algorithm that may be perceived as a $d$-dimensional variation of Shor's factoring algorithm. In this work, we extend Regev's factoring algorithm to an algorithm for computing discrete logarithms in a natural way. Furthermore, we discuss natural extensions of Regev's factoring algorithm to order finding, and to factoring completely via order finding. For all of these algorithms, we discuss various practical implementation considerations, including in particular the robustness of the post-processing.
Firing rate distributions in plastic networks of spiking neurons

In recurrent networks of leaky integrate-and-fire (LIF) neurons, mean-field theory has proven successful in describing various statistical properties of neuronal activity at equilibrium, such as firing rate distributions. Mean-field theory has been applied to networks in which either the synaptic weights are homogeneous across synapses and the number of incoming connections of individual neurons is heterogeneous, or vice versa. Here we extend the previous mean-field formalisms to treat networks in which these two sources of structural heterogeneity occur simultaneously, including networks whose synapses are subject to plastic, activity-dependent modulation. The plasticity in our model is mediated by the introduction of one spike trace per neuron: a chemical signal that is released every time the neuron emits a spike and which is degraded over time. The temporal evolution of the trace is controlled by its degradation rate $r_p$ and by the neuron's underlying firing rate $\nu$. When the ratio $\alpha=\nu / r_p$ tends to infinity, the trace can be rescaled to be a reliable estimation of the neuron's firing rate. In this regime, the value of any synaptic weight at equilibrium is a function of the pre- and post-synaptic firing rates, and this relation can be used in the mean-field formalism. The solution to the mean-field equations specifies the firing rate and synaptic weight distributions at equilibrium. These equations are exact in the limit of reliable traces but they already provide accurate results when the degradation rate lies within a reasonable range, as we show by comparison with simulations of the full neuronal dynamics in networks composed of excitatory and inhibitory LIF neurons. Overall, this work offers a way to explore and better understand the way in which plasticity shapes both activity and structure in neuronal networks.
Exploring Emotion Expression Recognition in Older Adults Interacting with a Virtual Coach

The EMPATHIC project aimed to design an emotionally expressive virtual coach capable of engaging healthy seniors to improve well-being and promote independent aging. One of the core aspects of the system is its human sensing capabilities, allowing for the perception of emotional states to provide a personalized experience. This paper outlines the development of the emotion expression recognition module of the virtual coach, encompassing data collection, annotation design, and a first methodological approach, all tailored to the project requirements. With the latter, we investigate the role of various modalities, individually and combined, for discrete emotion expression recognition in this context: speech from audio, and facial expressions, gaze, and head dynamics from video. The collected corpus includes users from Spain, France, and Norway, and was annotated separately for the audio and video channels with distinct emotional labels, allowing for a performance comparison across cultures and label types. Results confirm the informative power of the modalities studied for the emotional categories considered, with multimodal methods generally outperforming others (around 68% accuracy with audio labels and 72-74% with video labels). The findings are expected to contribute to the limited literature on emotion recognition applied to older adults in conversational human-machine interaction.
LooGLE: Can Long-Context Language Models Understand Long Contexts?

Large language models (LLMs), despite their impressive performance in various language tasks, are typically limited to processing texts within context-window size. This limitation has spurred significant research efforts to enhance LLMs' long-context understanding with high-quality long-sequence benchmarks. However, prior datasets in this regard suffer from shortcomings, such as short context length compared to the context window of modern LLMs; outdated documents that have data leakage problems; and an emphasis on short dependency tasks rather than long dependency tasks. In this paper, we present LooGLE, a Long Context Generic Language Evaluation benchmark for LLMs' long context understanding. LooGLE features relatively new documents post-2022, with over 24,000 tokens per document and 6,000 newly generated questions spanning diverse domains. Human annotators meticulously crafted more than 1,100 high-quality question-answer pairs to meet the long dependency requirements. These pairs underwent thorough cross-validation, yielding the most precise assessment of LLMs' long dependency capabilities. The evaluation of eight state-of-the-art LLMs on LooGLE revealed key findings: (i) commercial models outperformed open-sourced models; (ii) LLMs excelled in short dependency tasks like short question-answering and cloze tasks but struggled with more intricate long dependency tasks; (iii) in-context learning and chaining thoughts offered only marginal improvements; (iv) retrieval-based techniques demonstrated substantial benefits for short question-answering, while strategies for extending context window length had limited impact on long context understanding. As such, LooGLE not only provides a systematic and comprehensive evaluation schema on long-context LLMs, but also sheds light on future development of enhanced models towards"true long-context understanding".
Nonlinear propagation and localisation in photonic crystal waveguides

Anderson predicted that electrons diffused by a disordered potential in doped semiconductors see a metal-to-insulator transition of the material when the disorder is sufficiently high. In this manuscript we demonstrate an equivalent metal-to-insulator transition for nonlinear waves, namely Gap-Solitons, in disordered photonic crystal waveguides. Light localization is described by introducing a new metric able to track the wavepacket center of mass. By statistical averaging this quantity over many realizations of the disorder, we demonstrate that in linear regime the ensemble averaged barycenter matches the localization length lloc as defined in literature. Then, by applying our barycenter method, we prove that for Gap-Solitons the transition from localized to ballistic transport goes faster than the vg2 law ruling the linear regime. By overcoming this scaling law, improved robustness to disorder of nonlinear waves is demonstrated.
Out‐of‐phase relationships between convection over northwest India and warm pool region during the winter season

In this paper, we investigate the effect of convection over warm pool region of Indonesia on northwest India winter precipitation (NWIWP) using an analysis of latest available observed climate data. The analysis is based on correlations and composites performed using India Meteorological Department (IMD) records of station data. We find that there is an out‐of‐phase relationship between convection over warm pool region and northwest India during the winter season. In particular, a positive precipitation anomaly over northwest India is typically found in correspondence to subdued convection over warm pool region. A physical mechanism for such effect is proposed, by which during excess years of NWIWP the tropical cooling over the warm pool region due to suppressed convection generates an upper‐level convergence (cyclonic circulation) anomaly over south Asia which intensifies the westerly jet stream over Indian region. Also, the reduced convection over the warm pool may be due to reduced Madden‐Julian oscillation (MJO) activities on the intra‐seasonal timescale. The Rossby‐gyre dynamics with the strong vertical and westward meridional tilts in the tropical baroclinic atmosphere forms cyclonic circulation anomalies at the upper troposphere due to a weak MJO which intensifies the westerly jet stream over the Indian region. The jet stream guides the western disturbances (WDs) and hence increases precipitation over northwest India. The seasonal persistence in migration of convective maximum from south Asia (foothills of the Himalayas) to Indonesia during the peak months of boreal summer to winter increases the seasonal predictability of NWIWP. Copyright © 2008 Royal Meteorological Society
A failure to reject: Testing the correlation between large igneous provinces and deep mantle structures with EDF statistics

Absolute reconstructions of large igneous provinces (LIPs) for the past 300 Ma reveal a remarkable spatial pattern suggesting that almost all LIPs have erupted over the margins of the two large‐scale structures in the Earth's lower mantle commonly referred to as the Large Low Shear‐wave Velocity Provinces (LLSVPs). This correlation suggests that mantle plumes that have triggered LIP eruptions rose from the margins of LLSVPs, implying long‐term stability of these structures and suggesting that they may be chemically distinct from the bulk of the mantle. Yet, some researchers consider the LLSVPs to be purely thermal upwellings, arguing that the observed distribution of LIPs can be explained by plumes randomly forming over the entire areas of LLSVPs. Here we examine the correlation between the LIPs and LLSVPs using nonparametric statistical tests, updated plate reconstructions, and a large number of alternative definitions of LLSVPs based on seismic tomography. We show that probability models assuming plume sources originating at the margins of LLSVPs adequately explain the observed distribution of reconstructed LIPs. In contrast, we find strong evidence against the models seeking to link LIPs with plumes randomly forming over the entire LLSVP areas. However, the hypothesis proposing that the correlation can be explained by plumes randomly forming over a larger area of slower‐than‐average shear wave velocities in the lowermost mantle cannot be ruled out formally. Our analysis suggests that there is no statistically sound reason for questioning the hypothesis that the LIPs correlate with the margins of LLSVP globally.
Theory of topological exciton insulators and condensates in flat Chern bands

Excitons are the neutral quasiparticles that form when Coulomb interactions create bound states between electrons and holes. Due to their bosonic nature, excitons are expected to condense and exhibit superfluidity at sufficiently low temperatures. In interacting Chern insulators, excitons may inherit the nontrivial topology and quantum geometry from the underlying electron wavefunctions. We theoretically investigate the excitonic bound states and superfluidity in flat-band insulators pumped with light. We find that the exciton wavefunctions exhibit vortex structures in momentum space, with the total vorticity being equal to the difference of Chern numbers between the conduction and valence bands. Moreover, both the exciton binding energy and the exciton superfluid density are proportional to the Brillouin-zone average of the quantum metric and the Coulomb potential energy per unit cell. Spontaneous emission of circularly polarized light from radiative decay is a detectable signature of the vorticity of excitons. We propose that the exciton vorticity can also be experimentally measured by the nonlinear anomalous Hall effect, whereas the exciton superfluidity can be detected by voltage-drop quantization through a combination of the quantum geometry and the Aharonov-Casher effect. Topological excitons and their superfluid phase could be realized in flat bands of twisted Van der Waals heterostructures.
Unobscured Central Broad-line Regions in Type-1.9 AGN SDSS J1241+2602

Strong evidence is reported to support unobscured broad-line regions (BLRs) in Type-1.9 active galactic nucleus (AGN) SDSS J1241+2602 with reliable broad Hα but no broad Hβ. Commonly, the disappearance of broad Hβ can be explained by the AGN unified model, in which heavily obscured BLRs are expected in Type-1.9 AGNs. Here, based on properties of two kinds of BH masses, the virial BH mass and the BH mass obtained through the M BH–σ relation, an independent method is proposed to test whether there are unobscured central BLRs in a Type-1.9 AGN. By the reliable measurement of stellar velocity dispersion of about 110 ± 12 km s −1 through the host galaxy absorption features in SDSS J1241+2602, the BH mass obtained through the M BH–σ relation is consistent with the virial BH mass (3.43 ± 1.25) × 107 M ⊙ determined through properties of the observed broad Hα without considering the effects of obscurations in SDSS J1241+2602. Meanwhile, if considering heavily obscured BLRs in SDSS J1241+2602, the reddening-corrected virial BH mass is tens of times larger than the value expected from M BH–σ, leading SDSS J1241+2602 to be an outlier in the M BH–σ space with a confidence level higher than 5σ. Therefore, unobscured BLRs are preferred in the Type-1.9 AGN SDSS J1241+2602. The results indicate that it is necessary to check whether unobscured central BLRs are common in Type-1.9 AGNs when testing the unified model of AGNs through properties of Type-1.9 AGNs.
Group actions on multitrees and the $K$-theory of their crossed products

We study group actions on multitrees, which are directed graphs in which there is at most one directed path between any two vertices. In our main result we describe a six-term exact sequence in $K$-theory for the reduced crossed product $C_0(\partial E)\rtimes_r G$ induced from the action of a countable discrete group $G$ on a row-finite, finitely-aligned multitree $E$ with no sources. We provide formulas for the $K$-theory of $C_0(\partial E) \rtimes_r G$ in the case where $G$ acts freely on $E$, and in the case where all vertex stabilisers are infinite cyclic. We study the action $G\curvearrowright \partial E$ in a range of settings, and describe minimality, local contractivity, topological freeness, and amenability in terms of properties of the underlying data. In an application of our main theorem, we describe a six-term exact sequence in $K$-theory for the crossed product induced from a group acting on the boundary of an undirected tree.
Particle and pair spectra for strongly correlated Fermi gases: A real-frequency solver

The strongly attractive Fermi gas in the BCS-BEC crossover is efficiently described in terms of coupled fermions and fermion pairs, or molecules. We compute the spectral functions of both fermions and pairs in the normal state near the superfluid transition using a Keldysh formulation in real frequency. The mutual influence between fermions and pairs is captured by solving the self-consistent Luttinger-Ward equations: these include both the damping of fermions by scattering off dressed pairs, as well as the decay of pair states by dissociation into two dressed fermions. The pair spectra encode contact correlations between fermions and form the basis for computing dynamical response functions and transport properties.