Miracle-dz/UniFAR-FTUs huggingface dataset:
A Theory of Emergent In-Context Learning as Implicit Structure Induction

Scaling large language models (LLMs) leads to an emergent capacity to learn in-context from example demonstrations. Despite progress, theoretical understanding of this phenomenon remains limited. We argue that in-context learning relies on recombination of compositional operations found in natural language data. We derive an information-theoretic bound showing how in-context learning abilities arise from generic next-token prediction when the pretraining distribution has sufficient amounts of compositional structure, under linguistically motivated assumptions. A second bound provides a theoretical justification for the empirical success of prompting LLMs to output intermediate steps towards an answer. To validate theoretical predictions, we introduce a controlled setup for inducing in-context learning; unlike previous approaches, it accounts for the compositional nature of language. Trained transformers can perform in-context learning for a range of tasks, in a manner consistent with the theoretical results. Mirroring real-world LLMs in a miniature setup, in-context learning emerges when scaling parameters and data, and models perform better when prompted to output intermediate steps. Probing shows that in-context learning is supported by a representation of the input's compositional structure. Taken together, these results provide a step towards theoretical understanding of emergent behavior in large language models.
A functorial approach to monomorphism categories II: Indecomposables

We investigate the (separated) monomorphism category $\operatorname{mono}(Q,\Lambda)$ of a quiver $Q$ over an Artin algebra $\Lambda$. We construct an epivalence from $\overline{\operatorname{mono}}(Q,\Lambda)$ to $\operatorname{rep}(Q,\overline{\operatorname{mod}}\, \Lambda)$, where $\operatorname{mod}\Lambda$ is the category of finitely generated modules and $\overline{\operatorname{mod}}\, \Lambda$ and $\overline{\operatorname{mono}}(Q,\Lambda)$ denote the respective injectively stable categories. Furthermore, if $Q$ has at least one arrow, then we show that this is an equivalence if and only if $\Lambda$ is hereditary. In general, it induces a bijection between indecomposable objects in $\operatorname{rep}(Q,\overline{\operatorname{mod}}\, \Lambda)$ and non-injective indecomposable objects in $\operatorname{mono}(Q,\Lambda)$. We show that the generalized Mimo-construction, an explicit minimal right approximation into $\operatorname{mono}{(Q,\Lambda)}$, gives an inverse to this bijection. Using this, we describe the indecomposables in the monomorphism category of a radical-square-zero Nakayama algebra, and give a bijection between the indecomposables in the monomorphism category of two artinian uniserial rings of Loewy length $3$ with the same residue field. These results are proved using free monads on an abelian category, in order to avoid the technical combinatorics arising from quiver representations. The setup also specializes to representations of modulations. In particular, we obtain new results on the singularity category of the algebras $H$ which were introduced by Geiss, Leclerc, and Schr\"oer in order to extend their results relating cluster algebras and Lusztig's semicanonical basis to symmetrizable Cartan matrices. We also recover results on the $\iota$quivers algebras which were introduced by Lu and Wang to realize $\iota$quantum groups via semi-derived Hall algebras.
Superconducting Gap Structure of the Cage Compound Sc5Rh6Sn18

We report on the superconducting properties of the cage compound Sc 5 Rh 6 Sn 18 with T c = 5 K determined by magnetic susceptibility, electrical resistivity, and specific heat measurements. Measurements in magnetic fields indicate type-II superconductivity with the upper critical field µ 0 H c2 (0) = 7.24 T. H c2 ( T ) shows a clear enhancement over the Werthamer–Helfand–Hohenberg prediction, suggesting a strong electron–phonon coupling. Specific heat data reveal that Sc 5 Rh 6 Sn 18 is indeed a strong-coupling BCS superconductor having an isotropic superconducting gap.
Phenomena, dynamics and instabilities of vortex pairs

Our motivation for studying the dynamics of vortex pairs stems initially from an interest in the trailing wake vortices from aircraft and the dynamics of longitudinal vortices close to a vehicle surface. However, our motivation also comes from the fact that vortex–vortex interactions and vortex–wall interactions are fundamental to many turbulent flows. The intent of the paper is to present an overview of some of our recent work concerning the formation and structure of counter-rotating vortex pairs. We are interested in the long-wave and short-wave three-dimensional instabilities that evolve for an isolated vortex pair, but also we would like to know how vortex pairs interact with a wall, including both two-dimensional interactions, and also the influence of the surface on the three-dimensional instabilities. The emphasis of this presentation is on physical mechanisms by which vortices interact with each other and with surfaces, principally from an experimental approach, but also coupled with analytical studies.
Query2doc: Query Expansion with Large Language Models

This paper introduces a simple yet effective query expansion approach, denoted as query2doc, to improve both sparse and dense retrieval systems. The proposed method first generates pseudo-documents by few-shot prompting large language models (LLMs), and then expands the query with generated pseudo-documents. LLMs are trained on web-scale text corpora and are adept at knowledge memorization. The pseudo-documents from LLMs often contain highly relevant information that can aid in query disambiguation and guide the retrievers. Experimental results demonstrate that query2doc boosts the performance of BM25 by 3% to 15% on ad-hoc IR datasets, such as MS-MARCO and TREC DL, without any model fine-tuning. Furthermore, our method also benefits state-of-the-art dense retrievers in terms of both in-domain and out-of-domain results.
Arrhenius temperature dependence of the crystallization time of deeply supercooled liquids

Usually, supercooled liquids and glasses are thermodynamically unstable against crystallization. Classical nucleation theory (CNT) has been used to describe the crystallization dynamics of supercooled liquids. However, recent studies on overcompressed hard spheres show that their crystallization dynamics are intermittent and mediated by avalanche-like rearrangements of particles, which largely differ from the CNT. These observations suggest that the crystallization times of deeply supercooled liquids or glasses cannot be described by the CNT, but this point has not yet been studied in detail. In this paper, we use molecular dynamics simulations to study the crystallization dynamics of soft spheres just after an instantaneous quench. We show that although the equilibrium relaxation time increases in a super-Arrhenius manner with decreasing temperature, the crystallization time shows an Arrhenius temperature dependence at very low temperatures. This is contrary to the conventional formula based on the CNT. Furthermore, the estimated energy barrier for the crystallization is surprisingly small compared to that for the equilibrium dynamics. By comparing the crystallization and aging dynamics quantitatively, we show that a coupling between aging and crystallization is the key for understanding the rapid crystallization of deeply supercooled liquids or glasses.
FPUS23: An Ultrasound Fetus Phantom Dataset With Deep Neural Network Evaluations for Fetus Orientations, Fetal Planes, and Anatomical Features

Ultrasound imaging is one of the most prominent technologies to evaluate the growth, progression, and overall health of a fetus during its gestation. However, the interpretation of the data obtained from such studies is best left to expert physicians and technicians who are trained and well-versed in analyzing such images. To improve the clinical workflow and potentially develop an at-home ultrasound-based fetal monitoring platform, we present a novel fetus phantom ultrasound dataset, FPUS23, which can be used to identify (1) the correct diagnostic planes for estimating fetal biometric values, (2) fetus orientation, (3) their anatomical features, and (4) bounding boxes of the fetus phantom anatomies at 23 weeks gestation. The entire dataset is composed of 15,728 images, which are used to train four different Deep Neural Network models, built upon a ResNet34 backbone, for detecting aforementioned fetus features and use-cases. We have also evaluated the models trained using our FPUS23 dataset, to show that the information learned by these models can be used to substantially increase the accuracy on real-world ultrasound fetus datasets. We make the FPUS23 dataset and the pre-trained models publicly accessible at https://github.com/bharathprabakaran/FPUS23, which will further facilitate future research on fetal ultrasound imaging.
Using VAEs to Learn Latent Variables: Observations on Applications in cryo-EM

Variational autoencoders (VAEs) are a popular generative model used to approximate distributions. The encoder part of the VAE is used in amortized learning of latent variables, producing a latent representation for data samples. Recently, VAEs have been used to characterize physical and biological systems. In this case study, we qualitatively examine the amortization properties of a VAE used in biological applications. We find that in this application the encoder bears a qualitative resemblance to more traditional explicit representation of latent variables.
The Cosmic Microwave Background Radiation Temperature Signifying the Existence of the Thought-Carrying Particle, Thought Retaining Particle and Thought Force

We developed a consciousness model. Our consciousness model incorporating Thought-carrying particle (TCP), Thought-retaining particle (TRP) and Thought Force (TF) signifies the existence of the universal consciousness that exists along with the universe. The universal consciousness is a functional state of the universal mind (UM). We indicated the existence of thought force (TF) that is an expression of the universal consciousness. This TF is carried by the TCP in the presence of its super-symmetrical partner TRP. Many physicists believe that unifying all the forces, including gravity, into a single theory would require a phenomenon called super-symmetry. With super-symmetry, every fermion would have a boson twin, and vice-versa. TCP that behaves like boson should accompany its super-symmetrical partner TRP that functions like fermion in the generalized simpler way. The UM as well as the individual mind is constituted by these TCP and TRP. The quantized energy of the TCP is found to cause the universal consciousness as well as the cosmic microwave background radiation temperature. This TF appears to be the primordial quantum field that, in turn, exerts its functions both in vitro and in vivo. The individual consciousness owes its origin to the universal consciousness created by the same quantized energy. The individual consciousness is generated and maintained by the quantum mechanical activities of these postulated TCP and TRP in the presence of the thought force (TF) in vivo. The existence of TCP, TRP and Thought force (TF) is indicated here to form a possible bridge between mind and matter. NeuroQuantology | September 2012 | Volume 10 | Issue 3 | Page 428-442
Higher-order topological superconductors characterized by Fermi level crossings

We demonstrate that level crossings at the Fermi energy serve as robust indicators for higher-order topology in two-dimensional superconductors of symmetry class D. These crossings occur when the boundary condition in one direction is continuously varied from periodic to open, revealing the topological distinction between opposite edges. The associated Majorana numbers acquire nontrivial values whenever the system supports two Majorana zero modes distributed at its corners. Owing to their immunity to perturbations that break crystalline symmetries, Fermi level crossings are able to characterize a wide range of higher-order topological superconductors. By directly identifying the level-crossing points from the bulk Hamiltonian, we establish the correspondence between gapped bulk and Majorana corner states in higher-order phases. In the end, we illustrate this correspondence using two toy models. Our findings suggest that Fermi level crossings offer a possible avenue for characterizing higher-order topological superconductors in a unifying framework.
Flares, Jets, and Quasiperiodic Outbursts from Neutron Star Merger Remnants

Using numerical relativity simulations with a subgrid dynamo prescription to generate strong initial magnetic fields, we investigate the possibility of launching a jet-like outflow from the hypermassive neutron star (HMNS) during the early stages of the merger, prior to the remnant’s collapse to a black hole. We demonstrate that buoyant instabilities in the strongly magnetized HMNS can lead to a periodic emission of powerful electromagnetic flares shortly after the merger. These are followed by a collimated mildly relativistic outflow. Both types of outflows feature quasiperiodic kilohertz substructure. These early-time outflows may power precursors to short-duration gamma-ray bursts (sGRBs) or in some cases the entire sGRB. While the overall temporal power spectrum we find broadly agrees with the one recently reported for quasiperiodic oscillations in the sGRB GRB910711, our simulations suggest that the periodic electromagnetic substructure is dominated by magnetohydrodynamic shearing processes rather than correlating with the corresponding postmerger gravitational-wave signal.
Synthesizing and multiplexing autonomous quantum coherences

Quantum coherence is a crucial prerequisite for quantum technologies. Therefore, the robust generation, as autonomous as possible, of quantum coherence remains the essential problem for developing this field. We consider a method of synthesizing and multiplexing quantum coherence from spin systems without any direct drives only coupled to bosonic baths. The previous studies in this field have demonstrated that a back-action of the bath to the spin subsystem is important to generate it, however, it simultaneously gives significant limits to the generated coherence. We propose a viable approach with the bosonic bath that allows overcoming these limits by avoiding the destructive effect of the back-action processes. Using this approach, we suggest an advanced synthesis of the quantum coherence non-perturbatively in the spin-boson coupling parameters of multiple bosonic baths to increase and multiplex it for upcoming proof-of-principle experiments.
Simfluence: Modeling the Influence of Individual Training Examples by Simulating Training Runs

Training data attribution (TDA) methods offer to trace a model's prediction on any given example back to specific influential training examples. Existing approaches do so by assigning a scalar influence score to each training example, under a simplifying assumption that influence is additive. But in reality, we observe that training examples interact in highly non-additive ways due to factors such as inter-example redundancy, training order, and curriculum learning effects. To study such interactions, we propose Simfluence, a new paradigm for TDA where the goal is not to produce a single influence score per example, but instead a training run simulator: the user asks, ``If my model had trained on example $z_1$, then $z_2$, . , then $z_n$, how would it behave on $z_{test}$? ''; the simulator should then output a simulated training run, which is a time series predicting the loss on $z_{test}$ at every step of the simulated run. This enables users to answer counterfactual questions about what their model would have learned under different training curricula, and to directly see where in training that learning would occur. We present a simulator, Simfluence-Linear, that captures non-additive interactions and is often able to predict the spiky trajectory of individual example losses with surprising fidelity. Furthermore, we show that existing TDA methods such as TracIn and influence functions can be viewed as special cases of Simfluence-Linear. This enables us to directly compare methods in terms of their simulation accuracy, subsuming several prior TDA approaches to evaluation. In experiments on large language model (LLM) fine-tuning, we show that our method predicts loss trajectories with much higher accuracy than existing TDA methods (doubling Spearman's correlation and reducing mean-squared error by 75%) across several tasks, models, and training methods.
Merged beam measurements of mutual neutralization of H+ and H- ions

A merged beam apparatus is described that has been developed to study mutual neutralization. Initially, it has been used to measure cross sections for H+/H- neutralization for centre-of-mass energies between 3 and 500 eV. The results are in excellent agreement with those of Szucs et al. (1984), which covered the range from 5 to 2000 eV, and with inclined beam measurements of Peart et al in the overlapping range of energies from 30 to 500 eV. There is also good accord with a variety of recent theoretical results, notably those by Sidis et al. (1983) Fussen and Kubach (1986) and Shingal and Bransden.
Experimental certification of more than one bit of quantum randomness in the two inputs and two outputs scenario

One of the striking properties of quantum mechanics is the occurrence of the Bell-type non-locality. They are a fundamental feature of the theory that allows two parties that share an entangled quantum system to observe correlations stronger than possible in classical physics. In addition to their theoretical significance, non-local correlations have practical applications, such as device-independent randomness generation, providing private unpredictable numbers even when they are obtained using devices delivered by an untrusted vendor. Thus, determining the quantity of certifiable randomness that can be produced using a specific set of non-local correlations is of significant interest. In this paper, we present an experimental realization of recent Bell-type operators designed to provide private random numbers that are secure against adversaries with quantum resources. We use semi-definite programming to provide lower bounds on the generated randomness in terms of both min-entropy and von Neumann entropy in a device-independent scenario. We compare experimental setups providing Bell violations close to the Tsirelson’s bound with lower rates of events, with setups having slightly worse levels of violation but higher event rates. Our results demonstrate the first experiment that certifies close to two bits of randomness from binary measurements of two parties. Apart from single-round certification, we provide an analysis of finite-key protocol for quantum randomness expansion using the Entropy Accumulation theorem and show its advantages compared to existing solutions.
Editing Implicit Assumptions in Text-to-Image Diffusion Models

Text-to-image diffusion models often make implicit assumptions about the world when generating images. While some assumptions are useful (e.g., the sky is blue), they can also be outdated, incorrect, or reflective of social biases present in the training data. Thus, there is a need to control these assumptions without requiring explicit user input or costly re-training. In this work, we aim to edit a given implicit assumption in a pre-trained diffusion model. Our Text-to-Image Model Editing method, TIME for short, receives a pair of inputs: a "source" under-specified prompt for which the model makes an implicit assumption (e.g., "a pack of roses"), and a "destination" prompt that describes the same setting, but with a specified desired attribute (e.g., "a pack of blue roses"). TIME then updates the model’s cross-attention layers, as these layers assign visual meaning to textual tokens. We edit the projection matrices in these layers such that the source prompt is projected close to the destination prompt. Our method is highly efficient, as it modifies a mere 2.2% of the model’s parameters in under one second. To evaluate model editing approaches, we introduce TIMED (TIME Dataset), containing 147 source and destination prompt pairs from various domains. Our experiments (using Stable Diffusion) show that TIME is successful in model editing, generalizes well for related prompts unseen during editing, and imposes minimal effect on unrelated generations.1
Ultra-High-Resolution Detector Simulation with Intra-Event Aware GAN and Self-Supervised Relational Reasoning

Simulating high-resolution detector responses is a storage-costly and computationally intensive process that has long been challenging in particle physics. Despite the ability of deep generative models to make this process more cost-efficient, ultra-high-resolution detector simulation still proves to be difficult as it contains correlated and fine-grained mutual information within an event. To overcome these limitations, we propose Intra-Event Aware GAN (IEA-GAN), a novel fusion of Self-Supervised Learning and Generative Adversarial Networks. IEA-GAN presents a Relational Reasoning Module that approximates the concept of an ''event'' in detector simulation, allowing for the generation of correlated layer-dependent contextualized images for high-resolution detector responses with a proper relational inductive bias. IEA-GAN also introduces a new intra-event aware loss and a Uniformity loss, resulting in significant enhancements to image fidelity and diversity. We demonstrate IEA-GAN's application in generating sensor-dependent images for the high-granularity Pixel Vertex Detector (PXD), with more than 7.5M information channels and a non-trivial geometry, at the Belle II Experiment. Applications of this work include controllable simulation-based inference and event generation, high-granularity detector simulation such as at the HL-LHC (High Luminosity LHC), and fine-grained density estimation and sampling. To the best of our knowledge, IEA-GAN is the first algorithm for faithful ultra-high-resolution detector simulation with event-based reasoning.
General Loss Functions Lead to (Approximate) Interpolation in High Dimensions

We provide a unified framework, applicable to a general family of convex losses and across binary and multiclass settings in the overparameterized regime, to approximately characterize the implicit bias of gradient descent in closed form. Specifically, we show that the implicit bias is approximated (but not exactly equal to) the minimum-norm interpolation in high dimensions, which arises from training on the squared loss. In contrast to prior work which was tailored to exponentially-tailed losses and used the intermediate support-vector-machine formulation, our framework directly builds on the primal-dual analysis of Ji and Telgarsky (2021), allowing us to provide new approximate equivalences for general convex losses through a novel sensitivity analysis. Our framework also recovers existing exact equivalence results for exponentially-tailed losses across binary and multiclass settings. Finally, we provide evidence for the tightness of our techniques, which we use to demonstrate the effect of certain loss functions designed for out-of-distribution problems on the closed-form solution.