Miracle-dz/UniFAR-FTUs huggingface dataset:
Smoothing the Edges: A General Framework for Smooth Optimization in Sparse Regularization using Hadamard Overparametrization

This paper presents a framework for smooth optimization of objectives with $\ell_q$ and $\ell_{p,q}$ regularization for (structured) sparsity. Finding solutions to these non-smooth and possibly non-convex problems typically relies on specialized optimization routines. In contrast, the method studied here is compatible with off-the-shelf (stochastic) gradient descent that is ubiquitous in deep learning, thereby enabling differentiable sparse regularization without approximations. The proposed optimization transfer comprises an overparametrization of selected model parameters followed by a change of penalties. In the overparametrized problem, smooth and convex $\ell_2$ regularization induces non-smooth and non-convex regularization in the original parametrization. We show that the resulting surrogate problem not only has an identical global optimum but also exactly preserves the local minima. This is particularly useful in non-convex regularization, where finding global solutions is NP-hard and local minima often generalize well. We provide an integrative overview that consolidates various literature strands on sparsity-inducing parametrizations in a general setting and meaningfully extend existing approaches. The feasibility of our approach is evaluated through numerical experiments, demonstrating its effectiveness by matching or outperforming common implementations of convex and non-convex regularizers.
Three-dimensional Vorticity Effects on Extinction Behavior of Laminar Flamelets

A recently proposed rotational flamelet model is developed and tested in an improved framework of detailed chemistry and transport models. The rotational flamelet model, in contrast to standard two-dimensional or axisymmetric irrotational models, incorporates the effects of shear strain and vorticity on local flame behavior and is three-dimensional by nature. A similarity solution is used to reduce the three-dimensional governing equations to a set of ODEs involving a transformation to a non-Newtonian reference frame. Multiple flamelet cases including both diffusion and premixed flames are tested to demonstrate the impact of vorticity on flame structure, extinction, and local mixture composition. For diluted hydrogen flames where the location of minimum density coincides with the location of peak temperature, the centrifugal force induced by vorticity reduces mass flow rate in the flame zone, effectively lowering the local strain rate. This results in increased residence time, thus extending flammability limits and reducing burning rates. For pure-hydrogen/oxygen flames where minimum density lies between the flame zone and the fuel inlet boundary, centrifugal forces do not significantly modify flame behavior. Stable and unstable branches of extinction curves (S-curves) are computed which showcase how vorticity extends flammability limits. These capabilities of the rotational flamelet model reveal that vital physics are currently missing from two-dimensional, irrotational, constant-density, constant normal strain rate flamelet models. The improvements noted regarding detailed chemical kinetics, transport formulation, and thermo-physical properties bring the new flamelet model to par in these areas with existing models, while surpassing existing models in terms of physical emulation.
Dynamic Return and Star-Shaped Risk Measures via BSDEs

This paper establishes characterization results for dynamic return and star-shaped risk measures induced via backward stochastic differential equations (BSDEs). We first characterize a general family of static star-shaped functionals in a locally convex Fr\'echet lattice. Next, employing the Pasch-Hausdorff envelope, we build a suitable family of convex drivers of BSDEs inducing a corresponding family of dynamic convex risk measures of which the dynamic return and star-shaped risk measures emerge as the essential minimum. Furthermore, we prove that if the set of star-shaped supersolutions of a BSDE is not empty, then there exists, for each terminal condition, at least one convex BSDE with a non-empty set of supersolutions, yielding the minimal star-shaped supersolution. We illustrate our theoretical results in a few examples and demonstrate their usefulness in two applications, to capital allocation and portfolio choice.
One Step of Gradient Descent is Provably the Optimal In-Context Learner with One Layer of Linear Self-Attention

Recent works have empirically analyzed in-context learning and shown that transformers trained on synthetic linear regression tasks can learn to implement ridge regression, which is the Bayes-optimal predictor, given sufficient capacity [Aky\"urek et al., 2023], while one-layer transformers with linear self-attention and no MLP layer will learn to implement one step of gradient descent (GD) on a least-squares linear regression objective [von Oswald et al., 2022]. However, the theory behind these observations remains poorly understood. We theoretically study transformers with a single layer of linear self-attention, trained on synthetic noisy linear regression data. First, we mathematically show that when the covariates are drawn from a standard Gaussian distribution, the one-layer transformer which minimizes the pre-training loss will implement a single step of GD on the least-squares linear regression objective. Then, we find that changing the distribution of the covariates and weight vector to a non-isotropic Gaussian distribution has a strong impact on the learned algorithm: the global minimizer of the pre-training loss now implements a single step of $\textit{pre-conditioned}$ GD. However, if only the distribution of the responses is changed, then this does not have a large effect on the learned algorithm: even when the response comes from a more general family of $\textit{nonlinear}$ functions, the global minimizer of the pre-training loss still implements a single step of GD on a least-squares linear regression objective.
Quantum cohomology of projective bundles

We construct an I-function of the projective bundle P(V) associated with a not necessarily split vector bundle V\to B as a Fourier transform of the S^1-equivariant J-function of the total space of V and show that it lies on the Givental Lagrangian cone of P(V). Using this result, we show that the quantum cohomology D-module of P(V) splits into the direct sum of the quantum cohomology D-modules of the base space B. This has applications to the semisimplicity of big quantum cohomology.
Stochastic star formation in early galaxies: Implications for the James Webb Space Telescope

The star formation rate (SFR) in high redshift galaxies is expected to be time-variable due to competing physical processes. Such stochastic variability might boost the luminosity of galaxies, possibly explaining the over-abundance seen at $z\gtrsim 10$ by JWST. We aim at quantifying the amplitude and timescales of such variability, and identifying the key driving physical processes. We select 245 $z=7.7$ galaxies with stellar mass $5\times 10^{6}\lesssim M_\star/{\rm M}_\odot\lesssim 5\times 10^{10}$ from SERRA, a suite of high-resolution, radiation-hydrodynamic cosmological simulations. After fitting the average SFR trend, $\langle {\rm SFR} \rangle$, we quantify the time-dependent variation, $\delta(t) \equiv \log [\rm SFR/\langle {\rm SFR} \rangle]$ for each system, and perform a periodogram analysis to search for periodicity modulations. We find that $\delta(t)$ is distributed as a zero-mean Gaussian, with standard deviation $\sigma_\delta \simeq 0.24$ (corresponding to a UV magnitude s.d. $\sigma_{\rm UV} \simeq 0.61$) that is independent of $M_\star$. However, the modulation timescale increases with stellar mass: $t_\delta \sim (9, 50, 100)\, \rm Myr$ for $M_\star \sim (0.1, 1, 5)\times 10^9\, {\rm M}_\odot$, respectively. These timescales are imprinted on the SFR by different processes: (i) photoevaporation, (ii) supernova explosions, and (iii) cosmological accretion/merging dominating in low, intermediate, and high mass systems, respectively. The predicted SFR variations cannot account for the required $z\gtrsim 10$ UV luminosity function boost. Other processes, such as radiation-driven outflows clearing the dust, must then be invoked to explain the enhanced luminosity of super-early systems.
Scalable High-Dimensional Multivariate Linear Regression for Feature-Distributed Data

Feature-distributed data, referred to data partitioned by features and stored across multiple computing nodes, are increasingly common in applications with a large number of features. This paper proposes a two-stage relaxed greedy algorithm (TSRGA) for applying multivariate linear regression to such data. The main advantage of TSRGA is that its communication complexity does not depend on the feature dimension, making it highly scalable to very large data sets. In addition, for multivariate response variables, TSRGA can be used to yield low-rank coefficient estimates. The fast convergence of TSRGA is validated by simulation experiments. Finally, we apply the proposed TSRGA in a financial application that leverages unstructured data from the 10-K reports, demonstrating its usefulness in applications with many dense large-dimensional matrices.
Hydrodynamical simulations of galaxy formation with non-Gaussian initial conditions

Collisionless simulations of structure formation with significant local primordial non-Gaussianities at Mpc scales have shown that a non-Gaussian tail favouring underdensities, with a negative f NL parameter, can significantly change the merging history of galaxy-sized dark matter halos, which then typically assemble later than in vanilla ΛCDM. Moreover, such a small-scale negative f NL could have interesting consequences for the cosmological S 8 tension. Here, we complement our previous work on collisionless simulations with new hydrodynamical simulations of galaxy formation in boxes of 30 Mpc/h, using the RAMSES code. In particular, we show that all feedback prescriptions being otherwise identical, simulations with a negative f NL ∼ -1000 on small scales, hence forming galaxies a bit later than in vanilla ΛCDM, allow to form simulated galaxies with more disky kinematics than in the vanilla case. Therefore, such small-scale primordial non-Gaussianities could potentially help alleviate, simultaneously, tensions in cosmology and galaxy formation. These hydrodynamical simulations on small scales will need to be complemented with larger box simulations with scale-dependent non-Gaussianities, to statistically confirm these trends and explore their observational consequences in further detail.
Personalized Prediction of Recurrent Stress Events Using Self-Supervised Learning on Multimodal Time-Series Data

Chronic stress can significantly affect physical and mental health. The advent of wearable technology allows for the tracking of physiological signals, potentially leading to innovative stress prediction and intervention methods. However, challenges such as label scarcity and data heterogeneity render stress prediction difficult in practice. To counter these issues, we have developed a multimodal personalized stress prediction system using wearable biosignal data. We employ self-supervised learning (SSL) to pre-train the models on each subject's data, allowing the models to learn the baseline dynamics of the participant's biosignals prior to fine-tuning the stress prediction task. We test our model on the Wearable Stress and Affect Detection (WESAD) dataset, demonstrating that our SSL models outperform non-SSL models while utilizing less than 5% of the annotations. These results suggest that our approach can personalize stress prediction to each user with minimal annotations. This paradigm has the potential to enable personalized prediction of a variety of recurring health events using complex multimodal data streams.
GEANN: Scalable Graph Augmentations for Multi-Horizon Time Series Forecasting

Encoder-decoder deep neural networks have been increasingly studied for multi-horizon time series forecasting, especially in real-world applications. However, to forecast accurately, these sophisticated models typically rely on a large number of time series examples with substantial history. A rapidly growing topic of interest is forecasting time series which lack sufficient historical data -- often referred to as the ``cold start'' problem. In this paper, we introduce a novel yet simple method to address this problem by leveraging graph neural networks (GNNs) as a data augmentation for enhancing the encoder used by such forecasters. These GNN-based features can capture complex inter-series relationships, and their generation process can be optimized end-to-end with the forecasting task. We show that our architecture can use either data-driven or domain knowledge-defined graphs, scaling to incorporate information from multiple very large graphs with millions of nodes. In our target application of demand forecasting for a large e-commerce retailer, we demonstrate on both a small dataset of 100K products and a large dataset with over 2 million products that our method improves overall performance over competitive baseline models. More importantly, we show that it brings substantially more gains to ``cold start'' products such as those newly launched or recently out-of-stock.
Design of a Majorana trijunction

Braiding of Majorana states demonstrates their non-Abelian exchange statistics. One implementation of braiding requires control of the pairwise couplings between all Majorana states in a trijunction device. To have adiabaticity, a trijunction device requires the desired pair coupling to be sufficiently large and the undesired couplings to vanish. In this work, we design and simulate a trijunction device in a two-dimensional electron gas with a focus on the normal region that connects three Majorana states. We use an optimisation approach to find the operational regime of the device in a multi-dimensional voltage space. Using the optimization results, we simulate a braiding experiment by adiabatically coupling different pairs of Majorana states without closing the topological gap. We then evaluate the feasibility of braiding in a trijunction device for different shapes and disorder strengths.
Quantifying the perceptual value of lexical and non-lexical channels in speech

Speech is a fundamental means of communication that can be seen to provide two channels for transmitting information: the lexical channel of which words are said, and the non-lexical channel of how they are spoken. Both channels shape listener expectations of upcoming communication; however, directly quantifying their relative effect on expectations is challenging. Previous attempts require spoken variations of lexically-equivalent dialogue turns or conspicuous acoustic manipulations. This paper introduces a generalised paradigm to study the value of non-lexical information in dialogue across unconstrained lexical content. By quantifying the perceptual value of the non-lexical channel with both accuracy and entropy reduction, we show that non-lexical information produces a consistent effect on expectations of upcoming dialogue: even when it leads to poorer discriminative turn judgements than lexical content alone, it yields higher consensus among participants.
The imprint of galaxy mergers on satellite planes in a cosmological context

Flattened and kinematically correlated planes of dwarf satellite galaxies have been observed in the Local Volume. The slinging out of satellites during host galaxy mergers has been suggested as a formation mechanism for these peculiar structures. We statistically examined the impact of major mergers on present-time satellite systems for the first time in a full cosmological context using the IllustrisTNG suite of hydrodynamic simulations. Mergers with mass ratios above 1/3 generally have a negligible or adverse impact on the phase-space correlation of observationally motivated satellites. Even high angular momentum mergers are inefficient at slinging satellites outward due to the extended nature of simulated satellite distributions. Furthermore, any potential merger imprint is partially washed out by post-merger accretion of satellites, while satellites bound to the merging haloes since the merger’s beginning are disrupted and stripped of mass – minimizing the merger’s influence on the present-time distribution of the most massive satellites after $2{\! -\! }5\, \mathrm{Gyr}$. Constraining our sample to satellites bound to their host throughout the full duration of their system’s last merger, we recover no particular improvement in their phase-space correlation. Instead, such participant satellites experience a contraction of their radial distribution during and after the merger, resulting in smaller absolute plane heights (but comparable axial ratios). Overall, major mergers do not appear to form correlated planes in a statistical sample. Mergers that efficiently transfer their angular momentum to satellite distributions can marginally enhance their phase-space correlation, but cannot form highly flattened and orbitally coherent configurations as observed in our local Universe.
Distilled Pruning: Using Synthetic Data to Win the Lottery

This work introduces a novel approach to pruning deep learning models by using distilled data. Unlike conventional strategies which primarily focus on architectural or algorithmic optimization, our method reconsiders the role of data in these scenarios. Distilled datasets capture essential patterns from larger datasets, and we demonstrate how to leverage this capability to enable a computationally efficient pruning process. Our approach can find sparse, trainable subnetworks (a.k.a. Lottery Tickets) up to 5x faster than Iterative Magnitude Pruning at comparable sparsity on CIFAR-10. The experimental results highlight the potential of using distilled data for resource-efficient neural network pruning, model compression, and neural architecture search.
Unsupervised Domain Adaptation for Medical Image Segmentation via Feature-space Density Matching

Semantic segmentation is a critical step in automated image interpretation and analysis where pixels are classified into one or more predefined semantically meaningful classes. Deep learning approaches for semantic segmentation rely on harnessing the power of annotated images to learn features indicative of these semantic classes. Nonetheless, they often fail to generalize when there is a significant domain (i.e., distributional) shift between the training (i.e., source) data and the dataset(s) encountered when deployed (i.e., target), necessitating manual annotations for the target data to achieve acceptable performance. This is especially important in medical imaging because different image modalities have significant intra- and inter-site variations due to protocol and vendor variability. Current techniques are sensitive to hyperparameter tuning and target dataset size. This paper presents an unsupervised domain adaptation approach for semantic segmentation that alleviates the need for annotating target data. Using kernel density estimation, we match the target data distribution to the source in the feature space, particularly when the number of target samples is limited (3% of the target dataset size). We demonstrate the efficacy of our proposed approach on 2 datasets, multisite prostate MRI and histopathology images.
A Survey of Nanoflare Properties in Active Regions Observed with the Solar Dynamics Observatory

In this paper, we examine 15 different active regions (ARs) observed with the Solar Dynamics Observatory and analyze their nanoflare properties. We have recently developed a technique that systematically identifies and measures plasma temperature dynamics by computing time lags between light curves. The time lag method tests whether the plasma is maintained at a steady temperature, or if it is dynamic, undergoing heating and cooling cycles. An important aspect of our technique is that it analyzes both observationally distinct coronal loops as well as the much more prevalent diffuse emission between them. We find that the widespread cooling reported previously for NOAA AR 11082 is a generic property of all ARs. The results are consistent with impulsive nanoflare heating followed by slower cooling. Only occasionally, however, is there full cooling from above 7 MK to well below 1 MK. More often, the plasma cools to approximately 1–2 MK before being reheated by another nanoflare. These same 15 ARs were first studied by Warren et al. We find that the degree of cooling is not well correlated with the reported slopes of the emission measure distribution. We also conclude that the Fe xviii emitting plasma that they measured is mostly in a state of cooling. These results support the idea that nanoflares have a distribution of energies and frequencies, with the average delay between successive events on an individual flux tube being comparable to the plasma cooling timescale.
RObotic MAnipulation Network (ROMAN) - Hybrid Hierarchical Learning for Solving Complex Sequential Tasks

Solving long sequential tasks poses a significant challenge in embodied artificial intelligence. Enabling a robotic system to perform diverse sequential tasks with a broad range of manipulation skills is an active area of research. In this work, we present a Hybrid Hierarchical Learning framework, the Robotic Manipulation Network (ROMAN), to address the challenge of solving multiple complex tasks over long time horizons in robotic manipulation. ROMAN achieves task versatility and robust failure recovery by integrating behavioural cloning, imitation learning, and reinforcement learning. It consists of a central manipulation network that coordinates an ensemble of various neural networks, each specialising in distinct re-combinable sub-tasks to generate their correct in-sequence actions for solving complex long-horizon manipulation tasks. Experimental results show that by orchestrating and activating these specialised manipulation experts, ROMAN generates correct sequential activations for accomplishing long sequences of sophisticated manipulation tasks and achieving adaptive behaviours beyond demonstrations, while exhibiting robustness to various sensory noises. These results demonstrate the significance and versatility of ROMAN's dynamic adaptability featuring autonomous failure recovery capabilities, and highlight its potential for various autonomous manipulation tasks that demand adaptive motor skills.