Miracle-dz/UniFAR-FTUs huggingface dataset:
Continuous Pseudo-Labeling from the Start

Self-training (ST), or pseudo-labeling has sparked significant interest in the automatic speech recognition (ASR) community recently because of its success in harnessing unlabeled data. Unlike prior semi-supervised learning approaches that relied on iteratively regenerating pseudo-labels (PLs) from a trained model and using them to train a new model, recent state-of-the-art methods perform `continuous training' where PLs are generated using a very recent version of the model being trained. Nevertheless, these approaches still rely on bootstrapping the ST using an initial supervised learning phase where the model is trained on labeled data alone. We believe this has the potential for over-fitting to the labeled dataset in low resource settings and that ST from the start of training should reduce over-fitting. In this paper we show how we can do this by dynamically controlling the evolution of PLs during the training process in ASR. To the best of our knowledge, this is the first study that shows the feasibility of generating PLs from the very start of the training. We are able to achieve this using two techniques that avoid instabilities which lead to degenerate models that do not generalize. Firstly, we control the evolution of PLs through a curriculum that uses the online changes in PLs to control the membership of the cache of PLs and improve generalization. Secondly, we find that by sampling transcriptions from the predictive distribution, rather than only using the best transcription, we can stabilize training further. With these techniques, our ST models match prior works without an external language model.
An electronic-based model of the optical nonlinearity of low density Drude materials

Low electron density Drude (LEDD) materials such as indium tin oxide (ITO) are receiving considerable attention because of their combination of CMOS compatibility, unique epsilon-near-zero (ENZ) behavior, and giant ultrafast nonlinear thermo-optic response. However, the understanding of the electronic and optical response of LEDD materials is so far based on simplistic extensions of known models of noble metals, frequently without the inclusion of the interplay among the lower electron density, relatively high Debye energy, and the non-parabolic band structure. To bridge this knowledge gap, this work provides a complete understanding of the nonlinear electronic-thermal-optical response of LEDD materials. In particular, we rely on state-of-the-art electron dynamics modeling, as well as the newly derived time-dependent permittivity model for LEDD materials under optical pumping within the adiabatic approximation. We ﬁnd that unlike noble metals, the electron temperatures can reach the Fermi temperature, in which case the eﬀective chemical potential dramatically decreases and even becomes negative, thus, transiently converting the Drude metal into a semiconductor. We further show that the nonlinear optical response of LEDD materials originating from the changes to the real part of the permittivity is because of changes to population. This resolves the argument about the rise time of the permittivity and shows that it is instantaneous. In this vein, we show that referring to the LEDD permittivity as having a “saturable” nonlinearity is unsuitable since its permittivity dynamics does not originate from population inversion. Finally, we analyze the probe pulse dynamics and unlike previous work, we obtain a quantitative agreement with the results of recent experiments.
On Distributionally Robust Multistage Convex Optimization: Data-driven Models and Performance

This paper presents a novel algorithmic study with extensive numerical experiments of distributionally robust multistage convex optimization (DR-MCO). Following the previous work on dual dynamic programming (DDP) algorithmic framework for DR-MCO [48], we focus on data-driven DR-MCO models with Wasserstein ambiguity sets that allow probability measures with infinite supports. These data-driven Wasserstein DR-MCO models have out-of-sample performance guarantees and adjustable in-sample conservatism. Then by exploiting additional concavity or convexity in the uncertain cost functions, we design exact single stage subproblem oracle (SSSO) implementations that ensure the convergence of DDP algorithms. We test the data-driven Wasserstein DR-MCO models against multistage robust convex optimization (MRCO), risk-neutral and risk-averse multistage stochastic convex optimization (MSCO) models on multi-commodity inventory problems and hydro-thermal power planning problems. The results show that our DR-MCO models could outperform MRCO and MSCO models when the data size is small.
Flux Eruption Events Drive Angular Momentum Transport in Magnetically Arrested Accretion Flows

We evolve two high-resolution general relativistic magnetohydrodynamic simulations of advection-dominated accretion flows around nonspinning black holes (BHs), each over a duration ∼3 × 105 GM BH/c 3. One model captures the evolution of a weakly magnetized (SANE) disk and the other that of a magnetically arrested disk (MAD). Magnetic flux eruptions in the MAD model push out gas from the disk and launch strong winds with outflow efficiencies at times reaching 10% of the incoming accretion power. Despite the substantial power in these winds, average mass outflow rates remain low out to a radius ∼100GM BH/c 2, only reaching ∼60%–80% of the horizon accretion rate. The average outward angular momentum transport is primarily radial in both modes of accretion, but with a clear distinction: magnetic flux eruption–driven disk winds cause a strong vertical flow of angular momentum in the MAD model, while for the SANE model, the magnetorotational instability (MRI) moves angular momentum mostly equatorially through the disk. Further, we find that the MAD state is highly transitory and nonaxisymmetric, with the accretion mode often changing to a SANE-like state following an eruption before reattaining magnetic flux saturation with time. The Reynolds stress changes directions during such transitions, with the MAD (SANE) state showing an inward (outward) stress, possibly pointing to intermittent MRI-driven accretion in MADs. Pinning down the nature of flux eruptions using next-generation telescopes will be crucial in understanding the flow of mass, magnetic flux, and angular momentum in sub-Eddington accreting BHs like M87* and Sagittarius A*.
Unraveling information about supranuclear-dense matter from the complete binary neutron star coalescence process using future gravitational-wave detector networks

Gravitational waves provide us with an extraordinary tool to study the matter inside neutron stars. In particular, the postmerger signal probes an extreme temperature and density regime and will help reveal information about the equation of state of supranuclear-dense matter. Although current detectors are most sensitive to the signal emitted by binary neutron stars before the merger, the upgrades of existing detectors and the construction of the next generation of detectors will make postmerger detections feasible. For this purpose, we present a new analytical, frequency-domain model for the inspiral-merger-postmerger signal emitted by binary neutron stars systems. The inspiral and merger part of the signals are modeled with IMRPhenomD_NRTidalv2, and we describe the main emission peak of postmerger with a three-parameter Lorentzian, using two different approaches: one in which the Lorentzian parameters are kept free, and one in which we model them via quasi-universal relations. We test the performance of our new complete waveform model in parameter estimation analyses, studying simulated signals obtained from both our developed model and by injecting numerical relativity waveforms. We investigate the performance of different detector networks to determine the improvement that future detectors will bring to our analysis. We consider Advanced LIGO+ and Advanced Virgo+, KAGRA, and LIGO-India. We also study the possible impact of a detector with high sensitivity in the kilohertz band like NEMO, and finally we compare these results to the ones we obtain with third-generation detectors, the Einstein Telescope and the Cosmic Explorer.
Confirmation of a cluster of galaxies hidden behind the Galactic bulge using the VVV Survey

Suzaku and Chandra X-ray observations detected a new cluster of galaxies, Suzaku J1759-3450, at a redshift z=0.13. It is located behind the Milky Way, and the high Galactic dust extinction renders it nearly invisible at optical wavelengths. We attempt here to confirm the galaxy cluster with near-infrared imaging observations, and to characterize its central member galaxies. Images from the VVV survey were used to detect candidate member galaxies of Suzaku J1759-3450 within the central region of the cluster, up to 350 kpc from the X-ray peak emission. Color-magnitude and color-color diagrams and morphology criteria allowed us to select the galaxies among the numerous foreground sources. Fifteen candidate cluster members were found very close to a modeled red-sequence at the redshift of the cluster. Five members are extremely bright, and one is possibly a cD galaxy. The asymmetry in the spatial distribution of the galaxies respect to the X-ray peak emission is an indicator of that this cluster is still suffering a virialization process. Our investigation of Suzaku J1759-3450 demonstrates the potential of the VVV Survey to study the hidden population of galaxies in the Zone of Avoidance.
Towards Summary Candidates Fusion

Sequence-to-sequence deep neural models fine-tuned for abstractive summarization can achieve great performance on datasets with enough human annotations. Yet, it has been shown that they have not reached their full potential, with a wide gap between the top beam search output and the oracle beam. Recently, re-ranking methods have been proposed, to learn to select a better summary candidate. However, such methods are limited by the summary quality aspects captured by the first-stage candidates. To bypass this limitation, we propose a new paradigm in second-stage abstractive summarization called SummaFusion that fuses several summary candidates to produce a novel abstractive second-stage summary. Our method works well on several summarization datasets, improving both the ROUGE scores and qualitative properties of fused summaries. It is especially good when the candidates to fuse are worse, such as in the few-shot setup where we set a new state-of-the art. We will make our code and checkpoints available at https://github.com/ntunlp/SummaFusion/.
Keep Me Updated! Memory Management in Long-term Conversations

Remembering important information from the past and continuing to talk about it in the present are crucial in long-term conversations. However, previous literature does not deal with cases where the memorized information is outdated, which may cause confusion in later conversations. To address this issue, we present a novel task and a corresponding dataset of memory management in long-term conversations, in which bots keep track of and bring up the latest information about users while conversing through multiple sessions. In order to support more precise and interpretable memory, we represent memory as unstructured text descriptions of key information and propose a new mechanism of memory management that selectively eliminates invalidated or redundant information. Experimental results show that our approach outperforms the baselines that leave the stored memory unchanged in terms of engagingness and humanness, with larger performance gap especially in the later sessions.
THREE-DIMENSIONAL ADAPTIVE MESH REFINEMENT SIMULATIONS OF LONG-DURATION GAMMA-RAY BURST JETS INSIDE MASSIVE PROGENITOR STARS

We present the results of special relativistic, adaptive mesh refinement, 3D simulations of gamma-ray burst jets expanding inside a realistic stellar progenitor. Our simulations confirm that relativistic jets can propagate and break out of the progenitor star while remaining relativistic. This result is independent of the resolution, even though the amount of turbulence and variability observed in the simulations is greater at higher resolutions. We find that the propagation of the jet head inside the progenitor star is slightly faster in 3D simulations compared to 2D ones at the same resolution. This behavior seems to be due to the fact that the jet head in 3D simulations can wobble around the jet axis, finding the spot of least resistance to proceed. Most of the average jet properties, such as density, pressure, and Lorentz factor, are only marginally affected by the dimensionality of the simulations and therefore results from 2D simulations can be considered reliable.
Parametric estimation of stochastic differential equations via online gradient descent

We propose an online parametric estimation method of stochastic diﬀerential equations with discrete observations and misspeciﬁed modelling based on online gradient descent. Our study provides uniform upper bounds for the risks of the estimators over a family of stochastic diﬀerential equations. The derivation of the bounds involves three underlying theoretical results: the analysis of the stochastic mirror descent algorithm based on dependent and biased subgradients, the simultaneous exponential ergodicity of classes of diﬀusion processes, and the proposal of loss functions whose approximated stochastic subgradients are dependent only on the known model and observations.
Augmentation-Free Graph Contrastive Learning of Invariant-Discriminative Representations

Graph contrastive learning (GCL) is a promising direction toward alleviating the label dependence, poor generalization and weak robustness of graph neural networks, learning representations with invariance, and discriminability by solving pretasks. The pretasks are mainly built on mutual information estimation, which requires data augmentation to construct positive samples with similar semantics to learn invariant signals and negative samples with dissimilar semantics to empower representation discriminability. However, an appropriate data augmentation configuration depends heavily on lots of empirical trials such as choosing the compositions of data augmentation techniques and the corresponding hyperparameter settings. We propose an augmentation-free GCL method, invariant-discriminative GCL (iGCL), that does not intrinsically require negative samples. iGCL designs the invariant-discriminative loss (ID loss) to learn invariant and discriminative representations. On the one hand, ID loss learns invariant signals by directly minimizing the mean square error (MSE) between the target samples and positive samples in the representation space. On the other hand, ID loss ensures that the representations are discriminative by an orthonormal constraint forcing the different dimensions of representations to be independent of each other. This prevents representations from collapsing to a point or subspace. Our theoretical analysis explains the effectiveness of ID loss from the perspectives of the redundancy reduction criterion, canonical correlation analysis (CCA), and information bottleneck (IB) principle. The experimental results demonstrate that iGCL outperforms all baselines on five node classification benchmark datasets. iGCL also shows superior performance for different label ratios and is capable of resisting graph attacks, which indicates that iGCL has excellent generalization and robustness. The source code is available at https://github.com/lehaifeng/ T-GCN/tree/master/iGCL.
Parameter-free Dynamic Graph Embedding for Link Prediction

Dynamic interaction graphs have been widely adopted to model the evolution of user-item interactions over time. There are two crucial factors when modelling user preferences for link prediction in dynamic interaction graphs: 1) collaborative relationship among users and 2) user personalized interaction patterns. Existing methods often implicitly consider these two factors together, which may lead to noisy user modelling when the two factors diverge. In addition, they usually require time-consuming parameter learning with back-propagation, which is prohibitive for real-time user preference modelling. To this end, this paper proposes FreeGEM, a parameter-free dynamic graph embedding method for link prediction. Firstly, to take advantage of the collaborative relationships, we propose an incremental graph embedding engine to obtain user/item embeddings, which is an Online-Monitor-Offline architecture consisting of an Online module to approximately embed users/items over time, a Monitor module to estimate the approximation error in real time and an Offline module to calibrate the user/item embeddings when the online approximation errors exceed a threshold. Meanwhile, we integrate attribute information into the model, which enables FreeGEM to better model users belonging to some under represented groups. Secondly, we design a personalized dynamic interaction pattern modeller, which combines dynamic time decay with attention mechanism to model user short-term interests. Experimental results on two link prediction tasks show that FreeGEM can outperform the state-of-the-art methods in accuracy while achieving over 36X improvement in efficiency. All code and datasets can be found in https://github.com/FudanCISL/FreeGEM.
Broadband X-Ray Spectral Analysis of the Dual AGN System Mrk 739

We present the result of a broadband (0.5–70 keV) X-ray spectral analysis of the late-merger galaxy Mrk 739, which contains a dual active galactic nucleus (AGN), Mrk 739E and Mrk 739W, with a separation of ≈3.4 kpc. The spectra obtained with NuSTAR, Chandra, XMM-Newton, and Swift/BAT are simultaneously analyzed by separating the contributions from the two AGNs and extended emission with the Chandra data. To evaluate the reflection components from the AGN tori, we consider two models, a phenomenological one (pexrav and zgauss) and a more physically motivated one (XCLUMPY). On the basis of the results with XCLUMPY, we find that the AGNs in Mrk 739E and Mrk 739W have intrinsic 2–10 keV luminosities of 1.0 × 1043 and 7.5 × 1041 erg s−1 absorbed by hydrogen column densities of N H < 6.5 × 1019 cm−2 and NH=6.9−1.7+3.2×1021cm−2 , respectively. The torus-covering fraction of the material with N H > 1022cm−2 in Mrk 739E, CT(22)<0.50 at a 90% confidence limit, is found to be smaller than those found for late-merger ultra/luminous infrared galaxies, CT(22)=0.71±0.16 (mean and standard deviation). Considering the small star formation rate of Mrk 739E, we suggest that the gas-to-mass ratio of the host galaxy is an important parameter to determine the circumnuclear environment of an AGN in a late merger.
Resolving the Mixing Time of the Langevin Algorithm to its Stationary Distribution for Log-Concave Sampling

Sampling from a high-dimensional distribution is a fundamental task in statistics, engineering, and the sciences. A canonical approach is the Langevin Algorithm, i.e., the Markov chain for the discretized Langevin Diffusion. This is the sampling analog of Gradient Descent. Despite being studied for several decades in multiple communities, tight mixing bounds for this algorithm remain unresolved even in the seemingly simple setting of log-concave distributions over a bounded domain. This paper completely characterizes the mixing time of the Langevin Algorithm to its stationary distribution in this setting (and others). This mixing result can be combined with any bound on the discretization bias in order to sample from the stationary distribution of the continuous Langevin Diffusion. In this way, we disentangle the study of the mixing and bias of the Langevin Algorithm. Our key insight is to introduce a technique from the differential privacy literature to the sampling literature. This technique, called Privacy Amplification by Iteration, uses as a potential a variant of R\'enyi divergence that is made geometrically aware via Optimal Transport smoothing. This gives a short, simple proof of optimal mixing bounds and has several additional appealing properties. First, our approach removes all unnecessary assumptions required by other sampling analyses. Second, our approach unifies many settings: it extends unchanged if the Langevin Algorithm uses projections, stochastic mini-batch gradients, or strongly convex potentials (whereby our mixing time improves exponentially). Third, our approach exploits convexity only through the contractivity of a gradient step -- reminiscent of how convexity is used in textbook proofs of Gradient Descent. In this way, we offer a new approach towards further unifying the analyses of optimization and sampling algorithms.
PI-QT-Opt: Predictive Information Improves Multi-Task Robotic Reinforcement Learning at Scale

The predictive information, the mutual information between the past and future, has been shown to be a useful representation learning auxiliary loss for training reinforcement learning agents, as the ability to model what will happen next is critical to success on many control tasks. While existing studies are largely restricted to training specialist agents on single-task settings in simulation, in this work, we study modeling the predictive information for robotic agents and its importance for general-purpose agents that are trained to master a large repertoire of diverse skills from large amounts of data. Specifically, we introduce Predictive Information QT-Opt (PI-QT-Opt), a QT-Opt agent augmented with an auxiliary loss that learns representations of the predictive information to solve up to 297 vision-based robot manipulation tasks in simulation and the real world with a single set of parameters. We demonstrate that modeling the predictive information significantly improves success rates on the training tasks and leads to better zero-shot transfer to unseen novel tasks. Finally, we evaluate PI-QT-Opt on real robots, achieving substantial and consistent improvement over QT-Opt in multiple experimental settings of varying environments, skills, and multi-task configurations.
Singular positive mass theorem with arbitrary ends

Motivated by the recent progress on positive mass theorem for asymptotically flat manifolds with arbitrary ends and the Gromov's definition of scalar curvature lower bound for continuous metrics, we start a program on the positive mass theorem for asymptotically flat manifolds with $C^0$ arbitrary ends. In this work as the first step, we establish the positive mass theorem of asymptotically flat manifolds with $C^0$ arbitrary ends when the metric is $W^{1,p}_{\mathrm{loc}}$ for some $p\in(n,\infty]$ and is smooth away from a non-compact closed subset with Hausdorff dimension $n-\frac{p}{p-1}$. New techniques are developed to deal with non-compactness of the singular set.
StoryER: Automatic Story Evaluation via Ranking, Rating and Reasoning

Existing automatic story evaluation methods place a premium on story lexical level coherence, deviating from human preference.We go beyond this limitation by considering a novel Story Evaluation method that mimics human preference when judging a story, namely StoryER, which consists of three sub-tasks: Ranking, Rating and Reasoning.Given either a machine-generated or a human-written story, StoryER requires the machine to output 1) a preference score that corresponds to human preference, 2) specific ratings and their corresponding confidences and 3) comments for various aspects (e.g., opening, character-shaping). To support these tasks, we introduce a well-annotated dataset comprising (i) 100k ranked story pairs; and (ii) a set of 46k ratings and comments on various aspects of the story.We finetune Longformer-Encoder-Decoder (LED) on the collected dataset, with the encoder responsible for preference score and aspect prediction and the decoder for comment generation.Our comprehensive experiments result a competitive benchmark for each task, showing the high correlation to human preference.In addition, we have witnessed the joint learning of the preference scores, the aspect ratings, and the comments brings gain each single task.Our dataset and benchmarks are publicly available to advance the research of story evaluation tasks.