Miracle-dz/UniFAR-FTUs huggingface dataset:
Instance Regularization for Discriminative Language Model Pre-training

Discriminative pre-trained language models (PrLMs) can be generalized as denoising auto-encoders that work with two procedures, ennoising and denoising. First, an ennoising process corrupts texts with arbitrary noising functions to construct training instances. Then, a denoising language model is trained to restore the corrupted tokens. Existing studies have made progress by optimizing independent strategies of either ennoising or denosing. They treat training instances equally throughout the training process, with little attention on the individual contribution of those instances. To model explicit signals of instance contribution, this work proposes to estimate the complexity of restoring the original sentences from corrupted ones in language model pre-training. The estimations involve the corruption degree in the ennoising data construction process and the prediction confidence in the denoising counterpart. Experimental results on natural language understanding and reading comprehension benchmarks show that our approach improves pre-training efficiency, effectiveness, and robustness. Code is publicly available at https://github.com/cooelf/InstanceReg.
DHRL: A Graph-Based Approach for Long-Horizon and Sparse Hierarchical Reinforcement Learning

Hierarchical Reinforcement Learning (HRL) has made notable progress in complex control tasks by leveraging temporal abstraction. However, previous HRL algorithms often suffer from serious data inefficiency as environments get large. The extended components, $i.e.$, goal space and length of episodes, impose a burden on either one or both high-level and low-level policies since both levels share the total horizon of the episode. In this paper, we present a method of Decoupling Horizons Using a Graph in Hierarchical Reinforcement Learning (DHRL) which can alleviate this problem by decoupling the horizons of high-level and low-level policies and bridging the gap between the length of both horizons using a graph. DHRL provides a freely stretchable high-level action interval, which facilitates longer temporal abstraction and faster training in complex tasks. Our method outperforms state-of-the-art HRL algorithms in typical HRL environments. Moreover, DHRL achieves long and complex locomotion and manipulation tasks.
Star Formation History and Transition Epoch of Cluster Galaxies Based on the Horizon-AGN Simulation

Cluster galaxies exhibit substantially lower star formation rates than field galaxies today, but it is conceivable that clusters were sites of more active star formation in the early universe. Herein, we present an interpretation of the star formation history (SFH) of group/cluster galaxies based on the large-scale cosmological hydrodynamic simulation, Horizon-AGN. We find that massive galaxies in general have small values of e-folding timescales of star formation decay (i.e., “mass quenching”) regardless of their environment, while low-mass galaxies exhibit prominent environmental dependence. In massive host halos (i.e., clusters), the e-folding timescales of low-mass galaxies are further decreased if they reside in such halos for a longer period of time. This “environmental quenching” trend is consistent with the theoretical expectation from ram pressure stripping. Furthermore, we define a “transition epoch” as where cluster galaxies become less star-forming than field galaxies. The transition epoch of group/cluster galaxies varies according to their stellar and host-cluster halo masses. Low-mass galaxies in massive clusters show the earliest transition epoch of ∼7.6 Gyr ago in lookback time. However, this decreases to ∼5.2 Gyr for massive galaxies in low-mass clusters. Based on our findings, we can describe a cluster galaxy’s SFH with regard to the cluster halo-to-stellar mass ratio.
Finite-temperature Yang-Mills theory in the Hamiltonian approach in Coulomb gauge from a compactified spatial dimension

Yang-Mills theory is studied at finite temperature within the Hamiltonian approach in Coulomb gauge by means of the variational principle using a Gaussian-type Ansatz for the vacuum wave functional. Temperature is introduced by compactifying one spatial dimension. As a consequence the finite-temperature behavior is encoded in the vacuum wave functional calculated on the spatial manifold ${\mathbb{R}}^{2}\ifmmode\times\else\texttimes\fi{}{\mathrm{S}}^{1}(L)$ where ${L}^{\ensuremath{-}1}$ is the temperature. The finite-temperature equations of motion are obtained by minimizing the vacuum energy density to two-loop order. We show analytically that these equations yield the correct zero-temperature limit while at infinite temperature they reduce to the equations of the $2+1$-dimensional theory in accordance with dimensional reduction. The resulting propagators are compared to those obtained from the grand canonical ensemble where an additional Ansatz for the density matrix is required.
Modelling the Relationship Between Post Encroachment Time and Signal Timings Using UAV Video data

Intersection safety often relies on the correct modelling of signal phasing and timing parameters. A slight increase in yellow time or red time can have significant impact on the rear end crashes or conflicts. This paper aims to identify the relationship between surrogate safety measures and signal phasing. Unmanned Aerial Vehicle (UAV) video data has been used to study an intersection. Post Encroachment Time (PET) between vehicles was calculated from the video data as well as speed, heading and relevant signal timing parameters such as all red time, red clearance time, yellow time, etc. Random Parameter Ordered Logit Model was used to model the relationship between PET and these signal timing parameters. Overall, the results showed that yellow time and red clearance time is positively related to PETs. The model was also able to idendity certain signal phases that could be a potential safety hazard and would need to be retimed by considering the PETs. The odds ratios from the models also indicates that increasing the yellow and red clearance times by one second can improve the PET levels by 16% and 3% respectively.
Far-ultraviolet Dust Extinction and Molecular Hydrogen in the Diffuse Milky Way Interstellar Medium

We aim to compare variations in the full-UV dust extinction curve (912–3000 Å), with the H i/H2/total H content along diffuse Milky Way sightlines, to investigate possible connections between ISM conditions and dust properties. We combine an existing sample of 75 UV extinction curves based on IUE and FUSE data, with atomic and molecular column densities measured through UV absorption. The H2 column density data are based on existing Lyman–Werner absorption band models from earlier work on the extinction curves. Literature values for the H i column density were compiled, and improved for 23 stars by fitting a Lyα profile to archived spectra. We discover a strong correlation between the H2 column and the far-UV extinction, and the underlying cause is a linear relationship between H2 and the strength of the far-UV rise feature. This extinction does not scale with H i, and the total H column scales best with AV instead. The carrier of the far-UV rise therefore coincides with molecular gas, and further connections are shown by comparing the UV extinction features to the molecular fraction. Variations in the gas-to-extinction ratio N(H)/AV correlate with the UV-to-optical extinction ratio, and we speculate this could be due to coagulation or shattering effects. Based on the H2 temperature, the strongest far-UV rise strengths are found to appear in colder and denser sightlines.
Stronger 
<mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" display="inline"><mml:mrow><mml:mi>C</mml:mi></mml:mrow></mml:math>
-odd color charge correlations in the proton at higher energy

The non-forward eikonal scattering matrix for dipole-proton scattering at high energy obtains an imaginary part due to a $C$-odd three gluon exchange. We present numerical estimates for the perturbative Odderon amplitude as a function of dipole size, impact parameter, their relative azimuthal angle, and light-cone momentum cutoff $x$. The proton is approximated as $\psi_\mathrm{qqq}|qqq\rangle + \psi_\mathrm{qqqg}|qqqg\rangle$, where $\psi_\mathrm{qqq}$ is a non-perturbative three quark model wave function while the gluon emission is computed in light-cone perturbation theory. We find that the Odderon amplitude increases as $x$ decreases from 0.1 to 0.01. At yet lower $x$, the reversal of this energy dependence would reflect the onset of universal small-$x$ renormalization group evolution.
Meta-Learning with Self-Improving Momentum Target

The idea of using a separately trained target model (or teacher) to improve the performance of the student model has been increasingly popular in various machine learning domains, and meta-learning is no exception; a recent discovery shows that utilizing task-wise target models can significantly boost the generalization performance. However, obtaining a target model for each task can be highly expensive, especially when the number of tasks for meta-learning is large. To tackle this issue, we propose a simple yet effective method, coined Self-improving Momentum Target (SiMT). SiMT generates the target model by adapting from the temporal ensemble of the meta-learner, i.e., the momentum network. This momentum network and its task-specific adaptations enjoy a favorable generalization performance, enabling self-improving of the meta-learner through knowledge distillation. Moreover, we found that perturbing parameters of the meta-learner, e.g., dropout, further stabilize this self-improving process by preventing fast convergence of the distillation loss during meta-training. Our experimental results demonstrate that SiMT brings a significant performance gain when combined with a wide range of meta-learning methods under various applications, including few-shot regression, few-shot classification, and meta-reinforcement learning. Code is available at https://github.com/jihoontack/SiMT.
Three-dimensional core-collapse supernovae with complex magnetic structures: II. Rotational instabilities and multi-messenger signatures

The gravitational collapse of rapidly rotating massive stars can lead to the onset of the low T/|W| instability within the central proto-neutron star (PNS), which leaves strong signatures in both the gravitational wave (GW) and neutrino emission. Strong large-scale magnetic fields are usually invoked to explain outstanding stellar explosions of rapidly rotating progenitors, but their impact on the growth of such instability has not yet been cleared. We analyze a series of three-dimensional magnetohydrodynamic models to characterize the effects of different magnetic configurations on the development of the low T/|W| and the related multi-messenger features. In the absence of magnetic fields, we observe the growth on dynamical time scales of the low T/|W|, associated with a strong burst of GW and a correlated modulation of the neutrino emission. However, models with a strong magnetic field show a quenching of the low T/|W|, due to a flattening of the rotation profile in the first ∼100 ms after shock formation caused by the magnetic transport of angular momentum. The associated GW emission is weakened by an order of magnitude, exhibits a broader spectral shape, and has no dominant feature associated with the PNS large-scale oscillation modes. Neutrino luminosities are damped along the equatorial plane due to a more oblate PNS, and the only clear modulation in the signal is due to SASI activity. Finally, magnetized models produce lower luminosities for νe than for $\bar{\nu }_e$, which is connected to a higher concentration of neutron-rich material in the PNS surroundings.
MTet: Multi-domain Translation for English and Vietnamese

We introduce MTet, the largest publicly available parallel corpus for English-Vietnamese translation. MTet consists of 4.2M high-quality training sentence pairs and a multi-domain test set reﬁned by the Vietnamese research community. Combining with previous works on English-Vietnamese translation, we grow the existing parallel dataset to 6.2M sentence pairs. We also release the ﬁrst pretrained model EnViT5 for English and Vietnamese languages. Combining both resources, our model signiﬁcantly outperforms previous state-of-the-art results by up to 2 points in translation BLEU score, while being 1.6 times smaller.
The Astrophysical Gravitational Wave Stochastic Background

A stochastic background of gravitational waves with astrophysical origins may have resulted from the superposition of a large number of unresolved sources since the beginning of stellar activity. Its detection would put very strong constraints on the physical properties of compact objects, the initial mass function and star formation history. On the other hand, it could be a 'noise' that would mask the stochastic background of its cosmological origin. We review the main astrophysical processes which are able to produce a stochastic background and discuss how they may differ from the primordial contribution in terms of statistical properties. Current detection methods are also presented.
The radial distribution of galaxies in groups and clusters

We present a new catalogue of 55121 groups and clusters centred on Luminous Red Galaxies from SDSS DR7 in the redshift range 0.15 6 z 6 0.4. We provide halo mass (M500) estimates for each of these groups derived from a calibration between the optical richness of bright galaxies (Mr 6 −20.5) within 1 Mpc, and X-ray-derived mass for a small subset of 129 groups and clusters with X-ray measurements. For 20157 high-mass groups and clusters with M500 > 10 13.7 M⊙, we find that the catalogue has a purity of > 97 per cent and a completeness of ∼ 90 per cent. We derive the mean (stacked) surface number density profiles of galaxies as a function of total halo mass in different mass bins. We find that derived profiles can be well-described by a projected NFW profile with a concentration parameter (hci ≡ hr200/rsi ≈2.6) that is approximately a factor of two lower than that of the dark matter (as predicted by Nbody cosmological simulations) and nearly independent of halo mass. Interestingly, in spite of the difference in shape between the galaxy and dark matter radial distributions, both exhibit a high degree of self-similarity. We also stack the satellite profiles based on other observables, namely redshift, BCG luminosity, and satellite luminosity and colour. We see no evidence for strong variation in profile shape with redshift over the range we probe or with BCG luminosity (or BCG luminosity fraction), but we do find a strong dependence on satellite luminosity and colours, in agreement with previous studies. A self-consistent comparison to several recent semi-analytic models of galaxy formation indicates that: (1) beyond ≈ 0.3r500 current models are able to reproduce both the shape and normalisation of the satellite profiles; and (2) within ≈ 0.3r500 the predicted profiles are sensitive to the details of the satellite-BCG merger timescale calculation. The former is a direct result of the models being tuned to match the global galaxy luminosity function combined with the assumption that the satellite galaxies do not suffer significant tidal stripping, even though their surrounding dark matter haloes can be removed through this process. Combining our results with measurements of the intracluster light should provide a way to inform theoretical models on the efficacy of the tidal stripping and merging processes.
Understanding the Failure of Batch Normalization for Transformers in NLP

Batch Normalization (BN) is a core and prevalent technique in accelerating the training of deep neural networks and improving the generalization on Computer Vision (CV) tasks. However, it fails to defend its position in Natural Language Processing (NLP), which is dominated by Layer Normalization (LN). In this paper, we are trying to answer why BN usually performs worse than LN in NLP tasks with Transformer models. We find that the inconsistency between training and inference of BN is the leading cause that results in the failure of BN in NLP. We define Training Inference Discrepancy (TID) to quantitatively measure this inconsistency and reveal that TID can indicate BN's performance, supported by extensive experiments, including image classification, neural machine translation, language modeling, sequence labeling, and text classification tasks. We find that BN can obtain much better test performance than LN when TID keeps small through training. To suppress the explosion of TID, we propose Regularized BN (RBN) that adds a simple regularization term to narrow the gap between batch statistics and population statistics of BN. RBN improves the performance of BN consistently and outperforms or is on par with LN on 17 out of 20 settings, involving ten datasets and two common variants of Transformer Our code is available at https://github.com/wjxts/RegularizedBN.
ZIN: When and How to Learn Invariance Without Environment Partition?

It is commonplace to encounter heterogeneous data, of which some aspects of the data distribution may vary but the underlying causal mechanisms remain constant. When data are divided into distinct environments according to the heterogeneity, recent invariant learning methods have proposed to learn robust and invariant models based on this environment partition. It is hence tempting to utilize the inherent heterogeneity even when environment partition is not provided. Unfortunately, in this work, we show that learning invariant features under this circumstance is fundamentally impossible without further inductive biases or additional information. Then, we propose a framework to jointly learn environment partition and invariant representation, assisted by additional auxiliary information. We derive sufficient and necessary conditions for our framework to provably identify invariant features under a fairly general setting. Experimental results on both synthetic and real world datasets validate our analysis and demonstrate an improved performance of the proposed framework over existing methods. Finally, our results also raise the need of making the role of inductive biases more explicit in future works, when considering learning invariant models without environment partition. Codes are available at https://github.com/linyongver/ZIN_official .
Point Transformer V2: Grouped Vector Attention and Partition-based Pooling

As a pioneering work exploring transformer architecture for 3D point cloud understanding, Point Transformer achieves impressive results on multiple highly competitive benchmarks. In this work, we analyze the limitations of the Point Transformer and propose our powerful and efficient Point Transformer V2 model with novel designs that overcome the limitations of previous work. In particular, we first propose group vector attention, which is more effective than the previous version of vector attention. Inheriting the advantages of both learnable weight encoding and multi-head attention, we present a highly effective implementation of grouped vector attention with a novel grouped weight encoding layer. We also strengthen the position information for attention by an additional position encoding multiplier. Furthermore, we design novel and lightweight partition-based pooling methods which enable better spatial alignment and more efficient sampling. Extensive experiments show that our model achieves better performance than its predecessor and achieves state-of-the-art on several challenging 3D point cloud understanding benchmarks, including 3D point cloud segmentation on ScanNet v2 and S3DIS and 3D point cloud classification on ModelNet40. Our code will be available at https://github.com/Gofinge/PointTransformerV2.
Label Noise-Robust Learning using a Confidence-Based Sieving Strategy

In learning tasks with label noise, improving model robustness against overfitting is a pivotal challenge because the model eventually memorizes labels, including the noisy ones. Identifying the samples with noisy labels and preventing the model from learning them is a promising approach to address this challenge. When training with noisy labels, the per-class confidence scores of the model, represented by the class probabilities, can be reliable criteria for assessing whether the input label is the true label or the corrupted one. In this work, we exploit this observation and propose a novel discriminator metric called confidence error and a sieving strategy called CONFES to differentiate between the clean and noisy samples effectively. We provide theoretical guarantees on the probability of error for our proposed metric. Then, we experimentally illustrate the superior performance of our proposed approach compared to recent studies on various settings, such as synthetic and real-world label noise. Moreover, we show CONFES can be combined with other state-of-the-art approaches, such as Co-teaching and DivideMix to further improve model performance.
Correlations and linewidth of the atomic beam continuous superradiant laser

We propose a minimalistic model to account for the main properties of a continuous superradiant laser, in which a beam of atoms crosses the mode of a high-finesse Fabry-Perot cavity, and collectively emits light into the cavity mode. We focus on the case of weak single atom - cavity cooperativity, and highlight the relevant regime where decoherence due to the finite transit time dominates over spontaneous emission. We propose an original approach where the dynamics of atoms entering and leaving the cavity is described by a Hamiltonian process. This allows deriving the main dynamical equations for the superradiant laser, without the need for a stochastic approach. We derive analytical conditions for a sustained emission and show that the ultimate linewidth is set by the fundamental quantum fluctuations of the collective atomic dipole. We calculate steady-state values of the two-body correlators and show that the continuous superradiant regime is tied to the growth of atom-atom correlations, although these correlations only have a small impact on the laser linewidth.
MASSIV: Mass Assembly Survey with SINFONI in VVDS , V. The major merger rate of star-forming galaxies at 0.9 < z < 1.8 from IFS-based close pairs

Context. The contribution of the merging process to the early phase of galaxy assembly at z > 1 and, in particular, to the build-up of the red sequence, still needs to be accurately assessed. Aims. We aim to measure the major merger rate of star-forming galaxies at 0.9 < z < 1.8, using close pairs identified from integral field spectroscopy (IFS). Methods. We use the velocity field maps obtained with SINFONI/VLT on the MASSIV sample, selected from the star-forming population in the VVDS. We identify physical pairs of galaxies from the measurement of the relative velocity and the projected separation (rp) of the galaxies in the pair. Using the well constrained selection function of the MASSIV sample, we derive at a mean redshift up to z = 1.54 the gas-rich major merger fraction (luminosity ratio μ = L2/L1 ≥ 1/4), and the gas-rich major merger rate using merger time scales from cosmological simulations.