Miracle-dz/UniFAR-FTUs huggingface dataset:
Nonstandard Modeling of a Possible Blue Straggler Star, KIC 11145123

Nonstandard modeling of KIC 11145123, a possible blue straggler star, has been asteroseismically carried out based on a scheme to compute stellar models with the chemical compositions in their envelopes arbitrarily modified, mimicking the effects of some interactions with other stars through which blue straggler stars are thought to be born. We have constructed a nonstandard model of the star with the following parameters: M = 1.36 M ⊙, Y init = 0.26, Z init = 0.002, and f ovs = 0.027, where f ovs is the extent of overshooting described as an exponentially decaying diffusive process. The modification is down to the depth of r/R ∼ 0.6 and the extent ΔX, which is a difference in surface hydrogen abundance between the envelope-modified and unmodified models, is 0.06. The residuals between the model and the observed frequencies are comparable with those for the previous model computed assuming standard single-star evolution, suggesting that it is possible that the star was born with a relatively ordinary initial helium abundance of ∼0.26 compared with that of the previous models (∼0.30–0.40), then experienced some modification of the chemical compositions and gained helium in the envelope. Detailed analyses of the nonstandard model have implied that the elemental diffusion in the deep radiative region of the star might be much weaker than that assumed in current stellar evolutionary calculations; we need some extra mechanisms inside the star, rendering the star a much more intriguing target to be further investigated.
Constraints on Barrow Entropy from M87 and S2 Star Observations*

We use data from M87* central black hole shadow, as well as from the S2 star observations, in order to extract constraints on Barrow entropy. The latter is a modified entropy arising from quantum-gravitational effects on the black hole horizon, quantified by the new parameter Δ. Such a change in entropy leads to a change in temperature, as well as to the properties of the black hole and its shadow. We investigate the photon sphere and the shadow of a black hole with Barrow entropy, and assuming a simple model for infalling and radiating gas we estimate the corresponding intensity. Furthermore, we use the radius in order to extract the real part of the quasinormal modes, and for completeness we investigate the spherical accretion of matter onto the black hole, focusing on isothermal and polytropic test fluids. We extract the allowed parameter region, and by applying a Monte-Carlo-Markov Chains analysis we find that Δ≃0.0036−0.0145+0.0792. Hence, our results place the upper bound Δ≲0.0828 at 1σ, a constraint that is less strong than the Big Bang Nucleosynthesis one, but significantly stronger than the late-time cosmological constraints.
bert2BERT: Towards Reusable Pretrained Language Models

In recent years, researchers tend to pre-train ever-larger language models to explore the upper limit of deep models. However, large language model pre-training costs intensive computational resources, and most of the models are trained from scratch without reusing the existing pre-trained models, which is wasteful. In this paper, we propose bert2BERT, which can effectively transfer the knowledge of an existing smaller pre-trained model to a large model through parameter initialization and significantly improve the pre-training efficiency of the large model. Specifically, we extend the previous function-preserving method proposed in computer vision on the Transformer-based language model, and further improve it by proposing a novel method, advanced knowledge for large model’s initialization. In addition, a two-stage learning method is proposed to further accelerate the pre-training. We conduct extensive experiments on representative PLMs (e.g., BERT and GPT) and demonstrate that (1) our method can save a significant amount of training cost compared with baselines including learning from scratch, StackBERT and MSLT; (2) our method is generic and applicable to different types of pre-trained models. In particular, bert2BERT saves about 45% and 47% computational cost of pre-training BERT_{\rm BASE} and GPT_{\rm BASE} by reusing the models of almost their half sizes.
Distributed and Localized Model-Predictive Control–Part I: Synthesis and Implementation

The increasing presence of large-scale distributed systems highlights the need for scalable control strategies where only local communication is required. Moreover, in safety-critical systems, it is imperative that such control strategies handle constraints in the presence of disturbances. In response to this need, in this article, we present the distributed and localized model-predictive control (DLMPC) algorithm for large-scale linear systems. DLMPC is a distributed closed-loop model-predictive control (MPC) scheme, wherein only local state and model information needs to be exchanged between subsystems for the computation and implementation of control actions. We use the system-level synthesis framework to reformulate the centralized MPC problem and show that this allows us to naturally impose localized communication constraints between subcontrollers. The structure of the resulting problem can be exploited to develop an alternating-direction-method-of-multipliers-based algorithm that allows for a distributed and localized computation of closed-loop control policies. We demonstrate that the computational complexity of the subproblems solved by each subsystem in DLMPC is independent of the size of the global system. DLMPC is the first MPC algorithm that allows for the scalable computation and implementation of distributed closed-loop control policies and deals with additive disturbances. In our companion paper, we show that this approach enjoys recursive feasibility and asymptotic stability.
SaFeRDialogues: Taking Feedback Gracefully after Conversational Safety Failures

Current open-domain conversational models can easily be made to talk in inadequate ways. Online learning from conversational feedback given by the conversation partner is a promising avenue for a model to improve and adapt, so as to generate fewer of these safety failures. However, current state-of-the-art models tend to react to feedback with defensive or oblivious responses. This makes for an unpleasant experience and may discourage conversation partners from giving feedback in the future. This work proposes SaFeRDialogues, a task and dataset of graceful responses to conversational feedback about safety failures.We collect a dataset of 8k dialogues demonstrating safety failures, feedback signaling them, and a response acknowledging the feedback. We show how fine-tuning on this dataset results in conversations that human raters deem considerably more likely to lead to a civil conversation, without sacrificing engagingness or general conversational ability.
Study of Positional Encoding Approaches for Audio Spectrogram Transformers

Transformers have revolutionized the world of deep learning, specially in the field of natural language processing. Recently, the Audio Spectrogram Transformer (AST) was proposed for audio classification, leading to state of the art results in several datasets. However, in order for ASTs to outperform CNNs, pretraining with ImageNet is needed. In this paper, we study one component of the AST, the positional encoding, and propose several variants to improve the performance of ASTs trained from scratch, without ImageNet pretraining. Our best model, which incorporates conditional positional encodings, significantly improves performance on Audioset and ESC-50 compared to the original AST.
Bandits Don't Follow Rules: Balancing Multi-Facet Machine Translation with Multi-Armed Bandits

Training data for machine translation (MT) is often sourced from a multitude of large corpora that are multi-faceted in nature, e.g. containing contents from multiple domains or different levels of quality or complexity. Naturally, these facets do not occur with equal frequency, nor are they equally important for the test scenario at hand. In this work, we propose to optimize this balance jointly with MT model parameters to relieve system developers from manual schedule design. A multi-armed bandit is trained to dynamically choose between facets in a way that is most beneficial for the MT system. We evaluate it on three different multi-facet applications: balancing translationese and natural training data, or data from multiple domains or multiple language pairs. We find that bandit learning leads to competitive MT systems across tasks, and our analysis provides insights into its learned strategies and the underlying data sets.
Context-gloss Augmentation for Improving Word Sense Disambiguation

The goal of Word Sense Disambiguation (WSD) is to identify the sense of a polysemous word in a specific context. Deep-learning techniques using BERT have achieved very promising results in the field and different methods have been proposed to integrate structured knowledge to enhance performance. At the same time, an increasing number of data augmentation techniques have been proven to be useful for NLP tasks. Building upon previous works leveraging BERT and WordNet knowledge, we explore different data augmentation techniques on context-gloss pairs to improve the performance of WSD. In our experiment, we show that both sentence-level and word-level augmentation methods are effective strategies for WSD. Also, we find out that performance can be improved by adding hypernyms' glosses obtained from a lexical knowledge base. We compare and analyze different context-gloss augmentation techniques, and the results show that applying back translation on gloss performs the best.
Same-diff ? Part I : Conceptual similarities ( and one difference ) between gauge transformations and diffeomorphisms

The following questions are germane to our understanding of gauge-(in)variant quantities and physical possibility: in which ways are gauge transformations and spacetime diffeomorphisms similar, and in which are they different? To what extent are we justified in endorsing different attitudes—sophistication, quidditism/haecceitism, or full elimination—towards each? In a companion paper, I assess new and old contrasts between the two types of symmetries. In this one, I propose a new contrast: whether the symmetry changes pointwise the dynamical properties of a given field. This contrast distinguishes states that are related by a gauge-symmetry from states related by generic spacetime diffeomorphisms, as being ‘pointwise dynamically indiscernible’. Only the rigid isometries of homogeneous spacetimes fall in the same category, but they are neither local nor modally robust, in the way that gauge transformations are. In spite of this difference, I argue that for both gauge transformations and spacetime diffeomorphisms, symmetry-related models are best understood through the doctrine of ‘sophistication’.
Mapping Obscuration to Reionization with ALMA (MORA): 2 mm Efficiently Selects the Highest-redshift Obscured Galaxies

We present the characteristics of 2 mm selected sources from the largest Atacama Large Millimeter/submillimeter Array (ALMA) blank-field contiguous survey conducted to date, the Mapping Obscuration to Reionization with ALMA (MORA) survey covering 184 arcmin2 at 2 mm. Twelve of 13 detections above 5σ are attributed to emission from galaxies, 11 of which are dominated by cold dust emission. These sources have a median redshift of 〈z2mm〉=3.6−0.3+0.4 primarily based on optical/near-infrared photometric redshifts with some spectroscopic redshifts, with 77% ± 11% of sources at z > 3 and 38% ± 12% of sources at z > 4. This implies that 2 mm selection is an efficient method for identifying the highest-redshift dusty star-forming galaxies (DSFGs). Lower-redshift DSFGs (z < 3) are far more numerous than those at z > 3 yet are likely to drop out at 2 mm. MORA shows that DSFGs with star formation rates in excess of 300 M ⊙ yr−1 and a relative rarity of ∼10−5 Mpc−3 contribute ∼30% to the integrated star formation rate density at 3 < z < 6. The volume density of 2 mm selected DSFGs is consistent with predictions from some cosmological simulations and is similar to the volume density of their hypothesized descendants: massive, quiescent galaxies at z > 2. Analysis of MORA sources’ spectral energy distributions hint at steeper empirically measured dust emissivity indices than reported in typical literature studies, with 〈β〉=2.2−0.4+0.5 . The MORA survey represents an important step in taking census of obscured star formation in the universe’s first few billion years, but larger area 2 mm surveys are needed to more fully characterize this rare population and push to the detection of the universe’s first dusty galaxies.
SpongeCake: A Layered Microflake Surface Appearance Model

In this article, we propose SpongeCake: A layered BSDF model where each layer is a volumetric scattering medium, defined using microflake or other phase functions. We omit any reflecting and refracting interfaces between the layers. The first advantage of this formulation is that an exact and analytic solution for single scattering, regardless of the number of volumetric layers, can be derived. We propose to approximate multiple scattering by an additional single-scattering lobe with modified parameters and a Lambertian lobe. We use a parameter mapping neural network to find the parameters of the newly added lobes to closely approximate the multiple scattering effect. Despite the absence of layer interfaces, we demonstrate that many common material effects can be achieved with layers of SGGX microflake and other volumes with appropriate parameters. A normal mapping effect can also be achieved through mapping of microflake orientations, which avoids artifacts common in standard normal maps. Thanks to the analytical formulation, our model is very fast to evaluate and sample. Through various parameter settings, our model is able to handle many types of materials, like plastics, wood, cloth, and so on, opening a number of practical applications.
Retrieval-guided Counterfactual Generation for QA

Deep NLP models have been shown to be brittle to input perturbations. Recent work has shown that data augmentation using counterfactuals — i.e. minimally perturbed inputs — can help ameliorate this weakness. We focus on the task of creating counterfactuals for question answering, which presents unique challenges related to world knowledge, semantic diversity, and answerability. To address these challenges, we develop a Retrieve-Generate-Filter(RGF) technique to create counterfactual evaluation and training data with minimal human supervision. Using an open-domain QA framework and question generation model trained on original task data, we create counterfactuals that are fluent, semantically diverse, and automatically labeled. Data augmentation with RGF counterfactuals improves performance on out-of-domain and challenging evaluation sets over and above existing methods, in both the reading comprehension and open-domain QA settings. Moreover, we find that RGF data leads to significant improvements in a model’s robustness to local perturbations.
Quenches to the critical point of the three-state Potts model: Matrix product state simulations and conformal field theory

Conformal Field Theories (CFTs) have been used extensively to understand the physics of critical lattice models at equilibrium. However, the applicability of CFT calculations to the behavior of the lattice systems in the out-of-equilibrium setting is not entirely understood. In this work, we compare the CFT results of the evolution of the entanglement spectrum after a quantum quench with numerical calculations of the entanglement spectrum of the three-state Potts model using matrix product state simulations. Our results lead us to conjecture that CFT does not describe the entanglement spectrum of the three-state Potts model at long times, contrary to what happens in the Ising model. We thus numerically simulate the out-of-equilibrium behaviour of the Potts model according to the CFT protocol - i.e. by taking a particular product state and"cooling"it, then quenching to the critical point and find that, in this case, the entanglement spectrum is indeed described by the CFT at long times.
Playing for 3D Human Recovery

—Image- and video-based 3D human recovery ( i.e ., pose and shape estimation) have achieved substantial progress. However, due to the prohibitive cost of motion capture, existing datasets are often limited in scale and diversity. In this work, we obtain massive human sequences by playing the video game with automatically annotated 3D ground truths. Speciﬁcally, we contribute GTA-Human , a large-scale 3D human dataset generated with the GTA-V game engine, featuring a highly diverse set of subjects, actions, and scenarios. More importantly, we study the use of game-playing data and obtain ﬁve major insights. First , game-playing data is surprisingly effective. A simple frame-based baseline trained on GTA-Human outperforms more sophisticated methods by a large margin. For video-based methods, GTA-Human is even on par with the in-domain training set. Second , we discover that synthetic data provides critical complements to the real data that is typically collected indoor. Our investigation into domain gap provides explanations for our data mixture strategies that are simple yet useful. Third , the scale of the dataset matters. The performance boost is closely related to the additional data available. A systematic study reveals the model sensitivity to data density from multiple key aspects. Fourth , the effectiveness of GTA-Human is also attributed to the rich collection of strong supervision labels (SMPL parameters), which are otherwise expensive to acquire in real datasets. Fifth , the beneﬁts of synthetic data extend to larger models such as deeper convolutional neural networks (CNNs) and Transformers, for which a signiﬁcant impact is also observed. We hope our work could pave the way for scaling up 3D human recovery to the real world. Homepage: https://caizhongang.github. io/projects/GTA-Human/.
Spectroscopically Identified Emission Line Galaxy Pairs in the WISP Survey

We identify a sample of spectroscopically measured emission line galaxy (ELG) Pairs up to z = 1.6 from the Wide Field Camera 3 (WFC3) Infrared Spectroscopic Parallels (WISP) survey. WISP obtained slitless, near-infrared grism spectroscopy along with direct imaging in the J and H bands by observing in the pure-parallel mode with the WFC3 on board the Hubble Space Telescope. From our search of 419 WISP fields covering an area of ∼0.5 deg2, we find 413 ELG pair systems, mostly H α emitters. We then derive reliable star formation rates (SFRs) based on the attenuation-corrected H α fluxes. Compared to isolated galaxies, we find an average SFR enhancement of 40%–65%, which is stronger for major Pairs and Pairs with smaller velocity separations (Δ v < 300 km s−1). Based on the stacked spectra from various subsamples, we study the trends of emission line ratios in pairs, and find a general consistency with enhanced lower ionization lines. We study the pair fraction among ELGs, and find a marginally significant increase with redshift f ∝ (1 + z) α , where the power-law index α = 0.58 ± 0.17 from z ∼ 0.2 to ∼1.6. The fraction of active galactic nuclei is found to be the same in the ELG Pairs as compared to the isolated ELGs.
Non-Autoregressive Translation with Layer-Wise Prediction and Deep Supervision

How do we perform efficient inference while retaining high translation quality? Existing neural machine translation models, such as Transformer, achieve high performance, but they decode words one by one, which is inefficient. Recent non-autoregressive translation models speed up the inference, but their quality is still inferior. In this work, we propose DSLP, a highly efficient and high-performance model for machine translation. The key insight is to train a non-autoregressive Transformer with Deep Supervision and feed additional Layer-wise Predictions. We conducted extensive experiments on four translation tasks (both directions of WMT'14 EN-DE and WMT'16 EN-RO). Results show that our approach consistently improves the BLEU scores compared with respective base models. Specifically, our best variant outperforms the autoregressive model on three translation tasks, while being 14.8 times more efficient in inference.
S 5: The Orbital and Chemical Properties of One Dozen Stellar Streams

We report the kinematic, orbital, and chemical properties of 12 stellar streams with no evident progenitors using line-of-sight velocities and metallicities from the Southern Stellar Stream Spectroscopic Survey (S 5), proper motions from Gaia EDR3, and distances derived from distance tracers or the literature. This data set provides the largest homogeneously analyzed set of streams with full 6D kinematics and metallicities. All streams have heliocentric distances between ∼10 and 50 kpc. The velocity and metallicity dispersions show that half of the stream progenitors were disrupted dwarf galaxies (DGs), while the other half originated from disrupted globular clusters (GCs), hereafter referred to as DG and GC streams. Based on the mean metallicities of the streams and the mass–metallicity relation, the luminosities of the progenitors of the DG streams range between those of Carina and Ursa Major I (−9.5 ≲ M V ≲ −5.5). Four of the six GC streams have mean metallicities of [Fe/H] < −2, more metal poor than typical Milky Way (MW) GCs at similar distances. Interestingly, the 300S and Jet GC streams are the only streams on retrograde orbits in our dozen-stream sample. Finally, we compare the orbital properties of the streams with known DGs and GCs in the MW, finding several possible associations. Some streams appear to have been accreted with the recently discovered Gaia–Enceladus–Sausage system, and others suggest that GCs were formed in and accreted together with the progenitors of DG streams whose stellar masses are similar to those of Draco to Carina (∼105–106 M ⊙).
From sum of two squares to arithmetic Siegel–Weil formulas

The main goal of this expository article is to survey recent progress on the arithmetic Siegel–Weil formula and its applications. We begin with the classical sum of two squares problem and put it in the context of the Siegel–Weil formula. We then motivate the geometric and arithmetic Siegel–Weil formula using the classical example of the product of modular curves. After explaining the recent result on the arithmetic Siegel–Weil formula for Shimura varieties of arbitrary dimension, we discuss some aspects of the proof and its application to the arithmetic inner product formula and the Beilinson–Bloch conjecture. Rather than being intended as a complete survey of this vast field, this article focuses more on examples and background to provide easier access to several recent works by the author with W. Zhang and Y. Liu.
P-Tuning v2: Prompt Tuning Can Be Comparable to Fine-tuning Universally Across Scales and Tasks

Prompt tuning, which only tunes continuous prompts with a frozen language model, substantially reduces per-task storage and memory usage at training. However, in the context of NLU, prior work reveals that prompt tuning does not perform well for normal-sized pretrained models. We also find that existing methods of prompt tuning cannot handle hard sequence labeling tasks, indicating a lack of universality. We present a novel empirical finding that properly optimized prompt tuning can be universally effective across a wide range of model scales and NLU tasks. It matches the performance of finetuning while having only 0.1%-3% tuned parameters. Our method P-Tuning v2 is an implementation of Deep Prompt Tuning \cite{li2021prefix,qin2021learning} optimized and adapted for NLU. Given the universality and simplicity of P-Tuning v2, we believe it can serve as an alternative to finetuning and a strong baseline for future research.Our code and data are released at https://github.com/THUDM/P-tuning-v2.