Miracle-dz/UniFAR-FTUs huggingface dataset:
Chemical evolution during the formation of a protoplanetary disk

Context.The chemical composition of protoplanetary disks is expected to impact the composition of the forming planets. Characterizing the diversity of chemical composition in disks and the physicochemical factors that lead to this diversity is consequently of high interest.Aims. The aim of this study is to investigate the chemical evolution from the prestellar phase to the formation of the disk, and to determine the impact that the chemical composition of the cold and dense core has on the final composition of the disk.Methods. We performed 3D nonideal magneto-hydrodynamic (MHD) simulations of a dense core collapse using the adaptive-mesh-refinement RAMSES code. For each particle ending in the young rotationally supported disk, we ran chemical simulations with the three-phase gas-grain chemistry code Nautilus. Two different sets of initial abundances, which are characteristic of cold cores, were considered. The final distributions of the abundances of common species were compared to each other, as well as with the initial abundances of the cold core.Results. We find that the spatial distributions of molecules reflect their sensitivity to the temperature distribution. The main carriers of the chemical elements in the disk are usually the same as the ones in the cold core, except for the S-bearing species, where HS is replaced by H2S3, and the P-bearing species, where atomic P leads to the formation of PO, PN, HCP, and CP. However, the abundances of less abundant species change over time. This is especially the case for “large” complex organic molecules (COMs) such as CH3CHO, CH3NH2, CH3OCH3, and HCOOCH3which see their abundances significantly increase during the collapse. These COMs often present similar abundances in the disk despite significantly different abundances in the cold core. In contrast, the abundances of many radicals decrease with time. A significant number of species still show the same abundances in the cold core and the disk, which indicates efficient formation of these molecules in the cold core. This includes H2O, H2CO, HNCO, and “small” COMs such as CH3OH, CH3CN, and NH2CHO. We computed the MHD resistivities within the disk for the full gas–grain chemical evolution and find results in qualitative agreement with the literature assuming simpler chemical networks.Conclusions. In conclusion, the chemical content of prestellar cores is expected to affect the chemical composition of disks. The impact is more or less important depending on the type of species. Users of stand-alone chemical models of disks should pay special attention to the initial abundances they choose.
Probing Pretrained Language Models for Lexical Semantics

The success of large pretrained language models (LMs) such as BERT and RoBERTa has sparked interest in probing their representations, in order to unveil what types of knowledge they implicitly capture. While prior research focused on morphosyntactic, semantic, and world knowledge, it remains unclear to which extent LMs also derive lexical type-level knowledge from words in context. In this work, we present a systematic empirical analysis across six typologically diverse languages and five different lexical tasks, addressing the following questions: 1) How do different lexical knowledge extraction strategies (monolingual versus multilingual source LM, out-of-context versus in-context encoding, inclusion of special tokens, and layer-wise averaging) impact performance? How consistent are the observed effects across tasks and languages? 2) Is lexical knowledge stored in few parameters, or is it scattered throughout the network? 3) How do these representations fare against traditional static word vectors in lexical tasks? 4) Does the lexical information emerging from independently trained monolingual LMs display latent similarities? Our main results indicate patterns and best practices that hold universally, but also point to prominent variations across languages and tasks. Moreover, we validate the claim that lower Transformer layers carry more type-level lexical knowledge, but also show that this knowledge is distributed across multiple layers.
Fast Convergence of Langevin Dynamics on Manifold: Geodesics meet Log-Sobolev

Sampling is a fundamental and arguably very important task with numerous applications in Machine Learning. One approach to sample from a high dimensional distribution $e^{-f}$ for some function $f$ is the Langevin Algorithm (LA). Recently, there has been a lot of progress in showing fast convergence of LA even in cases where $f$ is non-convex, notably [53], [39] in which the former paper focuses on functions $f$ defined in $\mathbb{R}^n$ and the latter paper focuses on functions with symmetries (like matrix completion type objectives) with manifold structure. Our work generalizes the results of [53] where $f$ is defined on a manifold $M$ rather than $\mathbb{R}^n$. From technical point of view, we show that KL decreases in a geometric rate whenever the distribution $e^{-f}$ satisfies a log-Sobolev inequality on $M$.
Gradient-based Analysis of NLP Models is Manipulable

Gradient-based analysis methods, such as saliency map visualizations and adversarial input perturbations, have found widespread use in interpreting neural NLP models due to their simplicity, flexibility, and most importantly, the fact that they directly reflect the model internals. In this paper, however, we demonstrate that the gradients of a model are easily manipulable, and thus bring into question the reliability of gradient-based analyses. In particular, we merge the layers of a target model with a Facade Model that overwhelms the gradients without affecting the predictions. This Facade Model can be trained to have gradients that are misleading and irrelevant to the task, such as focusing only on the stop words in the input. On a variety of NLP tasks (sentiment analysis, NLI, and QA), we show that the merged model effectively fools different analysis tools: saliency maps differ significantly from the original model’s, input reduction keeps more irrelevant input tokens, and adversarial perturbations identify unimportant tokens as being highly important.
Local Region Knowledge Distillation

Knowledge distillation (KD) is an effective technique to transfer knowledge from one neural network (teacher) to another (student), thus improving the performance of the student. The existing work trains the student to mimic the outputs of the teacher on training data. We argue that transferring knowledge at sparse training data points cannot enable the student to well capture the local shape of the teacher function. To address this issue, we propose locally linear region knowledge distillation ( L 2 RKD) which transfers the knowledge in local, liner regions from a teacher to a student. L 2 RKD enforces the student to mimic the local shape of the teacher function in linear regions. Extensive experiments with various network architectures demonstrate that L 2 RKD outperforms the state-of-the-art approaches by a large margin and is more data-efﬁcient. Moreover, L 2 RKD is compatible with the existing distillation methods and further improves their performances signiﬁcantly.
Evaluating and Characterizing Human Rationales

Two main approaches for evaluating the quality of machine-generated rationales are: 1) using human rationales as a gold standard; and 2) automated metrics based on how rationales affect model behavior. An open question, however, is how human rationales fare with these automatic metrics. Analyzing a variety of datasets and models, we find that human rationales do not necessarily perform well on these metrics. To unpack this finding, we propose improved metrics to account for model-dependent baseline performance. We then propose two methods to further characterize rationale quality, one based on model retraining and one on using "fidelity curves" to reveal properties such as irrelevance and redundancy. Our work leads to actionable suggestions for evaluating and characterizing rationales.
Causal learning with sufficient statistics: an information bottleneck approach

The inference of causal relationships using observational data from partially observed multivariate systems with hidden variables is a fundamental question in many scientific domains. Methods extracting causal information from conditional independencies between variables of a system are common tools for this purpose, but are limited in the lack of independencies. To surmount this limitation, we capitalize on the fact that the laws governing the generative mechanisms of a system often result in substructures embodied in the generative functional equation of a variable, which act as sufficient statistics for the influence that other variables have on it. These functional sufficient statistics constitute intermediate hidden variables providing new conditional independencies to be tested. We propose to use the Information Bottleneck method, a technique commonly applied for dimensionality reduction, to find underlying sufficient sets of statistics. Using these statistics we formulate new additional rules of causal orientation that provide causal information not obtainable from standard structure learning algorithms, which exploit only conditional independencies between observable variables. We validate the use of sufficient statistics for structure learning both with simulated systems built to contain specific sufficient statistics and with benchmark data from regulatory rules previously and independently proposed to model biological signal transduction networks.
Robust Optimal Transport with Applications in Generative Modeling and Domain Adaptation

Optimal Transport (OT) distances such as Wasserstein have been used in several areas such as GANs and domain adaptation. OT, however, is very sensitive to outliers (samples with large noise) in the data since in its objective function, every sample, including outliers, is weighed similarly due to the marginal constraints. To remedy this issue, robust formulations of OT with unbalanced marginal constraints have previously been proposed. However, employing these methods in deep learning problems such as GANs and domain adaptation is challenging due to the instability of their dual optimization solvers. In this paper, we resolve these issues by deriving a computationally-efficient dual form of the robust OT optimization that is amenable to modern deep learning applications. We demonstrate the effectiveness of our formulation in two applications of GANs and domain adaptation. Our approach can train state-of-the-art GAN models on noisy datasets corrupted with outlier distributions. In particular, our optimization computes weights for training samples reflecting how difficult it is for those samples to be generated in the model. In domain adaptation, our robust OT formulation leads to improved accuracy compared to the standard adversarial adaptation methods. Our code is available at this https URL.
Galaxy evolution across environments as probed by the ages, stellar metallicities, and [α /Fe]  of central and satellite galaxies

We explore how the star formation and metal enrichment histories of present-day galaxies have been affected by environment combining stellar population parameter estimates and group environment characterization for SDSS DR7. We compare stellar ages, stellar metallicities and element abundance ratios [alpha/Fe] of satellite and central galaxies, as a function of their stellar and host group halo mass, controlling for the current star formation rate and for the infall epoch. We confirm that below log(Mstar/Msun)=10.5 satellites are older and metal-richer than equally-massive central galaxies. On the contrary, we do not detect any difference in their [alpha/Fe]: this depends primarily on stellar mass and not on group hierarchy nor host halo mass. We also find that the differences in the median age and metallicity of satellites and centrals at stellar mass below 10^{10.5}Msun are largely due to the higher fraction of passive galaxies among satellites and as a function of halo mass. We argue that the observed trends at low masses reveal the action of satellite-specific environmental effects in a `delayed-then-rapid' fashion. When accounting for the varying quiescent fraction, small residual excess in age, metallicity and [alpha/Fe] emerge for satellites dominated by old stellar populations and residing in halos more massive than 10^{14}Msun, compared to equally-massive central galaxies. This excess in age, metallicity and [alpha/Fe] pertain to ancient infallers, i.e. satellites that have accreted onto the current halo more than 5 Gyr ago. This result points to the action of environment in the early phases of star formation in galaxies located close to cosmic density peaks.
Impact Evaluation of Falsified Data Attacks on Connected Vehicle Based Traffic Signal Control

Connected vehicle (CV) technology enables data exchange between vehicles and transportation infrastructure and therefore has great potentials to improve current traffic signal control systems. However, this connectivity might also bring cyber security concerns. As the first step in investigating the cyber security of CV-based traffic signal control (CV-TSC) systems, potential cyber threats need to be identified and corresponding impact needs to be evaluated. In this paper, we aim to evaluate the impact of cyber attacks on CV-TSC systems by considering a realistic attack scenario in which the control logic of a CV-TSC system is unavailable to attackers. Our threat model presumes that an attacker may learn the control logic using a surrogate model. Based on the surrogate model, the attacker may launch falsified data attacks to influence signal control decisions. In the case study, we realistically evaluate the impact of falsified data attacks on an existing CV-TSC system (i.e., I-SIG).
Automated Concatenation of Embeddings for Structured Prediction

Pretrained contextualized embeddings are powerful word representations for structured prediction tasks. Recent work found that better word representations can be obtained by concatenating different types of embeddings. However, the selection of embeddings to form the best concatenated representation usually varies depending on the task and the collection of candidate embeddings, and the ever-increasing number of embedding types makes it a more difficult problem. In this paper, we propose Automated Concatenation of Embeddings (ACE) to automate the process of finding better concatenations of embeddings for structured prediction tasks, based on a formulation inspired by recent progress on neural architecture search. Specifically, a controller alternately samples a concatenation of embeddings, according to its current belief of the effectiveness of individual embedding types in consideration for a task, and updates the belief based on a reward. We follow strategies in reinforcement learning to optimize the parameters of the controller and compute the reward based on the accuracy of a task model, which is fed with the sampled concatenation as input and trained on a task dataset. Empirical results on 6 tasks and 21 datasets show that our approach outperforms strong baselines and achieves state-of-the-art performance with fine-tuned embeddings in all the evaluations.
The emergence of explainability of intelligent systems: Delivering explainable and personalized recommendations for energy efficiency

The recent advances in artificial intelligence namely in machine learning and deep learning, have boosted the performance of intelligent systems in several ways. This gave rise to human expectations, but also created the need for a deeper understanding of how intelligent systems think and decide. The concept of explainability appeared, in the extent of explaining the internal system mechanics in human terms. Recommendation systems are intelligent systems that support human decision making, and as such, they have to be explainable to increase user trust and improve the acceptance of recommendations. In this study, we focus on a context‐aware recommendation system for energy efficiency and develop a mechanism for explainable and persuasive recommendations, which are personalized to user preferences and habits. The persuasive facts either emphasize on the economical saving prospects (Econ) or on a positive ecological impact (Eco) and explanations provide the reason for recommending an energy saving action. Based on a study conducted using a Telegram bot, different scenarios have been validated with actual data and human feedback. Current results show a total increase of 19% on the recommendation acceptance ratio when both economical and ecological persuasive facts are employed. This revolutionary approach on recommendation systems, demonstrates how intelligent recommendations can effectively encourage energy saving behavior.
On the criteria of large cavitation bubbles in a tube during a transient process

Abstract Extreme cavitation scenarios, such as water column separations in hydraulic systems during transient processes caused by large cavitation bubbles, can lead to catastrophic destruction. In the present paper, we study the onset criteria and dynamics of large cavitation bubbles in a tube. A new cavitation number $Ca_2 = {l^*}^{-1} Ca_0$ is proposed to describe the maximum length $L_{max}$ of the cavitation bubble, where $l^*$ is a non-dimensional length of the water column indicating its slenderness, and $Ca_0$ is the classic cavitation number. Combined with the onset criteria for acceleration-induced cavitation ($Ca_1<1$, Pan et al., Proc. Natl Acad. Sci. USA, vol. 114, 2017, pp. 8470–8474), we show that the occurrence of large cylindrical cavitation bubbles requires both $Ca_2<1$ and $Ca_1<1$ simultaneously. We also establish a Rayleigh-type model for the dynamics of large cavitation bubbles in a tube. The bubbles collapse at a finite end speed, and the time from the maximum bubble size to collapse is $T_c=\sqrt {2}\sqrt {lL_{max}}\sqrt {{\rho }/{p_\infty }}$, where $l$ is the length of the water column, $L_{max}$ is the maximum bubble length, $\rho$ is the liquid density and $p_{\infty }$ is the reference pressure in the far field. The analytical results are validated against systematic experiments using a modified ‘tube-arrest’ apparatus, which can decouple acceleration and velocity. The results in the current work can guide design and operation of hydraulic systems encountering transient processes.
OBSERVABILITY OF BAOUENDI–GRUSHIN-TYPE EQUATIONS THROUGH RESOLVENT ESTIMATES

Abstract In this article, we study the observability (or equivalently, the controllability) of some subelliptic evolution equations depending on their step. This sheds light on the speed of propagation of these equations, notably in the ‘degenerated directions’ of the subelliptic structure. First, for any $\gamma \geq 1$ , we establish a resolvent estimate for the Baouendi–Grushin-type operator $\Delta _{\gamma }=\partial _x^2+\left \lvert x\right \rvert ^{2\gamma }\partial _y^2$ , which has step $\gamma +1$ . We then derive consequences for the observability of the Schrödinger-type equation $i\partial _tu-\left (-\Delta _{\gamma }\right )^{s}u=0$ , where $s\in \mathbb N$ . We identify three different cases: depending on the value of the ratio $(\gamma +1)/s$ , observability may hold in arbitrarily small time or only for sufficiently large times or may even fail for any time. As a corollary of our resolvent estimate, we also obtain observability for heat-type equations $\partial _tu+\left (-\Delta _{\gamma }\right )^su=0$ and establish a decay rate for the damped wave equation associated with $\Delta _{\gamma }$ .
Analysis of XMM-Newton Observations of Supernova Remnant W49B and Clues to the Progenitor

W49B is a supernova remnant (SNR) discovered over 60 yr ago in early radio surveys. It has since been observed over the entire wavelength range, with the X-ray morphology resembling a centrally filled SNR. The nature of its progenitor star is still debated. Applying Smoothed Particle Inference techniques to analyze the X-Ray emission from W49B, we characterize the morphology and abundance distribution over the entire remnant. We also infer the density structure and derive the mass of individual elements present in the plasma. The morphology is consistent with an interaction between the remnant and a dense medium along the eastern edge, and some obstruction toward the west. We find a total mass of 130 M and an estimated ejecta mass of 1.2 M . Comparison of the inferred abundance values and individual element masses with a wide selection of SN models suggests that deflagration-to-detonation (DDT) Type Ia models are the most compatible, with Fe abundance being the major discriminating factor. The general agreement between our abundance measurements and those from previous studies suggests that disagreement between various authors is more likely due to the choice of models used for comparison, rather than the abundance values themselves. While our abundance results lean toward a Type Ia origin, ambiguities in the interpretation of various morphological and spectral characteristics of W49B do not allow us to provide a definitive classification.
COGS: A Compositional Generalization Challenge Based on Semantic Interpretation

Natural language is characterized by compositionality: the meaning of a complex expression is constructed from the meanings of its constituent parts. To facilitate the evaluation of the compositional abilities of language processing architectures, we introduce COGS, a semantic parsing dataset based on a fragment of English. The evaluation portion of COGS contains multiple systematic gaps that can only be addressed by compositional generalization; these include new combinations of familiar syntactic structures, or new combinations of familiar words and familiar structures. In experiments with Transformers and LSTMs, we found that in-distribution accuracy on the COGS test set was near-perfect (96--99%), but generalization accuracy was substantially lower (16--35%) and showed high sensitivity to random seed ($\pm$6--8%). These findings indicate that contemporary standard NLP models are limited in their compositional generalization capacity, and position COGS as a good way to measure progress.
Learning Helpful Inductive Biases from Self-Supervised Pretraining

One reason pretraining on self-supervised linguistic tasks is effective is that it teaches models features that are helpful for language understanding. However, we want pretrained models to learn not only to represent linguistic features, but also to use those features preferentially during fine-turning. With this goal in mind, we introduce a new English-language diagnostic set called MSGS (the Mixed Signals Generalization Set), which consists of 20 ambiguous binary classification tasks that we use to test whether a pretrained model prefers linguistic or surface generalizations during fine-tuning. We pretrain RoBERTa models from scratch on quantities of data ranging from 1M to 1B words and compare their performance on MSGS to the publicly available RoBERTa-base. We find that models can learn to represent linguistic features with little pretraining data, but require far more data to learn to prefer linguistic generalizations over surface ones. Eventually, with about 30B words of pretraining data, RoBERTa-base does demonstrate a linguistic bias with some regularity. We conclude that while self-supervised pretraining is an effective way to learn helpful inductive biases, there is likely room to improve the rate at which models learn which features matter.