Miracle-dz/UniFAR-FTUs huggingface dataset:
The Landscape of Galaxies Harboring Changing-look Active Galactic Nuclei in the Local Universe

We study the properties of the host galaxies of changing-look active galactic nuclei (CL AGNs) with the aim of understanding the conditions responsible for triggering CL activity. We find that CL AGN hosts primarily reside in the so-called green valley that is located between spiral-like star-forming galaxies and dead ellipticals, implying that CL AGNs are activated during distinct periods of quenching and galaxy transformation processes. CL AGN hosts have low galaxy asymmetry indicators, suggesting that secular evolutionary processes (the influence of bars and spirals, and possibly minor mergers) might be the primary mechanism for transporting gas to the vicinity of the supermassive black hole (SMBH) rather than major mergers. Similar to tidal disruption events and highly variable AGNs, we find that CL AGN hosts are associated with SMBHs residing in high-density pseudo-bulges and appear to overlap most significantly with the population of low-ionization nuclear emission-line region galaxies. As such, CL AGNs are likely fueled by strong episodic bursts of accretion activity, which appear to take place preferentially as the amount of material accessible for star formation and accretion dwindles. We also identify that CL AGN hosts are characterized by either large Sérsic indices or high bulge fractions, which suggests a simple metric for identifying candidates for spectroscopic follow-up observations in forthcoming synoptic surveys.
German’s Next Language Model

In this work we present the experiments which lead to the creation of our BERT and ELECTRA based German language models, GBERT and GELECTRA. By varying the input training data, model size, and the presence of Whole Word Masking (WWM) we were able to attain SoTA performance across a set of document classification and named entity recognition (NER) tasks for both models of base and large size. We adopt an evaluation driven approach in training these models and our results indicate that both adding more data and utilizing WWM improve model performance. By benchmarking against existing German models, we show that these models are the best German models to date. All trained models will be made publicly available to the research community.
SCOP: Scientific Control for Reliable Neural Network Pruning

This paper proposes a reliable neural network pruning algorithm by setting up a scientific control. Existing pruning methods have developed various hypotheses to approximate the importance of filters to the network and then execute filter pruning accordingly. To increase the reliability of the results, we prefer to have a more rigorous research design by including a scientific control group as an essential part to minimize the effect of all factors except the association between the filter and expected network output. Acting as a control group, knockoff feature is generated to mimic the feature map produced by the network filter, but they are conditionally independent of the example label given the real feature map. We theoretically suggest that the knockoff condition can be approximately preserved given the information propagation of network layers. Besides the real feature map on an intermediate layer, the corresponding knockoff feature is brought in as another auxiliary input signal for the subsequent layers. Redundant filters can be discovered in the adversarial process of different features. Through experiments, we demonstrate the superiority of the proposed algorithm over state-of-the-art methods. For example, our method can reduce 57.8% parameters and 60.2% FLOPs of ResNet-101 with only 0.01% top-1 accuracy loss on ImageNet.
PARENTing via Model-Agnostic Reinforcement Learning to Correct Pathological Behaviors in Data-to-Text Generation

In language generation models conditioned by structured data, the classical training via maximum likelihood almost always leads models to pick up on dataset divergence (i.e., hallucinations or omissions), and to incorporate them erroneously in their own generations at inference. In this work, we build on top of previous Reinforcement Learning based approaches and show that a model-agnostic framework relying on the recently introduced PARENT metric is efficient at reducing both hallucinations and omissions. Evaluations on the widely used WikiBIO and WebNLG benchmarks demonstrate the effectiveness of this framework compared to state-of-the-art models.
Emergence of a nonconstant entropy for a fast-slow Hamiltonian system in its second-order asymptotic expansion

A system of ordinary differential equations describing the interaction of a fast and a slow particle is studied, where the interaction potential $U_\epsilon$ depends on a small parameter $\epsilon$. The parameter $\epsilon$ can be interpreted as the mass ratio of the two particles. For positive $\epsilon$, the equations of motion are Hamiltonian. It is known (Bornemann, Springer Lecture Notes in Mathematics 1687) that the homogenised limit $\epsilon\to0$ results again in a Hamiltonian system with homogenised potential $U_{\hom}$. In this article, we are interested in the situation where $\epsilon$ is small but positive. In the first part of this work, we rigorously derive the second-order correction to the homogenised degrees of freedom, notably for the slow particle $y_\epsilon = y_0 +\epsilon^2 (\bar{y}_2 +[y_2]^\epsilon)$. In the second part, we give the resulting asymptotic expansion of the energy associated with the fast particle $E_\epsilon^\perp = E_0^\perp + \epsilon [E_1^\perp]^\epsilon+ \epsilon^2 (\bar{E}_2^\perp+[E_2^\perp]^\epsilon)$, a thermodynamic interpretation. In particular, we note that to leading-order $\epsilon\to0$, the dynamics of the fast particle can be identified as an \emph{adiabatic} process with \emph{constant entropy}, $dS_0 =0$. This limit $\epsilon\to0$ is characterised by an energy relation that describes equilibrium thermodynamic processes, $d E_0^\perp = F_0 d y_0 + T_0 dS_0 = F_0 d y_0$, where $T_0$ an $F_0$ are the leading-order temperature and external force terms respectively. In contrast, we find that to second-order $\epsilon^2$, a \emph{non-constant entropy} emerges, $d \bar{S}_2 \neq 0$, effectively describing a non-adiabatic process. Remarkably, this process satisfies on average (in the weak$^\ast$ limit) a similar thermodynamic energy relation, i.e., $d \bar{E}^\perp_2 = F_0 d \bar{y}_2 + T_0 d \bar{\bar{S}}_2$.
Gemini Planet Imager Spectroscopy of the Dusty Substellar Companion HD 206893 B

We present new near-infrared Gemini Planet Imager (GPI) spectroscopy of HD 206893 B, a substellar companion orbiting within the debris disk of its F5V star. The J, H, K1, and K2 spectra from GPI demonstrate the extraordinarily red colors of the object, confirming it as the reddest substellar object observed to date. The significant flux increase throughout the infrared presents a challenging atmosphere to model with existing grids. Best-fit values vary from 1200 to 1800 K for effective temperature and from 3.0 to 5.0 for log(g), depending on which individual wavelength band is fit and which model suite is applied. The extreme redness of the companion can be partially reconciled by invoking a high-altitude layer of submicron dust particles, similar to dereddening approaches applied to the peculiar red field L dwarf population. However, reconciling the HD 206893 B spectra with even those of the reddest low-gravity L dwarf spectra still requires the contribution of additional atmospheric dust, potentially due to the debris disk environment in which the companion resides. Orbit fitting from 4 yr of astrometric monitoring is consistent with a ∼30 yr period, an orbital inclination of 147°, and a semimajor axis of 10 au, well within the estimated disk inner radius of ∼50 au. As one of a very few substellar companions imaged interior to a circumstellar disk, the properties of this system offer important dynamical constraints on companion–disk interaction and provide a benchmark for substellar and planetary atmospheric study.
Convectively Driven 3D Turbulence in Massive Star Envelopes. I. A 1D Implementation of Diffusive Radiative Transport

Massive (M > 30 M ⊙) stars exhibit luminosities that are near the Eddington limit for electron scattering, causing the increase in opacity associated with iron at T≈180,000K to trigger supersonic convection in their outer envelopes. Three-dimensional radiative hydrodynamics simulations by Jiang and collaborators with the Athena++ computational tool have found order-of-magnitude density and radiative flux fluctuations in these convective regions, even at optical depths ≫100 . We show here that radiation can diffuse out of a parcel during the timescale of convection in these optically thick parts of the star, motivating our use of a “pseudo” Mach number to characterize both the fluctuation amplitudes and their correlations. In this first paper, we derive the impact of these fluctuations on the radiative pressure gradient needed to carry a given radiative luminosity. This implementation leads to a remarkable improvement between 1D and 3D radiative pressure gradients, and builds confidence in our path to an eventual 1D implementation of these intrinsically 3D envelopes. However, simply reducing the radiation pressure gradient is not enough to implement a new 1D model. Rather, we must also account for the impact of two other aspects of turbulent convection: the substantial pressure and the ability to transport an appreciable fraction of the luminosity, which will be addressed in upcoming works. This turbulent convection also arises in other instances where the stellar luminosity approaches the Eddington luminosity. Hence, our effort should apply to other astrophysical situations where an opacity peak arises in a near Eddington-limited, radiation pressure-dominated plasma.
Optimization and Generalization of Regularization-Based Continual Learning: a Loss Approximation Viewpoint.

Neural networks have achieved remarkable success in many cognitive tasks. However, when they are trained sequentially on multiple tasks without access to old data, their performance on early tasks tend to drop significantly. This problem is often referred to as catastrophic forgetting, a key challenge in continual learning of neural networks. The regularization-based approach is one of the primary classes of methods to alleviate catastrophic forgetting. In this paper, we provide a novel viewpoint of regularization-based continual learning by formulating it as a second-order Taylor approximation of the loss function of each task. This viewpoint leads to a unified framework that can be instantiated to derive many existing algorithms such as Elastic Weight Consolidation and Kronecker factored Laplace approximation. Based on this viewpoint, we study the optimization aspects (i.e., convergence) as well as generalization properties (i.e., finite-sample guarantees) of regularization-based continual learning. Our theoretical results indicate the importance of accurate approximation of the Hessian matrix. The experimental results on several benchmarks provide empirical validation of our theoretical findings.
Complaint Identification in Social Media with Transformer Networks

Complaining is a speech act extensively used by humans to communicate a negative inconsistency between reality and expectations. Previous work on automatically identifying complaints in social media has focused on using feature-based and task-specific neural network models. Adapting state-of-the-art pre-trained neural language models and their combinations with other linguistic information from topics or sentiment for complaint prediction has yet to be explored. In this paper, we evaluate a battery of neural models underpinned by transformer networks which we subsequently combine with linguistic information. Experiments on a publicly available data set of complaints demonstrate that our models outperform previous state-of-the-art methods by a large margin achieving a macro F1 up to 87.
Eccentric Black Hole Mergers in Active Galactic Nuclei

The astrophysical origin of gravitational wave transients is a timely open question in the wake of discoveries by the Laser Interferometer Gravitational-Wave Observatory (LIGO)/Virgo. In active galactic nuclei (AGNs), binaries form and evolve efficiently by interaction with a dense population of stars and the gaseous AGN disk. Previous studies have shown that stellar-mass black hole (BH) mergers in such environments can explain the merger rate and the number of suspected hierarchical mergers observed by LIGO/Virgo. The binary eccentricity distribution can provide further information to distinguish between astrophysical models. Here we derive the eccentricity distribution of BH mergers in AGN disks. We find that eccentricity is mainly due to binary–single (BS) interactions, which lead to most BH mergers in AGN disks having a significant eccentricity at 0.01 Hz, detectable by the Laser Interferometer Space Antenna. If BS interactions occur in isotropic-3D directions, then 8%–30% of the mergers in AGN disks will have eccentricities at 10 Hz above e10 Hz ≳ 0.03, detectable by LIGO/Virgo/Kamioka Gravitational Wave Detector, while 5%–17% of mergers have e10 Hz ≥ 0.3. On the other hand, if BS interactions are confined to the AGN–disk plane due to torques from the disk, with 1–20 intermediate binary states during each interaction, or if BHs can migrate to ≲ 10−3 pc from the central supermassive BH, then 10%–70% of the mergers will be highly eccentric (e10 Hz ≥ 0.3), consistent with the possible high eccentricity in GW190521.
TweetBERT: A Pretrained Language Representation Model for Twitter Text Analysis

Twitter is a well-known microblogging social site where users express their views and opinions in real-time. As a result, tweets tend to contain valuable information. With the advancements of deep learning in the domain of natural language processing, extracting meaningful information from tweets has become a growing interest among natural language researchers. Applying existing language representation models to extract information from Twitter does not often produce good results. Moreover, there is no existing language representation models for text analysis specific to the social media domain. Hence, in this article, we introduce two TweetBERT models, which are domain specific language presentation models, pre-trained on millions of tweets. We show that the TweetBERT models significantly outperform the traditional BERT models in Twitter text mining tasks by more than 7% on each Twitter dataset. We also provide an extensive analysis by evaluating seven BERT models on 31 different datasets. Our results validate our hypothesis that continuously training language models on twitter corpus help performance with Twitter.
On robust probabilistic principal component analysis using multivariate t-distributions

Abstract Probabilistic principal component analysis (PPCA) is a probabilistic reformulation of principal component analysis (PCA), under the framework of a Gaussian latent variable model. To improve the robustness of PPCA, it has been proposed to change the underlying Gaussian distributions to multivariate t-distributions. Based on the representation of t-distribution as a scale mixture of Gaussian distributions, a hierarchical model is used for implementation. However, in the existing literature, the hierarchical model implemented does not yield the equivalent interpretation. In this paper, we present two sets of equivalent relationships between the high-level multivariate t-PPCA framework and the hierarchical model used for implementation. In doing so, we clarify a current misrepresentation in the literature, by specifying the correct correspondence. In addition, we discuss the performance of different multivariate t robust PPCA methods both in theory and simulation studies, and propose a new Monte Carlo expectation-maximization (MCEM) algorithm to implement one general type of such models.
RECONSIDER: Re-Ranking using Span-Focused Cross-Attention for Open Domain Question Answering

State-of-the-art Machine Reading Comprehension (MRC) models for Open-domain Question Answering (QA) are typically trained for span selection using distantly supervised positive examples and heuristically retrieved negative examples. This training scheme possibly explains empirical observations that these models achieve a high recall amongst their top few predictions, but a low overall accuracy, motivating the need for answer re-ranking. We develop a simple and effective re-ranking approach (RECONSIDER) for span-extraction tasks, that improves upon the performance of large pre-trained MRC models. RECONSIDER is trained on positive and negative examples extracted from high confidence predictions of MRC models, and uses in-passage span annotations to perform span-focused re-ranking over a smaller candidate set. As a result, RECONSIDER learns to eliminate close false positive passages, and achieves a new state of the art on four QA tasks, including 45.5% Exact Match accuracy on Natural Questions with real user questions, and 61.7% on TriviaQA.
Influence of natal host and oviposition experience on sex allocation in a solitary egg parasitoid, Anastatus disparis (Hymenoptera, Eupelmidae)

Constraints on adaptation are a major topic in evolutionary biology. Sex allocation, in particular the ratio of the sexes, has often been used as a key process for studying constraints on adaptation. Anastatus disparis Ruschka (Hymenoptera: Eupelmidae) is a solitary egg parasitoid of gypsy moth, Lymantria dispar (L.) (Lepidoptera: Lymantriidae), and several other lepidopteran forest pests. Here, we compared two different sized substitute hosts, the smaller one Dictyoploca japonica Moore (Lepidoptera: Saturnidae) and the larger one Antheraea pernyi Guerin-Meneville (Lepidoptera: Saturnidae), and investigated the influence of natal host and oviposition experience on sex allocation by A. disparis. Results showed that natal host had almost no impact on sex allocation by A. disparis, but oviposition experience did influence sex allocation of A. disparis on D. japonica eggs. This suggests that information females obtain from the environment influences how they allocate sex in their offspring. However, the sex ratios of A. disparis emerging from A. pernyi eggs were consistent irrespective of oviposition experience of female A. disparis. This indicates that the eggs of A. pernyi are large enough to maximize female progeny of A. disparis.
Self-supervised Graph Learning for Recommendation

Representation learning on user-item graph for recommendation has evolved from using single ID or interaction history to exploiting higher-order neighbors. This leads to the success of graph convolution networks (GCNs) for recommendation such as PinSage and LightGCN. Despite effectiveness, we argue that they suffer from two limitations: (1) high-degree nodes exert larger impact on the representation learning, deteriorating the recommendations of low-degree (long-tail) items; and (2) representations are vulnerable to noisy interactions, as the neighborhood aggregation scheme further enlarges the impact of observed edges. In this work, we explore self-supervised learning on user-item graph, so as to improve the accuracy and robustness of GCNs for recommendation. The idea is to supplement the classical supervised task of recommendation with an auxiliary self-supervised task, which reinforces node representation learning via self-discrimination. Specifically, we generate multiple views of a node, maximizing the agreement between different views of the same node compared to that of other nodes. We devise three operators to generate the views --- node dropout, edge dropout, and random walk --- that change the graph structure in different manners. We term this new learning paradigm asSelf-supervised Graph Learning (SGL), implementing it on the state-of-the-art model LightGCN. Through theoretical analyses, we find that SGL has the ability of automatically mining hard negatives. Empirical studies on three benchmark datasets demonstrate the effectiveness of SGL, which improves the recommendation accuracy, especially on long-tail items, and the robustness against interaction noises. Our implementations are available at \urlhttps://github.com/wujcan/SGL.
Bayesian Attention Modules

Attention modules, as simple and effective tools, have not only enabled deep neural networks to achieve state-of-the-art results in many domains, but also enhanced their interpretability. Most current models use deterministic attention modules due to their simplicity and ease of optimization. Stochastic counterparts, on the other hand, are less popular despite their potential benefits. The main reason is that stochastic attention often introduces optimization issues or requires significant model changes. In this paper, we propose a scalable stochastic version of attention that is easy to implement and optimize. We construct simplex-constrained attention distributions by normalizing reparameterizable distributions, making the training process differentiable. We learn their parameters in a Bayesian framework where a data-dependent prior is introduced for regularization. We apply the proposed stochastic attention modules to various attention-based models, with applications to graph node classification, visual question answering, image captioning, machine translation, and language understanding. Our experiments show the proposed method brings consistent improvements over the corresponding baselines.
Complex glycan utilization preferences of human gut bacteria

Tuncil, Yunus Emre. Ph.D., Purdue University, December 2016. Complex Glycan Utilization Preferences of Human Colonic Bacteria. Major Professor: Bruce R. Hamaker. Complex glycans, making up dietary fiber, have gained significance in recent years as they are the main energy source for the colonic microbiota which are physiologically important for the host health. Understanding glycan utilization strategies of the members of this community is essential to maximize beneficial health outcomes obtained from them. In order to investigate how the members of our gut microbiota utilize glycans and activate their respective polysaccharide utilization loci (PULs), we performed a series of time-course assays in which two model organisms, Bacteroides thetaiotamicron (Bt) and B. ovatus (Bo), were individually grown in a medium containing carbohydrates utilized by both bacteria: amylopectin (AP), arabinan (ARAB), chondroitin sulphate (CS), pectic galactan (PG), polygalacturonic acid (PGA), and rhamnogalacturonan I (RGI), and measured remaining substrates over time as well as PULs expression. Bt and Bo utilized some glycans before others, but with different prioritizations supporting past work that bacterial species show species-specific hierarchical preference to dietary fibers. Bacteria sensitivities to some glycans are tightly tuned to the residual concentration of polysaccharide in the medium, while others remain highly expressed even after most of the target substrates have been depleted. For