Miracle-dz/UniFAR-FTUs huggingface dataset:
Compton-thick AGN in the NuSTAR Era. IX. A Joint NuSTAR and XMM-Newton Analysis of Four Local AGN

We present the results of the broadband X-ray spectral analysis of simultaneous NuSTAR and XMM-Newton observations of four nearby Compton-thick active galactic nuclei (AGN) candidates selected from the Swift Burst Alert Telescope 150 month catalog. This work is part of a larger effort to identify and characterize all Compton-thick (N H ≥ 1024 cm−2) AGN in the local universe (z ≤ 0.05). We used three physically motivated models—MYTorus, borus02, and UXClumpy—to fit and characterize these sources. Of the four candidates analyzed, 2MASX J02051994-0233055 was found to be an unobscured (N H < 1022 cm−2) AGN, 2MASX J04075215-6116126 and IC 2227 to be Compton-thin (1022 cm−2 < N H < 1024 cm−2) AGN, and one, ESO 362−8, was confirmed to be a Compton-thick AGN. Additionally, every source was found to have a statistically significant difference between their line-of-sight and average torus hydrogen column density, further supporting the idea that the obscuring material in AGN is inhomogeneous. Furthermore, half of the sources in our sample (2MASX J02051994-0233055 and 2MASX J04075215-6116126) exhibited significant luminosity variation in the last decade, suggesting that this might be a common feature of AGN.
Lo-fi: Distributed Fine-tuning without Communication

When fine-tuning large neural networks, it is common to use multiple nodes and to communicate gradients at each optimization step. By contrast, we investigate completely local fine-tuning, which we refer to as lo-fi. During lo-fi, each node is fine-tuned independently without any communication. Then, the weights are averaged across nodes at the conclusion of fine-tuning. When fine-tuning DeiT-base and DeiT-large on ImageNet, this procedure matches accuracy in-distribution and improves accuracy under distribution shift compared to the baseline, which observes the same amount of data but communicates gradients at each step. We also observe that lo-fi matches the baseline's performance when fine-tuning OPT language models (up to 1.3B parameters) on Common Crawl. By removing the communication requirement, lo-fi reduces resource barriers for fine-tuning large models and enables fine-tuning in settings with prohibitive communication cost.
Using Large Language Models to Enhance Programming Error Messages

A key part of learning to program is learning to understand programming error messages. They can be hard to interpret and identifying the cause of errors can be time-consuming. One factor in this challenge is that the messages are typically intended for an audience that already knows how to program, or even for programming environments that then use the information to highlight areas in code. Researchers have been working on making these errors more novice friendly since the 1960s, however progress has been slow. The present work contributes to this stream of research by using large language models to enhance programming error messages with explanations of the errors and suggestions on how to fix them. Large language models can be used to create useful and novice-friendly enhancements to programming error messages that sometimes surpass the original programming error messages in interpretability and actionability. These results provide further evidence of the benefits of large language models for computing educators, highlighting their use in areas known to be challenging for students. We further discuss the benefits and downsides of large language models and highlight future streams of research for enhancing programming error messages.
Learning in RKHM: a C*-Algebraic Twist for Kernel Machines

Supervised learning in reproducing kernel Hilbert space (RKHS) and vector-valued RKHS (vvRKHS) has been investigated for more than 30 years. In this paper, we provide a new twist to this rich literature by generalizing supervised learning in RKHS and vvRKHS to reproducing kernel Hilbert $C^*$-module (RKHM), and show how to construct effective positive-definite kernels by considering the perspective of $C^*$-algebra. Unlike the cases of RKHS and vvRKHS, we can use $C^*$-algebras to enlarge representation spaces. This enables us to construct RKHMs whose representation power goes beyond RKHSs, vvRKHSs, and existing methods such as convolutional neural networks. Our framework is suitable, for example, for effectively analyzing image data by allowing the interaction of Fourier components.
TransLIST: A Transformer-Based Linguistically Informed Sanskrit Tokenizer

Sanskrit Word Segmentation (SWS) is essential in making digitized texts available and in deploying downstream tasks. It is, however, non-trivial because of the sandhi phenomenon that modifies the characters at the word boundaries, and needs special treatment. Existing lexicon driven approaches for SWS make use of Sanskrit Heritage Reader, a lexicon-driven shallow parser, to generate the complete candidate solution space, over which various methods are applied to produce the most valid solution. However, these approaches fail while encountering out-of-vocabulary tokens. On the other hand, purely engineering methods for SWS have made use of recent advances in deep learning, but cannot make use of the latent word information on availability. To mitigate the shortcomings of both families of approaches, we propose Transformer based Linguistically Informed Sanskrit Tokenizer (TransLIST) consisting of (1) a module that encodes the character input along with latent-word information, which takes into account the sandhi phenomenon specific to SWS and is apt to work with partial or no candidate solutions, (2) a novel soft-masked attention to prioritize potential candidate words and (3) a novel path ranking algorithm to rectify the corrupted predictions. Experiments on the benchmark datasets for SWS show that TransLIST outperforms the current state-of-the-art system by an average 7.2 points absolute gain in terms of perfect match (PM) metric. The codebase and datasets are publicly available at https://github.com/rsingha108/TransLIST
Is Encoder-Decoder Redundant for Neural Machine Translation?

Encoder-decoder architecture is widely adopted for sequence-to-sequence modeling tasks. For machine translation, despite the evolution from long short-term memory networks to Transformer networks, plus the introduction and development of attention mechanism, encoder-decoder is still the de facto neural network architecture for state-of-the-art models. While the motivation for decoding information from some hidden space is straightforward, the strict separation of the encoding and decoding steps into an encoder and a decoder in the model architecture is not necessarily a must. Compared to the task of autoregressive language modeling in the target language, machine translation simply has an additional source sentence as context. Given the fact that neural language models nowadays can already handle rather long contexts in the target language, it is natural to ask whether simply concatenating the source and target sentences and training a language model to do translation would work. In this work, we investigate the aforementioned concept for machine translation. Specifically, we experiment with bilingual translation, translation with additional target monolingual data, and multilingual translation. In all cases, this alternative approach performs on par with the baseline encoder-decoder Transformer, suggesting that an encoder-decoder architecture might be redundant for neural machine translation.
Anatomy of the AGN in NGC 5548 IX. Photoionized emission features in the soft X-ray spectra

The X-ray narrow emission line region (NELR) of the archetypal Seyfert 1 galaxy NGC\,5548 has been interpreted as a single-phase photoionized plasma that is absorbed by some of the warm absorber components. This scenario requires those overlaying warm absorber components to have larger distance (to the central engine) than the X-ray NELR, which is not fully consistent with the distance estimates found in the literature. Therefore, we reanalyze the high-resolution spectra obtained in 2013--2014 with the Reflection Grating Spectrometer (RGS) aboard \textit{XMM}-Newton to provide an alternative interpretation of the X-ray narrow emission features. We find that the X-ray narrow emission features in NGC\,5548 can be described by a two-phase photoionized plasma with different ionization parameters ($\log \xi=1.3$ and $0.1$) and kinematics ($v_{\rm out}=-50$ and $-400~{\rm km~s^{-1}}$), and no further absorption by the warm absorber components. The X-ray and optical NELR might be the same multi-phase photoionized plasma. Both X-ray and optical NELR have comparable distances, asymmetric line profiles, and the underlying photoionized plasma is turbulent and compact in size. The X-ray NELR is not the counterpart of the UV/X-ray absorber outside the line of sight because their distances and kinematics are not consistent. In addition, X-ray broad emission features that we find in the spectrum can be accounted for by a third photoionized emission component. The RGS spectrum obtained in 2016 is analyzed as well, where the luminosity of most prominent emission lines (the \ion{O}{vii} forbidden line and \ion{O}{viii} Ly$\alpha$ line) are the same (at a 1 $\sigma$ confidence level) as in 2013--2014.
Learning Graphical Factor Models with Riemannian Optimization

Graphical models and factor analysis are well-established tools in multivariate statistics. While these models can be both linked to structures exhibited by covariance and precision matrices, they are generally not jointly leveraged within graph learning processes. This paper therefore addresses this issue by proposing a flexible algorithmic framework for graph learning under low-rank structural constraints on the covariance matrix. The problem is expressed as penalized maximum likelihood estimation of an elliptical distribution (a generalization of Gaussian graphical models to possibly heavy-tailed distributions), where the covariance matrix is optionally constrained to be structured as low-rank plus diagonal (low-rank factor model). The resolution of this class of problems is then tackled with Riemannian optimization, where we leverage geometries of positive definite matrices and positive semi-definite matrices of fixed rank that are well suited to elliptical models. Numerical experiments on real-world data sets illustrate the effectiveness of the proposed approach.
Unsupervised Image Semantic Segmentation through Superpixels and Graph Neural Networks

Unsupervised image segmentation is an important task in many real-world scenarios where labelled data is of scarce availability. In this paper we propose a novel approach that harnesses recent advances in unsupervised learning using a combination of Mutual Information Maximization (MIM), Neural Superpixel Segmentation and Graph Neural Networks (GNNs) in an end-to-end manner, an approach that has not been explored yet. We take advantage of the compact representation of superpixels and combine it with GNNs in order to learn strong and semantically meaningful representations of images. Specifically, we show that our GNN based approach allows to model interactions between distant pixels in the image and serves as a strong prior to existing CNNs for an improved accuracy. Our experiments reveal both the qualitative and quantitative advantages of our approach compared to current state-of-the-art methods over four popular datasets.
Linear mixed model vs two-stage methods: Developing prognostic models of diabetic kidney disease progression

Identifying prognostic factors for disease progression is a cornerstone of medical research. Repeated assessments of a marker outcome are often used to evaluate disease progression, and the primary research question is to identify factors associated with the longitudinal trajectory of this marker. Our work is motivated by diabetic kidney disease (DKD), where serial measures of estimated glomerular filtration rate (eGFR) are the longitudinal measure of kidney function, and there is notable interest in identifying factors, such as metabolites, that are prognostic for DKD progression. Linear mixed models (LMM) with serial marker outcomes (e.g., eGFR) are a standard approach for prognostic model development, namely by evaluating the time × prognostic factor (e.g., metabolite) interaction. However, two-stage methods that first estimate individual-specific eGFR slopes, and then use these as outcomes in a regression framework with metabolites as predictors are easy to interpret and implement for applied researchers. Herein, we compared the LMM and two-stage methods, in terms of bias and mean squared error via analytic methods and simulations, allowing for irregularly spaced measures and missingness. Our findings provide novel insights into when two-stage methods are suitable longitudinal prognostic modeling alternatives to the LMM. Notably, our findings generalize to other disease studies.
If Dark Matter is Fuzzy, the First Stars Form in Massive Pancakes

Fuzzy dark matter (FDM) is a proposed modification for the standard cold dark matter (CDM) model motivated by small-scale discrepancies in low-mass galaxies. Composed of ultralight (mass ∼ 1022 eV) axions with kiloparsec-scale de Broglie wavelengths, this is one of a class of candidates that predicts that the first collapsed objects form in relatively massive dark matter halos. This implies that the formation history of the first stars and galaxies would be very different, potentially placing strong constraints on such models. Here we numerically simulate the formation of the first stars in an FDM cosmology, following the collapse in a representative volume all the way down to primordial protostar formation including a primordial nonequilibrium chemical network and cooling for the first time. We find two novel results: first, the large-scale collapse results in a very thin and flat gas “pancake”; second, despite the very different cosmology, this pancake fragments until it forms protostellar objects indistinguishable from those in CDM. Combined, these results indicate that the first generation of stars in this model are also likely to be massive and, because of the sheet morphology, do not self-regulate, resulting in a massive Population III starburst. We estimate the total number of first stars forming in this extended structure to be 104 over 20 Myr using a simple model to account for the ionizing feedback from the stars, and should be observable with the James Webb Space Telescope. These predictions provide a potential smoking gun signature of FDM and similar dark matter candidates.
Quiver varieties and finite dimensional representations of quantum affine algebras

Introduction 145 1. Quantum affine algebra 150 2. Quiver variety 155 3. Stratification of M0 163 4. Fixed point subvariety 167 5. Hecke correspondence and induction of quiver varieties 169 6. Equivariant K-theory 174 7. Freeness 178 8. Convolution 185 9. A homomorphism Uq(Lg)→ KGw×C ∗ (Z(w))⊗Z[q,q−1] Q(q) 192 10. Relations (I) 194 11. Relations (II) 202 12. Integral structure 214 13. Standard modules 218 14. Simple modules 224 15. The Ue(g)-module structure 233 Added in proof 236 References 236
Do Vision-and-Language Transformers Learn Grounded Predicate-Noun Dependencies?

Recent advances in vision-and-language modeling have seen the development of Transformer architectures that achieve remarkable performance on multimodal reasoning tasks.Yet, the exact capabilities of these black-box models are still poorly understood. While much of previous work has focused on studying their ability to learn meaning at the word-level, their ability to track syntactic dependencies between words has received less attention.We take a first step in closing this gap by creating a new multimodal task targeted at evaluating understanding of predicate-noun dependencies in a controlled setup.We evaluate a range of state-of-the-art models and find that their performance on the task varies considerably, with some models performing relatively well and others at chance level. In an effort to explain this variability, our analyses indicate that the quality (and not only sheer quantity) of pretraining data is essential. Additionally, the best performing models leverage fine-grained multimodal pretraining objectives in addition to the standard image-text matching objectives.This study highlights that targeted and controlled evaluations are a crucial step for a precise and rigorous test of the multimodal knowledge of vision-and-language models.
Improved Normalizing Flow-Based Speech Enhancement Using an all-Pole Gammatone Filterbank for Conditional Input Representation

Deep generative models for Speech Enhancement (SE) received increasing attention in recent years. The most prominent example are Generative Adversarial Networks (GANs), while normalizing flows (NF) received less attention despite their potential. Building on previous work, architectural modifications are proposed, along with an investigation of different conditional input representations. Despite being a common choice in related works, Mel-spectrograms demonstrate to be inadequate for the given scenario. Alternatively, a novel All-Pole Gammatone filterbank (APG) with high temporal resolution is proposed. Although computational evaluation metric results would suggest that state-of-the-art GAN-based methods perform best, a perceptual evaluation via a listening test indicates that the presented NF approach (based on time domain and APG) performs best, especially at lower SNRs. On average, APG outputs are rated as having good quality, which is unmatched by the other methods, including GAN.
BioLORD: Learning Ontological Representations from Definitions (for Biomedical Concepts and their Textual Descriptions)

This work introduces BioLORD, a new pre-training strategy for producing meaningful representations for clinical sentences and biomedical concepts. State-of-the-art methodologies operate by maximizing the similarity in representation of names referring to the same concept, and preventing collapse through contrastive learning. However, because biomedical names are not always self-explanatory, it sometimes results in non-semantic representations. BioLORD overcomes this issue by grounding its concept representations using definitions, as well as short descriptions derived from a multi-relational knowledge graph consisting of biomedical ontologies. Thanks to this grounding, our model produces more semantic concept representations that match more closely the hierarchical structure of ontologies. BioLORD establishes a new state of the art for text similarity on both clinical sentences (MedSTS) and biomedical concepts (MayoSRS).
Boosting Natural Language Generation from Instructions with Meta-Learning

Recent work has shown that language models (LMs) trained with multi-task instructional learning (MTIL) can solve diverse NLP tasks in zero- and few-shot settings with improved performance compared to prompt tuning. MTIL illustrates that LMs can extract and use information about the task from instructions beyond the surface patterns of the inputs and outputs. This suggests that meta-learning may further enhance the utilization of instructions for effective task transfer. In this paper we investigate whether meta-learning applied to MTIL can further improve generalization to unseen tasks in a zero-shot setting. Specifically, we propose to adapt meta-learning to MTIL in three directions: 1) Model Agnostic Meta Learning (MAML), 2) Hyper-Network (HNet) based adaptation to generate task specific parameters conditioned on instructions, and 3) an approach combining HNet and MAML. Through extensive experiments on the large scale Natural Instructions V2 dataset, we show that our proposed approaches significantly improve over strong baselines in zero-shot settings. In particular, meta-learning improves the effectiveness of instructions and is most impactful when the test tasks are strictly zero-shot (i.e. no similar tasks in the training set) and are “hard” for LMs, illustrating the potential of meta-learning for MTIL for out-of-distribution tasks.
The invariant measure of a walking droplet in hydrodynamic pilot-wave theory

We study the long time statistics of a walker in a hydrodynamic pilot-wave system, which is a stochastic Langevin dynamics with an external potential and memory kernel. While prior experiments and numerical simulations have indicated that the system may reach a statistically steady state, its long-time behavior has not been studied rigorously. For a broad class of external potentials and pilot-wave forces, we construct the solutions as a dynamics evolving on suitable path spaces. Then, under the assumption that the pilot-wave force is dominated by the potential, we demonstrate that the walker possesses a unique statistical steady state. We conclude by presenting an example of such an invariant measure, as obtained from a numerical simulation of a walker in a harmonic potential.
Comparison of REML methods for the study of phenome-wide genetic variation

It is now well documented that genetic covariance between functionally related traits leads to an uneven distribution of genetic variation across multivariate trait combinations, and possibly a large part of phenotype-space that is inaccessible to evolution. How the size of this nearly-null genetic space translates to the broader phenome level is unknown. High dimensional phenotype data to address these questions are now within reach, however, incorporating these data into genetic analyses remains a challenge. Multi-trait genetic analyses, of more than a handful of traits, are slow and often fail to converge when fit with REML. This makes it challenging to estimate the genetic covariance ($\mathbf{G}$) underlying thousands of traits, let alone study its properties. We present a previously proposed REML algorithm that is feasible for high dimensional genetic studies in the specific setting of a balanced nested half-sib design, common of quantitative genetics. We show that it substantially outperforms other common approaches when the number of traits is large, and we use it to investigate the bias in estimated eigenvalues of $\mathbf{G}$ and the size of the nearly-null genetic subspace. We show that the high-dimensional biases observed are qualitatively similar to those substantiated by asymptotic approximation in a simpler setting of a sample covariance matrix based on i.i. d. vector observation, and that interpreting the estimated size of the nearly-null genetic subspace requires considerable caution in high-dimensional studies of genetic variation. Our results provide the foundation for future research characterizing the asymptotic approximation of estimated genetic eigenvalues, and a statistical null distribution for phenome-wide studies of genetic variation.