Miracle-dz/UniFAR-FTUs huggingface dataset:
The Effect of Sport in Online Dating: Evidence from Causal Machine Learning

Online dating emerged as a key platform for human mating. Previous research focused on socio-demographic characteristics to explain human mating in online dating environments, neglecting the commonly recognized relevance of sport. This research investigates the effect of sport activity on human mating by exploiting a unique data set from an online dating platform. Thereby, we leverage recent advances in the causal machine learning literature to estimate the causal effect of sport frequency on the contact chances. We find that for male users, doing sport on a weekly basis increases the probability to receive a first message from a woman by 50%, relatively to not doing sport at all. For female users, we do not find evidence for such an effect. In addition, for male users the effect increases with higher income.
Better Feature Integration for Named Entity Recognition

It has been shown that named entity recognition (NER) could benefit from incorporating the long-distance structured information captured by dependency trees. We believe this is because both types of features - the contextual information captured by the linear sequences and the structured information captured by the dependency trees may complement each other. However, existing approaches largely focused on stacking the LSTM and graph neural networks such as graph convolutional networks (GCNs) for building improved NER models, where the exact interaction mechanism between the two types of features is not very clear, and the performance gain does not appear to be significant. In this work, we propose a simple and robust solution to incorporate both types of features with our Synergized-LSTM (Syn-LSTM), which clearly captures how the two types of features interact. We conduct extensive experiments on several standard datasets across four languages. The results demonstrate that the proposed model achieves better performance than previous approaches while requiring fewer parameters. Our further analysis demonstrates that our model can capture longer dependencies compared with strong baselines.
Relics of Supermassive Black Hole Seeds: The Discovery of an Accreting Black Hole in an Optically Normal, Low Metallicity Dwarf Galaxy

The detection and characterization of supermassive black holes (SMBHs) in local low mass galaxies is crucial to our understanding of the origins of SMBHs. This statement assumes that low mass galaxies have had a relatively quiet cosmic history, so that their black holes have not undergone significant growth and therefore can be treated as relics of the original SMBH seeds. While recent studies have found optical signatures of active galactic nuclei (AGNs) in a growing population of dwarf galaxies, these studies are biased against low metallicity and relatively merger-free galaxies, thus missing precisely the demographic in which to search for the relics of SMBH seeds. Here, we report the detection of the [Si vi]1.963 μm coronal line (CL), a robust indicator of an AGN in the galaxy SDSS J160135.95+311353.7, a nearby (z = 0.031) low metallicity galaxy with a stellar mass approximately an order of magnitude lower than the LMC (M * ≈ 108.56 M ⊙) and no optical evidence for an AGN. The AGN bolometric luminosity implied by the CL detection is ≈1042 erg s−1, precisely what is predicted from its near-infrared continuum emission based on well-studied AGNs. Our results are consistent with a black hole of mass ≈105 M ⊙, in line with expectations based on its stellar mass. This is the first time a near-infrared CL has been detected in a low mass, low metallicity galaxy with no optical evidence for AGN activity, providing confirmation of the utility of infrared CLs in finding AGNs in low mass galaxies when optical diagnostics fail. These observations highlight a powerful avenue of investigation to hunt for low mass black holes in the James Webb Space Telescope era.
LoTSS jellyfish galaxies. I. Radio tails in low redshift clusters

In this paper we present a large sample of jellyfish galaxies in low redshift clusters (z<0.05), identified through 120-168 MHz radio continuum from the LOFAR Two-metre Sky Survey (LoTSS). From a parent sample of 29 X-ray-detected SDSS galaxy clusters and their spectroscopic members, we visually identify 95 star-forming, LoTSS jellyfish galaxies with 144 MHz radio tails. Star formation rates (SFRs) and stellar masses are obtained for all galaxies from SED fits. For each jellyfish galaxy we determine the tail orientation with respect to the cluster centre and quantify the prominence of the radio tails with the 144 MHz shape asymmetry. After carefully accounting for redshift-dependent selection effects, we find that the frequency of jellyfish galaxies is relatively constant from cluster to cluster. LoTSS jellyfish galaxies are preferentially found at small clustercentric radius and large velocity offsets within their host clusters and have radio tails that are oriented away from the cluster centre. These galaxies also show enhanced star formation, relative to both 'normal' cluster galaxies and isolated field galaxies, but generally fall within the scatter of the L144MHz - SFR relation. The properties of the LoTSS jellyfish galaxies identified in this work are fully consistent with expectations from ram pressure stripping. This large sample of jellyfish galaxies will be valuable for further constraining ram pressure stripping and star formation quenching in nearby galaxy clusters. We show that LOFAR is a powerful instrument for identifying ram pressure stripped galaxies across extremely wide fields. Moving forward we will push the search for jellyfish galaxies beyond this initial cluster sample, including a comprehensive survey of the galaxy group regime.
UNIT-DDPM: UNpaired Image Translation with Denoising Diffusion Probabilistic Models

We propose a novel unpaired image-to-image translation method that uses denoising diffusion probabilistic models without requiring adversarial training. Our method, UNpaired Image Translation with Denoising Diffusion Probabilistic Models (UNIT-DDPM), trains a generative model to infer the joint distribution of images over both domains as a Markov chain by minimising a denoising score matching objective conditioned on the other domain. In particular, we update both domain translation models simultaneously, and we generate target domain images by a denoising Markov Chain Monte Carlo approach that is conditioned on the input source domain images, based on Langevin dynamics. Our approach provides stable model training for image-to-image translation and generates high-quality image outputs. This enables state-of-the-art Fr\'echet Inception Distance (FID) performance on several public datasets, including both colour and multispectral imagery, significantly outperforming the contemporary adversarial image-to-image translation methods.
Patient contrastive learning: A performant, expressive, and practical approach to electrocardiogram modeling

Supervised machine learning applications in health care are often limited due to a scarcity of labeled training data. To mitigate the effect of small sample size, we introduce a pre-training approach, Patient Contrastive Learning of Representations (PCLR), which creates latent representations of electrocardiograms (ECGs) from a large number of unlabeled examples using contrastive learning. The resulting representations are expressive, performant, and practical across a wide spectrum of clinical tasks. We develop PCLR using a large health care system with over 3.2 million 12-lead ECGs and demonstrate that training linear models on PCLR representations achieves a 51% performance increase, on average, over six training set sizes and four tasks (sex classification, age regression, and the detection of left ventricular hypertrophy and atrial fibrillation), relative to training neural network models from scratch. We also compared PCLR to three other ECG pre-training approaches (supervised pre-training, unsupervised pre-training with an autoencoder, and pre-training using a contrastive multi ECG-segment approach), and show significant performance benefits in three out of four tasks. We found an average performance benefit of 47% over the other models and an average of a 9% performance benefit compared to best model for each task. We release PCLR to enable others to extract ECG representations at https://github.com/broadinstitute/ml4h/tree/master/model_zoo/PCLR.
Domain-wall roughness in GdFeCo thin films: Crossover length scales and roughness exponents

Domain-wall dynamics and spatial fluctuations are closely related to each other and to universal features of disordered systems. Experimentally measured roughness exponents characterizing spatial fluctuations have been reported for magnetic thin films, with values generally different from those predicted by the equilibrium, depinning and thermal reference states. Here, we study the roughness of domain walls in GdFeCo thin films over a large range of magnetic field and temperature. Our analysis is performed in the framework of a model considering length-scale crossovers between the reference states, which is shown to bridge the differences between experimental results and theoretical predictions. We also quantify for the first time the size of the depinning avalanches below the depinning field at finite temperatures.
Pinch-off of viscoelastic particulate suspensions

The formation of drops of a complex fluid, for instance including dissolved polymers and/or solid particles, has practical implications in several industrial and biophysical processes. In this Letter, we experimentally investigate the generation of drops of a viscoelastic suspension, made of non-Brownian spherical particles dispersed in a dilute polymer solution. Using high-speed imaging, we characterize the different stages of the detachment. Our experiments show that the particles primarily affect the initial Newtonian necking by increasing the fluid viscosity. In the viscoelastic regime, particles do not affect the thinning until the onset of the blistering instability, which they accelerate. We find that the transition from one regime to another, which corresponds to the coil-stretch transition of the polymer chains, strongly depends on the particle content. Considering that the presence of rigid particles increases the deformation of the liquid phase, we propose an expression for the local strain rate in the suspension, which rationalizes our experimental results. This method could enable the precise measurement of local stresses in particulate suspensions.
A Learnable Self-supervised Task for Unsupervised Domain Adaptation on Point Clouds

Deep neural networks have achieved promising performance in supervised point cloud applications, but manual annotation is extremely expensive and time-consuming in supervised learning schemes. Unsupervised domain adaptation (UDA) addresses this problem by training a model with only labeled data in the source domain but making the model generalize well in the target domain. Existing studies show that self-supervised learning using both source and target domain data can help improve the adaptability of trained models, but they all rely on hand-crafted designs of the self-supervised tasks. In this paper, we propose a learnable self-supervised task and integrate it into a self-supervision-based point cloud UDA architecture. Specifically, we propose a learnable nonlinear transformation that transforms a part of a point cloud to generate abundant and complicated point clouds while retaining the original semantic information, and the proposed self-supervised task is to reconstruct the original point cloud from the transformed ones. In the UDA architecture, an encoder is shared between the networks for the self-supervised task and the main task of point cloud classification or segmentation, so that the encoder can be trained to extract features suitable for both the source and the target domain data. Experiments on PointDA-10 and PointSegDA datasets show that the proposed method achieves new state-of-the-art performance on both classification and segmentation tasks of point cloud UDA. Code will be made publicly available.
Couplings for Multinomial Hamiltonian Monte Carlo

Hamiltonian Monte Carlo (HMC) is a popular sampling method in Bayesian inference. Recently, Heng&Jacob (2019) studied Metropolis HMC with couplings for unbiased Monte Carlo estimation, establishing a generic parallelizable scheme for HMC. However, in practice a different HMC method, multinomial HMC, is considered as the go-to method, e.g. as part of the no-U-turn sampler. In multinomial HMC, proposed states are not limited to end-points as in Metropolis HMC; instead points along the entire trajectory can be proposed. In this paper, we establish couplings for multinomial HMC, based on optimal transport for multinomial sampling in its transition. We prove an upper bound for the meeting time - the time it takes for the coupled chains to meet - based on the notion of local contractivity. We evaluate our methods using three targets: 1,000 dimensional Gaussians, logistic regression and log-Gaussian Cox point processes. Compared to Heng&Jacob (2019), coupled multinomial HMC generally attains a smaller meeting time, and is more robust to choices of step sizes and trajectory lengths, which allows re-use of existing adaptation methods for HMC. These improvements together paves the way for a wider and more practical use of coupled HMC methods.
Predicting Pedestrian Crossing Intention With Feature Fusion and Spatio-Temporal Attention

Predicting vulnerableroad user behavior is an essential prerequisite for deploying Automated Driving Systems (ADS) in the real-world. Pedestrian crossing intention should be recognized in real-time, especially for urban driving. Recent works have shown the potential of using vision-based deep neural network models for this task. However, these models are not robust and certain issues still need to be resolved. First, the global spatio-temporal context that accounts for the interaction between the target pedestrian and the scene has not been properly utilized. Second, the optimal strategy for fusing different sensor data has not been thoroughly investigated. This work addresses the above limitations by introducing a novel neural network architecture to fuse inherently different spatio-temporal features for pedestrian crossing intention prediction. We fuse different phenomena such as sequences of RGB imagery, semantic segmentation masks, and ego-vehicle speed in an optimal way using attention mechanisms and a stack of recurrent neural networks. The optimal architecture was obtained through exhaustive ablation and comparison studies. Extensive comparative experiments on the JAAD and PIE pedestrian action prediction benchmarks demonstrate the effectiveness of the proposed method, where state-of-the-art performance was achieved. Our code is open-source and publicly available: https://github.com/OSU-Haolin/Pedestrian_Crossing_Intention_Prediction.
Escaping the Big Data Paradigm with Compact Transformers

With the rise of Transformers as the standard for language processing, and their advancements in computer vision, there has been a corresponding growth in parameter size and amounts of training data. Many have come to believe that because of this, transformers are not suitable for small sets of data. This trend leads to concerns such as: limited availability of data in certain scientific domains and the exclusion of those with limited resource from research in the field. In this paper, we aim to present an approach for small-scale learning by introducing Compact Transformers. We show for the first time that with the right size, convolutional tokenization, transformers can avoid overfitting and outperform state-of-the-art CNNs on small datasets. Our models are flexible in terms of model size, and can have as little as 0.28M parameters while achieving competitive results. Our best model can reach 98% accuracy when training from scratch on CIFAR-10 with only 3.7M parameters, which is a significant improvement in data-efficiency over previous Transformer based models being over 10x smaller than other transformers and is 15% the size of ResNet50 while achieving similar performance. CCT also outperforms many modern CNN based approaches, and even some recent NAS-based approaches. Additionally, we obtain a new SOTA result on Flowers-102 with 99.76% top-1 accuracy, and improve upon the existing baseline on ImageNet (82.71% accuracy with 29% as many parameters as ViT), as well as NLP tasks. Our simple and compact design for transformers makes them more feasible to study for those with limited computing resources and/or dealing with small datasets, while extending existing research efforts in data efficient transformers. Our code and pre-trained models are publicly available at https://github.com/SHI-Labs/Compact-Transformers.
ALT-MAS: A Data-Efficient Framework for Active Testing of Machine Learning Algorithms

Machine learning models are being used extensively in many important areas, but there is no guarantee a model will always perform well or as its developers intended. Understanding the correctness of a model is crucial to prevent potential failures that may have significant detrimental impact in critical application areas. In this paper, we propose a novel framework to efficiently test a machine learning model using only a small amount of labeled test data. The idea is to estimate the metrics of interest for a model-under-test using Bayesian neural network (BNN). We develop a novel data augmentation method helping to train the BNN to achieve high accuracy. We also devise a theoretic information based sampling strategy to sample data points so as to achieve accurate estimations for the metrics of interest. Finally, we conduct an extensive set of experiments to test various machine learning models for different types of metrics. Our experiments show that the metrics estimations by our method are significantly better than existing baselines.
The structure of online social networks modulates the rate of lexical change

New words are regularly introduced to communities, yet not all of these words persist in a community’s lexicon. Among the many factors contributing to lexical change, we focus on the understudied effect of social networks. We conduct a large-scale analysis of over 80k neologisms in 4420 online communities across a decade. Using Poisson regression and survival analysis, our study demonstrates that the community’s network structure plays a significant role in lexical change. Apart from overall size, properties including dense connections, the lack of local clusters, and more external contacts promote lexical innovation and retention. Unlike offline communities, these topic-based communities do not experience strong lexical leveling despite increased contact but accommodate more niche words. Our work provides support for the sociolinguistic hypothesis that lexical change is partially shaped by the structure of the underlying network but also uncovers findings specific to online communities.
Macro-Average: Rare Types Are Important Too

While traditional corpus-level evaluation metrics for machine translation (MT) correlate well with fluency, they struggle to reflect adequacy. Model-based MT metrics trained on segment-level human judgments have emerged as an attractive replacement due to strong correlation results. These models, however, require potentially expensive re-training for new domains and languages. Furthermore, their decisions are inherently non-transparent and appear to reflect unwelcome biases. We explore the simple type-based classifier metric, MacroF1, and study its applicability to MT evaluation. We find that MacroF1 is competitive on direct assessment, and outperforms others in indicating downstream cross-lingual information retrieval task performance. Further, we show that MacroF1 can be used to effectively compare supervised and unsupervised neural machine translation, and reveal significant qualitative differences in the methods’ outputs.
Comparing the Benefit of Synthetic Training Data for Various Automatic Speech Recognition Architectures

Recent publications on automatic-speech-recognition (ASR) have a strong focus on attention encoder-decoder (AED) architectures which tend to suffer from over-fitting in low resource scenarios. One solution to tackle this issue is to generate synthetic data with a trained text-to-speech system (TTS) if additional text is available. This was successfully applied in many publications with AED systems, but only very limited in the context of other ASR architectures. We investigate the effect of varying pre-processing, the speaker embedding and input encoding of the TTS system w.r. t. the effectiveness of the synthesized data for AED-ASR training. Additionally, we also consider internal language model subtraction for the first time, resulting in up to 38% relative improvement. We compare the AED results to a state-of-the-art hybrid ASR system, a monophone based system using connectionist-temporal-classification (CTC) and a monotonic transducer based system. We show that for the later systems the addition of synthetic data has no relevant effect, but they still outperform the AED systems on LibriSpeech-100h. We achieve a final word-error-rate of 3.3%/10.0% with a hybrid system on the clean/noisy test-sets, surpassing any previous state-of-the-art systems on Librispeech-100h that do not include unlabeled audio data.
NorDial: A Preliminary Corpus of Written Norwegian Dialect Use

Norway has a large amount of dialectal variation, as well as a general tolerance to its use in the public sphere. There are, however, few available resources to study this variation and its change over time and in more informal areas, on social media. In this paper, we propose a first step to creating a corpus of dialectal variation of written Norwegian. We collect a small corpus of tweets and manually annotate them as Bokmål, Nynorsk, any dialect, or a mix. We further perform preliminary experiments with state-of-the-art models, as well as an analysis of the data to expand this corpus in the future. Finally, we make the annotations available for future work.
Semi-Infinite Linear Regression and Its Applications

Finite linear least squares is one of the core problems of numerical linear algebra, with countless applications across science and engineering. Consequently, there is a rich and ongoing literature on algorithms for solving linear least squares problems. In this paper, we explore a variant in which the system's matrix has one infinite dimension (i.e., it is a quasimatrix). We call such problems semi-infinite linear regression problems. As we show, the semi-infinite case arises in several applications, such as supervised learning and function approximation, and allows for novel interpretations of existing algorithms. We explore semi-infinite linear regression rigorously and algorithmically. To that end, we give a formal framework for working with quasimatrices, and generalize several algorithms designed for the finite problem to the infinite case. Finally, we suggest the use of various sampling methods for obtaining an approximate solution.