Miracle-dz/UniFAR-FTUs huggingface dataset:
Discovery of a directly imaged planet to the young solar analog YSES 2

Context. To understand the origin and formation pathway of wide-orbit gas giant planets, it is necessary to expand the limited sample of these objects. The mass of exoplanets derived with spectrophotometry, however, varies strongly as a function of the age of the system and the mass of the primary star. Aims. By selecting stars with similar ages and masses, the Young Suns Exoplanet Survey (YSES) aims to detect and characterize planetary-mass companions to solar-type host stars in the Scorpius-Centaurus association. Methods. Our survey is carried out with VLT/SPHERE with short exposure sequences on the order of 5 min per star per filter. The subtraction of the stellar point spread function (PSF) is based on reference star differential imaging using the other targets (with similar colors and magnitudes) in the survey in combination with principal component analysis. Two astrometric epochs that are separated by more than one year are used to confirm co-moving companions by proper motion analysis. Results. We report the discovery of YSES 2b, a co-moving, planetary-mass companion to the K1 star YSES 2 (TYC 8984-2245-1, 2MASS J11275535-6626046). The primary has a Gaia EDR3 distance of 110 pc, and we derive a revised mass of 1.1 M⊙ and an age of approximately 14 Myr. We detect the companion in two observing epochs southwest of the star at a position angle of 205° and with a separation of ~1. ′′05, which translates to a minimum physical separation of 115 au at the distance of the system. Photometric measurements in the H and Ks bands are indicative of a late L spectral type, similar to the innermost planets around HR 8799. We derive a photometric planet mass of 6.3−0.9+1.6 MJup using AMES-COND and AMES-dusty evolutionary models; this mass corresponds to a mass ratio of q = (0.5 ± 0.1)% with the primary. This is the lowest mass ratio of a direct imaging planet around a solar-type star to date. We discuss potential formation mechanisms and find that the current position of the planet is compatible with formation by disk gravitational instability, but its mass is lower than expected from numerical simulations. Formation via core accretion must have occurred closer to the star, yet we do not find evidence that supports the required outward migration, such as via scattering off another undiscovered companion in the system. We can exclude additional companions with masses greater than 13 MJup in the full field of view of the detector (0.′′152′′ we are sensitive to planets with masses as low as 2 MJup. Conclusions. YSES 2b is an ideal target for follow-up observations to further the understanding of the physical and chemical formation mechanisms of wide-orbit Jovian planets. The YSES strategy of short snapshot observations (≤5 min) and PSF subtraction based on a large reference library proves to be extremely efficient and should be considered for future direct imaging surveys.
Coarse-to-Fine Searching for Efficient Generative Adversarial Networks

This paper studies the neural architecture search (NAS) problem for developing efficient generator networks. Compared with deep models for visual recognition tasks, generative adversarial network (GAN) are usually designed to conduct various complex image generation. We first discover an intact search space of generator networks including three dimensionalities, i.e., path, operator, channel for fully excavating the network performance. To reduce the huge search cost, we explore a coarse-to-fine search strategy which divides the overall search process into three sub-optimization problems accordingly. In addition, a fair supernet training approach is utilized to ensure that all sub-networks can be updated fairly and stably. Experiments results on benchmarks show that we can provide generator networks with better image quality and lower computational costs over the state-of-the-art methods. For example, with our method, it takes only about 8 GPU hours on the entire edges-to-shoes dataset to get a 2.56 MB model with a 24.13 FID score and 10 GPU hours on the entire Urban100 dataset to get a 1.49 MB model with a 24.94 PSNR score.
GPT3Mix: Leveraging Large-scale Language Models for Text Augmentation

Large-scale language models such as GPT-3 are excellent few-shot learners, allowing them to be controlled via natural text prompts. Recent studies report that prompt-based direct classification eliminates the need for fine-tuning but lacks data and inference scalability. This paper proposes a novel data augmentation technique that leverages large-scale language models to generate realistic text samples from a mixture of real samples. We also propose utilizing soft-labels predicted by the language models, effectively distilling knowledge from the large-scale language models and creating textual perturbations simultaneously. We perform data augmentation experiments on diverse classification tasks and show that our method hugely outperforms existing text augmentation methods. Ablation studies and a qualitative analysis provide more insights into our approach.
Classification of head impacts based on the spectral density of measurable kinematics

Traumatic brain injury can be caused by head impacts, but many brain injury risk estimation models are less accurate across the variety of impacts that patients may undergo. In this study, we investigated the spectral characteristics of different head impact types with kinematics classification. Data was analyzed from 3262 head impacts from head model simulations, on-field data from American football and mixed martial arts (MMA) using our instrumented mouthguard, and publicly available car crash data. A random forest classifier with spectral densities of linear acceleration and angular velocity was built to classify different types of head impacts (e.g., football, MMA), reaching a median accuracy of 96% over 1000 random partitions of training and test sets. Furthermore, to test the classifier on data from different measurement devices, another 271 lab-reconstructed impacts were obtained from 5 other instrumented mouthguards with the classifier reaching over 96% accuracy from these devices. The most important features in classification included both low-frequency and high-frequency features, both linear acceleration features and angular velocity features. It was found that different head impact types had different distributions of spectral densities in low-frequency and high-frequency ranges (e.g., the spectral densities of MMA impacts were higher in high-frequency range than in the low-frequency range). Finally, with head impact classification, type-specific, nearest-neighbor regression models were built for 95th percentile maximum principal strain, 95th percentile maximum principal strain in corpus callosum, and cumulative strain damage (15th percentile). This showed a generally higher R-value than baseline models without classification.
Finding Closure Terms Directly from Coarse Data for 2D Turbulent Flow

Machine learning is used to develop closure terms for coarse grained model of two-dimensional turbulent flow directly from the coarse grained data by adding a source term to the Navier-Stokes equations to ensure that the coarse-grained flow evolves in the correct way. The source term is related to the average flow using a Neural Network with a relatively simple structure and smoothed slightly to prevent instabilities in a posteriori test. The time dependent coarse grained flow field is generated by filtering fully resolved results and the predicted coarse field evolution agrees well with the filtered results, both for the flow used to learn the closure terms and for flows not used
PARE: Part Attention Regressor for 3D Human Body Estimation

Despite significant progress, we show that state of the art 3D human pose and shape estimation methods remain sensitive to partial occlusion and can produce dramatically wrong predictions although much of the body is observable. To address this, we introduce a soft attention mechanism, called the Part Attention REgressor (PARE), that learns to predict body-part-guided attention masks. We observe that state-of-the-art methods rely on global feature representations, making them sensitive to even small occlusions. In contrast, PARE’s part-guided attention mechanism overcomes these issues by exploiting information about the visibility of individual body parts while leveraging information from neighboring body-parts to predict occluded parts. We show qualitatively that PARE learns sensible attention masks, and quantitative evaluation confirms that PARE achieves more accurate and robust reconstruction results than existing approaches on both occlusion-specific and standard benchmarks. The code and data are available for research purposes at https://pare.is. tue.mpg. de/
Risk prediction models for discrete ordinal outcomes: Calibration and the impact of the proportional odds assumption

Calibration is a vital aspect of the performance of risk prediction models, but research in the context of ordinal outcomes is scarce. This study compared calibration measures for risk models predicting a discrete ordinal outcome, and investigated the impact of the proportional odds assumption on calibration and overfitting. We studied the multinomial, cumulative, adjacent category, continuation ratio, and stereotype logit/logistic models. To assess calibration, we investigated calibration intercepts and slopes, calibration plots, and the estimated calibration index. Using large sample simulations, we studied the performance of models for risk estimation under various conditions, assuming that the true model has either a multinomial logistic form or a cumulative logit proportional odds form. Small sample simulations were used to compare the tendency for overfitting between models. As a case study, we developed models to diagnose the degree of coronary artery disease (five categories) in symptomatic patients. When the true model was multinomial logistic, proportional odds models often yielded poor risk estimates, with calibration slopes deviating considerably from unity even on large model development datasets. The stereotype logistic model improved the calibration slope, but still provided biased risk estimates for individual patients. When the true model had a cumulative logit proportional odds form, multinomial logistic regression provided biased risk estimates, although these biases were modest. Nonproportional odds models require more parameters to be estimated from the data, and hence suffered more from overfitting. Despite larger sample size requirements, we generally recommend multinomial logistic regression for risk prediction modeling of discrete ordinal outcomes.
A multilabel approach to morphosyntactic probing

We introduce a multilabel probing task to assess the morphosyntactic representations of word embeddings from multilingual language models. We demonstrate this task with multilingual BERT (Devlin et al., 2018), training probes for seven typologically diverse languages of varying morphological complexity: Afrikaans, Croatian, Finnish, Hebrew, Korean, Spanish, and Turkish. Through this simple but robust paradigm, we show that multilingual BERT renders many morphosyntactic features easily and simultaneously extractable (e.g., gender, grammatical case, pronominal type). We further evaluate the probes on six"held-out"languages in a zero-shot transfer setting: Arabic, Chinese, Marathi, Slovenian, Tagalog, and Yoruba. This style of probing has the added benefit of revealing the linguistic properties that language models recognize as being shared across languages. For instance, the probes performed well on recognizing nouns in the held-out languages, suggesting that multilingual BERT has a conception of noun-hood that transcends individual languages; yet, the same was not true of adjectives.
Electron transport in dual-gated three-layer 
MoS2

Michele Masseroni, Tim Davatz, Riccardo Pisoni, Folkert K. de Vries, Peter Rickhaus, Takashi Taniguchi, Kenji Watanabe, Vladimir Fal’ko, Thomas Ihn, and Klaus Ensslin Solid State Physics Laboratory, ETH Zürich, 8093 Zürich, Switzerland International Center for Materials Nanoarchitectonics, 1-1 Namiki, Tsukuba 305-0044, Japan Research Center for Functional Materials, 1-1 Namiki, Tsukuba 305-0044, Japan National Graphene Institute, University of Manchester, Booth St. E. Manchester M13 9PL, United Kingdom (Dated: April 30, 2021)
Surface Form Competition: Why the Highest Probability Answer Isn’t Always Right

Large language models have shown promising results in zero-shot settings. For example, they can perform multiple choice tasks simply by conditioning on a question and selecting the answer with the highest probability. However, ranking by string probability can be problematic due to surface form competition—wherein different surface forms compete for probability mass, even if they represent the same underlying concept in a given context, e.g. “computer” and “PC. ” Since probability mass is finite, this lowers the probability of the correct answer, due to competition from other strings that are valid answers (but not one of the multiple choice options). We introduce Domain Conditional Pointwise Mutual Information, an alternative scoring function that directly compensates for surface form competition by simply reweighing each option according to its a priori likelihood within the context of a specific task. It achieves consistent gains in zero-shot performance over both calibrated and uncalibrated scoring functions on all GPT-2 and GPT-3 models on a variety of multiple choice datasets.
Cycle-Free CycleGAN Using Invertible Generator for Unsupervised Low-Dose CT Denoising

Recently, CycleGAN was shown to provide high-performance, ultra-fast denoising for low-dose X-ray computed tomography (CT) without the need for a paired training dataset. Although this was possible thanks to cycle consistency, CycleGAN requires two generators and two discriminators to enforce cycle consistency, demanding significant GPU resources and technical skills for training. A recent proposal of switchable CycleGAN with Adaptive Instance Normalization (AdaIN) alleviates the problem in part by using a single generator. However, two discriminators and an additional AdaIN code generator are still required for training. To solve this problem, here we present a novel cycle-free CycleGAN architecture, which consists of a single generator and a discriminator but still guarantees the cycle consistency. The main innovation comes from the observation that the use of an invertible generator automatically fulfills the cycle consistency condition and eliminates the additional discriminator in the CycleGAN formulation. Extensive experiments using various levels of low-dose CT images confirm that our method can improve denoising performance using only 24% of learnable parameters, twice faster training time, and showed better trade-offs between the complexity versus reconstruction quality, compared to the existing methods.
Accelerating Sparse Deep Neural Networks

As neural network model sizes have dramatically increased, so has the interest in various techniques to reduce their parameter counts and accelerate their execution. An active area of research in this field is sparsity - encouraging zero values in parameters that can then be discarded from storage or computations. While most research focuses on high levels of sparsity, there are challenges in universally maintaining model accuracy as well as achieving significant speedups over modern matrix-math hardware. To make sparsity adoption practical, the NVIDIA Ampere GPU architecture introduces sparsity support in its matrix-math units, Tensor Cores. We present the design and behavior of Sparse Tensor Cores, which exploit a 2:4 (50%) sparsity pattern that leads to twice the math throughput of dense matrix units. We also describe a simple workflow for training networks that both satisfy 2:4 sparsity pattern requirements and maintain accuracy, verifying it on a wide range of common tasks and model architectures. This workflow makes it easy to prepare accurate models for efficient deployment on Sparse Tensor Cores.
Simple and Efficient ways to Improve REALM

Dense retrieval has been shown to be effective for Open Domain Question Answering, surpassing sparse retrieval methods like BM25. One such model, REALM, (Guu et al., 2020) is an end-to-end dense retrieval system that uses MLM based pretraining for improved downstream QA performance. However, the current REALM setup uses limited resources and is not comparable in scale to more recent systems, contributing to its lower performance. Additionally, it relies on noisy supervision for retrieval during fine-tuning. We propose REALM++, where we improve upon the training and inference setups and introduce better supervision signal for improving performance, without any architectural changes. REALM++ achieves ~5.5% absolute accuracy gains over the baseline while being faster to train. It also matches the performance of large models which have 3x more parameters demonstrating the efficiency of our setup.
SimCSE: Simple Contrastive Learning of Sentence Embeddings

This paper presents SimCSE, a simple contrastive learning framework that greatly advances the state-of-the-art sentence embeddings. We first describe an unsupervised approach, which takes an input sentence and predicts itself in a contrastive objective, with only standard dropout used as noise. This simple method works surprisingly well, performing on par with previous supervised counterparts. We find that dropout acts as minimal data augmentation and removing it leads to a representation collapse. Then, we propose a supervised approach, which incorporates annotated pairs from natural language inference datasets into our contrastive learning framework, by using “entailment” pairs as positives and “contradiction” pairs as hard negatives. We evaluate SimCSE on standard semantic textual similarity (STS) tasks, and our unsupervised and supervised models using BERT base achieve an average of 76.3% and 81.6% Spearman’s correlation respectively, a 4.2% and 2.2% improvement compared to previous best results. We also show—both theoretically and empirically—that contrastive learning objective regularizes pre-trained embeddings’ anisotropic space to be more uniform, and it better aligns positive pairs when supervised signals are available.
Nonlinear Functional Modeling Using Neural Networks

Abstract We introduce a new class of nonlinear models for functional data based on neural networks. Deep learning has been very successful in nonlinear modeling, but there has been little work done in the functional data setting. We propose two variations of our framework: a functional neural network with continuous hidden layers, called the Functional Direct Neural Network (FDNN), and a second version that uses basis expansions and continuous hidden layers, called the Functional Basis Neural Network (FBNN). Both are designed explicitly to exploit the structure inherent in functional data. To fit these models we derive a functional gradient based optimization algorithm. The effectiveness of the proposed methods in handling complex functional models is demonstrated by comprehensive simulation studies and real data examples. Supplementary materials for this article are available online.
Concadia: Tackling image accessibility with context

Images have become an integral part of online media. This has enhanced self-expression and the dissemination of knowledge, but it poses serious accessibility challenges. Adequate textual descriptions are rare. Captions are more abundant, but they do not consistently provide the needed descriptive details, and systems trained on such texts inherit these shortcomings. To address this, we introduce the publicly available Wikipedia-based corpus Concadia , which consists of 96,918 images with corresponding English-language descriptions, cap-tions, and surrounding context. We use Conca-dia to further characterize the commonalities and differences between descriptions and cap-tions, and this leads us to the hypothesis that captions, while not substitutes for descriptions, can provide a useful signal for creating effective descriptions. We substantiate this hypothesis by showing that image captioning systems trained on Concadia beneﬁt from having caption embeddings as part of their inputs. These experiments also begin to show how Concadia can be a powerful tool in addressing the underlying accessibility issues posed by image data.
Bayesian Active Learning with Pretrained Language Models

Active Learning (AL) is a method to iteratively select data for annotation from a pool of unlabeled data, aiming to achieve better model performance than random selection. Previous AL approaches in Natural Language Processing (NLP) have been limited to either task-speciﬁc models that are trained from scratch at each iteration using only the labeled data at hand or using off-the-shelf pretrained language models (LMs) that are not adapted effectively to the downstream task. In this paper, we address these limitations by introducing B ALM ; Bayesian Active Learning with pre-trained language Models. We ﬁrst propose to adapt the pretrained LM to the downstream task by continuing training with all the available unlabeled data and then use it for AL. We also suggest a simple yet effective ﬁne-tuning method to ensure that the adapted LM is properly trained in both low and high resource scenarios during AL. We ﬁnally apply Monte Carlo dropout to the downstream model to obtain well-calibrated conﬁdence scores for data selection with uncertainty sampling. Our experiments in ﬁve standard natural language understanding tasks demonstrate that B ALM provides substantial data efﬁciency improvements compared to various combinations of acquisition functions, models and ﬁne-tuning meth-ods proposed in recent AL literature.
A major asymmetric ice trap in a planet-forming disk

Context. The chemistry of planet-forming disks sets the exoplanet atmosphere composition and the prebiotic molecular content. Dust traps are of particular importance as pebble growth and transport are crucial for setting the chemistry where giant planets form. Aims. The asymmetric Oph IRS 48 dust trap located at 60 au radius provides a unique laboratory for studying chemistry in pebble-concentrated environments in warm Herbig disks with gas-to-dust ratios as low as 0.01. Methods. We use deep ALMA Band 7 line observations to search the IRS 48 disk for H2CO and CH3OH line emission, the first steps of complex organic chemistry. Results. We report the detection of seven H2CO and six CH3OH lines with energy levels between 17 and 260 K. The line emission shows a crescent morphology, similar to the dust continuum, suggesting that the icy pebbles play an important role in the delivery of these molecules. Rotational diagrams and line ratios indicate that both molecules originate from warm molecular regions in the disk with temperatures > 100 K and column densities ∼1014 cm−2 or a fractional abundance of ∼10−8 and with H2CO/CH3OH ∼0.2, indicative of ice chemistry. Based on arguments from a physical-chemical model with low gas-to-dust ratios, we propose a scenario where the dust trap provides a huge icy grain reservoir in the disk midplane, or an ‘ice trap’, which can result in high gas-phase abundances of warm complex organic molecules through efficient vertical mixing. Conclusions. This is the first time that complex organic molecules have been clearly linked to the presence of a dust trap. These results demonstrate the importance of including dust evolution and vertical transport in chemical disk models as icy dust concentrations provide important reservoirs for complex organic chemistry in disks.