Miracle-dz/UniFAR-FTUs huggingface dataset:
Revealing Ionization Conditions of Sz 102 with Spatially Resolved [Ne III] Microjets

Forbidden neon emission lines from small-scale microjets can probe high-energy processes in low-mass young stellar systems. We obtained spatially resolved [Ne iii] spectra in the microjets from the classical T Tauri Star Sz 102 using the Hubble Space Telescope Imaging Spectrograph at a spatial resolution of ∼0. ″1. The blueshifted and redshifted [Ne iii] emission both peak in intensity within ∼0. ″1 of the star and gradually decay along the flow outward to ∼0. ″24. The spatial distribution and extent of the [Ne iii] microjet is consistent with a jet that is ionized close to the base and subsequently recombines on a longer timescale than the flow time. Ca ii H and K lines are also detected from the redshifted microjet with a line full-width at half-maximum of ∼170 km s−1, consistent with those of other forbidden emission lines, atop a 300 km s−1 wide stellar component. The launching radius of the Sz 102 jet, inferred from the observed line centroids and the range of inclination angles and stellar masses from the literature, is on the order of ∼0.03 au. The possible proximity of the launching region to the star allows immediate ionization without distance dilution from the circumstellar ionization sources, most likely keV X-ray flares generated by magnetic reconnection events in the star--disk system, to sustain the observed [Ne iii] flux.
BinaryBERT: Pushing the Limit of BERT Quantization

The rapid development of large pre-trained language models has greatly increased the demand for model compression techniques, among which quantization is a popular solution. In this paper, we propose BinaryBERT, which pushes BERT quantization to the limit by weight binarization. We find that a binary BERT is hard to be trained directly than a ternary counterpart due to its complex and irregular loss landscape. Therefore, we propose ternary weight splitting, which initializes BinaryBERT by equivalently splitting from a half-sized ternary network. The binary model thus inherits the good performance of the ternary one, and can be further enhanced by fine-tuning the new architecture after splitting. Empirical results show that our BinaryBERT has only a slight performance drop compared with the full-precision model while being 24x smaller, achieving the state-of-the-art compression results on the GLUE and SQuAD benchmarks. Code will be released.
Average-distance problem with curvature penalization for data parameterization: regularity of minimizers

We propose a model for finding one-dimensional structure in a given measure. Our approach is based on minimizing an objective functional which combines the average-distance functional to measure the quality of the approximation and penalizes the curvature, similarly to the elastica functional. Introducing the curvature penalization overcomes some of the shortcomings of the average-distance functional, in particular the lack of regularity of minimizers. We establish existence, uniqueness and regularity of minimizers of the proposed functional. In particular we establish C1,1 estimates on the minimizers.
Semi-parametric estimation of incubation and generation times by means of Laguerre polynomials

In epidemics many interesting quantities, like the reproduction number, depend on the incubation period (time from infection to symptom onset) and/or the generation time (time until a new person is infected from another infected person). Therefore, estimation of the distribution of these two quantities is of distinct interest. However, this is a challenging problem since it is normally not possible to obtain precise observations of these two variables. Instead, in the beginning of a pandemic, it is possible to observe for transmission pairs the time of symptom onset for both people as well as a window for infection of the first person (e.g. because of travel to a risk area). In this paper we suggest a simple semi-parametric sieve-estimation method based on Laguerre-Polynomials for estimation of these distributions. We provide detailed theory for consistency and illustrate the finite sample performance for small datasets via a simulation study.
HateCheck: Functional Tests for Hate Speech Detection Models

Detecting online hate is a difficult task that even state-of-the-art models struggle with. Typically, hate speech detection models are evaluated by measuring their performance on held-out test data using metrics such as accuracy and F1 score. However, this approach makes it difficult to identify specific model weak points. It also risks overestimating generalisable model performance due to increasingly well-evidenced systematic gaps and biases in hate speech datasets. To enable more targeted diagnostic insights, we introduce HateCheck, a suite of functional tests for hate speech detection models. We specify 29 model functionalities motivated by a review of previous research and a series of interviews with civil society stakeholders. We craft test cases for each functionality and validate their quality through a structured annotation process. To illustrate HateCheck’s utility, we test near-state-of-the-art transformer models as well as two popular commercial models, revealing critical model weaknesses.
Is Pessimism Provably Efficient for Offline RL?

We study offline reinforcement learning (RL), which aims to learn an optimal policy based on a dataset collected a priori. Due to the lack of further interactions with the environment, offline RL suffers from the insufficient coverage of the dataset, which eludes most existing theoretical analysis. In this paper, we propose a pessimistic variant of the value iteration algorithm (PEVI), which incorporates an uncertainty quantifier as the penalty function. Such a penalty function simply flips the sign of the bonus function for promoting exploration in online RL, which makes it easily implementable and compatible with general function approximators. Without assuming the sufficient coverage of the dataset, we establish a data-dependent upper bound on the suboptimality of PEVI for general Markov decision processes (MDPs). When specialized to linear MDPs, it matches the information-theoretic lower bound up to multiplicative factors of the dimension and horizon. In other words, pessimism is not only provably efficient but also minimax optimal. In particular, given the dataset, the learned policy serves as the"best effort"among all policies, as no other policies can do better. Our theoretical analysis identifies the critical role of pessimism in eliminating a notion of spurious correlation, which emerges from the"irrelevant"trajectories that are less covered by the dataset and not informative for the optimal policy.
ERICA: Improving Entity and Relation Understanding for Pre-trained Language Models via Contrastive Learning

Pre-trained Language Models (PLMs) have shown superior performance on various downstream Natural Language Processing (NLP) tasks. However, conventional pre-training objectives do not explicitly model relational facts in text, which are crucial for textual understanding. To address this issue, we propose a novel contrastive learning framework ERICA to obtain a deep understanding of the entities and their relations in text. Specifically, we define two novel pre-training tasks to better understand entities and relations: (1) the entity discrimination task to distinguish which tail entity can be inferred by the given head entity and relation; (2) the relation discrimination task to distinguish whether two relations are close or not semantically, which involves complex relational reasoning. Experimental results demonstrate that ERICA can improve typical PLMs (BERT and RoBERTa) on several language understanding tasks, including relation extraction, entity typing and question answering, especially under low-resource settings.
A Letter Highlighting Matrix Mapping in Minimal 4D, ${\mathbf {\cal N}}$ = 1 On-Shell Supermultiplet Representations

On the basis of comparing eigenvalues of an operator Ĉ(R), that proved useful in distinguishing how off-shell 4D, N = 1 supermultiplets become offshell 4D, N = 2 supermultiplets, the double tensor supermultiplet is shown to be radically different for other known multiplets. This suggests difficulties, if not impossibilities, to embed it into an off-shell structure. PACS: 11.30. Pb, 12.60. Jv
A Shape Newton Scheme for Deforming Shells with Application to Capillary Bridges

We present a second order numerical scheme to compute capillary bridges between arbitrary solids by minimizing the total energy of all interfaces. From a theoretical point of view, this approach can be interpreted as the computation of generalized minimal surfaces using a Newton-scheme utilizing the shape Hessian. In particular, we give an explicit representation of the shape Hessian for functionals on shells involving the normal vector without reverting back to a volume formulation or approximating curvature. From an algorithmic perspective, we combine a resolved interface via a triangulated surface for the liquid with a level set description for the constraints stemming from the arbitrary geometry. The actual shape of the capillary bridge is then computed via finite elements provided by the FEniCS environment, minimizing the shape derivative of the total interface energy.
Topological obstructions in neural networks learning

We apply topological data analysis methods to loss functions to gain insights into learning of deep neural networks and deep neural networks generalization properties. We use the Morse complex of the loss function to relate the local behavior of gradient descent trajectories with global properties of the loss surface. We define the neural network Topological Obstructions score,"TO-score", with the help of robust topological invariants, barcodes of the loss function, that quantify the"badness"of local minima for gradient-based optimization. We have made experiments for computing these invariants for fully-connected, convolutional and ResNet-like neural networks on different datasets: MNIST, Fashion MNIST, CIFAR10, CIFAR100 and SVHN. Our two principal observations are as follows. Firstly, the neural network barcode and TO score decrease with the increase of the neural network depth and width, thus the topological obstructions to learning diminish. Secondly, in certain situations there is an intriguing connection between the lengths of minima segments in the barcode and the minima generalization errors.
The APOGEE Library of Infrared SSP Templates (A-LIST): High-resolution Simple Stellar Population Spectral Models in the H Band

Integrated light spectroscopy from galaxies can be used to study the stellar populations that cannot be resolved into individual stars. This analysis relies on stellar population synthesis (SPS) techniques to study the formation history and structure of galaxies. However, the spectral templates available for SPS are limited, especially in the near-infrared (near-IR). We present A-LIST (APOGEE Library of Infrared SSP Templates), a new set of high-resolution, near-IR SSP spectral templates spanning a wide range of ages (2–12 Gyr), metallicities ( − 2.2 < [M/H] < + 0.4) and α abundances ( − 0.2 < [α/M] < + 0.4). This set of SSP templates is the highest resolution (R ∼ 22, 500) available in the near-IR, and the first such based on an empirical stellar library. Our models are generated using spectra of ∼300,000 stars spread across the Milky Way, with a wide range of metallicities and abundances, from the APOGEE survey. We show that our model spectra provide accurate fits to M31 globular cluster spectra taken with APOGEE, with best-fit metallicities agreeing with those of previous estimates to within ∼0.1 dex. We also compare these model spectra to lower-resolution E-MILES models and demonstrate that we recover the ages of these models to within ∼1.5 Gyr. This library is available in https://github.com/aishashok/ALIST-library.
Accelerating Pre-trained Language Models via Calibrated Cascade

Dynamic early exiting aims to accelerate pre-trained language models’ (PLMs) inference by exiting in shallow layer without passing through the entire model. In this paper, we analyze the working mechanism of dynamic early exiting and ﬁnd it cannot achieve a satisfying trade-off between inference speed and performance. On one hand, the PLMs’ representations in shallow layers are not sufﬁ-cient for accurate prediction. One the other hand, the internal off-ramps cannot provide reliable exiting decisions. To remedy this, we instead propose CascadeBERT, which dynamically selects a proper-sized, complete model in a cascading manner. To obtain more reliable model selection, we further devise a difﬁculty-aware objective, encouraging the model output class probability to reﬂect the real difﬁculty of each instance. Extensive experimental re-sults demonstrate the superiority of our proposal over strong baseline models of PLMs’ acceleration including both dynamic early exiting and knowledge distillation methods.
Almost sure large fluctuations of random multiplicative functions

We prove that if f ( n ) is a Steinhaus or Rademacher random multiplicative function, there almost surely exist arbitrarily large values of x for which | (cid:80) n ≤ x f ( n ) | ≥ √ x (log log x ) 1 / 4+ o (1) . This is the ﬁrst such bound that grows faster than √ x , answering a question of Hal´asz and proving a conjecture of Erd˝os. It is plausible that the exponent 1 / 4 is sharp in this problem. The proofs work by establishing a multivariate Gaussian approximation for the sums (cid:80) n ≤ x f ( n ) at a sequence of x , conditional on the behaviour of f ( p ) for all except the largest primes p . The most diﬃcult aspect is showing that the conditional covariances of the sums are usually small, so the corresponding Gaussians are usually roughly independent. These covariances are related to an Euler product (or multiplicative chaos) type integral twisted by additive characters, which we study using various tools including mean value estimates for Dirichlet polynomials, high mixed moment estimates for random Euler products, and barrier arguments with random walks.
First-order conditions for the optimal control of the obstacle problem with state constraints

We consider an optimal control problem in which the state is governed by an unilateral obstacle problem (with obstacle from below) and restricted by a pointwise state constraint (from above). In the presence of control constraints, we prove, via regularization of the state constraints, that a system of C-stationarity is necessary for optimality. In the absence of control constraints, we show that local minimizers are even strongly stationary by a careful discussion of the primal first-order conditions of B-stationary type.
Improving Unsupervised Domain Adaptation by Reducing Bi-level Feature Redundancy

Reducing feature redundancy has shown beneficial effects for improving the accuracy of deep learning models, thus it is also indispensable for the models of unsupervised domain adaptation (UDA). Nevertheless, most recent efforts in the field of UDA ignores this point. Moreover, main schemes realizing this in general independent of UDA purely involve a single domain, thus might not be effective for cross-domain tasks. In this paper, we emphasize the significance of reducing feature redundancy for improving UDA in a bi-level way. For the first level, we try to ensure compact domain-specific features with a transferable decorrelated normalization module, which preserves specific domain information whilst easing the side effect of feature redundancy on the sequel domain-invariance. In the second level, domain-invariant feature redundancy caused by domain-shared representation is further mitigated via an alternative brand orthogonality for better generalization. These two novel aspects can be easily plugged into any BN-based backbone neural networks. Specifically, simply applying them to ResNet50 has achieved competitive performance to the state-of-the-arts on five popular benchmarks. Our code will be available at https://github.com/dreamkily/gUDA.
Making Pre-trained Language Models Better Few-shot Learners

The recent GPT-3 model (Brown et al., 2020) achieves remarkable few-shot performance solely by leveraging a natural-language prompt and a few task demonstrations as input context. Inspired by their findings, we study few-shot learning in a more practical scenario, where we use smaller language models for which fine-tuning is computationally efficient. We present LM-BFF—better few-shot fine-tuning of language models—a suite of simple and complementary techniques for fine-tuning language models on a small number of annotated examples. Our approach includes (1) prompt-based fine-tuning together with a novel pipeline for automating prompt generation; and (2) a refined strategy for dynamically and selectively incorporating demonstrations into each context. Finally, we present a systematic evaluation for analyzing few-shot performance on a range of NLP tasks, including classification and regression. Our experiments demonstrate that our methods combine to dramatically outperform standard fine-tuning procedures in this low resource setting, achieving up to 30% absolute improvement, and 11% on average across all tasks. Our approach makes minimal assumptions on task resources and domain expertise, and hence constitutes a strong task-agnostic method for few-shot learning.
Universal Silicon Microwave Photonic Spectral Shaper

Optical modulation plays arguably the utmost important role in microwave photonic (MWP) systems. Precise synthesis of modulated optical spectra dictates virtually all aspects of MWP system quality including loss, noise figure, linearity, and the types of functionality that can be executed. But for such a critical function, the versatility to generate and transform analog optical modulation is severely lacking, blocking the pathways to truly unique MWP functions including ultra-linear links and low-loss high rejection filters. Here we demonstrate versatile RF photonic spectrum synthesis in an all-integrated silicon photonic circuit, enabling electrically-tailorable universal analog modulation transformation. We show a series of unprecedented RF filtering experiments through monolithic integration of the spectrum-synthesis circuit with a network of reconfigurable ring resonators.
Intrinsic Bias Metrics Do Not Correlate with Application Bias

Natural Language Processing (NLP) systems learn harmful societal biases that cause them to amplify inequality as they are deployed in more and more situations. To guide efforts at debiasing these systems, the NLP community relies on a variety of metrics that quantify bias in models. Some of these metrics are intrinsic, measuring bias in word embedding spaces, and some are extrinsic, measuring bias in downstream tasks that the word embeddings enable. Do these intrinsic and extrinsic metrics correlate with each other? We compare intrinsic and extrinsic metrics across hundreds of trained models covering different tasks and experimental conditions. Our results show no reliable correlation between these metrics that holds in all scenarios across tasks and languages. We urge researchers working on debiasing to focus on extrinsic measures of bias, and to make using these measures more feasible via creation of new challenge sets and annotated test data. To aid this effort, we release code, a new intrinsic metric, and an annotated test set focused on gender bias in hate speech.
Generating Wikipedia Article Sections from Diverse Data Sources

Datasets for data-to-text generation typically focus either on multi-domain, single-sentence generation or on single-domain, long-form generation. In this work, we create a large-scale dataset, W IKI T ABLE T, that pairs Wikipedia sections with their corresponding tabular data and various metadata. W IKI - T ABLE T contains millions of instances, covering a broad range of topics, as well as a variety of ﬂavors of generation tasks with different levels of ﬂexibility. We benchmark several training and decoding strategies on W IKI T ABLE T. Our qualitative analysis shows that the best approaches can generate ﬂuent and high quality texts but they sometimes struggle with coherence. 1
Why does individual learning endure when crowds are wiser?

The ability to learn from others (social learning) is often deemed a cause of human species success. But if social learning is indeed more efficient (whether less costly or more accurate) than individual learning, it raises the question of why would anyone engage in individual information seeking, which is a necessary condition for social learning's efficacy. We propose an evolutionary model solving this paradox, provided agents (i) aim not only at information quality but also vie for audience and prestige, and (ii) do not only value accuracy but also reward originality -- allowing them to alleviate herding effects. We find that under some conditions (large enough success rate of informed agents and intermediate taste for popularity), both social learning's higher accuracy and the taste for original opinions are evolutionary-stable, within a mutually beneficial division of labour-like equilibrium. When such conditions are not met, the system most often converges towards mutually detrimental equilibria.
COIN: Contrastive Identifier Network for Breast Mass Diagnosis in Mammography

Computer-aided breast cancer diagnosis in mammography is a challenging problem, stemming from mammographical data scarcity and data entanglement. In particular, data scarcity is attributed to the privacy and expensive annotation. And data entanglement is due to the high similarity between benign and malignant masses, of which manifolds reside in lower dimensional space with very small margin. To address these two challenges, we propose a deep learning framework, named Contrastive Identifier Network (\textsc{COIN}), which integrates adversarial augmentation and manifold-based contrastive learning. Firstly, we employ adversarial learning to create both on- and off-distribution mass contained ROIs. After that, we propose a novel contrastive loss with a built Signed graph. Finally, the neural network is optimized in a contrastive learning manner, with the purpose of improving the deep model's discriminativity on the extended dataset. In particular, by employing COIN, data samples from the same category are pulled close whereas those with different labels are pushed further in the deep latent space. Moreover, COIN outperforms the state-of-the-art related algorithms for solving breast cancer diagnosis problem by a considerable margin, achieving 93.4\% accuracy and 95.0\% AUC score. The code will release on ***.
Testing models of periodically modulated FRB activity

The activity of the repeating FRB 20180916B is periodically modulated with a period of 16.3 d, and FRB 121102 may be similarly modulated with a period of about 160 d. In some models of this modulation, the period derivative is insensitive to the uncertain parameters; these models can be tested by measurement of or bounds on the derivative. In other models, values of the uncertain parameters can be constrained. Periodic modulation of aperiodic bursting activity may result from emission by a narrow beam wandering within a cone or funnel along the axis of a precessing disc, such as the accretion discs in X-ray binaries. The production of FRB 200428 by a neutron star that is neither accreting nor in a binary then shows universality: coherent emission occurring in a wide range of circumstances.