Miracle-dz/UniFAR-FTUs huggingface dataset:
Study of pair production in inhomogeneous two-color electric fields using the computational quantum field theory

We first demonstrate theoretically that the computational quantum field theory is equivalent to the quantum kinetic theory for pair creation in a spatially homogeneous and time-dependent electric field, then verify numerically their equivalence for pair creation in one-dimensional time-dependent electric fields, and finally investigate detailedly the effects of the field frequency, spatial width, pulse duration, and relative phase on dynamically assisted Schwinger pair production in an inhomogeneous two-color electric field. It is found that the enhancement effect of pair creation is very sensitive to the field frequency and generally very obvious for a shorter field width, a longer pulse duration, and a relative phase of maximizing the field strength. These results can provide a significant reference for the optimal control theory of pair creation which aims to maximize the created pair yield within a given scope of field parameters.
Obtaining Better Static Word Embeddings Using Contextual Embedding Models

The advent of contextual word embeddings — representations of words which incorporate semantic and syntactic information from their context—has led to tremendous improvements on a wide variety of NLP tasks. However, recent contextual models have prohibitively high computational cost in many use-cases and are often hard to interpret. In this work, we demonstrate that our proposed distillation method, which is a simple extension of CBOW-based training, allows to significantly improve computational efficiency of NLP applications, while outperforming the quality of existing static embeddings trained from scratch as well as those distilled from previously proposed methods. As a side-effect, our approach also allows a fair comparison of both contextual and static embeddings via standard lexical evaluation tasks.
Coarse-to-Fine Curriculum Learning

When faced with learning challenging new tasks, humans often follow sequences of steps that allow them to incrementally build up the necessary skills for performing these new tasks. However, in machine learning, models are most often trained to solve the target tasks directly.Inspired by human learning, we propose a novel curriculum learning approach which decomposes challenging tasks into sequences of easier intermediate goals that are used to pre-train a model before tackling the target task. We focus on classification tasks, and design the intermediate tasks using an automatically constructed label hierarchy. We train the model at each level of the hierarchy, from coarse labels to fine labels, transferring acquired knowledge across these levels. For instance, the model will first learn to distinguish animals from objects, and then use this acquired knowledge when learning to classify among more fine-grained classes such as cat, dog, car, and truck. Most existing curriculum learning algorithms for supervised learning consist of scheduling the order in which the training examples are presented to the model. In contrast, our approach focuses on the output space of the model. We evaluate our method on several established datasets and show significant performance gains especially on classification problems with many labels. We also evaluate on a new synthetic dataset which allows us to study multiple aspects of our method.
Provably Adversarially Robust Detection of Out-of-Distribution Data (Almost) for Free

The application of machine learning in safety-critical systems requires a reliable assessment of uncertainty. However, deep neural networks are known to produce highly overconfident predictions on out-of-distribution (OOD) data. Even if trained to be non-confident on OOD data, one can still adversarially manipulate OOD data so that the classifier again assigns high confidence to the manipulated samples. We show that two previously published defenses can be broken by better adapted attacks, highlighting the importance of robustness guarantees around OOD data. Since the existing method for this task is hard to train and significantly limits accuracy, we construct a classifier that can simultaneously achieve provably adversarially robust OOD detection and high clean accuracy. Moreover, by slightly modifying the classifier's architecture our method provably avoids the asymptotic overconfidence problem of standard neural networks. We provide code for all our experiments.
Evaluating low-depth quantum algorithms for time evolution on fermion-boson systems

Simulating time evolution of quantum systems is one of the most promising applications of quantum computing and also appears as a subroutine in many applications such as Green's function methods. In the current era of NISQ machines we assess the state of algorithms for simulating time dynamics with limited resources. We propose the Jaynes-Cummings model and extensions to it as useful toy models to investigate time evolution algorithms on near-term quantum computers. Using these simple models, direct Trotterisation of the time evolution operator produces deep circuits, requiring coherence times out of reach on current NISQ hardware. Therefore we test two alternative responses to this problem: variational compilation of the time evolution operator, and variational quantum simulation of the wavefunction ansatz. We demonstrate numerically to what extent these methods are successful in time evolving this system. The costs in terms of circuit depth and number of measurements are compared quantitatively, along with other drawbacks and advantages of each method. We find that computational requirements for both methods make them suitable for performing time evolution simulations of our models on NISQ hardware. Our results also indicate that variational quantum compilation produces more accurate results than variational quantum simulation, at the cost of a larger number of measurements.
MViT: Mask Vision Transformer for Facial Expression Recognition in the wild

Facial Expression Recognition (FER) in the wild is an extremely challenging task in computer vision due to variant backgrounds, low-quality facial images, and the subjectiveness of annotators. These uncertainties make it difficult for neural networks to learn robust features on limited-scale datasets. Moreover, the networks can be easily distributed by the above factors and perform incorrect decisions. Recently, vision transformer (ViT) and data-efficient image transformers (DeiT) present their significant performance in traditional classification tasks. The self-attention mechanism makes transformers obtain a global receptive field in the first layer which dramatically enhances the feature extraction capability. In this work, we first propose a novel pure transformer-based mask vision transformer (MVT) for FER in the wild, which consists of two modules: a transformer-based mask generation network (MGN) to generate a mask that can filter out complex backgrounds and occlusion of face images, and a dynamic relabeling module to rectify incorrect labels in FER datasets in the wild. Extensive experimental results demonstrate that our MVT outperforms state-of-the-art methods on RAF-DB with 88.62%, FERPlus with 89.22%, and AffectNet-7 with 64.57%, respectively, and achieves a comparable result on AffectNet-8 with 61.40%.
Description and Discussion on DCASE 2021 Challenge Task 2: Unsupervised Anomalous Sound Detection for Machine Condition Monitoring under Domain Shifted Conditions

We present the task description and discussion on the results of the DCASE 2021 Challenge Task 2. In 2020, we organized an unsupervised anomalous sound detection (ASD) task, identifying whether a given sound was normal or anomalous without anomalous training data. In 2021, we organized an advanced unsupervised ASD task under domain-shift conditions, which focuses on the inevitable problem of the practical use of ASD systems. The main challenge of this task is to detect unknown anomalous sounds where the acoustic characteristics of the training and testing samples are different, i.e., domain-shifted. This problem frequently occurs due to changes in seasons, manufactured products, and/or environmental noise. We received 75 submissions from 26 teams, and several novel approaches have been developed in this challenge. On the basis of the analysis of the evaluation results, we found that there are two types of remarkable approaches that TOP-5 winning teams adopted: 1) ensemble approaches of ``outlier exposure'' (OE)-based detectors and ``inlier modeling'' (IM)-based detectors and 2) approaches based on IM-based detection for features learned in a machine-identification task.
What Data Augmentation Do We Need for Deep-Learning-Based Finance?

The main task we consider is portfolio construction in a speculative market, a fundamental problem in modern ﬁnance. While various empirical works now exist to explore deep learning in ﬁnance, the theory side is almost non-existent. In this work, we focus on developing a theoretical framework for understanding the use of data augmentation for deep-learning-based approaches to quantitative ﬁnance. The proposed theory clariﬁes the role and necessity of data augmentation for ﬁnance; moreover, our theory motivates a simple algorithm of injecting a random noise of strength √∣ r t − 1 ∣ to the observed return r t . This algorithm is shown to work well in practice
On the role of feedback in visual processing: a predictive coding perspective

Brain-inspired machine learning is gaining increasing consideration, particularly in computer vision. Several studies investigated the inclusion of top-down feedback connections in convolutional networks; however, it remains unclear how and when these connections are functionally helpful. Here we address this question in the context of object recognition under noisy conditions. We consider deep convolutional networks (CNNs) as models of feed-forward visual processing and implement Predictive Coding (PC) dynamics through feedback connections (predictive feedback) trained for reconstruction or classification of clean images. To directly assess the computational role of predictive feedback in various experimental situations, we optimize and interpret the hyper-parameters controlling the network's recurrent dynamics. That is, we let the optimization process determine whether top-down connections and predictive coding dynamics are functionally beneficial. Across different model depths and architectures (3-layer CNN, ResNet18, and EfficientNetB0) and against various types of noise (CIFAR100-C), we find that the network increasingly relies on top-down predictions as the noise level increases; in deeper networks, this effect is most prominent at lower layers. In addition, the accuracy of the network implementing PC dynamics significantly increases over time-steps, compared to its equivalent forward network. All in all, our results provide novel insights relevant to Neuroscience by confirming the computational role of feedback connections in sensory systems, and to Machine Learning by revealing how these can improve the robustness of current vision models.
Designing Toxic Content Classification for a Diversity of Perspectives

In this work, we demonstrate how existing classifiers for identifying toxic comments online fail to generalize to the diverse concerns of Internet users. We survey 17,280 participants to understand how user expectations for what constitutes toxic content differ across demographics, beliefs, and personal experiences. We find that groups historically at-risk of harassment - such as people who identify as LGBTQ+ or young adults - are more likely to to flag a random comment drawn from Reddit, Twitter, or 4chan as toxic, as are people who have personally experienced harassment in the past. Based on our findings, we show how current one-size-fits-all toxicity classification algorithms, like the Perspective API from Jigsaw, can improve in accuracy by 86% on average through personalized model tuning. Ultimately, we highlight current pitfalls and new design directions that can improve the equity and efficacy of toxic content classifiers for all users.
On the relation between statistical learning and perceptual distances

we to the non-trivial relationships between the probability distribution of the perceptual distances, and unsupervised machine learning. To this end, we show that perceptual sensitivity is correlated with the probability of an image in its close neighborhood. We also explore the relation between distances induced by autoencoders and the probability distribution of the training data, as well as how these induced distances are correlated with human perception. Finally, we perceptual distances do not always lead to noticeable gains in performance over Euclidean distance in common image processing tasks, except when data is scarce and the perceptual distance provides regularization. We propose this may be due to a double-counting effect of the image statistics, once in the perceptual distance and once in the training procedure.
Demystifying Local Vision Transformer: Sparse Connectivity, Weight Sharing, and Dynamic Weight

Vision Transformer (ViT) attains state-of-the-art performance in visual recognition, and the variant, Local Vision Transformer, makes further improvements. The major component in Local Vision Transformer, local attention, performs the attention separately over small local windows. We rephrase local attention as a channel-wise locally-connected layer and analyze it from two network regularization manners, sparse connectivity and weight sharing, as well as weight computation. Sparse connectivity : there is no connection across channels, and each position is connected to the positions within a small local window. Weight sharing : the connection weights for one position are shared across channels or within each group of channels. Dynamic weight : the connection weights are dynamically predicted according to each image instance. We point out that local attention resembles depth-wise convolution and its dynamic version in sparse connectivity. The main difference lies in weight sharing - depth-wise convolution shares connection weights (kernel weights) across spatial positions. We empirically observe that the models based on depth-wise convolution and the dynamic variant with lower computation complexity perform on-par with or sometimes slightly better than Swin Transformer, an instance of Local Vision Transformer, for ImageNet classiﬁcation, COCO object detection and ADE semantic segmentation. These observations suggest that Local Vision Transformer takes advantage of two regularization forms and dynamic weight to increase the network capacity.
From Center-Vortex Ensembles to the Confining Flux Tube

In this review, we discuss the present status of the description of confining flux tubes in SU(N) pure Yang–Mills theory in terms of ensembles of percolating center vortices. This is based on three main pillars: modeling in the continuum the ensemble components detected in the lattice, the derivation of effective field representations, and contrasting the associated properties with Monte Carlo lattice results. The integration of the present knowledge about these points is essential to get closer to a unified physical picture for confinement. Here, we shall emphasize the last advances, which point to the importance of including the non-oriented center-vortex component and non-Abelian degrees of freedom when modeling the center-vortex ensemble measure. These inputs are responsible for the emergence of topological solitons and the possibility of accommodating the asymptotic scaling properties of the confining string tension.
Broadband X-ray spectral variability of the pulsing ULX NGC 1313 X-2

Context. It is thought that ultraluminous X-ray sources (ULXs) are mainly powered by super-Eddington accreting neutron stars or black holes as shown by the recent discovery of X-ray pulsations and relativistic winds. Aims. This work presents a follow-up study of the spectral evolution over two decades of the pulsing ULX NGC 1313 X-2 in order to understand the structure of the accretion disc. The primary objective is to determine the shape and nature of the dominant spectral components by investigating their variability with the changes in the source luminosity. Methods. We performed a spectral analysis over the canonical 0.3–10.0 keV energy band of all the high signal-to-noise XMM-Newton observations (96% of the available data), and we tested a number of different spectral models, which should approximate super-Eddington accretion discs. The baseline model consists of two thermal blackbody components with different temperatures plus an exponential cutoff powerlaw. Results. The baseline model provides a good description of the X-ray spectra. In particular, the hotter and brighter (LX ∼ 6–9 × 1039 erg s−1) thermal component describes the emission from the super-Eddington inner disc and the cutoff powerlaw describes the contribution from the accretion column of the neutron star. Instead, the cooler component describes the emission from the outer region of the disc close to the spherisation radius and the wind. The luminosity-temperature relation for the cool component follows a negative trend, which is not consistent with L ∝ T4, as is expected from a sub-Eddington thin disc of Shakura-Sunayev. This is not consistent with L ∝ T2 either, as is expected for an advection-dominated disc. However, this would rather agree with a wind-dominated X-ray emitting region. Instead, the (Lx, Tdisk) relation for the hotter component is somewhere in between the first two theoretical scenarios. Conclusions. Our findings agree with the super-Eddington scenario and provide further detail on the disc structure. The source spectral evolution is qualitatively similar to that seen in NGC 1313 X-1 and Holmberg IX X-1, indicating a common structure and evolution among archetypal ULXs.
$*$-logarithm for slice regular functions

In this paper, we study the (possible) solutions of the equation $\exp_{*}(f)=g$, where $g$ is a slice regular never vanishing function on a circular domain of the quaternions $\mathbb{H}$ and $\exp_{*}$ is the natural generalization of the usual exponential to the algebra of slice regular functions. Any function $f$ which satisfies $\exp_{*}(f)=g$ is called a $*$-logarithm of $g$. We provide necessary and sufficient conditions, expressed in terms of the zero set of the ``vector'' part $g_{v}$ of $g$, for the existence of a $*$-logarithm of $g$, under a natural topological condition on the domain $\Omega$. By the way, we prove an existence result if $g_{v}$ has no non-real isolated zeroes; we are also able to give a comprehensive approach to deal with more general cases. We are thus able to obtain an existence result when the non-real isolated zeroes of $g_{v}$ are finite, the domain is either the unit ball, or $\mathbb{H}$, or $\mathbb{D}$ and a further condition on the ``real part'' $g_{0}$ of $g$ is satisfied (see Theorem 6.19 for a precise statement). We also find some unexpected uniqueness results, again related to the zero set of $g_{v}$, in sharp contrast with the complex case. A number of examples are given throughout the paper in order to show the sharpness of the required conditions.
Flow through Fluorescence Detection of Phosphate in Human Saliva Based on Sensitized Turn-On Photoluminescence of CdS Quantum Dots

ABSTRACT The development of fluorescent turn-on probes for the determination of inorganic anions based on fluorescent semiconductor nanocrystals (quantum dots) has recently attracted great attention. Methods reported so far enable anion sensing at elevated concentration levels and usually with low selectivity against other abundant matrix components. In this work, we describe an alternative approach that combines the appropriate modulation of nanocrystal surface chemistry (i.e., stabilization) with rational design and modulation of the experimental conditions (i.e., flow injection analysis) in order to deliver an expedient flow through method that enables the determination of phosphate at low concentration levels and with good selectivity. The developed method was successfully applied to the determination of phosphate in artificial and human saliva with good analytical features in terms of reproducibility and repeatability (2.3–9.2%), recoveries from fortified samples (96.0–112.0%), linearity of response curves to phosphate concentrations (R2 > 0.99), and satisfactory sensitivity (<1 µM), providing a significant improvement to previous fluorescent turn-on quantum dot probes.
Investigating Transfer Learning in Multilingual Pre-trained Language Models through Chinese Natural Language Inference

Multilingual transformers (XLM, mT5) have been shown to have remarkable transfer skills in zero-shot settings. Most transfer studies, however, rely on automatically translated resources (XNLI, XQuAD), making it hard to discern the particular linguistic knowledge that is being transferred, and the role of expert annotated monolingual datasets when developing task-specific models. We investigate the cross-lingual transfer abilities of XLM-R for Chinese and English natural language inference (NLI), with a focus on the recent large-scale Chinese dataset OCNLI. To better understand linguistic transfer, we created 4 categories of challenge and adversarial tasks (totaling 17 new datasets) for Chinese that build on several well-known resources for English (e.g., HANS, NLI stress-tests). We find that cross-lingual models trained on English NLI do transfer well across our Chinese tasks (e.g., in 3/4 of our challenge categories, they perform as well/better than the best monolingual models, even on 3/5 uniquely Chinese linguistic phenomena such as idioms, pro drop). These results, however, come with important caveats: cross-lingual models often perform best when trained on a mixture of English and high-quality monolingual NLI data (OCNLI), and are often hindered by automatically translated resources (XNLI-zh). For many phenomena, all models continue to struggle, highlighting the need for our new diagnostics to help benchmark Chinese and cross-lingual models. All new datasets/code are released at https://github.com/huhailinguist/ChineseNLIProbing.
Global solutions to an initial boundary problem for the compressible 3D MHD equations with Navier-slip and perfectly conducting boundary conditions in exterior domains

An initial boundary value problem for compressible magnetohydrodynamics (MHD) is considered on an exterior domain (with the vanishing first Betti number) in R3 in this paper. The global existence of smooth solutions near a given constant state for compressible MHD with the boundary conditions of Navier-slip for the velocity filed and perfect conduction for the magnetic field is established. Moreover the explicit decay rate is given. In particular, the results obtained in this paper also imply the global existence of classical solutions for the full compressible Navier–Stokes equations with Navier-slip boundary conditions on exterior domains in three dimensions, which was not available in literature prior to the work in this paper, to the best of knowledge of the authors’.