Miracle-dz/UniFAR-FTUs huggingface dataset:
Single-Side Domain Generalization for Face Anti-Spoofing

Existing domain generalization methods for face anti-spoofing endeavor to extract common differentiation features to improve the generalization. However, due to large distribution discrepancies among fake faces of different domains, it is difficult to seek a compact and generalized feature space for the fake faces. In this work, we propose an end-to-end single-side domain generalization framework (SSDG) to improve the generalization ability of face anti-spoofing. The main idea is to learn a generalized feature space, where the feature distribution of the real faces is compact while that of the fake ones is dispersed among domains but compact within each domain. Specifically, a feature generator is trained to make only the real faces from different domains undistinguishable, but not for the fake ones, thus forming a single-side adversarial learning. Moreover, an asymmetric triplet loss is designed to constrain the fake faces of different domains separated while the real ones aggregated. The above two points are integrated into a unified framework in an end-to-end training manner, resulting in a more generalized class boundary, especially good for samples from novel domains. Feature and weight normalization is incorporated to further improve the generalization ability. Extensive experiments show that our proposed approach is effective and outperforms the state-of-the-art methods on four public databases. The code is released online.
Reevaluating Adversarial Examples in Natural Language

State-of-the-art attacks on NLP models lack a shared definition of a what constitutes a successful attack. We distill ideas from past work into a unified framework: a successful natural language adversarial example is a perturbation that fools the model and follows some linguistic constraints. We then analyze the outputs of two state-of-the-art synonym substitution attacks. We find that their perturbations often do not preserve semantics, and 38% introduce grammatical errors. Human surveys reveal that to successfully preserve semantics, we need to significantly increase the minimum cosine similarities between the embeddings of swapped words and between the sentence encodings of original and perturbed sentences.With constraints adjusted to better preserve semantics and grammaticality, the attack success rate drops by over 70 percentage points.
Automatically Identifying Gender Issues in Machine Translation using Perturbations

The successful application of neural methods to machine translation has realized huge quality advances for the community. With these improvements, many have noted outstanding challenges, including the modeling and treatment of gendered language. While previous studies have identified issues using synthetic examples, we develop a novel technique to mine examples from real world data to explore challenges for deployed systems. We use our method to compile an evaluation benchmark spanning examples for four languages from three language families, which we publicly release to facilitate research. The examples in our benchmark expose where model representations are gendered, and the unintended consequences these gendered representations can have in downstream application.
On Bi-Modal Constrained Coding

Bi-modal (respectively, multi-modal) constrained coding refers to an encoding model whereby a user input block can be mapped to two (respectively, multiple) codewords. In current storage applications, such as optical disks, multi-modal coding allows to achieve DC control, in addition to satisfying the runlength limited (RLL) constraint specified by the recording channel. In this work, a study is initiated on bi-modal fixed-length constrained encoders. Necessary and sufficient conditions are presented for the existence of such encoders for a given constraint. It is also shown that under somewhat stronger conditions, one can guarantee a bi-modal encoder with finite decoding delay.
Efficient Document Re-Ranking for Transformers by Precomputing Term Representations

Deep pretrained transformer networks are effective at various ranking tasks, such as question answering and ad-hoc document ranking. However, their computational expenses deem them cost-prohibitive in practice. Our proposed approach, called PreTTR (Precomputing Transformer Term Representations), considerably reduces the query-time latency of deep transformer networks (up to a 42x speedup on web document ranking) making these networks more practical to use in a real-time ranking scenario. Specifically, we precompute part of the document term representations at indexing time (without a query), and merge them with the query representation at query time to compute the final ranking score. Due to the large size of the token representations, we also propose an effective approach to reduce the storage requirement by training a compression layer to match attention scores. Our compression technique reduces the storage required up to 95% and it can be applied without a substantial degradation in ranking performance.
Avoiding catastrophic forgetting in mitigating model biases in sentence-pair classification with elastic weight consolidation

The biases present in training datasets have been shown to be affecting models for a number of tasks such as natural language inference(NLI) and fact verification. While fine-tuning models on additional data has been used to mitigate such biases, a common issue is that of catastrophic forgetting of the original task. In this paper, we show that elastic weight consolidation (EWC) allows fine-tuning of models to mitigate biases for NLI and fact verification while being less susceptible to catastrophic forgetting. In our evaluation on fact verification systems, we show that fine-tuning with EWC Pareto dominates standard fine-tuning, yielding models lower levels of forgetting on the original task for equivalent gains in accuracy on the fine-tuned task. Additionally, we show that systems trained on NLI can be fine-tuned to improve their accuracy on stress test challenge tasks with minimal loss in accuracy on the MultiNLI dataset despite greater domain shift.
Twisted differential operators and q-crystals

We describe explicitly the q-PD-envelopes considered by Bhatt and Scholze in their recent theory of q-crystalline cohomology and explain the relation with our notion of a divided polynomial twisted algebra. Together with an interpretation of crystals on the q-crystalline site, that we call q-crystals, as modules endowed with some kind of stratification, it allows us to associate a module on the ring of twisted differential operators to any q-crystal.
Degenerating sequences of conformal classes and the conformal Steklov spectrum

Abstract Let $\Sigma $ be a compact surface with boundary. For a given conformal class c on $\Sigma $ the functional $\sigma _k^*(\Sigma ,c)$ is defined as the supremum of the kth normalized Steklov eigenvalue over all metrics in c. We consider the behavior of this functional on the moduli space of conformal classes on $\Sigma $ . A precise formula for the limit of $\sigma _k^*(\Sigma ,c_n)$ when the sequence $\{c_n\}$ degenerates is obtained. We apply this formula to the study of natural analogs of the Friedlander–Nadirashvili invariants of closed manifolds defined as $\inf _{c}\sigma _k^*(\Sigma ,c)$ , where the infimum is taken over all conformal classes c on $\Sigma $ . We show that these quantities are equal to $2\pi k$ for any surface with boundary. As an application of our techniques we obtain new estimates on the kth normalized Steklov eigenvalue of a nonorientable surface in terms of its genus and the number of boundary components.
Utterance Pair Scoring for Noisy Dialogue Data Filtering

Filtering noisy training data is one of the key approaches to improving the quality of neural network-based language generation. The dialogue research community especially suffers from a lack of less-noisy and sufficiently large data. In this work, we propose a scoring function that is specifically designed to identify low-quality utterance--response pairs to filter noisy training data. Our scoring function models the naturalness of the interconnection within dialogue pairs and their content-relatedness, which is based on previous findings in dialogue response generation and linguistics. We then demonstrate the effectiveness of our scoring function by confirming (i) the correlation between automatic scoring by the proposed function and human evaluation, and (ii) the performance of a dialogue response generator trained with filtered data. Furthermore, we experimentally confirm that our scoring function potentially works as a language-independent method.
Gender Diversity in Computer Science at a Large Research University

With the number of Computer Science (CS) jobs on the rise, there is a greater need for Computer Science graduates than ever. At the same time, most CS departments across the country are only seeing 25 to 30 percent of female students in their classes, meaning that we are failing to draw interest from a large portion of the population. In this work, we explore the gender gap in CS at a large public research university, using three data sets that span thousands of students across 5 and a half academic years. By combining these data sets, we can explore many issues such as retention as students progress through the CS major. For example, we find that a large percentage of women taking the Introductory CS1 course for majors do not intend to major in CS, which contributes to a large increase in the gender gap immediately after CS1. This finding implies that a large part of the retention task is attracting these women to further explore the major. We report findings in three areas of research in the context of the CS department at our university: the CS environment, the computing background of our students, and the students' grades. These findings may also be applicable to computing programs at other large public research universities.
Tropical Fukaya Algebras

We introduce a tropical version of the Fukaya algebra of a Lagrangian submanifold and use it to show that tropical Lagrangian tori are weakly unobstructed. Tropical graphs arise by studying the behavior of pseudoholomorphic disks under a multiple cut operation on a symplectic manifold that produces a collection of cut spaces each containing relative normal crossing divisors, following works of Ionel and Brett Parker. Given a Lagrangian submanifold in the complement of the relative divisors in one of the cut spaces, we first study a broken Fukaya algebra whose structure maps count broken disks associated to rigid tropical graphs. We introduce a further degeneration of the matching conditions (similar in spirit to Bourgeois' version of symplectic field theory) which results in a tropical Fukaya algebra in which the structure maps are sums of products over vertices of tropical graphs. We show the tropical Fukaya algebra is homotopy equivalent to the original Fukaya algebra. In the case of toric Lagrangians contained in a toric component of the degeneration, an invariance argument implies the existence of projective Maurer-Cartan solutions.
Constraining the cosmological evolution of scalar-tensor theories with local measurements of the time variation of G

We determine the conditions for which the constraints from lunar laser ranging on the time evolution of the local gravitational constant can be extrapolated to impose constraints on the time evolution of the cosmological gravitational constant in scalar-tensor theories of modified gravity. We allow for the possibility that the scalar-tensor theories are non-linear and contain a screening mechanism. This results in strong late Universe constraints on the running of the cosmological Planck mass described by the Horndeski function |αM|≲ 0.002. We find that our assumptions are valid for most Vainshtein and kinetic screening models, where the shift symmetry ϕ→ϕ + c holds, but are violated by some Chameleon and Symmetron screening models, where the macroscopic equivalence principle is broken.
X-shooter survey of disk accretion in Upper Scorpius

Determining the mechanisms that drive the evolution of protoplanetary disks is a necessary step toward understanding how planets form. For this work, we measured the mass accretion rate for young stellar objects with disks at age > 5 Myr, a critical test for the current models of disk evolution. We present the analysis of the spectra of 36 targets in the ∼5–10 Myr old Upper Scorpius star-forming region for which disk masses were measured with ALMA. We find that the mass accretion rates in this sample of old but still surviving disks are similarly high as those of the younger (∼1−3 Myr old) star-forming regions of Lupus and Chamaeleon I, when considering the dependence on stellar and disk mass. In particular, several disks show high mass accretion rates ≳10−9 M⊙ yr−1 while having low disk masses. Furthermore, the median values of the measured mass accretion rates in the disk mass ranges where our sample is complete at a level ∼60−80% are compatible in these three regions. At the same time, the spread of mass accretion rates at any given disk mass is still > 0.9 dex, even at age > 5 Myr. These results are in contrast with simple models of viscous evolution, which would predict that the values of the mass accretion rate diminish with time, and a tighter correlation with disk mass at age > 5 Myr. Similarly, simple models of internal photoevaporation cannot reproduce the observed mass accretion rates, while external photoevaporation might explain the low disk masses and high accretion rates. A possible partial solution to the discrepancy with the viscous models is that the gas-to-dust ratio of the disks at ∼5–10 Myr is significantly different and higher than the canonical 100, as suggested by some dust and gas disk evolution models. The results shown here require the presence of several interplaying processes, such as detailed dust evolution, external photoevaporation, and possibly MHD winds, to explain the secular evolution of protoplanetary disks.
Constrained nonsmooth problems of the calculus of variations

The paper is devoted to an analysis of optimality conditions for nonsmooth multidimensional problems of the calculus of variations with various types of constraints, such as additional constraints at the boundary and isoperimetric constraints. To derive optimality conditions, we study generalised concepts of differentiability of nonsmooth functions called codifferentiability and quasidifferentiability. Under some natural and easily verifiable assumptions we prove that a nonsmooth integral functional defined on the Sobolev space is continuously codifferentiable and compute its codifferential and quasidifferential. Then we apply general optimality conditions for nonsmooth optimisation problems in Banach spaces to obtain optimality conditions for nonsmooth problems of the calculus of variations. Through a series of simple examples we demonstrate that our optimality conditions are sometimes better than existing ones in terms of various subdifferentials, in the sense that our optimality conditions can detect the non-optimality of a given point, when subdifferential-based optimality conditions fail to disqualify this point as non-optimal.
GePpeTto Carves Italian into a Language Model

In the last few years, pre-trained neural architectures have provided impressive improvements across several NLP tasks. Still, generative language models are available mainly for English. We develop GePpeTto, the first generative language model for Italian, built using the GPT-2 architecture. We provide a thorough analysis of GePpeTto's quality by means of both an automatic and a human-based evaluation. The automatic assessment consists in (i) calculating perplexity across different genres and (ii) a profiling analysis over GePpeTto's writing characteristics. We find that GePpeTto's production is a sort of bonsai version of human production, with shorter but yet complex sentences. Human evaluation is performed over a sentence completion task, where GePpeTto's output is judged as natural more often than not, and much closer to the original human texts than to a simpler language model which we take as baseline.
A Fast Radio Burst Discovered in FAST Drift Scan Survey

We report the discovery of a highly dispersed fast radio burst (FRB), FRB 181123, from an analysis of ∼1500 hr of drift scan survey data taken using the Five-hundred-meter Aperture Spherical radio Telescope (FAST). The pulse has three distinct emission components, which vary with frequency across our 1.0–1.5 GHz observing band. We measure the peak flux density to be >0.065 Jy and the corresponding fluence >0.2 Jy ms. Based on the observed dispersion measure of 1812 cm−3 pc, we infer a redshift of ∼1.9. From this, we estimate the peak luminosity and isotropic energy to be ≲2 × 1043 erg s−1 and ≲2 × 1040 erg, respectively. With only one FRB from the survey detected so far, our constraints on the event rate are limited. We derive a 95% confidence lower limit for the event rate of 900 FRBs per day for FRBs with fluences >0.025 Jy ms. We performed follow-up observations of the source with FAST for four hours and have not found a repeated burst. We discuss the implications of this discovery for our understanding of the physical mechanisms of FRBs.
Exploiting Structured Knowledge in Text via Graph-Guided Representation Learning

In this work, we aim at equipping pre-trained language models with structured knowledge. We present two self-supervised tasks learning over raw text with the guidance from knowledge graphs. Building upon entity-level masked language models, our first contribution is an entity masking scheme that exploits relational knowledge underlying the text. This is fulfilled by using a linked knowledge graph to select informative entities and then masking their mentions. In addition we use knowledge graphs to obtain distractors for the masked entities, and propose a novel distractor-suppressed ranking objective which is optimized jointly with masked language model. In contrast to existing paradigms, our approach uses knowledge graphs implicitly, only during pre-training, to inject language models with structured knowledge via learning from raw text. It is more efficient than retrieval-based methods that perform entity linking and integration during finetuning and inference, and generalizes more effectively than the methods that directly learn from concatenated graph triples. Experiments show that our proposed model achieves improved performance on five benchmark datasets, including question answering and knowledge base completion tasks.
The behaviour of moulting King Eiders Somateria spectabilis

Activity budgets and distribution patterns of King Eiders were studied at Disko Island, West Greenland during the moulting season of July to September, and compared with the activity budgets of non-moulting King Eiders. King Eiders were generally active in the daylight hours, and spent most of the night and a period at mid-day resting. The feeding intensity peaked early and late in the day, with no marked tidal influence on the feeding pattern. The amount of time devoted to diurnal feeding increased during the moulting period, and was negatively correlated with daylength. There was no evidence of increased feeding intensity by moulting birds compared to non-moulting birds. Moulting King Eiders devoted more time to comfort movements than non-moulting birds. Ice conditions and daylength are suggested to be ultimate factors affecting the timing of moult.
A deep X-ray spectral imaging of the bow-shock pulsar wind nebula associated with PSR B1929+10

In this work, we study the X-ray bow-shock nebula powered by the mature pulsar PSR B1929+10 using data from XMM-Newton, with an effective exposure of ∼300 ks, offering the deepest investigation of this system thus far. We found the X-ray axial outflow extends as long as ∼8 arc minute behind the proper motion direction, which is a factor of two longer than the result reported in the previous study. Furthermore, we found evidence of two faint lateral outflows extending laterally with respect to the proper motion. We also found indications of spectral hardening along the axial outflow, suggesting that certain acceleration processes might occur along this feature.
The size‐Ramsey number of short subdivisions

The r‐size‐Ramsey number R^r(H) of a graph H is the smallest number of edges a graph G can have such that for every edge‐coloring of G with r colors there exists a monochromatic copy of H in G. For a graph H, we denote by Hq the graph obtained from H by subdividing its edges with q − 1 vertices each. In a recent paper of Kohayakawa, Retter and Rödl, it is shown that for all constant integers q, r ≥ 2 and every graph H on n vertices and of bounded maximum degree, the r‐size‐Ramsey number of Hq is at most (logn)20(q−1)n1+1/q , for n large enough. We improve upon this result using a significantly shorter argument by showing that R^r(Hq)≤O(n1+1/q) for any such graph H.
How fine can fine-tuning be? Learning efficient language models

State-of-the-art performance on language understanding tasks is now achieved with increasingly large networks; the current record holder has billions of parameters. Given a language model pre-trained on massive unlabeled text corpora, only very light supervised fine-tuning is needed to learn a task: the number of fine-tuning steps is typically five orders of magnitude lower than the total parameter count. Does this mean that fine-tuning only introduces small differences from the pre-trained model in the parameter space? If so, can one avoid storing and computing an entire model for each task? In this work, we address these questions by using Bidirectional Encoder Representations from Transformers (BERT) as an example. As expected, we find that the fine-tuned models are close in parameter space to the pre-trained one, with the closeness varying from layer to layer. We show that it suffices to fine-tune only the most critical layers. Further, we find that there are surprisingly many good solutions in the set of sparsified versions of the pre-trained model. As a result, fine-tuning of huge language models can be achieved by simply setting a certain number of entries in certain layers of the pre-trained parameters to zero, saving both task-specific parameter storage and computational cost.
Families of sequences with good family complexity and cross-correlation measure

In this paper we study pseudorandomness of a family of sequences in terms of two measures, the family complexity ($f$-complexity) and the cross-correlation measure of order $\ell$. We consider sequences not only on binary alphabet but also on $k$-symbols ($k$-ary) alphabet. We first generalize some known methods on construction of the family of binary pseudorandom sequences. We prove a bound on the $f$-complexity of a large family of binary sequences of Legendre-symbols of certain irreducible polynomials. We show that this family as well as its dual family have both a large family complexity and a small cross-correlation measure up to a rather large order. Next, we present another family of binary sequences having high $f$-complexity and low cross-correlation measure. Then we extend the results to the family of sequences on $k$-symbols alphabet.