Miracle-dz/UniFAR-FTUs huggingface dataset:
Variance-reduced Language Pretraining via a Mask Proposal Network

Self-supervised learning, a.k. a. , pretraining, is important in natural language processing. Most of the pretraining methods first randomly mask some positions in a sentence and then train a model to recover the tokens at the masked positions. In such a way, the model can be trained without human labeling, and the massive data can be used with billion parameters. Therefore, the optimization efficiency becomes critical. In this paper, we tackle the problem from the view of gradient variance reduction. In particular, we first propose a principled gradient variance decomposition theorem, which shows that the variance of the stochastic gradient of the language pretraining can be naturally decomposed into two terms: the variance that arises from the sample of data in a batch, and the variance that arises from the sampling of the mask. The second term is the key difference between selfsupervised learning and supervised learning, which makes the pretraining slower. In order to reduce the variance of the second part, we leverage the importance sampling strategy, which aims at sampling the masks according to a proposal distribution instead of the uniform distribution. It can be shown that if the proposal distribution is proportional to the gradient norm, the variance of the sampling is reduced. To improve efficiency, we introduced a MAsk Proposal Network (MAPNet), which approximates the optimal mask proposal distribution and is trained end-to-end along with the model. According to the experimental result, our model converges much faster and achieves higher performance than the baseline BERT model.
Classical-Quantum Separations in Certain Classes of Boolean Functions-- Analysis using the Parity Decision Trees.

In this paper we study the separation between the deterministic (classical) query complexity ($D$) and the exact quantum query complexity ($Q_E$) of several Boolean function classes using the parity decision tree method. We first define the Query Friendly (QF) functions on $n$ variables as the ones with minimum deterministic query complexity $(D(f))$. We observe that for each $n$, there exists a non-separable class of QF functions such that $D(f)=Q_E(f)$. Further, we show that for some values of $n$, all the QF functions are non-separable. Then we present QF functions for certain other values of $n$ where separation can be demonstrated, in particular, $Q_E(f)=D(f)-1$. In a related effort, we also study the Maiorana McFarland (M-M) type Bent functions. We show that while for any M-M Bent function $f$ on $n$ variables $D(f) = n$, separation can be achieved as $\frac{n}{2} \leq Q_E(f) \leq \lceil \frac{3n}{4} \rceil$. Our results highlight how different classes of Boolean functions can be analyzed for classical-quantum separation exploiting the parity decision tree method.
Why escape is faster than expected

We consider chaotic (hyperbolic) dynamical systems which have a generating Markov partition. Then, open dynamical systems are built by making one element of a Markov partition a ‘hole’ through which orbits escape. We compare various estimates of the escape rate which correspond to a physical picture of leaking in the entire phase space. Moreover, we uncover a reason why the escape rate is faster than expected, which is the convexity of the function defining escape rate. Exact computations are present for the skewed tent map and Arnold’s cat map.
RAINFALL PATTERNS AS BACKGROUND TO PLANT PHENOLOGY IN NORTHERN TANZANIA

The rainfall pattern in time and space within the Masai Ecosystem of northern Tanzania was studied using rainfall data of sixteen stations within the area. Annual amounts of rainfall were found to be unpredictable, and the annual deviations of the long-term mean annual rainfall were without cyclic trends for the individual stations. However, the pooled rainfall data of the different stations show that series of wet and dry years alternate. Total annual rainfall increases with altitude and the variability in rainfall decreases with altitude. It appears that rainfall in the dry season is caused by thunderstorm activity, and no relation between rainfall and altitude exists during this time of the year. Rainfall in the wet season shows a very different spatial relationship, indicating the effects of the monsoon, and, especially during the long rains, rainfall increases with altitude. Different seasons can be discerned, and the data indicate that the amount of rain in the long rainy season is more predictable than the amount in the short rainy season. The differences are discussed briefly with regard to plant phenology patterns and grass growth.
Radiative neutrino masses in the 
νRMSSM

We present a complete analysis of scenarios with radiatively generated neutrino masses that may occur in the Minimal Supersymmetric Standard Model with low-scale right-handed neutrinos. For brevity, we call such a model the $\nu_R$MSSM. We pay particular attention to the impact of the non-renormalization theorem of supersymmetry (SUSY) on the loop-induced neutrino masses, by performing our computations in the weak and flavour bases. In particular, we find that the smallness of the observed light neutrino masses may naturally arise due to a soft SUSY-screening effect from a nearly supersymmetric singlet neutrino sector. The profound phenomenological and cosmological implications that may originate from this screening phenomenon in the $\nu_R$MSSM and its minimal extensions are discussed.
Polaritonic cylinders as multifunctional metamaterials: Single scattering and effective medium description

Polaritonic materials, owing to a strong phonon-polariton resonance in the THz and far-infrared parts of the electromagnetic spectrum, offer both high-index dielectric and metallic response in this regime. This complex response makes them suitable candidates for the design of metamaterial-related phenomena and applications. Here we show that one type of polaritonic-material-based structures that are particularly suitable for the achievement of a wide range of metamaterial properties are systems of polaritonic rods. To study the interplay between the material and the structural resonances in such systems we employ as model systems rods of LiF and SiC and we calculate first the scattering properties of a single rod, identifying and discussing the behavior of the different resonances for different rod diameters. To analyze the response of ensembles of polaritonic rods we employ an effective medium approach based on the Coherent Potential Approximation (CPA), which is shown to be superior to the simple Maxwell-Garnett approximation for polaritonic and high-index dielectric metamaterials. Calculating and analyzing the CPA effective parameters, we found that our systems exhibit a large variety of interesting metamaterial properties, including hyperbolic dispersion, epsilon-near-zero and negative refractive index response. This rich response, achievable in almost any system of polaritonic rods, is highly engineerable by properly selecting the radius and the filling ratio of the rods, making polaritonic rod systems an ideal platform for demonstration of multifunctional metamaterials.
Reactive Synthesis from Extended Bounded Response LTL Specifications

Reactive synthesis is a key technique for the design of correct-by-construction systems and has been thoroughly investigated in the last decades. It consists in the synthesis of a controller that reacts to environment's inputs satisfying a given temporal logic specification. Common approaches are based on the explicit construction of automata and on their determinization, which limit their scalability. In this paper, we introduce a new fragment of Linear Temporal Logic, called Extended Bounded Response LTL (LTLEBR), that allows one to combine bounded and universal unbounded temporal operators (thus covering a large set of practical cases), and we show that reactive synthesis from LTLEBRspecifications can be reduced to solving a safety game over a deterministic symbolic automaton built directly from the specification. We prove the correctness of the proposed approach and we successfully evaluate it on various benchmarks.
Photometric Classification of 2315 Pan-STARRS1 Supernovae with Superphot

The classification of supernovae (SNe) and its impact on our understanding of explosion physics and progenitors have traditionally been based on the presence or absence of certain spectral features. However, current and upcoming wide-field time-domain surveys have increased the transient discovery rate far beyond our capacity to obtain even a single spectrum of each new event. We must therefore rely heavily on photometric classification—connecting SN light curves back to their spectroscopically defined classes. Here, we present Superphot, an open-source Python implementation of the machine-learning classification algorithm of Villar et al., and apply it to 2315 previously unclassified transients from the Pan-STARRS1 Medium Deep Survey for which we obtained spectroscopic host-galaxy redshifts. Our classifier achieves an overall accuracy of 82%, with completenesses and purities of >80% for the best classes (SNe Ia and superluminous SNe). For the worst performing SN class (SNe Ibc), the completeness and purity fall to 37% and 21%, respectively. Our classifier provides 1257 newly classified SNe Ia, 521 SNe II, 298 SNe Ibc, 181 SNe IIn, and 58 SLSNe. These are among the largest uniformly observed samples of SNe available in the literature and will enable a wide range of statistical studies of each class.
Rethinking of the Image Salient Object Detection: Object-level Semantic Saliency Re-ranking First, Pixel-wise Saliency Refinement Latter

The real human attention is an interactive activity between our visual system and our brain, using both low-level visual stimulus and high-level semantic information. Previous image salient object detection (SOD) works conduct their saliency predictions in a multi-task manner, i.e., performing pixel-wise saliency regression and segmentation-like saliency refinement at the same time, which degenerates their feature backbones in revealing semantic information. However, given an image, we tend to pay more attention to those regions which are semantically salient even in the case that these regions are perceptually not the most salient ones at first glance. In this paper, we divide the SOD problem into two sequential tasks: 1) we propose a lightweight, weakly supervised deep network to coarsely locate those semantically salient regions first; 2) then, as a post-processing procedure, we selectively fuse multiple off-the-shelf deep models on these semantically salient regions as the pixel-wise saliency refinement. In sharp contrast to the state-of-the-art (SOTA) methods that focus on learning pixel-wise saliency in "single image" using perceptual clues mainly, our method has investigated the "object-level semantic ranks between multiple images", of which the methodology is more consistent with the real human attention mechanism. Our method is simple yet effective, which is the first attempt to consider the salient object detection mainly as an object-level semantic re-ranking problem.
Interlayer Link Prediction in Multiplex Social Networks Based on Multiple Types of Consistency Between Embedding Vectors

Online users are typically active on multiple social media networks (SMNs), which constitute a multiplex social network. With improvements in cybersecurity awareness, users increasingly choose different usernames and provide different profiles on different SMNs. Thus, it is becoming increasingly challenging to determine whether given accounts on different SMNs belong to the same user; this can be expressed as an interlayer link prediction problem in a multiplex network. To address the challenge of predicting interlayer links, feature or structure information is leveraged. Existing methods that use network embedding techniques to address this problem focus on learning a mapping function to unify all nodes into a common latent representation space for prediction; positional relationships between unmatched nodes and their common matched neighbors (CMNs) are not utilized. Furthermore, the layers are often modeled as unweighted graphs, ignoring the strengths of the relationships between nodes. To address these limitations, we propose a framework based on multiple types of consistency between embedding vectors (MulCEVs). In MulCEV, the traditional embedding-based method is applied to obtain the degree of consistency between the vectors representing the unmatched nodes, and a proposed distance consistency index based on the positions of nodes in each latent space provides additional clues for prediction. By associating these two types of consistency, the effective information in the latent spaces is fully utilized. In addition, MulCEV models the layers as weighted graphs to obtain representation. In this way, the higher the strength of the relationship between nodes, the more similar their embedding vectors in the latent representation space will be. The results of our experiments on several real-world and synthetic datasets demonstrate that the proposed MulCEV framework markedly outperforms current embedding-based methods, especially when the number of training iterations is small.
An Inter- and Intra-Band Loss for Pansharpening Convolutional Neural Networks

Pansharpening aims to fuse panchromatic and multispectral images from the satellite to generate images with both high spatial and spectral resolution. With the successful applications of deep learning in the computer vision field, a lot of scholars have proposed many convolutional neural networks (CNNs) to solve the pansharpening task. These pansharpening networks focused on various distinctive structures of CNNs, and most of them are trained by L2 loss between fused images and simulated desired multispectral images. However, L2 loss is designed to directly minimize the difference of spectral information of each band, which does not consider the inter-band relations in the training process. In this letter, we propose a novel inter- and intra-band (IIB) loss to overcome the drawback of original L2 loss. Our proposed IIB loss can effectively preserve both inter- and intra-band relations and can be directly applied to different pansharpening CNNs.
GEE-TGDR: A Longitudinal Feature Selection Algorithm and Its Application to lncRNA Expression Profiles for Psoriasis Patients Treated with Immune Therapies

With the fast evolution of high-throughput technology, longitudinal gene expression experiments have become affordable and increasingly common in biomedical fields. Generalized estimating equation (GEE) approach is a widely used statistical method for the analysis of longitudinal data. Feature selection is imperative in longitudinal omics data analysis. Among a variety of existing feature selection methods, an embedded method—threshold gradient descent regularization (TGDR)—stands out due to its excellent characteristics. An alignment of GEE with TGDR is a promising area for the purpose of identifying relevant markers that can explain the dynamic changes of outcomes across time. We proposed a new novel feature selection algorithm for longitudinal outcomes—GEE-TGDR. In the GEE-TGDR method, the corresponding quasilikelihood function of a GEE model is the objective function to be optimized, and the optimization and feature selection are accomplished by the TGDR method. Long noncoding RNAs (lncRNAs) are posttranscriptional and epigenetic regulators and have lower expression levels and are more tissue-specific compared with protein-coding genes. So far, the implication of lncRNAs in psoriasis remains largely unexplored and poorly understood even though some evidence in the literature supports that lncRNAs and psoriasis are highly associated. In this study, we applied the GEE-TGDR method to a lncRNA expression dataset that examined the response of psoriasis patients to immune treatments. As a result, a list including 10 relevant lncRNAs was identified with a predictive accuracy of 70% that is superior to the accuracies achieved by two competitive methods and meaningful biological interpretation. A widespread application of the GEE-TGDR method in omics longitudinal data analysis is anticipated.
Compact Speaker Embedding: lrx-vector

Deep neural networks (DNN) have recently been widely used in speaker recognition systems, achieving state-of-the-art performance on various benchmarks. The x-vector architecture is especially popular in this research community, due to its excellent performance and manageable computational complexity. In this paper, we present the lrx-vector system, which is the low-rank factorized version of the x-vector embedding network. The primary objective of this topology is to further reduce the memory requirement of the speaker recognition system. We discuss the deployment of knowledge distillation for training the lrx-vector system and compare against low-rank factorization with SVD. On the VOiCES 2019 far-field corpus we were able to reduce the weights by 28% compared to the full-rank x-vector system while keeping the recognition rate constant (1.83% EER).
Statistical Aspects of the Quantum Supremacy Demonstration

The notable claim of quantum supremacy presented by Google's team in 2019 consists of demonstrating the ability of a quantum circuit to generate, albeit with considerable noise, bitstrings from a distribution that is considered hard to simulate on classical computers. Verifying that the generated data is indeed from the claimed distribution and assessing the circuit's noise level and its fidelity is a purely statistical undertaking. The objective of this paper is to explain the relations between quantum computing and some of the statistical aspects involved in demonstrating quantum supremacy in terms that are accessible to statisticians, computer scientists, and mathematicians. Starting with the statistical analysis in Google's demonstration, which we explain, we study various estimators of the fidelity, and different approaches to testing the distributions generated by the quantum computer. We propose different noise models, and discuss their implications. A preliminary study of the Google data, focusing mostly on circuits of 12 and 14 qubits is discussed throughout the paper.
Formation pathway for lonely stripped-envelope supernova progenitors: implications for Cassiopeia A

We explore a new scenario for producing stripped-envelope supernova progenitors. In our scenario, the stripped-envelope supernova is the second supernova of the binary, in which the envelope of the secondary was removed during its red supergiant phase by the impact of the first supernova. Through 2D hydrodynamical simulations, we find that ∼50–90 ${{\ \rm per\ cent}}$ of the envelope can be unbound as long as the pre-supernova orbital separation is ≲5 times the stellar radius. Recombination energy plays a significant role in the unbinding, especially for relatively high mass systems (≳18 M⊙). We predict that more than half of the unbound mass should be distributed as a one-sided shell at about ∼10–100 pc away from the second supernova site. We discuss possible applications to known supernova remnants such as Cassiopeia A, RX J1713.7−3946, G11.2−0.3, and find promising agreements. The predicted rate is ∼0.35–1${{\ \rm per\ cent}}$ of the core-collapse population. This new scenario could be a major channel for the subclass of stripped-envelope or type IIL supernovae that lack companion detections like Cassiopeia A.
Forced perimeter in Elnitksy polygons

We study tiling-based perimeter and characterize when a given perimeter tile appears in all rhombic tilings of an Elnitsky polygon. Regardless of where on the perimeter this tile appears, its forcing can be described in terms of 321-patterns. We characterize the permutations with maximally many forced right-perimeter tiles, and show that they are enumerated by the Catalan numbers.
Active surveillance for low-risk prostate cancer – in pursuit of a standardized protocol

Introduction Active surveillance (AS) is a management option recommended by most guidelines for low risk clinically-localized prostate cancer (LR-CLPC). Data shows that AS is being increasingly adopted into clinical practice worldwide. Our aim was to review the up-to date guidelines and observational studies in regards to AS in LR-CLRPC to gain insight into principles of contemporary clinical practice. Material and methods Several guidelines on the management of low-risk prostate cancer were reviewed for evidence-based recommendations regarding the protocol of AS. We reviewed the available literature for most recent studies on AS in LR-CLPC. Results No uniform protocol of AS in LR-CLPC has been recommended up to date and available guidelines significantly differ in terms of protocol schedules and the role of particular tools in monitoring for disease progression. Nevertheless, recent studies on AS in LR-CLPC, in which various protocols were adopted, have demonstrated promising outcomes in regards to cancer-specific survival (99–100% at 5 years, 98.1–99.9% at 10 years, and 94.3–96% at 15 years), with high rates of men remaining within the protocols (23–39% at 10 years). Conclusions This article is a call for focusing further research on development and recommending a precise and standardized, evidence-based protocol for AS in LR-CLPC.
Cervical cancer in low and middle-income countries

Cervical cancer is a malignant tumour that occurs in the cervix and is classified into two histological types, adenocarcinoma and squamous cell carcinoma (SCC); SCC is more common and accounts for 70% of all cases. In 2018 there were ~569,000 new cases of cervical cancer diagnosed worldwide and ~311,000 deaths were attributed to cervical cancer. Of these, between 84 and 90% occurred in low- and middle-income countries (LMICs) such as South Africa, India, China and Brazil. The most common cause of cervical cancer is persistent infection caused by the sexually transmitted human papilloma virus. Other factors that contribute to the incidence of cervical cancer include geography, traditional practices and beliefs, the screening levels, socioeconomic status, healthcare access, public awareness, use of oral contraceptives, smoking and co-infection with HIV. An estimated 11 million women from LMICs will be diagnosed with cervical cancer in the next 10–20 years. The aim of this review was to explore various types of genetic and epigenetic factors that influence the development, progression or suppression of cervical cancer.
Optimizing Skeletal Muscle Anabolic Response to Resistance Training in Aging

Loss of muscle mass and strength with aging, also termed sarcopenia, results in a loss of mobility and independence. Exercise, particularly resistance training, has proven to be beneficial in counteracting the aging-associated loss of skeletal muscle mass and function. However, the anabolic response to exercise in old age is not as robust, with blunted improvements in muscle size, strength, and function in comparison to younger individuals. This review provides an overview of several physiological changes which may contribute to age-related loss of muscle mass and decreased anabolism in response to resistance training in the elderly. Additionally, the following supplemental therapies with potential to synergize with resistance training to increase muscle mass are discussed: nutrition, creatine, anti-inflammatory drugs, testosterone, and growth hormone (GH). Although these interventions hold some promise, further research is necessary to optimize the response to exercise in elderly patients.