Miracle-dz/UniFAR-FTUs huggingface dataset:
Existence and Stability of Localized Patterns in the Population Models with Large Advection and Strong Allee Effect

The strong Allee effect plays an important role on the evolution of population in ecological systems. One important concept is the Allee threshold that determines the persistence or extinction of the population in a long time. In general, a small initial population size is harmful to the survival of a species since when the initial data is below the Allee threshold the population tends to extinction, rather than persistence. Another interesting feature of population evolution is that a species whose movement strategy follows a conditional dispersal strategy is more likely to persist. In other words, the biased movement can be a benefit for the persistence of the population. The coexistence of the above two conflicting mechanisms makes the dynamics rather intricate. However, some numerical results obtained by Cosner et. al. (SIAM J. Appl. Math., Vol. 81, No. 2, 2021) show that the directed movement can invalidate the strong Allee effect and help the population survive. To study this intriguing phenomenon, we consider the pattern formation and local dynamics for a class of single species population models of that is subject to the strong Allee effect. We first rigorously show the existence of multiple localized solutions when the directed movement is strong enough. Next, the spectrum analysis of the associated linear eigenvalue problem is established and used to investigate the stability properties of these interior spikes. This analysis proves that there exists not only unstable but also linear stable steady states. Then, we extend results of the single equation to coupled systems, and also construct several non-constant steady states and analyze their stability. Finally, numerical simulations are performed to illustrate the theoretical results.
Deep Contrastive Learning is Provably (almost) Principal Component Analysis

We show that Contrastive Learning (CL) under a family of loss functions (including InfoNCE) has a game-theoretical formulation, where the max player ﬁnds representation to maximize contrastiveness, and the min player puts weights on pairs of samples with similar representation. We show that the max player who does representation learning reduces to Principal Component Analysis for deep linear network, and almost all local minima are global, recovering optimal PCA solutions. Experiments show that the formulation yields comparable (or better) performance on CIFAR10 and STL-10 when extending beyond InfoNCE, resulting in novel contrastive losses. Furthermore, we extend our theoretical analysis to 2-layer ReLU networks, showing its sharp difference from linear ones, and proving that feature composition is preferred over picking single dominant feature under strong augmentation.
Towards the 24-Hour Knowledge Factory: Offshore-Onsite Team Dynamics

Globalization and the spread of advanced information and communication technologies have encouraged a transition to distributed, virtual work practices. By reducing the costs of communication, these technologies now make it possible for more people than ever before to collaborate and compete in real time on different kinds of work from anywhere in the world - using computers, email, networks, teleconferencing, and dynamic new software applications. Thus, according to Thomas Friedman in his book, The World is Flat, "The global competitive playing field has being leveled" (Friedman, 2005, p.8). Connected to the flattening of the world is the increase in the diffusion of offshoring of knowledge intensive work towards emerging countries, such as India, China, and Romania. Although offshoring is becoming a part of our everyday social lexicon, we find little empirical evidence in the academic literature on the implications of offshoring for organizations and knowledge workers, or on the disruptive forces that offshoring brings to local, day-to-day work practices. Certainly, the literature on geographically dispersed or globally distributed teams (GDTs) shows the many organizational challenges of distributed collaboration, exploring issues like compatibility with existing hierarchical structures, awareness of other team member's activities, increased coordination costs, trust between distant members, status differentials, and leadership (e.g., Mohrman, 1998, Paul and McDaniel, 2004, Metiu, 2006, Weisband, in press). Moreover, previous studies explain how distance limits some of the important benefits that come from collocation, such as spontaneous conversation, collaborative social environments, mutual learning and influence (e.g. Hinds and Kiesler, 2002; Cummings, 2004). Some of this research was conducted in situations involving offshoring, but much of it reflects distributed work that remained onshore and does not distinguish between different types of globally distributed work arrangements (Saunders and Ahuja, 2006). Many of the conclusions and insights from these studies, therefore, may apply to the case of offshoring, but more work is needed to better define the specific upsetting effects of offshoring on knowledge workers and organizations.
AutoDistil: Few-shot Task-agnostic Neural Architecture Search for Distilling Large Language Models

Knowledge distillation (KD) methods compress large models into smaller students with manually-designed student architectures given pre-specified computational cost. This requires several trials to find a viable student, and further repeating the process for each student or computational budget change. We use Neural Architecture Search (NAS) to automatically distill several compressed students with variable cost from a large model. Current works train a single SuperLM consisting of millions of subnetworks with weight-sharing, resulting in interference between subnetworks of different sizes. Our framework AutoDistil addresses above challenges with the following steps: (a) Incorporates inductive bias and heuristics to partition Transformer search space into K compact sub-spaces (K=3 for typical student sizes of base, small and tiny); (b) Trains one SuperLM for each sub-space using task-agnostic objective (e.g., self-attention distillation) with weight-sharing of students; (c) Lightweight search for the optimal student without re-training. Fully task-agnostic training and search allow students to be reused for fine-tuning on any downstream task. Experiments on GLUE benchmark against state-of-the-art KD and NAS methods demonstrate AutoDistil to outperform leading compression techniques with upto 2.7x reduction in computational cost and negligible loss in task performance.
Vibrational fingerprints of ferroelectric hafnia

Shiyu Fan, 5 Sobhit Singh, 5 Xianghan Xu, 3 Kiman Park, Yubo Qi, S. W. Cheong, 3 David Vanderbilt, Karin M. Rabe, ∗ and J. L. Musfeldt 4, † Department of Physics and Astronomy, University of Tennessee, Knoxville, Tennessee, 37996, USA Department of Physics and Astronomy, Rutgers University, Piscataway, New Jersey 08854, USA Rutgers Center for Emergent Materials, Rutgers University, Piscataway, New Jersey 08854, USA Department of Chemistry, University of Tennessee, Knoxville, Tennessee, 37996, USA These authors contributed equally to this work Abstract Hafnia (HfO2) is a promising material for emerging chip applications due to its highκ dielectric behaviour, suitability for negative capacitance heterostructures, scalable ferroelectricity, and silicon compatibility. The lattice dynamics along with phononic properties such as thermal conductivity, contraction, and heat capacity are underexplored, primarily due to the absence of high quality single crystals. Herein, we report the vibrational properties of a series of HfO2 crystals stabilized with yttrium (chemical formula HfO2:xY, where x = 20, 12, 11, 8, and 0%) and compare our findings with a symmetry analysis and lattice dynamics calculations. We untangle the effects of Y by testing our calculations against the measured Raman and infrared spectra of the cubic, antipolar orthorhombic, and monoclinic phases and then proceed to reveal the signature modes of polar orthorhombic hafnia. This work provides a spectroscopic fingerprint for several different phases of HfO2 and paves the way for an analysis of mode contributions to high-κ dielectric and ferroelectric properties for chip technologies.
Polyphonic Audio Event Detection: Multi-Label or Multi-Class Multi-Task Classification Problem?

Polyphonic events are the main error source of audio event detection (AED) systems. In deep-learning context, the most common approach to deal with event overlaps is to treat the AED task as a multi-label classification problem. By doing this, we inherently consider multiple one-vs.-rest classification problems, which are jointly solved by a single (i.e. shared) network. In this work, to better handle polyphonic mixtures, we propose to frame the task as a multi-class classification problem by considering each possible label combination as one class. To circumvent the large number of arising classes due to combinatorial explosion, we divide the event categories into multiple groups and construct a multi-task problem in a divide-and-conquer fashion, where each of the tasks is a multi-class classification problem. A network architecture is then devised for multi-class multi-task modelling. The network is composed of a backbone subnet and multiple task-specific subnets. The task-specific subnets are designed to learn time-frequency and channel attention masks to extract features for the task at hand from the common feature maps learned by the backbone. Experiments on the TUT-SED-Synthetic-2016 with high degree of event overlap show that the proposed approach results in more favorable performance than the common multi-label approach.
Geometry of positive scalar curvature on complete manifold

Abstract In this paper, we study the interplay of geometry and positive scalar curvature on a complete, non-compact manifold with non-negative Ricci curvature. On three-dimensional manifold, we prove a minimal volume growth, an estimate of integral of scalar curvature and width. On higher-dimensional manifold, we obtain a volume growth with a stronger condition.
Hyperbolic Neural Networks for Molecular Generation

With the recent advance of deep learning, neural networks have been extensively used for the task of molecular generation. Many deep generators extract atomic relations from molecular graphs and ignore hierarchical information at both atom and molecule levels. In order to extract such hierarchical information, we propose a novel hyperbolic generative model. Our model contains three parts: first, a fully hyperbolic junction-tree encoder-decoder that embeds the hierarchical information of the molecules in the latent hyperbolic space; second, a latent generative adversarial network for generating the latent embeddings; third, a molecular generator that inherits the decoders from the first part and the latent generator from the second part. We evaluate our model on the ZINC dataset using the MOSES benchmarking platform and achieve competitive results, especially in metrics about structural similarity.
Deep learning in fNIRS: a review

Abstract. Significance: Optical neuroimaging has become a well-established clinical and research tool to monitor cortical activations in the human brain. It is notable that outcomes of functional near-infrared spectroscopy (fNIRS) studies depend heavily on the data processing pipeline and classification model employed. Recently, deep learning (DL) methodologies have demonstrated fast and accurate performances in data processing and classification tasks across many biomedical fields. Aim: We aim to review the emerging DL applications in fNIRS studies. Approach: We first introduce some of the commonly used DL techniques. Then, the review summarizes current DL work in some of the most active areas of this field, including brain–computer interface, neuro-impairment diagnosis, and neuroscience discovery. Results: Of the 63 papers considered in this review, 32 report a comparative study of DL techniques to traditional machine learning techniques where 26 have been shown outperforming the latter in terms of the classification accuracy. In addition, eight studies also utilize DL to reduce the amount of preprocessing typically done with fNIRS data or increase the amount of data via data augmentation. Conclusions: The application of DL techniques to fNIRS studies has shown to mitigate many of the hurdles present in fNIRS studies such as lengthy data preprocessing or small sample sizes while achieving comparable or improved classification accuracy.
Peaks and primordial black holes: the effect of non-Gaussianity

In light of recent developments in the field, we re-evaluate the effect of local-type non-Gaussianity on the primordial black hole (PBH) abundance (and consequently, upon constraints on the primordial power spectrum arising from PBHs). We apply peaks theory to the full, non-linear compaction, finding that, whilst the effect of non-Gaussianity is qualitatively similar to previous findings, the effect is much less significant. It is found the non-Gaussianity parameters f NL local and g NL local typically need to be approximately 1 or 2 orders of magntiude larger respectively to have a similar to that previously found. The effect will be to weaken the dependance of PBH constraints on the primordial power spectrum on the non-Gaussianity parameters, as well as to dramatically weaken constraints on the non-Gaussianity parameters (and/or PBH abundance) arising from the non-observation of dark matter isocurvature modes. We also consider the correlation between the curvature perturbation ζ and the compaction C, finding that, whilst PBHs may form at rare peaks in C these do not necessarily correspond to rare peaks in ζ — casting some doubt on many of the existing calculations of the PBH abundance.
On scale-invariant properties in natural images and their simulations

We study samples of natural images for which a set of statistical characteristics is computed and scale-invariant properties of samples are demonstrated computationally. Computations of the power spectrum are carried out and a power-law decaying power spectrum is observed on samples taken from van Hateren images of natural scenes. We propose a dynamic model to reproduce the observed slope in the power spectrum qualitatively. For two types of sources for this model the behaviour of power spectrum is investigated and scale-invariance confirmed numerically. We then discuss potential applications of scale-invariant properties of natural images.
Discovering Invariant Rationales for Graph Neural Networks

Intrinsic interpretability of graph neural networks (GNNs) is to find a small subset of the input graph's features -- rationale -- which guides the model prediction. Unfortunately, the leading rationalization models often rely on data biases, especially shortcut features, to compose rationales and make predictions without probing the critical and causal patterns. Moreover, such data biases easily change outside the training distribution. As a result, these models suffer from a huge drop in interpretability and predictive performance on out-of-distribution data. In this work, we propose a new strategy of discovering invariant rationale (DIR) to construct intrinsically interpretable GNNs. It conducts interventions on the training distribution to create multiple interventional distributions. Then it approaches the causal rationales that are invariant across different distributions while filtering out the spurious patterns that are unstable. Experiments on both synthetic and real-world datasets validate the superiority of our DIR in terms of interpretability and generalization ability on graph classification over the leading baselines. Code and datasets are available at https://github.com/Wuyxin/DIR-GNN.
The Donor of the Black Hole X-Ray Binary MAXI J1820+070

We estimate the parameters of the donor of the accreting black hole binary MAXI J1820+070. The measured values of the binary period, rotational and radial velocities, and constraints on the orbital inclination imply the donor is a subgiant with the mass of M2≈0.49−0.10+0.10M☉ and the radius of R2≈1.19−0.08+0.08R☉ . We reanalyze the previously obtained optical spectrum from the Gran Telescopio Canarias and found it yields a strict lower limit on the effective temperature of T > 4200 K. We compile optical and infrared fluxes observed during the quiescence of this system. From the minima r- and i-band fluxes found in Pan-STARSS1 Data Release 2 prediscovery imaging and for a distance of D ≈ 3 kpc, reddening of E(B – V) = 0.23, and R 2 ≈ 1.11R ⊙, we find T ≲ 4230 K, very close to the above lower limit. For a larger distance, the temperature can be higher, up to about 4500 K (corresponding to a K5 spectral type, preferred by previous studies) at D = 3.5 kpc, allowed by the Gaia parallax. We perform evolutionary calculations for the binary system and compare them to the observational constraints. Our model fitting the above temperature and radius constraints at D ≈ 3 kpc has a mass of 0.4M ⊙, T ≈ 4200 K, and solar metallicity. Two alternative models require D ≳ 3.3–3.4 kpc at 0.4M ⊙, T ≈ 4500 K and half-solar metallicity, and 0.5M ⊙, T ≈ 4300 K, and solar metallicity. These models yield mass-transfer rates of ∼10−10 M ⊙ yr−1, compatible with those based on the estimated accreted mass of ≈2 × 1025 g and the time between the 2018 discovery and the 1934 historical outburst.
On Improving Model-Free Algorithms for Decentralized Multi-Agent Reinforcement Learning

Multi-agent reinforcement learning (MARL) algorithms often suffer from an exponential sample complexity dependence on the number of agents, a phenomenon known as \emph{the curse of multiagents}. In this paper, we address this challenge by investigating sample-efficient model-free algorithms in \emph{decentralized} MARL, and aim to improve existing algorithms along this line. For learning (coarse) correlated equilibria in general-sum Markov games, we propose \emph{stage-based} V-learning algorithms that significantly simplify the algorithmic design and analysis of recent works, and circumvent a rather complicated no-\emph{weighted}-regret bandit subroutine. For learning Nash equilibria in Markov potential games, we propose an independent policy gradient algorithm with a decentralized momentum-based variance reduction technique. All our algorithms are decentralized in that each agent can make decisions based on only its local information. Neither communication nor centralized coordination is required during learning, leading to a natural generalization to a large number of agents. We also provide numerical simulations to corroborate our theoretical findings.
Hermite Polynomial Features for Private Data Generation

Kernel mean embedding is a useful tool to represent and compare probability measures. Despite its usefulness, kernel mean embedding consid-ers inﬁnite-dimensional features, which are chal-lenging to handle in the context of differentially private data generation. A recent work (Harder et al., 2021) proposes to approximate the kernel mean embedding of data distribution using ﬁnite-dimensional random features , which yields analyt-ically tractable sensitivity. However, the number of required random features is excessively high, often ten thousand to a hundred thousand, which worsens the privacy-accuracy trade-off. To im-prove the trade-off, we propose to replace random features with Hermite polynomial features. Unlike the random features, the Hermite polynomial features are ordered , where the features at the low orders contain more information on the distribution than those at the high orders. Hence, a relatively low order of Hermite polynomial features can more accurately approximate the mean embedding of the data distribution compared to a signiﬁcantly higher number of random features.
Tight Bounds on the Convergence of Noisy Random Circuits to the Uniform Distribution

We study the properties of output distributions of noisy, random circuits. We obtain upper and lower bounds on the expected distance of the output distribution from the"useless"uniform distribution. These bounds are tight with respect to the dependence on circuit depth. Our proof techniques also allow us to make statements about the presence or absence of anticoncentration for both noisy and noiseless circuits. We uncover a number of interesting consequences for hardness proofs of sampling schemes that aim to show a quantum computational advantage over classical computation. Specifically, we discuss recent barrier results for depth-agnostic and/or noise-agnostic proof techniques. We show that in certain depth regimes, noise-agnostic proof techniques might still work in order to prove an often-conjectured claim in the literature on quantum computational advantage, contrary to what was thought prior to this work.
A skew logistic distribution with application to modelling COVID-19 epidemic waves (preprint)/ en

A novel yet simple extension of the symmetric logistic distribution is proposed by introducing a skewness parameter. It is shown how the three parameters of the ensuing skew logistic distribution may be estimated using maximum likelihood. The skew logistic distribution is then extended to the skew bi-logistic distribution to allow the modelling of multiple waves in epidemic time series data. The proposed model is validated on COVID-19 data from the UK.
Federated Learning With Erroneous Communication Links

In this letter, we consider the federated learning (FL) problem in the presence of communication errors. We model the link between the devices and the central node (CN) by a packet erasure channel, where the local parameters from devices are either erased or received correctly by CN with probability <inline-formula> <tex-math notation="LaTeX">$\epsilon $ </tex-math></inline-formula> and <inline-formula> <tex-math notation="LaTeX">$1-\epsilon $ </tex-math></inline-formula>, respectively. We proved that the FL algorithm in the presence of communication errors, where the CN uses the past local update if the fresh one is not received from a device, converges to the same global parameter as that the FL algorithm converges to without any communication error. We provide several simulation results to validate our theoretical analysis. We also show that when the dataset is uniformly distributed among devices, the FL algorithm that only uses fresh updates and discards missing updates might converge faster than the FL algorithm that uses past local updates.
No-Regret Learning in Time-Varying Zero-Sum Games

Learning from repeated play in a fixed two-player zero-sum game is a classic problem in game theory and online learning. We consider a variant of this problem where the game payoff matrix changes over time, possibly in an adversarial manner. We first present three performance measures to guide the algorithmic design for this problem: 1) the well-studied individual regret, 2) an extension of duality gap, and 3) a new measure called dynamic Nash Equilibrium regret, which quantifies the cumulative difference between the player's payoff and the minimax game value. Next, we develop a single parameter-free algorithm that simultaneously enjoys favorable guarantees under all these three performance measures. These guarantees are adaptive to different non-stationarity measures of the payoff matrices and, importantly, recover the best known results when the payoff matrix is fixed. Our algorithm is based on a two-layer structure with a meta-algorithm learning over a group of black-box base-learners satisfying a certain property, along with several novel ingredients specifically designed for the time-varying game setting. Empirical results further validate the effectiveness of our algorithm.