Miracle-dz/UniFAR-FTUs huggingface dataset:
DiffWave: A Versatile Diffusion Model for Audio Synthesis

In this work, we propose DiffWave, a versatile Diffusion probabilistic model for conditional and unconditional Waveform generation. The model is non-autoregressive, and converts the white noise signal into structured waveform through a Markov chain with a constant number of steps at synthesis. It is efficiently trained by optimizing a variant of variational bound on the data likelihood. DiffWave produces high-fidelity audios in Different Waveform generation tasks, including neural vocoding conditioned on mel spectrogram, class-conditional generation, and unconditional generation. We demonstrate that DiffWave matches a strong WaveNet vocoder in terms of speech quality~(MOS: 4.44 versus 4.43), while synthesizing orders of magnitude faster. In particular, it significantly outperforms autoregressive and GAN-based waveform models in the challenging unconditional generation task in terms of audio quality and sample diversity from various automatic and human evaluations.
Semi-analytic pricing of double barrier options with time-dependent barriers and rebates at hit

We continue a series of papers devoted to construction of semi-analytic solutions for barrier options. These options are written on underlying following some simple one-factor diffusion model, but all the parameters of the model as well as the barriers are time-dependent. We managed to show that these solutions are systematically more efficient for pricing and calibration than, e.g., the corresponding finite-difference solvers. In this paper we extend this technique to pricing double barrier options and present two approaches to solving it: the General Integral transform method and the Heat Potential method. Our results confirm that for double barrier options these semi-analytic techniques are also more efficient than the traditional numerical methods used to solve this type of problems.
A Experimental Study of Weight Initialization and Weight Inheritance Effects on Neuroevolution

Weight initialization is critical in being able to successfully train artificial neural networks (ANNs), and even more so for recurrent neural networks (RNNs) which can easily suffer from vanishing and exploding gradients. In neuroevolution, where evolutionary algorithms are applied to neural architecture search, weights typically need to be initialized at three different times: when initial genomes (ANN architectures) are created at the beginning of the search, when offspring genomes are generated by crossover, and when new nodes or edges are created during mutation. This work explores the difference between using Xavier, Kaiming, and uniform random weight initialization methods, as well as novel Lamarckian weight inheritance methods for initializing new weights during crossover and mutation operations. These are examined using the Evolutionary eXploration of Augmenting Memory Models (EXAMM) neuroevolution algorithm, which is capable of evolving RNNs with a variety of modern memory cells (e.g., LSTM, GRU, MGU, UGRNN and Delta-RNN cells) as well recurrent connections with varying time skips through a high performance island based distributed evolutionary algorithm. Results show that with statistical significance, utilizing the Lamarckian strategies outperforms Kaiming, Xavier and uniform random weight initialization, and can speed neuroevolution by requiring less backpropagation epochs to be evaluated for each generated RNN.
Virtual equivariant Grothendieck-Riemann-Roch formula

For a $G$-scheme $X$ with a given equivariant perfect obstruction theory, we prove a virtual equivariant Grothendieck-Riemann-Roch formula, this is an extension of a result of Fantechi-Gottsche to the equivariant context. We also prove a virtual non-abelian localization theorem for schemes over $\mathbb{C}$ with proper actions.
ES Attack: Model Stealing Against Deep Neural Networks Without Data Hurdles

Deep neural networks (DNNs) have become the essential components for various commercialized machine learning services, such as Machine Learning as a Service (MLaaS). Recent studies show that machine learning services face severe privacy threats - well-trained DNNs owned by MLaaS providers can be stolen through public APIs, namely model stealing attacks. However, most existing works undervalued the impact of such attacks, where a successful attack has to acquire confidential training data or auxiliary data regarding the victim DNN. In this paper, we propose ES Attack, a novel model stealing attack without any data hurdles. By using heuristically generated synthetic data, ES Attack iteratively trains a substitute model and eventually achieves a functionally equivalent copy of the victim DNN. The experimental results reveal the severity of ES Attack: i) ES Attack successfully steals the victim model without data hurdles, and ES Attack even outperforms most existing model stealing attacks using auxiliary data in terms of model accuracy; ii) most countermeasures are ineffective in defending ES Attack; iii) ES Attack facilitates further attacks relying on the stolen model.
Repulsive Attention: Rethinking Multi-head Attention as Bayesian Inference

The neural attention mechanism plays an important role in many natural language processing applications. In particular, the use of multi-head attention extends single-head attention by allowing a model to jointly attend information from different perspectives. Without explicit constraining, however, multi-head attention may suffer from attention collapse, an issue that makes different heads extract similar attentive features, thus limiting the model's representation power. In this paper, for the first time, we provide a novel understanding of multi-head attention from a Bayesian perspective. Based on the recently developed particle-optimization sampling techniques, we propose a non-parametric approach that explicitly improves the repulsiveness in multi-head attention and consequently strengthens model's expressiveness. Remarkably, our Bayesian interpretation provides theoretical inspirations on the not-well-understood questions: why and how one uses multi-head attention. Extensive experiments on various attention models and applications demonstrate that the proposed repulsive attention can improve the learned feature diversity, leading to more informative representations with consistent performance improvement on various tasks.
Fast and Asymptotically Powerful Detection for Filamentary Objects in Digital Images.

Given an inhomogeneous chain embedded in a noisy image, we consider the conditions under which such an embedded chain is detectable. Many applications, such as detecting moving objects, detecting ship wakes, can be abstracted as the detection on the existence of chains. In this work, we provide the detection algorithm with low order of computation complexity to detect the chain and the optimal theoretical detectability regarding SNR (signal to noise ratio) under the normal distribution model. Specifically, we derive an analytical threshold that specifies what is detectable. We design a longest significant chain detection algorithm, with computation complexity in the order of $O(n\log n)$. We also prove that our proposed algorithm is asymptotically powerful, which means, as the dimension $n \rightarrow \infty$, the probability of false detection vanishes. We further provide some simulated examples and a real data example, which validate our theory.
Hidden Latent State Inference in a Spatio-Temporal Generative Model

Knowledge of the hidden factors that determine particular system dynamics is crucial for both explaining them and pursuing goal-directed, interventional actions. The inference of these factors without supervision given time series data remains an open challenge. Here, we focus on spatio-temporal processes, including wave propagations and weather dynamics, and assume that universal causes (e.g. physics) apply throughout space and time. We apply a novel DIstributed, Spatio-Temporal graph Artificial Neural network Architecture, DISTANA, which learns a generative model in such domains. DISTANA requires fewer parameters, and yields more accurate predictions than temporal convolutional neural networks and other related approaches on a 2D circular wave prediction task. We show that DISTANA, when combined with a retrospective latent state inference principle called active tuning, can reliably derive hidden local causal factors. In a current weather prediction benchmark, DISTANA infers our planet's land-sea mask solely by observing temperature dynamics and uses the self inferred information to improve its own prediction of temperature. We are convinced that the retrospective inference of latent states in generative RNN architectures will play an essential role in future research on causal inference and explainable systems.
Species limits in the Le Conte's Thrasher

We studied patterns of mitochondrial DNA (mtDNA) and colorimetric variation in the Le Conte's Thrasher (Toxostoma lecontei) to determine patterns of geographic variation, and to clarify subspecific and species limits. We sequenced 619 base pairs of mtDNA (parts of cytochrome b and ND6) for 14 Le Conte's Thrashers from 9 localities representing the three putative subspecies (lecontei, macmillanorum, and arenicola). Haplotypes were shared between lecontei and macmillanorum, and we do not support recognition of the latter. However, haplotypes of arenicola differed by 3.5% sequence divergence from those of lecontei and macmillanorum, and we suggest that arenicola be recognized as a species. Colorimetric analyses of 105 specimens indicated that arenicola differed consistently from lecontei and macmillanorum, corroborating the mtDNA results.
Learning Audio-Visual Representations with Active Contrastive Coding

Contrastive coding has achieved promising results in self-supervised representation learning. However, there are practical challenges given that obtaining a tight lower bound on mutual information (MI) requires a sample size exponential in MI and thus a large set of negative samples. We can incorporate more samples by building a large queue-based dictionary, but there are theoretical limits to performance improvements even with a large number of negative samples. We hypothesize that 'random negative sampling' leads to a highly redundant dictionary, which could result in representations that are suboptimal for downstream tasks. In this paper, we propose an active contrastive coding approach that builds an 'actively sampled' dictionary with diverse and informative items, which improves the quality of negative samples and achieves substantially improved results on tasks where there is high mutual information in the data, e.g., video classification. Our model achieves state-of-the-art performance on multiple challenging audio and visual downstream benchmarks including UCF101, HMDB51 and ESC50.
Distinguishing actions of symmetric groups and related graphs

The distinguishing number $D(G,X)$ of an action of a group $G$ on a set $X$ is the least size of a partition of $X$ such that no element of $G$ acting nontrivially on $X$ preserves this partition. In this paper we describe the distinguishing numbers for all actions of the symmetric group $S_n$, for any $n\geq 3$. This allows us to describe the distinguishing numbers for all graphs whose automorphism group is isomorphic with a symmetric group. Our description solves a few open problems posed by various authors in earlier papers on this topic.
CLOSURE PROPERTIES OF MEASURABLE ULTRAPOWERS

Abstract We study closure properties of measurable ultrapowers with respect to Hamkin’s notion of freshness and show that the extent of these properties highly depends on the combinatorial properties of the underlying model of set theory. In one direction, a result of Sakai shows that, by collapsing a strongly compact cardinal to become the double successor of a measurable cardinal, it is possible to obtain a model of set theory in which such ultrapowers possess the strongest possible closure properties. In the other direction, we use various square principles to show that measurable ultrapowers of canonical inner models only possess the minimal amount of closure properties. In addition, the techniques developed in the proofs of these results also allow us to derive statements about the consistency strength of the existence of measurable ultrapowers with non-minimal closure properties.
An experimental study of the effects of sheep grazing on vegetation change in a species-poor grassland and the role of seedling recruitment into gaps

An experiment was set up in 1986 on a species-poor grassland in Oxfordshire to determine the effect of sheep grazing management on vegetation change after cessation of fertilizer applications. Three seasons of grazing (winter, spring and summer) were applied, each with two grazing intensities, in a 2 × 2 × 2 factorial design with two blocks in 16 paddocks. Point quadrat surveys in 1990 showed that the grassland vegetation was dominated by perennial grasses and that the frequency distribution of species was highly skewed. Dicotyledonous species («dicots») were extremely rare, having an overall frequency of only 0.43%
Weak-shot Fine-grained Classification via Similarity Transfer

Recognizing fine-grained categories remains a challenging task, due to the subtle distinctions among different subordinate categories, which results in the need of abundant annotated samples. To alleviate the data-hungry problem, we consider the problem of learning novel categories from web data with the support of a clean set of base categories, which is referred to as weak-shot learning. Under this setting, we propose to transfer pairwise semantic similarity from base categories to novel categories, because this similarity is highly transferable and beneficial for learning from web data. Specifically, we firstly train a similarity net on clean data, and then employ two simple yet effective strategies to leverage the transferred similarity to denoise web training data. In addition, we apply adversarial loss on similarity net to enhance the transferability of similarity. Comprehensive experiments on three fine-grained datasets demonstrate that we could dramatically facilitate webly supervised learning by a clean set and similarity transfer is effective under this setting.
Energy-based Surprise Minimization for Multi-Agent Value Factorization

Multi-Agent Reinforcement Learning (MARL) has demonstrated significant success in training decentralised policies in a centralised manner by making use of value factorization methods. However, addressing surprise across spurious states and approximation bias remain open problems for multi-agent settings. We introduce the Energy-based MIXer (EMIX), an algorithm which minimizes surprise utilizing the energy across agents. Our contributions are threefold; (1) EMIX introduces a novel surprise minimization technique across multiple agents in the case of multi-agent partially-observable settings. (2) EMIX highlights the first practical use of energy functions in MARL (to our knowledge) with theoretical guarantees and experiment validations of the energy operator. Lastly, (3) EMIX presents a novel technique for addressing overestimation bias across agents in MARL. When evaluated on a range of challenging StarCraft II micromanagement scenarios, EMIX demonstrates consistent state-of-the-art performance for multi-agent surprise minimization. Moreover, our ablation study highlights the necessity of the energy-based scheme and the need for elimination of overestimation bias in MARL. Our implementation of EMIX and videos of agents are available at this https URL.
NetReduce: RDMA-Compatible In-Network Reduction for Distributed DNN Training Acceleration

We present NetReduce, a novel RDMA-compatible in-network reduction architecture to accelerate distributed DNN training. Compared to existing designs, NetReduce maintains a reliable connection between end-hosts in the Ethernet and does not terminate the connection in the network. The advantage of doing so is that we can fully reuse the designs of congestion control and reliability in RoCE. In the meanwhile, we do not need to implement a high-cost network protocol processing stack in the switch, as IB does. The prototype implemented by using FPGA is an out-of-box solution without modifying commodity devices such as NICs or switches. For the coordination between the end-host and the switch, NetReduce customizes the transport protocol only on the first packet in a data message to comply with RoCE v2. The special status monitoring module is designed to reuse the reliability mechanism of RoCE v2 for dealing with packet loss. A message-level credit-based flow control algorithm is also proposed to fully utilize bandwidth and avoid buffer overflow. We study the effects of intra bandwidth on the training performance in multi-machines multi-GPUs scenario and give sufficient conditions for hierarchical NetReduce to outperform other algorithms. We also extend the design from rack-level aggregation to more general spine-leaf topology in the data center. NetReduce accelerates the training up to 1.7x and 1.5x for CNN-based CV and transformer-based NLP tasks, respectively. Simulations on large-scale systems indicate the superior scalability of NetReduce to the state-of-the-art ring all-reduce.
Role of thyroxin versus Brewer’s yeast supplementation in amelioration of pancreatic alterations induced by hypothyroidism in adult male albino rats

Introduction Induction of hypothyroidism by carbimazole could affect many glands including the pancreas. Brewer’s yeast, a herbal product, is recently under research for its possible uses. Aim of the study The aim of the study was to study the microscopic, morphometric, and biochemical changes in the pancreas of hypothyroid rats and to evaluate the role of thyroxin versus Brewer’s yeast in amelioration of these changes. Materials and methods Sixty adult male albino rats were randomly divided into five groups. Group I rats served as a control group. Group II rats received carbimazole. Group III rats received carbimazole followed by thyroxin. Group IV rats received carbimazole followed by Brewer’s yeast. Group V rats received only Brewer’s yeast. At the end of the experimental period, specimens from the pancreas were processed for light and transmission electron microscopic examinations and immunostaining for Ki-67. Biochemical analysis for T3, T4, and insulin was performed. In addition, body mass gain and pancreatic mass were measured. All parameters were statistically analyzed. Results Group II specimens revealed congestion of blood vessels and cellular infiltration. Acinar cells showed pyknotic nuclei, hyalinization, and vacuolation of the cytoplasm, with few zymogen granules. &bgr; Cells of the islets of Langerhans revealed depletion of organelles and small granules without their characteristic halo. Immunostaining revealed significant decrease in the percentage of Ki-67-positive nuclei. In group III, the acini revealed numerous secretory granules, few vacuoles, and euchromatic nuclei. &bgr; Cells showed secretory granules with the characteristic halo. Group IV revealed less improvement in the histopathological changes as compared with group III. T3, T4, and insulin levels were significantly lower in hypothyroid rats in comparison with the control group, whereas administration of thyroxin showed restoration of these levels to near control values. Conclusion Hypothyroidism showed a deleterious effect on the histological structure of rat’s pancreas. Administration of thyroxin minimized these effects more than administration of Brewer’s yeast.
Defending against substitute model black box adversarial attacks with the 01 loss

Substitute model black box attacks can create adversarial examples for a target model just by accessing its output labels. This poses a major challenge to machine learning models in practice, particularly in security sensitive applications. The 01 loss model is known to be more robust to outliers and noise than convex models that are typically used in practice. Motivated by these properties we present 01 loss linear and 01 loss dual layer neural network models as a defense against transfer based substitute model black box attacks. We compare the accuracy of adversarial examples from substitute model black box attacks targeting our 01 loss models and their convex counterparts for binary classification on popular image benchmarks. Our 01 loss dual layer neural network has an adversarial accuracy of 66.2%, 58%, 60.5%, and 57% on MNIST, CIFAR10, STL10, and ImageNet respectively whereas the sigmoid activated logistic loss counterpart has accuracies of 63.5%, 19.3%, 14.9%, and 27.6%. Except for MNIST the convex counterparts have substantially lower adversarial accuracies. We show practical applications of our models to deter traffic sign and facial recognition adversarial attacks. On GTSRB street sign and CelebA facial detection our 01 loss network has 34.6% and 37.1% adversarial accuracy respectively whereas the convex logistic counterpart has accuracy 24% and 1.9%. Finally we show that our 01 loss network can attain robustness on par with simple convolutional neural networks and much higher than its convex counterpart even when attacked with a convolutional network substitute model. Our work shows that 01 loss models offer a powerful defense against substitute model black box attacks.
Semiclassical Gevrey operators in the complex domain

We study semiclassical Gevrey pseudodifferential operators, acting on exponentially weighted spaces of entire holomorphic functions. The symbols of such operators are Gevrey functions defined on suitable I-Lagrangian submanifolds of the complexified phase space, which are extended almost holomorphically in the same Gevrey class, or in some larger space, to complex neighborhoods of these submanifolds. Using almost holomorphic extensions, we obtain uniformly bounded realizations of such operators on a natural scale of exponentially weighted spaces of holomorphic functions for all Gevrey indices, with remainders that are optimally small, provided that the Gevrey index is $\leq 2$.
Conditionally Adaptive Multi-Task Learning: Improving Transfer Learning in NLP Using Fewer Parameters & Less Data

Multi-Task Learning (MTL) has emerged as a promising approach for transferring learned knowledge across different tasks. However, multi-task learning must deal with challenges such as: overfitting to low resource tasks, catastrophic forgetting, and negative task transfer, or learning interference. Additionally, in Natural Language Processing (NLP), MTL alone has typically not reached the performance level possible through per-task fine-tuning of pretrained models. However, many fine-tuning approaches are both parameter inefficient, e.g. potentially involving one new model per task, and highly susceptible to losing knowledge acquired during pretraining. We propose a novel transformer based architecture consisting of a new conditional attention mechanism as well as a set of task conditioned modules that facilitate weight sharing. Through this construction we achieve more efficient parameter sharing and mitigate forgetting by keeping half of the weights of a pretrained model fixed. We also use a new multi-task data sampling strategy to mitigate the negative effects of data imbalance across tasks. Using this approach we are able to surpass single-task fine-tuning methods while being parameter and data efficient. With our base model, we attain 2.2% higher performance compared to a full fine-tuned BERT large model on the GLUE benchmark, adding only 5.6% more trained parameters per task (whereas naive fine-tuning potentially adds 100% of the trained parameters per task) and needing only 64.6% of the data. We show that a larger variant of our single multi-task model approach performs competitively across 26 NLP tasks and yields state-of-the-art results on a number of test and development sets.