Miracle-dz/UniFAR-FTUs huggingface dataset:
Towards a Better Global Loss Landscape of GANs

Understanding of GAN training is still very limited. One major challenge is its non-convex-non-concave min-max objective, which may lead to sub-optimal local minima. In this work, we perform a global landscape analysis of the empirical loss of GANs. We prove that a class of separable-GAN, including the original JS-GAN, has exponentially many bad basins which are perceived as mode-collapse. We also study the relativistic pairing GAN (RpGAN) loss which couples the generated samples and the true samples. We prove that RpGAN has no bad basins. Experiments on synthetic data show that the predicted bad basin can indeed appear in training. We also perform experiments to support our theory that RpGAN has a better landscape than separable-GAN. For instance, we empirically show that RpGAN performs better than separable-GAN with relatively narrow neural nets. The code is available at this https URL.
Expressiveness of Neural Networks Having Width Equal or Below the Input Dimension

The expressiveness of deep neural networks of bounded width has recently been investigated in a series of articles. The understanding about the minimum width needed to ensure universal approximation for different kind of activation functions has progressively been extended (Park et al., 2020). In particular, it turned out that, with respect to approximation on general compact sets in the input space, a network width less than or equal to the input dimension excludes universal approximation. In this work, we focus on network functions of width less than or equal to the latter critical bound. We prove that in this regime the exact fit of partially constant functions on disjoint compact sets is still possible for ReLU network functions under some conditions on the mutual location of these components. Conversely, we conclude from a maximum principle that for all continuous and monotonic activation functions, universal approximation of arbitrary continuous functions is impossible on sets that coincide with the boundary of an open set plus an inner point of that set. We also show that some network functions of maximum width two, respectively one, allow universal approximation on finite sets.
Frobenius projective and affine geometry of varieties in positive characteristic.

The goal of the present paper is to lay the foundations for a theory of projective and affine structures on higher-dimensional varieties in positive characteristic. This theory deals with Frobenius-projective and Frobenius-affine structures, which have been previously investigated only in the case where the underlying varieties are curves. As the first step in expanding the theory, we prove various basic properties on Frobenius-projective and Frobenius-affine structures and study (the positive characteristic version of) the classification problem, starting with S. Kobayashi and T. Ochiai, of varieties admitting projective or affine structures. In the first half of the present paper, we construct bijective correspondences with several types of generalized indigenous bundles; one of them is defined in terms of Berthelot's higher-level differential operators. We also prove the positive characteristic version of Gunning's formulas, which give necessary conditions on Chern classes for the existence of Frobenius-projective or Frobenius-affine structures respectively. The second half of the present paper is devoted to studying, from the viewpoint of Frobenius-projective and Frobenius-affine structures, some specific classes of varieties, i.e., projective and affine spaces, abelian varieties, curves, and surfaces. For example, it is shown that the existence of Frobenius-projective structures of infinite level gives a characterization of ordinariness for abelian varieties. Also, we prove that any smooth projective curve of genus $>1$ admits a Frobenius-projective structure of infinite level, which induces a representation of the stratified fundamental group.
Attentive Social Recommendation: Towards User And Item Diversities

Social recommendation system is to predict unobserved user-item rating values by taking advantage of user-user social relation and user-item ratings. However, user/item diversities in social recommendations are not well utilized in the literature. Especially, inter-factor (social and rating factors) relations and distinct rating values need taking into more consideration. In this paper, we propose an attentive social recommendation system (ASR) to address this issue from two aspects. First, in ASR, Rec-conv graph network layers are proposed to extract the social factor, user-rating and item-rated factors and then automatically assign contribution weights to aggregate these factors into the user/item embedding vectors. Second, a disentangling strategy is applied for diverse rating values. Extensive experiments on benchmarks demonstrate the effectiveness and advantages of our ASR.
Anyonic Defect Braiding and Spontaneous Chiral Symmetry Breaking in Dihedral Liquid Crystals

Dihedral liquid crystals (DLCs) are assemblies of microscopic constituent particles that exhibit $k$-fold discrete rotational and reflection symmetries. Generalizing the half-integer defects in nematic liquid crystals, two-dimensional DLCs can host point defects of fractional topological charge $\pm m/k$. Starting from a generic microscopic model, we derive a unified hydrodynamic description of DLCs with aligning and anti-aligning short-range interactions in terms of Ginzburg-Landau and Swift-Hohenberg theories for a universal complex order-parameter field. Building on this framework, we demonstrate in both continuum and particle simulations how adiabatic braiding protocols, implemented through suitable boundary conditions, can realize classical counterparts of anyonic exchange symmetries. The theory further predicts a novel spontaneous chiral symmetry breaking transition in anti-aligning DLCs, in quantitative agreement with particle simulations. In view of recent advances in the design and controlled assembly of dihedral colloids, we expect that these theoretical predictions can be realized with currently available experimental technology, thereby promising a path to fractional topological information storage and processing in soft matter systems.
UmBERTo-MTSA @ AcCompl-It: Improving Complexity and Acceptability Prediction with Multi-task Learning on Self-Supervised Annotations (short paper)

This work describes a self-supervised data augmentation approach used to improve learning models' performances when only a moderate amount of labeled data is available. Multiple copies of the original model are initially trained on the downstream task. Their predictions are then used to annotate a large set of unlabeled examples. Finally, multi-task training is performed on the parallel annotations of the resulting training set, and final scores are obtained by averaging annotator-specific head predictions. Neural language models are fine-tuned using this procedure in the context of the AcCompl-it shared task at EVALITA 2020, obtaining considerable improvements in prediction quality.
A Variational Infinite Mixture for Probabilistic Inverse Dynamics Learning

Probabilistic regression techniques in control and robotics applications have to fulfill different criteria of data-driven adaptability, computational efficiency, scalability to high dimensions, and the capacity to deal with different modalities in the data. Classical regressors usually fulfill only a subset of these properties. In this work, we extend seminal work on Bayesian nonparametric mixtures and derive an efficient variational Bayes inference technique for infinite mixtures of probabilistic local polynomial models with well-calibrated certainty quantification. We highlight the model’s power in combining data-driven complexity adaptation, fast prediction, and the ability to deal with discontinuous functions and heteroscedastic noise. We benchmark this technique on a range of large real-world inverse dynamics datasets, showing that the infinite mixture formulation is competitive with classical Local Learning methods and regularizes model complexity by adapting the number of components based on data and without relying on heuristics. Moreover, to showcase the practicality of the approach, we use the learned models for online inverse dynamics control of a Barrett-WAM manipulator, significantly improving the trajectory tracking performance.
Gaussian Graphical Regression Models with High Dimensional Responses and Covariates

Though Gaussian graphical models have been widely used in many scientific fields, limited progress has been made to link graph structures to external covariates because of substantial challenges in theory and computation. We propose a Gaussian graphical regression model, which regresses both the mean and the precision matrix of a Gaussian graphical model on covariates. In the context of co-expression quantitative trait locus (QTL) studies, our framework facilitates estimation of both population- and subject-level gene regulatory networks, and detection of how subject-level networks vary with genetic variants and clinical conditions. Our framework accommodates high dimensional responses and covariates, and encourages covariate effects on both the mean and the precision matrix to be sparse. In particular for the precision matrix, we stipulate simultaneous sparsity, i.e., group sparsity and element-wise sparsity, on effective covariates and their effects on network edges, respectively. We establish variable selection consistency first under the case with known mean parameters and then a more challenging case with unknown means depending on external covariates, and show in both cases that the convergence rate of the estimated precision parameters is faster than that obtained by lasso or group lasso, a desirable property for the sparse group lasso estimation. The utility and efficacy of our proposed method is demonstrated through simulation studies and an application to a co-expression QTL study with brain cancer patients.
On Posterior Collapse and Encoder Feature Dispersion in Sequence VAEs.

Variational autoencoders (VAEs) hold great potential for modelling text, as they could in theory separate high-level semantic and syntactic properties from local regularities of natural language. Practically, however, VAEs with autoregressive decoders often suffer from posterior collapse, a phenomenon where the model learns to ignore the latent variables, causing the sequence VAE to degenerate into a language model. In this paper, we argue that posterior collapse is in part caused by the lack of dispersion in encoder features. We provide empirical evidence to verify this hypothesis, and propose a straightforward fix using pooling. This simple technique effectively prevents posterior collapse, allowing model to achieve significantly better data log-likelihood than standard sequence VAEs. Comparing to existing work, our proposed method is able to achieve comparable or superior performances while being more computationally efficient.
On the State of Social Media Data for Mental Health Research

Data-driven methods for mental health treatment and surveillance have become a major focus in computational science research in the last decade. However, progress in the domain remains bounded by the availability of adequate data. Prior systematic reviews have not necessarily made it possible to measure the degree to which data-related challenges have affected research progress. In this paper, we offer an analysis specifically on the state of social media data that exists for conducting mental health research. We do so by introducing an open-source directory of mental health datasets, annotated using a standardized schema to facilitate meta-analysis.
Lifting the curtain: The Seyfert galaxy Mrk 335 emerges from deep low-state in a sequence of rapid flare events

Context. The narrow-line Seyfert 1 galaxy Mrk 335 was one of the X-ray brightest active galactic nuclei, but it has systematically faded since 2007. Aims. We report the discovery with Swift of a sequence of bright and rapid X-ray flare events that reveal the emergence of Mrk 335 from its ultra-deep multiyear low state. Methods. Results are based on our dedicated multiyear monitoring of Mrk 335 with Swift. Results. Unlike other bright active galactic nuclei, the optical–UV is generally not correlated with the X-rays in Mrk 335 on a timescale of days to months. This fact either implies the absence of a direct link between the two emission components; or else implies that the observed X-rays are significantly affected by (dust-free) absorption along our line of sight. The UV and optical, however, are closely correlated at the 99.99% confidence level. The UV is leading the optical by Δt = 1.5 ± 1.5 d. The Swift X-ray spectrum shows strong deviations from a single power law in all brightness states of the outbursts, indicating that significant absorption or reprocessing is taking place. Mrk 335 displays a softer-when-brighter variability pattern at intermediate X-ray count rates, which has been seen in our Swift data since 2007 (based on a total of 590 observations). This pattern breaks down at the highest and lowest count rates. Conclusions. We interpret the 2020 brightening of Mrk 335 as a decrease in column density and covering factor of a partial-covering absorber along our line of sight in the form of a clumpy accretion-disk wind that reveals an increasing portion of the intrinsic emission of Mrk 335 from the disk and/or corona region, while the optical emission-line regions receive a less variable spectral energy distribution. This then also explains why Mrk 335 was never seen to change its optical Seyfert type (not “changing look”) despite its factor ∼50 X-ray variability with Swift.
A broadband signal recycling scheme for saturating the quantum limit from optical losses

Quantum noise limits the sensitivity of laser interferometric gravitational-wave detectors. Given the state-of-the-art optics, the optical losses define the lower bound of best possible quantum-limited detector sensitivity. In this work, we come up with the configuration which allows to saturate this lower bound by converting the signal recycling cavity to be a broadband signal amplifier using an active optomechanical filter. We will show the difference and advantage of such a broadband signal recycling scheme compared with the previous white-light-cavity scheme using the optomechanical filter in [Phys.Rev.Lett.115. 211104 (2015)]. The drawback is that the new scheme is more susceptible to the thermal noise of the mechanical oscillator. To suppress the radiation pressure noise which rises along with the signal amplification, squeezing with input/output filter cavities and heavier test mass are used in this work.
On the dynamics of extensions of free-abelian times free groups endomorphisms to the completion

Abstract We obtain conditions of uniform continuity for endomorphisms of free-abelian times free groups for the product metric defined by taking the prefix metric in each component and establish an equivalence between uniform continuity for this metric and the preservation of a coarse-median, a concept recently introduced by Fioravanti. Considering the extension of an endomorphism to the completion, we count the number of orbits for the action of the subgroup of fixed points (respectively periodic) points on the set of infinite fixed (respectively periodic) points. Finally, we study the dynamics of infinite points: for automorphisms and some endomorphisms, defined in a precise way, fitting a classification given by Delgado and Ventura, we prove that every infinite point is either periodic or wandering, which implies that the dynamics is asymptotically periodic.
Model-based Reinforcement Learning from Signal Temporal Logic Specifications

Techniques based on Reinforcement Learning (RL) are increasingly being used to design control policies for robotic systems. RL fundamentally relies on state-based reward functions to encode desired behavior of the robot and bad reward functions are prone to exploitation by the learning agent, leading to behavior that is undesirable in the best case and critically dangerous in the worst. On the other hand, designing good reward functions for complex tasks is a challenging problem. In this paper, we propose expressing desired high-level robot behavior using a formal specification language known as Signal Temporal Logic (STL) as an alternative to reward/cost functions. We use STL specifications in conjunction with model-based learning to design model predictive controllers that try to optimize the satisfaction of the STL specification over a finite time horizon. The proposed algorithm is empirically evaluated on simulations of robotic system such as a pick-and-place robotic arm, and adaptive cruise control for autonomous vehicles.
Classification of optics-free images with deep neural networks

The thinnest possible camera is achieved by removing all optics, leaving only the image sensor. We train deep neural networks to perform multi-class detection and binary classification (with accuracy of 92%) on optics-free images without the need for anthropocentric image reconstructions. Inferencing from optics-free images has the potential for enhanced privacy and power efficiency.
Measurements of 
W
 and 
Z/γ*
 cross sections and their ratios in 
p+p
 collisions at RHIC

We report on the $W$ and $Z/\gamma^*$ differential and total cross sections as well as the $W^+$/$W^-$ and $(W^+ + W^-)$/$(Z/\gamma^*)$ cross-section ratios measured by the STAR experiment at RHIC in $p+p$ collisions at $\sqrt{s} = 500$ GeV and $510$ GeV. The cross sections and their ratios are sensitive to quark and antiquark parton distribution functions. In particular, at leading order, the $W$ cross-section ratio is sensitive to the $\bar{d}/\bar{u}$ ratio. These measurements were taken at high $Q^2 \sim M_W^2,M_Z^2$ and can serve as input into global analyses to provide constraints on the sea quark distributions. The results presented here combine three STAR data sets from 2011, 2012, and 2013, accumulating an integrated luminosity of 350 pb$^{-1}$. We also assess the expected impact that our $W^+/W^-$ cross-section ratios will have on various quark distributions, and find sensitivity to the $\bar{u}-\bar{d}$ and $\bar{d}/\bar{u}$ distributions.
Learning with Optimized Random Features: Exponential Speedup by Quantum Machine Learning without Sparsity and Low-Rank Assumptions

Kernel methods augmented with random features give scalable algorithms for learning from big data. But it has been computationally hard to sample random features according to a probability distribution that is optimized for the data, so as to minimize the required number of features for achieving the learning to a desired accuracy. Here, we develop a quantum algorithm for sampling from this optimized distribution over features, in runtime $O(D)$ that is linear in the dimension $D$ of the input data. Our algorithm achieves an exponential speedup in $D$ compared to any known classical algorithm for this sampling task. In contrast to existing quantum machine learning algorithms, our algorithm circumvents sparsity and low-rank assumptions and thus has wide applicability. We also show that the sampled features can be combined with regression by stochastic gradient descent to achieve the learning without canceling out our exponential speedup. Our algorithm based on sampling optimized random features leads to an accelerated framework for machine learning that takes advantage of quantum computers.
Comparison of Statistical and Machine Learning Techniques for Physical Layer Authentication

In this article we consider authentication at the physical layer, in which the authenticator aims at distinguishing a legitimate supplicant from an attacker on the basis of the characteristics of a set of parallel wireless channels, which are affected by time-varying fading. Moreover, the attacker’s channel has a spatial correlation with the supplicant’s one. In this setting, we assess and compare the performance achieved by different approaches under different channel conditions. We first consider the use of two different statistical decision methods, and we prove that using a large number of references (in the form of channel estimates) affected by different levels of time-varying fading is not beneficial from a security point of view. We then consider classification methods based on machine learning. In order to face the worst case scenario of an authenticator provided with no forged messages during training, we consider one-class classifiers. When instead the training set includes some forged messages, we resort to more conventional binary classifiers, considering the cases in which such messages are either labelled or not. For the latter case, we exploit clustering algorithms to label the training set. The performance of both nearest neighbor (NN) and support vector machine (SVM) classification techniques is evaluated. Through numerical examples, we show that under the same probability of false alarm, one-class classification (OCC) algorithms achieve the lowest probability of missed detection when a small spatial correlation exists between the main channel and the adversary one, while statistical methods are advantageous when the spatial correlation between the two channels is large.
5G satellite networks for IoT: offloading and backhauling

One of the main drivers of 5G cellular networks is provision of connectivity service for various Internet of Things (IoT) devices. Considering the potential volume of IoT devices at a global scale, the next leap is to integrate Non-Terrestrial Networks (NTN) into 5G terrestrial systems, thereby extend the coverage and complement the terrestrial service. This paper focuses on the use of Low-Earth Orbit (LEO) satellite constellations for two specific purposes: offloading and backhauling. The former allows offloading IoT traffic from a congested terrestrial network, usually in a very dense area. In the latter application, the constellation provides a multi-hop backhaul that connects a remote terrestrial gNB to the 5G core network. After providing an overview of the status of the 3GPP standardization process, we model and analyze the user data performance, specifically in the uplink access and the satellite multi-hop constellation path. The evaluation of the collisions, the delay and the Age of Information, and the comparison of the terrestrial and the satellite access networks provide useful insights to understand the potential of LEO constellations for offloading and backhauling of IoT traffic.
Pretraining Strategies, Waveform Model Choice, and Acoustic Configurations for Multi-Speaker End-to-End Speech Synthesis

We explore pretraining strategies including choice of base corpus with the aim of choosing the best strategy for zero-shot multi-speaker end-to-end synthesis. We also examine choice of neural vocoder for waveform synthesis, as well as acoustic configurations used for mel spectrograms and final audio output. We find that fine-tuning a multi-speaker model from found audiobook data that has passed a simple quality threshold can improve naturalness and similarity to unseen target speakers of synthetic speech. Additionally, we find that listeners can discern between a 16kHz and 24kHz sampling rate, and that WaveRNN produces output waveforms of a comparable quality to WaveNet, with a faster inference time.