Miracle-dz/UniFAR-FTUs huggingface dataset:
Prototypical Graph Contrastive Learning

Graph-level representations are critical in various real-world applications, such as predicting the properties of molecules. However, in practice, precise graph annotations are generally very expensive and time-consuming. To address this issue, graph contrastive learning constructs an instance discrimination task, which pulls together positive pairs (augmentation pairs of the same graph) and pushes away negative pairs (augmentation pairs of different graphs) for unsupervised representation learning. However, since for a query, its negatives are uniformly sampled from all graphs, existing methods suffer from the critical sampling bias issue, i.e., the negatives likely having the same semantic structure with the query, leading to performance degradation. To mitigate this sampling bias issue, in this article, we propose a prototypical graph contrastive learning (PGCL) approach. Specifically, PGCL models the underlying semantic structure of the graph data via clustering semantically similar graphs into the same group and simultaneously encourages the clustering consistency for different augmentations of the same graph. Then, given a query, it performs negative sampling via drawing the graphs from those clusters that differ from the cluster of query, which ensures the semantic difference between query and its negative samples. Moreover, for a query, PGCL further reweights its negative samples based on the distance between their prototypes (cluster centroids) and the query prototype such that those negatives having moderate prototype distance enjoy relatively large weights. This reweighting strategy is proven to be more effective than uniform sampling. Experimental results on various graph benchmarks testify the advantages of our PGCL over state-of-the-art methods. The code is publicly available at https://github.com/ha-lins/PGCL.
On the Independence of Electromagnetism and Gravitation from Test Bodies Physical Properties

Does a physical field exist independently of the interaction between the field source and the test body used to measure it at a given point? What does propagate between two physical bodies when they interact? These are the fundamental questions we answer in the present work for electromagnetic and gravitational interactions. We come to the conclusion that gravitational fields as well as spacetime curvature cannot depend on the passive-gravitational to inertial mass ratio of the test bodies, because this would lead to energy conservation and causality violation except when this ratio is universally exactly equal to one. For what concerns the hypothetical dependence of electromagnetic fields on the charge to inertial-mass ratio of test bodies, this would call for the propagation of an operationally ill defined quantity (Coulomb . Meter2 / Second2) that would replace energy (Joule = Kilogram . Meter2 / Second2) transfer in the interaction between two electrically charged bodies. Hence electromagnetic fields cannot depend on the charge to inertial-mass ratio of test bodies. A consequence of these results is also the disqualification of Murat Ozer’s unification theory of gravitation and electromagnetism.
Theory of superconductivity due to Ngai's mechanism in lightly doped 
SrTiO3

We develop a theory of superconducting pairing in low-density Strontium titanate due to quadratic coupling of electron density to soft transverse optical phonons [1]. It leads to static attractive potential between electrons which decay length leff that scales inversely with soft optical gap ωT . For low electron densities n ≤ 1018cm−3 attraction between electrons is local and transition temperature Tc was found using Ref. [2]. The Tc(n) dependence in agreement with experimental data [3] for low doping was calculated. Next, we show that suppression of Tc by hydrostatic pressure [4] and strong increase of Tc due to isotop substitution O → O observed in [5] are explained within our theory.
Counting Discrete, Level-$1$, Quaternionic Automorphic Representations on $G_2$

Quaternionic automorphic representations are one attempt to generalize to other groups the special place holomorphic modular forms have among automorphic representations of $\mathrm{GL}_2$. Here, we use"hyperendoscopy"techniques to develop a general trace formula and understand them on an arbitrary group. Then we specialize this general formula to study quaternionic automorphic representations on the exceptional group $G_2$, eventually getting an analog of the Eichler-Selberg trace formula for classical modular forms. We finally use this together with some techniques of Chenevier, Renard, and Ta\"ibi to compute dimensions of spaces of level-$1$ quaternionic representations. On the way, we prove a Jacquet-Langlands-style result describing them in terms of classical modular forms and automorphic representations on the compact-at-infinity form $G_2^c$. The main technical difficulty is that the quaternionic discrete series that quaternionic automorphic representations are defined in terms of do not satisfy a condition of being"regular". A real representation theory argument shows that regularity miraculously does not matter for specifically the case of quaternionic discrete series. We hope that the techniques and shortcuts highlighted in this project are of interest in other computations about discrete-at-infinity automorphic representations on arbitrary reductive groups instead of just classical ones.
Asymptotic shape of the concave majorant of a Lévy process

We establish distributional limit theorems for the shape statistics of a concave majorant (i.e. the fluctuations of its length, its supremum, the time it is attained and its value at $T$) of any L\'evy process on $[0,T]$ as $T\to\infty$. The scale of the fluctuations of the length and other statistics, as well as their asymptotic dependence, vary significantly with the tail behaviour of the L\'evy measure. The key tool in the proofs is the recent representation of the concave majorant for all L\'evy processes using a stick-breaking representation.
Compactness for $\Omega$-Yang-Mills connections

On a Riemannian manifold of dimension $n$ we extend the known analytic results on Yang-Mills connections to the class of connections called $\Omega$-Yang-Mills connections, where $\Omega$ is a smooth, not necessarily closed, $(n-4)$-form. Special cases include $\Omega$-anti-self-dual connections and Hermitian-Yang-Mills connections over general complex manifolds. By a key observation, a weak compactness result is obtained for moduli space of smooth $\Omega$-Yang-Mills connections with uniformly $L^2$ bounded curvature, and it can be improved in the case of Hermitian-Yang-Mills connections over general complex manifolds. A removable singularity theorem for singular $\Omega$-Yang-Mills connections on a trivial bundle with small energy concentration is also proven. As an application, it is shown how to compactify the moduli space of smooth Hermitian-Yang-Mills connections on unitary bundles over a class of balanced manifolds of Hodge-Riemann type. This class includes the metrics coming from multipolarizations, and in particular, the Kaehler metrics. In the case of multipolarizations on a projective algebraic manifold, the compactification of smooth irreducible Hermitian-Yang-Mills connections with fixed determinant modulo gauge transformations inherits a complex structure from algebro-geometric considerations.
Narrow bands, electrostatic interactions and band topology in graphene stacks

The occurrence of superconducting and insulating phases is well-established in twisted graphene bilayers, and they have also been reported in other arrangements of graphene layers. We investigate three such arrangements: untwisted AB bilayer graphene on an hBN substrate, two graphene bilayers twisted with respect to each other, and a single ABC stacked graphene trilayer on an hBN substrate. Narrow bands with different topology occur in all cases, producing a high density of states which enhances the role of interactions. We investigate the effect of the long range Coulomb interaction, treated within the self consistent Hartree–Fock approximation. We find that the on-site part of the Fock potential strongly modifies the band structure at charge neutrality. The Hartree part does not significantly modify the shape and width of the bands in the three cases considered here, in contrast to the effect that such a potential has in twisted bilayer graphene.
Orthogonal-Padé Activation Functions: Trainable Activation functions for smooth and faster convergence in deep networks

We have proposed orthogonal-Pad\'e activation functions, which are trainable activation functions and show that they have faster learning capability and improves the accuracy in standard deep learning datasets and models. Based on our experiments, we have found two best candidates out of six orthogonal-Pad\'e activations, which we call safe Hermite-Pade (HP) activation functions, namely HP-1 and HP-2. When compared to ReLU, HP-1 and HP-2 has an increment in top-1 accuracy by 5.06% and 4.63% respectively in PreActResNet-34, by 3.02% and 2.75% respectively in MobileNet V2 model on CIFAR100 dataset while on CIFAR10 dataset top-1 accuracy increases by 2.02% and 1.78% respectively in PreActResNet-34, by 2.24% and 2.06% respectively in LeNet, by 2.15% and 2.03% respectively in Efficientnet B0.
Author Clustering and Topic Estimation for Short Texts

Analysis of short text, such as social media posts, is extremely diﬃcult because of their inherent brevity. In addition to classifying topics of such posts, a common downstream task is grouping the authors of these documents for subsequent analyses. We propose a novel model that expands on the Latent Dirichlet Allocation by modeling strong dependence among the words in the same document, with user-level topic distributions. We also simultaneously cluster users, removing the need for post-hoc cluster estimation and improving topic estimation by shrinking noisy user-level topic distributions towards typical values. Our method performs as well as — or better — than traditional approaches, and we demonstrate its usefulness on a dataset of tweets from United States Senators, recovering both meaningful topics and clusters that reﬂect partisan ideology. We also develop a novel measure of echo chambers among these politicians by characterizing insularity of topics discussed by groups of Senators and provide uncertainty quantiﬁcation.
Spherically Symmetric Exact Vacuum Solutions in Einstein-Aether Theory

We study spherically symmetric spacetimes in Einstein-aether theory in three different coordinate systems, the isotropic, Painlevè-Gullstrand, and Schwarzschild coordinates, in which the aether is always comoving, and present both time-dependent and time-independent exact vacuum solutions. In particular, in the isotropic coordinates we find a class of exact static solutions characterized by a single parameter c14 in closed forms, which satisfies all the current observational constraints of the theory, and reduces to the Schwarzschild vacuum black hole solution in the decoupling limit (c14=0). However, as long as c14≠0, a marginally trapped throat with a finite non-zero radius always exists, and on one side of it the spacetime is asymptotically flat, while on the other side the spacetime becomes singular within a finite proper distance from the throat, although the geometric area is infinitely large at the singularity. Moreover, the singularity is a strong and spacetime curvature singularity, at which both of the Ricci and Kretschmann scalars become infinitely large.
Band topology of pseudo-Hermitian phases through tensor Berry connections and quantum metric

Yan-Qing Zhu, 2, 3 Wen Zheng, Shi-Liang Zhu, 2, ∗ and Giandomenico Palumbo † Guangdong Provincial Key Laboratory of Quantum Engineering and Quantum Materials, School of Physics and Telecommunication Engineering, South China Normal University, Guangzhou 510006, China Guangdong-Hong Kong Joint Laboratory of Quantum Matter, Frontier Research Institute for Physics, South China Normal University, Guangzhou 510006, China The University of Hong Kong Shenzhen Institute of Research and Innovation, Shenzhen 518057, China National Laboratory of Solid State Microstructures, School of Physics, Nanjing University, Nanjing 210093, China School of Theoretical Physics, Dublin Institute for Advanced Studies, 10 Burlington Road, Dublin 4, Ireland (Dated: November 9, 2021)
WaveGrad 2: Iterative Refinement for Text-to-Speech Synthesis

This paper introduces WaveGrad 2, a non-autoregressive generative model for text-to-speech synthesis. WaveGrad 2 is trained to estimate the gradient of the log conditional density of the waveform given a phoneme sequence. The model takes an input phoneme sequence, and through an iterative refinement process, generates an audio waveform. This contrasts to the original WaveGrad vocoder which conditions on mel-spectrogram features, generated by a separate model. The iterative refinement process starts from Gaussian noise, and through a series of refinement steps (e.g., 50 steps), progressively recovers the audio sequence. WaveGrad 2 offers a natural way to trade-off between inference speed and sample quality, through adjusting the number of refinement steps. Experiments show that the model can generate high fidelity audio, approaching the performance of a state-of-the-art neural TTS system. We also report various ablation studies over different model configurations. Audio samples are available at https://wavegrad.github. io/v2.
Stellar Shocks From Dark Matter

Macroscopic dark matter is almost unconstrained over a wide “asteroid-like” mass range, where it could scatter on baryonic matter with geometric cross section. We show that when such an object travels through a star, it produces shock waves which reach the stellar surface, leading to a distinctive transient optical, UV and X-ray emission. This signature can be searched for on a variety of stellar types and locations. In a dense globular cluster, such events occur far more often than flare backgrounds, and an existing UV telescope could probe orders of magnitude in dark matter mass in one week of dedicated observation.
CoANE: Modeling Context Co-Occurrence for Attributed Network Embedding

Attributed network embedding (ANE) is to learn low-dimensional vectors so that not only the network structure but also node attributes can be preserved in the embedding space. Existing ANE models do not consider the specific combination between graph structure and attributes. While each node has its structural characteristics, such as highly-interconnected neighbors along with their certain patterns of attribute distribution, each node's neighborhood should be not only depicted by multi-hop nodes, but consider certain clusters or social circles. To model such information, in this paper, we propose a novel ANE model, Context Co-occurrence-aware Attributed Network Embedding (CoANE). The basic idea of CoANE is to model the context attributes that each node's involved diverse patterns, and apply the convolutional mechanism to encode positional information by treating each attribute as a channel. The learning of context co-occurrence can capture the latent social circles of each node. To better encode structural and semantic knowledge of nodes, we devise a three-way objective function, consisting of positive graph likelihood, contextual negative sampling, and attribute reconstruction. We conduct experiments on five real datasets in the tasks of link prediction, node label classification, and node clustering. The results exhibit that CoANE can significantly outperform state-of-the-art ANE models.
Rotation Invariant Graph Neural Networks using Spin Convolutions

Progress towards the energy breakthroughs needed to combat climate change can be significantly accelerated through the efficient simulation of atomic systems. Simulation techniques based on first principles, such as Density Functional Theory (DFT), are limited in their practical use due to their high computational expense. Machine learning approaches have the potential to approximate DFT in a computationally efficient manner, which could dramatically increase the impact of computational simulations on real-world problems. Approximating DFT poses several challenges. These include accurately modeling the subtle changes in the relative positions and angles between atoms, and enforcing constraints such as rotation invariance or energy conservation. We introduce a novel approach to modeling angular information between sets of neighboring atoms in a graph neural network. Rotation invariance is achieved for the network's edge messages through the use of a per-edge local coordinate frame and a novel spin convolution over the remaining degree of freedom. Two model variants are proposed for the applications of structure relaxation and molecular dynamics. State-of-the-art results are demonstrated on the large-scale Open Catalyst 2020 dataset. Comparisons are also performed on the MD17 and QM9 datasets.
Scientific Language Models for Biomedical Knowledge Base Completion: An Empirical Study

Biomedical knowledge graphs (KGs) hold rich information on entities such as diseases, drugs, and genes. Predicting missing links in these graphs can boost many important applications, such as drug design and repurposing. Recent work has shown that general-domain language models (LMs) can serve as"soft"KGs, and that they can be fine-tuned for the task of KG completion. In this work, we study scientific LMs for KG completion, exploring whether we can tap into their latent knowledge to enhance biomedical link prediction. We evaluate several domain-specific LMs, fine-tuning them on datasets centered on drugs and diseases that we represent as KGs and enrich with textual entity descriptions. We integrate the LM-based models with KG embedding models, using a router method that learns to assign each input example to either type of model and provides a substantial boost in performance. Finally, we demonstrate the advantage of LM models in the inductive setting with novel scientific entities. Our datasets and code are made publicly available.
Uniform Convergence of Interpolators: Gaussian Width, Norm Bounds, and Benign Overfitting

We consider interpolation learning in high-dimensional linear regression with Gaussian data, and prove a generic uniform convergence guarantee on the generalization error of interpolators in an arbitrary hypothesis class in terms of the class's Gaussian width. Applying the generic bound to Euclidean norm balls recovers the consistency result of Bartlett et al. (2020) for minimum-norm interpolators, and confirms a prediction of Zhou et al. (2020) for near-minimal-norm interpolators in the special case of Gaussian data. We demonstrate the generality of the bound by applying it to the simplex, obtaining a novel consistency result for minimum l1-norm interpolators (basis pursuit). Our results show how norm-based generalization bounds can explain and be used to analyze benign overfitting, at least in some settings.
A Survey on Semi-supervised Learning for Delayed Partially Labelled Data Streams

Unlabelled data appear in many domains and are particularly relevant to streaming applications, where even though data is abundant, labelled data is rare. To address the learning problems associated with such data, one can ignore the unlabelled data and focus only on the labelled data (supervised learning); use the labelled data and attempt to leverage the unlabelled data (semi-supervised learning); or assume some labels will be available on request (active learning). The first approach is the simplest, yet the amount of labelled data available will limit the predictive performance. The second relies on finding and exploiting the underlying characteristics of the data distribution. The third depends on an external agent to provide the required labels in a timely fashion. This survey pays special attention to methods that leverage unlabelled data in a semi-supervised setting. We also discuss the delayed labelling issue, which impacts both fully supervised and semi-supervised methods. We propose a unified problem setting, discuss the learning guarantees and existing methods, and explain the differences between related problem settings. Finally, we review the current benchmarking practices and propose adaptations to enhance them.
Meta-Calibration: Learning of Model Calibration Using Differentiable Expected Calibration Error

Calibration of neural networks is a topical problem that is becoming more and more important as neural networks increasingly underpin real-world applications. The problem is especially noticeable when using modern neural networks, for which there is a significant difference between the confidence of the model and the probability of correct prediction. Various strategies have been proposed to improve calibration, yet accurate calibration remains challenging. We propose a novel framework with two contributions: introducing a new differentiable surrogate for expected calibration error (DECE) that allows calibration quality to be directly optimised, and a meta-learning framework that uses DECE to optimise for validation set calibration with respect to model hyper-parameters. The results show that we achieve competitive performance with existing calibration approaches. Our framework opens up a new avenue and toolset for tackling calibration, which we believe will inspire further work on this important challenge.
Spiral Structure Boosts Star Formation in Disk Galaxies

We investigate the impact of spiral structure on global star formation using a sample of 2226 nearby bright disk galaxies. Examining the relationship between spiral arms, star formation rate (SFR), and stellar mass, we find that arm strength correlates well with the variation of SFR as a function of stellar mass. Arms are stronger above the star-forming galaxy main sequence (MS) and weaker below it: arm strength increases with higher log(SFR/SFRMS) , where SFRMS is the SFR along the MS. Likewise, stronger arms are associated with higher specific SFR. We confirm this trend using the optical colors of a larger sample of 4378 disk galaxies, whose position on the blue cloud also depends systematically on spiral arm strength. This link is independent of other galaxy structural parameters. For the subset of galaxies with cold gas measurements, arm strength positively correlates with H i and H2 mass fraction, even after removing the mutual dependence on log(SFR/SFRMS) , consistent with the notion that spiral arms are maintained by dynamical cooling provided by gas damping. For a given gas fraction, stronger arms lead to higher log(SFR/SFRMS) , resulting in a trend of increasing arm strength with shorter gas depletion time. We suggest a physical picture in which the dissipation process provided by gas damping maintains spiral structure, which, in turn, boosts the star formation efficiency of the gas reservoir.
Temporal Logic Planning for Minimum-Time Positioning of Multiple Threat-Seduction Decoys

Reusable decoys offer a cost-effective alternative to the single-use hardware commonly applied to protect surface assets from threats. Such decoys portray fake assets to lure threats away from the true asset. To deceive a threat, a decoy first has to position itself such that it can break the radar lock. Considering multiple simultaneous threats, this paper introduces an approach for controlling multiple decoys to minimise the time required to break the locks of all the threats. The method includes the optimal allocation of one decoy to every threat with an assignment procedure that provides local position constraints to guarantee collision avoidance and thereby decouples the control of the decoys. A crude model of a decoy with uncertainty is considered for motion planning. The task of a decoy reaching a state in which the lock of the assigned threat can be broken is formulated as a temporal logic specification. To this end, the requirements to complete the task are modelled as time-varying set-membership constraints. The temporal and logical combination of the constraints is encoded in a mixed-integer optimisation problem. To demonstrate the results a simulated case study is provided.