Miracle-dz/UniFAR-FTUs huggingface dataset:
Two-Hole Ground State: Dichotomy in Pairing Symmetry

A single-hole ground state Ansatz for the two-dimensional t-J model has been recently studied by the variational Monte Carlo (VMC) method. Such a doped hole behaves like a “twisted” non-Landau quasiparticle characterized by an emergent quantum number in agreement with exact numerics. In this work, we further investigate the ground state of two holes by VMC. It is found that the two holes strongly attract each other to form a pairing state with a new quantum number the same as obtained by the numerical exact diagonalization and density matrix renormalization group (DMRG) calculations. A unique feature of this pairing state is a dichotomy in the pairing symmetry, i.e., a d wave in terms of the electron c operators and an s wave in terms of the new quasiparticles, as explicitly illustrated in the ground state wave function. A similar VMC study of a two-hole wave function for the t-J two-leg ladder also yields a good agreement with the DMRG result. We demonstrate that the pairing mechanism responsible for the strong binding here is not due to the long-range antiferromagnetic order nor the resonating-valence-bound pairing in the spin background but is the consequence of the quantum phase-strings created by the hopping of holes. The resulting spin-current pattern mediating the pairing force is explicitly illustrated in the VMC calculation. Physical implications to superconductivity at finite doping are also discussed.
Predictive Control based on Equivalent Dynamic Linearization Model

Based on equivalent-dynamic-linearization model (EDLM), we propose a kind of model predictive control (MPC) for single-input and single-output (SISO) nonlinear or linear systems. After compensating the EDLM with disturbance for multiple-input and multiple-output nonlinear or linear systems, the MPC compensated with disturbance is proposed to address the disturbance rejection problem. The system performance analysis results are much clear compared with the system stability analyses on MPC in current works. And this may help the engineers understand how to design, analyze and apply the controller in practical.
Making the Most of Small Software Engineering Datasets With Modern Machine Learning

This paper provides a starting point for Software Engineering (SE) researchers and practitioners faced with the problem of training machine learning models on small datasets. Due to the high costs associated with labeling data, in Software Engineering, there exist many small (< 5,000 samples) and medium-sized (<100,000 samples) datasets. While deep learning has set the state of the art in many machine learning tasks, it is only recently that it has proven effective on small-sized datasets, primarily thanks to pre-training, a semi-supervised learning technique that leverages abundant unlabelled data alongside scarce labelled data. In this work, we evaluate pre-trained Transformer models on a selection of 13 smaller datasets from the SE literature, covering both, source code and natural language. Our results suggest that pre-trained Transformers are competitive and in some cases superior to previous models, especially for tasks involving natural language; whereas for source code tasks, in particular for very small datasets, traditional machine learning methods often has the edge. In addition, we experiment with several techniques that ought to aid training on small datasets, including active learning, data augmentation, soft labels, self-training and intermediate-task fine-tuning, and issue recommendations on when they are effective. We also release all the data, scripts, and most importantly pre-trained models for the community to reuse on their own datasets.
Subgroup Generalization and Fairness of Graph Neural Networks

Despite enormous successful applications of graph neural networks (GNNs), theoretical understanding of their generalization ability, especially for node-level tasks where data are not independent and identically-distributed (IID), has been sparse. The theoretical investigation of the generalization performance is beneficial for understanding fundamental issues (such as fairness) of GNN models and designing better learning methods. In this paper, we present a novel PAC-Bayesian analysis for GNNs under a non-IID semi-supervised learning setup. Moreover, we analyze the generalization performances on different subgroups of unlabeled nodes, which allows us to further study an accuracy-(dis)parity-style (un)fairness of GNNs from a theoretical perspective. Under reasonable assumptions, we demonstrate that the distance between a test subgroup and the training set can be a key factor affecting the GNN performance on that subgroup, which calls special attention to the training node selection for fair learning. Experiments across multiple GNN models and datasets support our theoretical results.
Convergent and Efficient Deep Q Network Algorithm

Despite the empirical success of the deep Q network (DQN) reinforcement learning algorithm and its variants, DQN is still not well understood and it does not guarantee convergence. In this work, we show that DQN can indeed diverge and cease to operate in realistic settings. Although there exist gradient-based convergent methods, we show that they actually have inherent problems in learning dynamics which cause them to fail even in simple tasks. To overcome these problems, we propose a convergent DQN algorithm (C-DQN) that is guaranteed to converge and can work with large discount factors (0.9998). It learns robustly in difficult settings and can learn several difficult games in the Atari 2600 benchmark that DQN fails to solve. Our codes have been publicly released and can be used to reproduce our results.
Geometry-aware Transformer for molecular property prediction

Recently, message passing neural networks (MPNNs) have achieved remarkable performances for various chemical tasks. However, the meesage passing scheme can only cover a local ﬁeld, and cannot capture long-range relationship between atoms. This behavior does not accord with chemical theory, which is one of the fundamental limitations of the MPNNs. In this work, we propose a novel Transformer-based framework for representation learning on molecules, named Geometry-aware Transformer (GeoT). GeoT learns sequential representations of molecule graphs based on global attentions. By virtue of the Transformer-based architecture and additional reﬁnements, the smoothing problem can be signiﬁcantly alleviated compared to graph convolution-based architectures. GeoT achieved comparable performances with other recent MPNN-based models. We also veriﬁed the intuition behind GeoT with respect to the reasonability and interpretability, through attention visualization.
Bottleneck Time Minimization for Distributed Iterative Processes: Speeding Up Gossip-Based Federated Learning on Networked Computers

We present a novel task scheduling scheme for accelerating computational applications involving distributed iterative processes that are executed on networked computing resources. Such an application consists of multiple tasks, each of which outputs data at each iteration to be processed by neighboring tasks; these dependencies between the tasks can be represented as a directed graph. We first mathematically formulate the problem as a Binary Quadratic Program (BQP), accounting for both computation and communication costs. We show that the problem is NP-hard. We then relax the problem as a Semi-Definite Program (SDP) and utilize a randomized rounding technique based on sampling from a suitably-formulated multi-variate Gaussian distribution. Furthermore, we derive the expected value of bottleneck time. Finally, we apply our proposed scheme on gossip-based federated learning as an application of iterative processes. Through numerical evaluations on the MNIST and CIFAR-10 datasets, we show that our proposed approach outperforms well-known scheduling techniques from distributed computing. In particular, for arbitrary settings, we show that it reduces bottleneck time by $91\%$ compared to HEFT and $84\%$ compared to throughput HEFT.
Parity Violating Non-Gaussianity from Axion-Gauge Field Dynamics

We study scalar-tensor-tensor and tensor-scalar-scalar three point cross correlations generated by the dynamics of a transiently rolling spectator axion-U(1) gauge field model during inflation. In this framework, tensor and scalar fluctuations are sourced by gauge fields at the non-linear level due to gravitational interactions, providing a chiral background of gravitational waves while keeping the level of scalar fluctuations at the observationally viable levels at CMB scales. We show that the gravitational couplings between the observable sector and gauge fields can also mediate strong correlations between scalar and tensor fluctuations, generating an amplitude for the mixed type three-point functions that is parametrically larger – fNL ' O(1− 10)(r/0.01)3/2– compared to the single-field realizations of inflation. As the amplification of the gauge field sources are localized around the time of horizon exit, the resulting mixed bispectra are peaked close to the equilateral configurations. The shape dependence, along with the scale dependence and the parity violating nature of the mixed bispectra can serve as a distinguishing feature of the underlying axion-gauge field dynamics and suggest a careful investigation of their signatures on the CMB observables including cross correlations between temperature T and E,B polarization modes. ar X iv :2 10 6. 14 89 5v 2 [ as tr oph . C O ] 7 D ec 2 02 1
Poisson double structures

We introduce Poisson double algebroids, and the equivalent concept of double Lie bialgebroid, which arise as second-order infinitesimal counterparts of Poisson double groupoids. We develop their underlying Lie theory, showing how these objects are related by differentiation and integration. We use these results to revisit Lie 2-bialgebras by means of Poisson double structures.
Exploring the Efficacy of Automatically Generated Counterfactuals for Sentiment Analysis

While state-of-the-art NLP models have been achieving the excellent performance of a wide range of tasks in recent years, important questions are being raised about their robustness and their underlying sensitivity to systematic biases that may exist in their training and test data. Such issues come to be manifest in performance problems when faced with out-of-distribution data in the field. One recent solution has been to use counterfactually augmented datasets in order to reduce any reliance on spurious patterns that may exist in the original data. Producing high-quality augmented data can be costly and time-consuming as it usually needs to involve human feedback and crowdsourcing efforts. In this work, we propose an alternative by describing and evaluating an approach to automatically generating counterfactual data for the purpose of data augmentation and explanation. A comprehensive evaluation on several different datasets and using a variety of state-of-the-art benchmarks demonstrate how our approach can achieve significant improvements in model performance when compared to models training on the original data and even when compared to models trained with the benefit of human-generated augmented data.
Joint Learning of Portrait Intrinsic Decomposition and Relighting

Inverse rendering is the problem of decomposing an image into its intrinsic components, i.e. albedo, normal and lighting. To solve this ill-posed problem from single image, state-of-the-art methods in shape from shading mostly resort to supervised training on all the components on either synthetic or real datasets. Here, we propose a new self-supervised training paradigm that 1) reduces the need for full supervision on the decomposition task and 2) takes into account the relighting task. We introduce new self-supervised loss terms that leverage the consistencies between multi-lit images (images of the same scene under different illuminations). Our approach is applicable to multi-lit datasets. We apply our training approach in two settings: 1) train on a mixture of synthetic and real data, 2) train on real datasets with limited supervision. We show-case the effectiveness of our training paradigm on both intrinsic decomposition and relighting and demonstrate how the model struggles in both tasks without the self-supervised loss terms in limited supervision settings. We provide results of comprehensive experiments on SfSNet, CelebA and Photoface datasets and verify the performance of our approach on images in the wild.
Stratification in tensor triangular geometry with applications to spectral Mackey functors

We systematically develop a theory of stratification in the context of tensor triangular geometry and apply it to classify the localizing tensor-ideals of certain categories of spectral $G$-Mackey functors for all finite groups $G$. Our theory of stratification is based on the approach of Stevenson which uses the Balmer-Favi notion of big support for tensor-triangulated categories whose Balmer spectrum is weakly noetherian. We clarify the role of the local-to-global principle and establish that the Balmer-Favi notion of support provides the universal approach to weakly noetherian stratification. This provides a uniform new perspective on existing classifications in the literature and clarifies the relation with the theory of Benson-Iyengar-Krause. Our systematic development of this approach to stratification, involving a reduction to local categories and the ability to pass through finite \'{e}tale extensions, may be of independent interest. Moreover, we strengthen the relationship between stratification and the telescope conjecture. The starting point for our equivariant applications is the recent computation by Patchkoria-Sanders-Wimmer of the Balmer spectrum of the category of derived Mackey functors, which was found to capture precisely the height $0$ and height $\infty$ chromatic layers of the spectrum of the equivariant stable homotopy category. We similarly study the Balmer spectrum of the category of $E(n)$-local spectral Mackey functors noting that it bijects onto the height $\le n$ chromatic layers of the spectrum of the equivariant stable homotopy category; conjecturally the topologies coincide. Despite our incomplete knowledge of the topology of the Balmer spectrum, we are able to completely classify the localizing tensor-ideals of these categories of spectral Mackey functors.
Multimodal Trajectory Prediction Conditioned on Lane-Graph Traversals

Accurately predicting the future motion of surrounding vehicles requires reasoning about the inherent uncertainty in driving behavior. This uncertainty can be loosely decoupled into lateral (e.g., keeping lane, turning) and longitudinal (e.g., accelerating, braking). We present a novel method that combines learned discrete policy rollouts with a focused decoder on subsets of the lane graph. The policy rollouts explore different goals given current observations, ensuring that the model captures lateral variability. Longitudinal variability is captured by our latent variable model decoder that is conditioned on various subsets of the lane graph. Our model achieves state-of-the-art performance on the nuScenes motion prediction dataset, and qualitatively demonstrates excellent scene compliance. Detailed ablations highlight the importance of the policy rollouts and the decoder architecture.
Coalition Formation Games for Improved Cell-Edge User Service in Downlink NOMA and MU-MIMO Small Cell Systems

In today’s wireless communication systems, the exponentially growing needs of mobile users require the combination of new and existing techniques to meet the demands for reliable and high-quality service provision. This may not always be possible, as the resources in wireless telecommunication systems are limited and a significant number of users, usually located at the cell edges, can suffer from severe interference. For this purpose, a new Joint Transmission Coordinated Multipoint (JT-CoMP) scheme, in which the transmission points’ clustering is based on a coalition formation game, is deployed alongside with Non-Orthogonal Multiple Access (NOMA) in a Cloud Radio Access Network (C-RAN) consisting of small cells. To further enhance the network’s performance, Multiuser Multiple-Input Multiple-Output (MU-MIMO) with Zero-Forcing (ZF) beamforming is applied. The proposed scheme’s performance, in terms of user throughput, is then compared to that of a scheme where JT-CoMP with static clustering is selected, a JT-CoMP scheme with clustering based on a greedy algorithm and a scenario where JT-CoMP is not deployed. Simulation results confirm that the proposed scheme reliably improves the throughput of users with poor wireless channels while guaranteeing that the performance of the rest is not severely undermined.
Striking the Right Balance: Recall Loss for Semantic Segmentation

Class imbalance is a fundamental problem in computer vision applications such as semantic segmentation. Specifically, uneven class distributions in a training dataset often result in unsatisfactory performance on under-represented classes. Many works have proposed to weight the standard cross entropy loss function with pre-computed weights based on class statistics, such as the number of samples and class margins. There are two major drawbacks to these methods: 1) constantly up-weighting minority classes can introduce excessive false positives in semantic segmentation; 2) a minority class is not necessarily a hard class. The consequence is low precision due to excessive false positives. In this regard, we propose a hard-class mining loss by reshaping the vanilla cross entropy loss such that it weights the loss for each class dynamically based on instantaneous recall performance. We show that the novel recall loss changes gradually between the standard cross entropy loss and the inverse frequency weighted loss. Recall loss also leads to improved mean accuracy while offering competitive mean Intersection over Union (IoU) performance. On Synthia dataset11Synthia-Sequence Summer split, recall loss achieves 9% relative improvement on mean accuracy with competitive mean IoU using DeepLab-ResNet18 compared to the cross entropy loss. Code available at https://github.com/PotatoTian/recall-semseg.
Restricted Adaptivity in Stochastic Scheduling

We consider the stochastic scheduling problem of minimizing the expected makespan on $m$ parallel identical machines. While the (adaptive) list scheduling policy achieves an approximation ratio of $2$, any (non-adaptive) fixed assignment policy has performance guarantee $\Omega\left(\frac{\log m}{\log \log m}\right)$. Although the performance of the latter class of policies are worse, there are applications in which non-adaptive policies are desired. In this work, we introduce the two classes of $\delta$-delay and $\tau$-shift policies whose degree of adaptivity can be controlled by a parameter. We present a policy - belonging to both classes - which is an $\mathcal{O}(\log \log m)$-approximation for reasonably bounded parameters. In other words, an exponential improvement on the performance of any fixed assignment policy can be achieved when allowing a small degree of adaptivity. Moreover, we provide a matching lower bound for any $\delta$-delay and $\tau$-shift policy when both parameters, respectively, are in the order of the expected makespan of an optimal non-anticipatory policy.
Learning from Miscellaneous Other-Class Words for Few-shot Named Entity Recognition

Few-shot Named Entity Recognition (NER) exploits only a handful of annotations to iden- tify and classify named entity mentions. Pro- totypical network shows superior performance on few-shot NER. However, existing prototyp- ical methods fail to differentiate rich seman- tics in other-class words, which will aggravate overfitting under few shot scenario. To address the issue, we propose a novel model, Mining Undefined Classes from Other-class (MUCO), that can automatically induce different unde- fined classes from the other class to improve few-shot NER. With these extra-labeled unde- fined classes, our method will improve the dis- criminative ability of NER classifier and en- hance the understanding of predefined classes with stand-by semantic knowledge. Experi- mental results demonstrate that our model out- performs five state-of-the-art models in both 1- shot and 5-shots settings on four NER bench- marks. We will release the code upon accep- tance. The source code is released on https: //github.com/shuaiwa16/OtherClassNER.git.
Fast Approximation of the Sliced-Wasserstein Distance Using Concentration of Random Projections

The Sliced-Wasserstein distance (SW) is being increasingly used in machine learning applications as an alternative to the Wasserstein distance and offers significant computational and statistical benefits. Since it is defined as an expectation over random projections, SW is commonly approximated by Monte Carlo. We adopt a new perspective to approximate SW by making use of the concentration of measure phenomenon: under mild assumptions, one-dimensional projections of a high-dimensional random vector are approximately Gaussian. Based on this observation, we develop a simple deterministic approximation for SW. Our method does not require sampling a number of random projections, and is therefore both accurate and easy to use compared to the usual Monte Carlo approximation. We derive nonasymptotical guarantees for our approach, and show that the approximation error goes to zero as the dimension increases, under a weak dependence condition on the data distribution. We validate our theoretical findings on synthetic datasets, and illustrate the proposed approximation on a generative modeling problem.
Small random initialization is akin to spectral learning: Optimization and generalization guarantees for overparameterized low-rank matrix reconstruction

Recently there has been significant theoretical progress on understanding the convergence and generalization of gradient-based methods on nonconvex losses with overparameterized models. Nevertheless, many aspects of optimization and generalization and in particular the critical role of small random initialization are not fully understood. In this paper, we take a step towards demystifying this role by proving that small random initialization followed by a few iterations of gradient descent behaves akin to popular spectral methods. We also show that this implicit spectral bias from small random initialization, which is provably more prominent for overparameterized models, also puts the gradient descent iterations on a particular trajectory towards solutions that are not only globally optimal but also generalize well. Concretely, we focus on the problem of reconstructing a low-rank matrix from a few measurements via a natural nonconvex formulation. In this setting, we show that the trajectory of the gradient descent iterations from small random initialization can be approximately decomposed into three phases: (I) a spectral or alignment phase where we show that that the iterates have an implicit spectral bias akin to spectral initialization allowing us to show that at the end of this phase the column space of the iterates and the underlying low-rank matrix are sufficiently aligned, (II) a saddle avoidance/refinement phase where we show that the trajectory of the gradient iterates moves away from certain degenerate saddle points, and (III) a local refinement phase where we show that after avoiding the saddles the iterates converge quickly to the underlying low-rank matrix. Underlying our analysis are insights for the analysis of overparameterized nonconvex optimization schemes that may have implications for computational problems beyond low-rank reconstruction.