Miracle-dz/UniFAR-FTUs huggingface dataset:
Spectroscopy in the 2+1$d$ Thirring Model with $N=1$ Domain Wall Fermions

We employ the domain wall fermion (DWF) formulation of the Thirring model on a lattice in 2+1+1 dimensions and perform $N=1$ flavor Monte Carlo simulations. At a critical interaction strength the model features a spontaneous $\mathrm{U}(2)\rightarrow\mathrm{U}(1)\otimes \mathrm{U}(1)$ symmetry breaking; we analyse the induced spin-0 mesons, both Goldstone and non-Goldstone, as well as the correlator of the fermion quasiparticles, in both resulting phases. Crucially, we determine the anomalous dimension $\eta_\psi\approx 3$ at the critical point, in stark contrast with the Gross-Neveu model in 3$d$ and with results obtained with staggered fermions. Our numerical simulations are complemented by an analytical treatment of the free fermion correlator, which exhibits large early-time artifacts due to branch cuts in the propagator stemming from unbound interactions of the fermion with its heavy doublers. These artifacts are generalisable beyond the Thirring model, being an intrinsic property of DWF, or more generally Ginsparg-Wilson fermions.
Precise Single-stage Detector

There are still two problems in SDD causing some inaccurate results: (1) In the process of feature extraction, with the layer-by-layer acquisition of semantic information, local information is gradually lost, resulting into less representative feature maps; (2) During the Non-Maximum Suppression (NMS) algorithm due to inconsistency in classification and regression tasks, the classification confidence and predicted detection position cannot accurately indicate the position of the prediction boxes. Methods: In order to address these aforementioned issues, we propose a new architecture, a modified version of Single Shot Multibox Detector (SSD), named Precise Single Stage Detector (PSSD). Firstly, we improve the features by adding extra layers to SSD. Secondly, we construct a simple and effective feature enhancement module to expand the receptive field step by step for each layer and enhance its local and semantic information. Finally, we design a more efficient loss function to predict the IOU between the prediction boxes and ground truth boxes, and the threshold IOU guides classification training and attenuates the scores, which are used by the NMS algorithm. Main Results: Benefiting from the above optimization, the proposed model PSSD achieves exciting performance in real-time. Specifically, with the hardware of Titan Xp and the input size of 320 pix, PSSD achieves 33.8 mAP at 45 FPS speed on MS COCO benchmark and 81.28 mAP at 66 FPS speed on Pascal VOC 2007 outperforming state-of-the-art object detection models. Besides, the proposed model performs significantly well with larger input size. Under 512 pix, PSSD can obtain 37.2 mAP with 27 FPS on MS COCO and 82.82 mAP with 40 FPS on Pascal VOC 2007. The experiment results prove that the proposed model has a better trade-off between speed and accuracy.
Long N-step Surrogate Stage Reward to Reduce Variances of Deep Reinforcement Learning in Complex Problems

High variances in reinforcement learning have shown impeding successful convergence and hurting task performance. As reward signal plays an important role in learning behavior, multi-step methods have been considered to mitigate the problem, and are believed to be more effective than single step methods. However, there is a lack of comprehensive and systematic study on this important aspect to demonstrate the effectiveness of multi-step methods in solving highly complex continuous control problems. In this study, we introduce a new long $N$-step surrogate stage (LNSS) reward approach to effectively account for complex environment dynamics while previous methods are usually feasible for limited number of steps. The LNSS method is simple, low computational cost, and applicable to value based or policy gradient reinforcement learning. We systematically evaluate LNSS in OpenAI Gym and DeepMind Control Suite to address some complex benchmark environments that have been challenging to obtain good results by DRL in general. We demonstrate performance improvement in terms of total reward, convergence speed, and coefficient of variation (CV) by LNSS. We also provide analytical insights on how LNSS exponentially reduces the upper bound on the variances of Q value from a respective single step method
Distilling Causal Effect from Miscellaneous Other-Class for Continual Named Entity Recognition

Continual Learning for Named Entity Recognition (CL-NER) aims to learn a growing number of entity types over time from a stream of data. However, simply learning Other-Class in the same way as new entity types amplifies the catastrophic forgetting and leads to a substantial performance drop. The main cause behind this is that Other-Class samples usually contain old entity types, and the old knowledge in these Other-Class samples is not preserved properly. Thanks to the causal inference, we identify that the forgetting is caused by the missing causal effect from the old data.To this end, we propose a unified causal framework to retrieve the causality from both new entity types and Other-Class.Furthermore, we apply curriculum learning to mitigate the impact of label noise and introduce a self-adaptive weight for balancing the causal effects between new entity types and Other-Class. Experimental results on three benchmark datasets show that our method outperforms the state-of-the-art method by a large margin. Moreover, our method can be combined with the existing state-of-the-art methods to improve the performance in CL-NER.
Stimulative Training of Residual Networks: A Social Psychology Perspective of Loafing

Residual networks have shown great success and become indispensable in today's deep models. In this work, we aim to re-investigate the training process of residual networks from a novel social psychology perspective of loafing, and further propose a new training strategy to strengthen the performance of residual networks. As residual networks can be viewed as ensembles of relatively shallow networks (i.e., \textit{unraveled view}) in prior works, we also start from such view and consider that the final performance of a residual network is co-determined by a group of sub-networks. Inspired by the social loafing problem of social psychology, we find that residual networks invariably suffer from similar problem, where sub-networks in a residual network are prone to exert less effort when working as part of the group compared to working alone. We define this previously overlooked problem as \textit{network loafing}. As social loafing will ultimately cause the low individual productivity and the reduced overall performance, network loafing will also hinder the performance of a given residual network and its sub-networks. Referring to the solutions of social psychology, we propose \textit{stimulative training}, which randomly samples a residual sub-network and calculates the KL-divergence loss between the sampled sub-network and the given residual network, to act as extra supervision for sub-networks and make the overall goal consistent. Comprehensive empirical results and theoretical analyses verify that stimulative training can well handle the loafing problem, and improve the performance of a residual network by improving the performance of its sub-networks. The code is available at https://github.com/Sunshine-Ye/NIPS22-ST .
Using Whole Slide Image Representations from Self-Supervised Contrastive Learning for Melanoma Concordance Regression

Although melanoma occurs more rarely than several other skin cancers, patients' long term survival rate is extremely low if the diagnosis is missed. Diagnosis is complicated by a high discordance rate among pathologists when distinguishing between melanoma and benign melanocytic lesions. A tool that provides potential concordance information to healthcare providers could help inform diagnostic, prognostic, and therapeutic decision-making for challenging melanoma cases. We present a melanoma concordance regression deep learning model capable of predicting the concordance rate of invasive melanoma or melanoma in-situ from digitized Whole Slide Images (WSIs). The salient features corresponding to melanoma concordance were learned in a self-supervised manner with the contrastive learning method, SimCLR. We trained a SimCLR feature extractor with 83,356 WSI tiles randomly sampled from 10,895 specimens originating from four distinct pathology labs. We trained a separate melanoma concordance regression model on 990 specimens with available concordance ground truth annotations from three pathology labs and tested the model on 211 specimens. We achieved a Root Mean Squared Error (RMSE) of 0.28 +/- 0.01 on the test set. We also investigated the performance of using the predicted concordance rate as a malignancy classifier, and achieved a precision and recall of 0.85 +/- 0.05 and 0.61 +/- 0.06, respectively, on the test set. These results are an important first step for building an artificial intelligence (AI) system capable of predicting the results of consulting a panel of experts and delivering a score based on the degree to which the experts would agree on a particular diagnosis. Such a system could be used to suggest additional testing or other action such as ordering additional stains or genetic tests.
KG-MTT-BERT: Knowledge Graph Enhanced BERT for Multi-Type Medical Text Classification

Medical text learning has recently emerged as a promising area to improve healthcare due to the wide adoption of electronic health record (EHR) systems. The complexity of the medical text such as diverse length, mixed text types, and full of medical jargon, poses a great challenge for developing effective deep learning models. BERT has presented state-of-the-art results in many NLP tasks, such as text classification and question answering. However, the standalone BERT model cannot deal with the complexity of the medical text, especially the lengthy clinical notes. Herein, we develop a new model called KG-MTT-BERT (Knowledge Graph Enhanced Multi-Type Text BERT) by extending the BERT model for long and multi-type text with the integration of the medical knowledge graph. Our model can outperform all baselines and other state-of-the-art models in diagnosis-related group (DRG) classification, which requires comprehensive medical text for accurate classification. We also demonstrated that our model can effectively handle multi-type text and the integration of medical knowledge graph can significantly improve the performance.
Differentially Private Deep Learning with ModelMix

Training large neural networks with meaningful/usable differential privacy security guarantees is a demanding challenge. In this paper, we tackle this problem by revisiting the two key operations in Differentially Private Stochastic Gradient Descent (DP-SGD): 1) iterative perturbation and 2) gradient clipping. We propose a generic optimization framework, called {\em ModelMix}, which performs random aggregation of intermediate model states. It strengthens the composite privacy analysis utilizing the entropy of the training trajectory and improves the $(\epsilon, \delta)$ DP security parameters by an order of magnitude. We provide rigorous analyses for both the utility guarantees and privacy amplification of ModelMix. In particular, we present a formal study on the effect of gradient clipping in DP-SGD, which provides theoretical instruction on how hyper-parameters should be selected. We also introduce a refined gradient clipping method, which can further sharpen the privacy loss in private learning when combined with ModelMix. Thorough experiments with significant privacy/utility improvement are presented to support our theory. We train a Resnet-20 network on CIFAR10 with $70.4\%$ accuracy via ModelMix given $(\epsilon=8, \delta=10^{-5})$ DP-budget, compared to the same performance but with $(\epsilon=145.8,\delta=10^{-5})$ using regular DP-SGD; assisted with additional public low-dimensional gradient embedding, one can further improve the accuracy to $79.1\%$ with $(\epsilon=6.1, \delta=10^{-5})$ DP-budget, compared to the same performance but with $(\epsilon=111.2, \delta=10^{-5})$ without ModelMix.
Categorial Independence and Lévy Processes

We generalize Franz' independence in tensor categories with inclusions from two morphisms (which represent generalized random variables) to arbitrary ordered families of morphisms. We will see that this only works consistently if the unit object is an initial object, in which case the inclusions can be defined starting from the tensor category alone. The obtained independence for morphisms is called categorial independence. We define categorial Lévy processes on every tensor category with initial unit object and present a construction generalizing the reconstruction of a Lévy process from its convolution semigroup via the Daniell-Kolmogorov theorem. Finally, we discuss examples showing that many known independences from algebra as well as from (noncommutative) probability are special cases of categorial independence.
Quantum Sequence States

In a recent paper it has been shown how to create a quantum state related to the prime number sequence using Grover's algorithm. Moreover, its multiqubit entanglement was analyzed. In the present work, we compare the multiqubit entanglement of several quantum sequence states as well we study the feasibility of producing such states using Grover's algorithm.
Compact Model of a Topological Transistor

The precession of a ferromagnet leads to the injection of spin current and heat into an adjacent non-magnetic material. Besides, spin-orbit entanglement causes an additional charge current injection. Such a device has been recently proposed where a quantum-spin hall insulator (QSHI) in proximity to a ferromagnetic insulator (FI) and superconductor (SC) leads to the pumping of charge, spin, and heat. Here we build a circuit-compatible Verilog-A-based compact model for the QSHI-FI-SC device capable of generating two topologically robust modes enabling the device operation. Our model also captures the dependence on the ferromagnetic precision, drain voltage, and temperature with an excellent (> 99%) accuracy.
Network polarization, filter bubbles, and echo chambers: an annotated review of measures and reduction methods

Polarization arises when the underlying network connecting the members of a community or society becomes characterized by highly connected groups with weak inter-group connectivity. The increasing polarization, the strengthening of echo chambers, and the isolation caused by information filters in social networks are increasingly attracting the attention of researchers from different areas of knowledge such as computer science, economics, social and political sciences. This work presents an annotated review of network polarization measures and models used to handle the polarization. Several approaches for measuring polarization in graphs and networks were identified, including those based on homophily, modularity, random walks, and balance theory. The strategies used for reducing polarization include methods that propose edge or node editions (including insertions or deletions, as well as edge weight modifications), changes in social network design, or changes in the recommendation systems embedded in these networks.
Effect of Brief Meditation Intervention on Attention: An ERP Investigation

Fast and efficient strategies for modulation of attention have been extensively studied recently. The present study has attempted to observe the effect of brief meditation practices on executive control of the attention system. The study recruits cognitive control by introducing conflict using Stroop task in twenty-six novice participants. Behavioral responses indicate a positive effect on response time and accuracy at Stroop task after ten minutes of meditation intervention. Neurophysiological findings suggest more efficient allocation of attentional resources. An increase in positive ERP components (P200, P300) and expected decrease in the inhibitory or negative component (N200) after intervention shows positive results. The findings suggest a positive impact of meditation intervention on attention even for brief periods in non-meditating population.
Quantifying quantum causal influences

Causal influences are at the core of any empirical science, the reason why its quantification is of paramount relevance for the mathematical theory of causality and applications. Quantum correlations, however, challenge our notion of cause and effect, implying that tools and concepts developed over the years having in mind a classical world, have to be reevaluated in the presence of quantum effects. Here, we propose the quantum version of the most common causality quantifier, the average causal effect (ACE), measuring how much a target quantum system is changed by interventions on its presumed cause. Not only it offers an innate manner to quantify causation in two-qubit gates but also in alternative quantum computation models such as the measurement-based version, suggesting that causality can be used as a proxy for optimizing quantum algorithms. Considering quantum teleportation, we show that any pure entangled state offers an advantage in terms of causal effects as compared to separable states. This broadness of different uses showcases that, just as in the classical case, the quantification of causal influence has foundational and applied consequences and can lead to a yet totally unexplored tool for quantum information science.
TransRepair: Context-aware Program Repair for Compilation Errors

Automatically fixing compilation errors can greatly raise the productivity of software development, by guiding the novice or AI programmers to write and debug code. Recently, learning-based program repair has gained extensive attention and became the state-of-the-art in practice. But it still leaves plenty of space for improvement. In this paper, we propose an end-to-end solution TransRepair to locate the error lines and create the correct substitute for a C program simultaneously. Superior to the counterpart, our approach takes into account the context of erroneous code and diagnostic compilation feedback. Then we devise a Transformer-based neural network to learn the ways of repair from the erroneous code as well as its context and the diagnostic feedback. To increase the effectiveness of TransRepair, we summarize 5 types and 74 fine-grained sub-types of compilations errors from two real-world program datasets and the Internet. Then a program corruption technique is developed to synthesize a large dataset with 1,821,275 erroneous C programs. Through the extensive experiments, we demonstrate that TransRepair outperforms the state-of-the-art in both single repair accuracy and full repair accuracy. Further analysis sheds light on the strengths and weaknesses in the contemporary solutions for future improvement.
Low Error-Rate Approximate Multiplier Design for DNNs with Hardware-Driven Co-Optimization

In this paper, two approximate 3 × 3 multipliers are proposed and the synthesis results of the ASAP-7nm process library justify that they can reduce the area by 31.38% and 36.17%, and the power consumption by 36.73% and 35.66% compared with the exact multiplier, respectively. They can be aggregated with a 2 × 2 multiplier to produce an 8 × 8 multiplier with low error-rate based on the distribution of DNN weights. We propose a hardware-driven software co-optimization method to improve the DNN accuracy by retraining. Based on the proposed two approximate 3-bit multipliers, three approximate 8-bit multipliers with low error-rate are designed for DNNs. Compared with the exact 8-bit unsigned multiplier, our design can achieve a significant advantage over other approximate multipliers on the public dataset.
Second-order regression models exhibit progressive sharpening to the edge of stability

Recent studies of gradient descent with large step sizes have shown that there is often a regime with an initial increase in the largest eigenvalue of the loss Hessian (progressive sharpening), followed by a stabilization of the eigenvalue near the maximum value which allows convergence (edge of stability). These phenomena are intrinsically non-linear and do not happen for models in the constant Neural Tangent Kernel (NTK) regime, for which the predictive function is approximately linear in the parameters. As such, we consider the next simplest class of predictive models, namely those that are quadratic in the parameters, which we call second-order regression models. For quadratic objectives in two dimensions, we prove that this second-order regression model exhibits progressive sharpening of the NTK eigenvalue towards a value that differs slightly from the edge of stability, which we explicitly compute. In higher dimensions, the model generically shows similar behavior, even without the specific structure of a neural network, suggesting that progressive sharpening and edge-of-stability behavior aren't unique features of neural networks, and could be a more general property of discrete learning algorithms in high-dimensional non-linear models.
Temporal Vectorization: A Compiler Approach to Automatic Multi-Pumping

The multi-pumping resource sharing technique can overcome the limitations commonly found in single-clocked FPGA designs by allowing hardware components to operate at a higher clock frequency than the surrounding system. However, this optimization cannot be expressed in high levels of abstraction, such as HLS, requiring the use of hand-optimized RTL. In this paper we show how to leverage multiple clock domains for computational subdomains on reconfigurable devices through data movement analysis on high-level programs. We offer a novel view on multi-pumping as a compiler optimization — a superclass of traditional vectorization. As multiple data elements are fed and consumed, the computations are packed temporally rather than spatially. The optimization is applied automatically using an intermediate representation that maps high-level code to HLS. Internally, the optimization injects modules into the generated designs, incorporating RTL for fine-grained control over the clock domains. We obtain a reduction of resource consumption by up to 50% on critical components and 23% on average. For scalable designs, this can enable further parallelism, increasing overall performance.
Rieoptax: Riemannian Optimization in JAX

We present Rieoptax, an open source Python library for Riemannian optimization in JAX. We show that many differential geometric primitives, such as Riemannian exponential and logarithm maps, are usually faster in Rieoptax than existing frameworks in Python, both on CPU and GPU. We support various range of basic and advanced stochastic optimization solvers like Riemannian stochastic gradient, stochastic variance reduction, and adaptive gradient methods. A distinguishing feature of the proposed toolbox is that we also support differentially private optimization on Riemannian manifolds.