Miracle-dz/UniFAR-FTUs huggingface dataset:
Phase Separation in Systems of Interacting Active Brownian Particles

The aim of this paper is to discuss the mathematical modeling of Brownian active particle systems, a recently popular paradigmatic system for self-propelled particles. We present four microscopic models with different types of repulsive interactions between particles and their associated macroscopic models, which are formally obtained using different coarse-graining methods. The macroscopic limits are integro-differential equations for the density in phase space (positions and orientations) of the particles and may include nonlinearities in both the diffusive and advective components. In contrast to passive particles, systems of active particles can undergo phase separation without any attractive interactions, a mechanism known as motility-induced phase separation (MIPS). We explore the onset of such a transition for each model in the parameter space of occupied volume fraction and P\'eclet number via a linear stability analysis and numerical simulations at both the microscopic and macroscopic levels. We establish that one of the models, namely the mean-field model which assumes long-range repulsive interactions, cannot explain the emergence of MIPS. In contrast, MIPS is observed for the remaining three models that assume short-range interactions that localize the interaction terms in space.
Network Representation Learning: From Preprocessing, Feature Extraction to Node Embedding

Network representation learning (NRL) advances the conventional graph mining of social networks, knowledge graphs, and complex biomedical and physics information networks. Dozens of NRL algorithms have been reported in the literature. Most of them focus on learning node embeddings for homogeneous networks, but they differ in the specific encoding schemes and specific types of node semantics captured and used for learning node embedding. This article reviews the design principles and the different node embedding techniques for NRL over homogeneous networks. To facilitate the comparison of different node embedding algorithms, we introduce a unified reference framework to divide and generalize the node embedding learning process on a given network into preprocessing steps, node feature extraction steps, and node embedding model training for an NRL task such as link prediction and node clustering. With this unifying reference framework, we highlight the representative methods, models, and techniques used at different stages of the node embedding model learning process. This survey not only helps researchers and practitioners gain an in-depth understanding of different NRL techniques but also provides practical guidelines for designing and developing the next generation of NRL algorithms and systems.
Comparison of SVD and factorized TDNN approaches for speech to text

This work concentrates on reducing the RTF and word error rate of a hybrid HMM-DNN. Our baseline system uses an architecture with TDNN and LSTM layers. We find this architecture particularly useful for lightly reverberated environments. However, these models tend to demand more computation than is desirable. In this work, we explore alternate architectures employing singular value decomposition (SVD) is applied to the TDNN layers to reduce the RTF, as well as to the affine transforms of every LSTM cell. We compare this approach with specifying bottleneck layers similar to those introduced by SVD before training. Additionally, we reduced the search space of the decoding graph to make it a better fit to operate in real-time applications. We report -61.57% relative reduction in RTF and almost 1% relative decrease in WER for our architecture trained on Fisher data along with reverberated versions of this dataset in order to match one of our target test distributions.
Output Space Entropy Search Framework for Multi-Objective Bayesian Optimization

We consider the problem of black-box multi-objective optimization (MOO) using expensive function evaluations (also referred to as experiments), where the goal is to approximate the true Pareto set of solutions by minimizing the total resource cost of experiments. For example, in hardware design optimization, we need to find the designs that trade-off performance, energy, and area overhead using expensive computational simulations. The key challenge is to select the sequence of experiments to uncover high-quality solutions using minimal resources. In this paper, we propose a general framework for solving MOO problems based on the principle of output space entropy (OSE) search: select the experiment that maximizes the information gained per unit resource cost about the true Pareto front. We appropriately instantiate the principle of OSE search to derive efficient algorithms for the following four MOO problem settings: 1) The most basic single-fidelity setting, where experiments are expensive and accurate; 2) Handling black-box constraints which cannot be evaluated without performing experiments; 3) The discrete multi-fidelity setting, where experiments can vary in the amount of resources consumed and their evaluation accuracy; and 4) The continuous-fidelity setting, where continuous function approximations result in a huge space of experiments. Experiments on diverse synthetic and real-world benchmarks show that our OSE search based algorithms improve over state-of-the-art methods in terms of both computational-efficiency and accuracy of MOO solutions.
A Mathematical Model of Thyroid Disease Response to Radiotherapy

We present a mechanistic biomathematical model of molecular radiotherapy of thyroid disease. The general model consists of a set of differential equations describing the dynamics of different populations of thyroid cells with varying degrees of damage caused by radiotherapy (undamaged cells, sub-lethally damaged cells, doomed cells, and dead cells), as well as the dynamics of thyroglobulin and antithyroglobulin autoantibodies, which are important surrogates of treatment response. The model is presented in two flavours: on the one hand, as a deterministic continuous model, which is useful to fit populational data, and on the other hand, as a stochastic Markov model, which is particularly useful to investigate tumor control probabilities and treatment individualization. The model was used to fit the response dynamics (tumor/thyroid volumes, thyroglobulin and antithyroglobulin autoantibodies) observed in experimental studies of thyroid cancer and Graves’ disease treated with 131I-radiotherapy. A qualitative adequate fitting of the model to the experimental data was achieved. We also used the model to investigate treatment individualization strategies for differentiated thyroid cancer, aiming to improve the tumor control probability. We found that simple individualization strategies based on the absorbed dose in the tumor and tumor radiosensitivity (which are both magnitudes that can potentially be individually determined for every patient) can lead to an important raise of tumor control probabilities.
Block Contextual MDPs for Continual Learning

In reinforcement learning (RL), when defining a Markov Decision Process (MDP), the environment dynamics is implicitly assumed to be stationary. This assumption of stationarity, while simplifying, can be unrealistic in many scenarios. In the continual reinforcement learning scenario, the sequence of tasks is another source of nonstationarity. In this work, we propose to examine this continual reinforcement learning setting through the block contextual MDP (BC-MDP) framework, which enables us to relax the assumption of stationarity. This framework challenges RL algorithms to handle both nonstationarity and rich observation settings and, by additionally leveraging smoothness properties, enables us to study generalization bounds for this setting. Finally, we take inspiration from adaptive control to propose a novel algorithm that addresses the challenges introduced by this more realistic BC-MDP setting, allows for zero-shot adaptation at evaluation time, and achieves strong performance on several nonstationary environments.
Bandwidth Utilization Side-Channel on ML Inference Accelerators

Accelerators used for machine learning (ML) inference provide great performance benefits over CPUs. Securing confidential model in inference against off-chip side-channel attacks is critical in harnessing the performance advantage in practice. Data and memory address encryption has been recently proposed to defend against off-chip attacks. In this paper, we demonstrate that bandwidth utilization on the interface between accelerators and the weight storage can serve a side-channel for leaking confidential ML model architecture. This side channel is independent of the type of interface, leaks even in the presence of data and memory address encryption and can be monitored through performance counters or through bus contention from an on-chip unprivileged process.
Region Semantically Aligned Network for Zero-Shot Learning

Zero-shot learning (ZSL) aims to recognize unseen classes based on the knowledge of seen classes. Previous methods focused on learning direct embeddings from global features to the semantic space in hope of knowledge transfer from seen classes to unseen classes. However, an unseen class shares local visual features with a set of seen classes and leveraging global visual features makes the knowledge transfer ineffective. To tackle this problem, we propose a Region Semantically Aligned Network (RSAN), which maps local features of unseen classes to their semantic attributes. Instead of using global features which are obtained by an average pooling layer after an image encoder, we directly utilize the output of the image encoder which maintains local information of the image. Concretely, we obtain each attribute from a specific region of the output and exploit these attributes for recognition. As a result, the knowledge of seen classes can be successfully transferred to unseen classes in a region-bases manner. In addition, we regularize the image encoder through attribute regression with a semantic knowledge to extract robust and attribute-related visual features. Experiments on several standard ZSL datasets reveal the benefit of the proposed RSAN method, outperforming state-of-the-art methods.
Treatment Effect Detection with Controlled FDR under Dependence for Large-Scale Experiments

Online controlled experiments (also known as A/B Testing) have been viewed as a golden standard for large data-driven companies since the last few decades. The most common A/B testing framework adopted by many companies use"average treatment effect"(ATE) as statistics. However, it remains a difficult problem for companies to improve the power of detecting ATE while controlling"false discovery rate"(FDR) at a predetermined level. One of the most popular FDR-control algorithms is BH method, but BH method is only known to control FDR under restrictive positive dependence assumptions with a conservative bound. In this paper, we propose statistical methods that can systematically and accurately identify ATE, and demonstrate how they can work robustly with controlled low FDR but a higher power using both simulation and real-world experimentation data. Moreover, we discuss the scalability problem in detail and offer comparison of our paradigm to other more recent FDR control methods, e.g., knockoff, AdaPT procedure, etc.
Dynamics of Negativity of a Wannier–Stark Many‐Body Localized System Coupled to a Bath

An interacting system subjected to a strong linear potential can host a many‐body localized (MBL) phase when being slightly perturbed. This so‐called Wannier–Stark or “tilted‐field” MBL phase inherits many properties from the well‐investigated disordered MBL phase, and provides an alternative route to experimentally engineer interacting localized systems without quenched disorder. Herein, the dynamics of entanglement in a Wannier–Stark MBL system coupled to a dephasing environment is investigated. As an accessible entanglement proxy, the third Rényi negativity R 3 is used, which reduces to the third Rényi entropy in case the system is isolated from the environment. This measure captures the characteristic logarithmic growth of interacting localized phases in the intermediate‐time regime, where the effects of the coupling to the environment are not yet dominating the dynamics. Thus, it forms a tool to distinguish Wannier–Stark MBL from noninteracting Wannier–Stark localization up to intermediate time‐scales, and to quantify quantum correlations in mixed‐state dynamics.
Contrastive Proposal Extension with Sequential Network for Weakly Supervised Object Detection

—Weakly supervised object detection (WSOD) has attracted more and more attention since it only uses image-level labels and can save huge annotation costs. Most of the WSOD methods use Multiple Instance Learning (MIL) as their basic framework, which regard it as an instance classiﬁcation problem. However, these methods based on MIL tends to converge only on the most discriminate regions of different instances, rather than their corresponding complete regions, that is, insufﬁcient integrity. Inspired by the habit of observing things by the human, we propose a new method by comparing the initial proposals and the extension ones to optimize those initial proposals. Speciﬁcally, we propose one new strategy for WSOD by involving contrastive proposal extension (CPE), which consists of multiple directional contrastive proposal extensions (D-CPE), and each D-CPE contains encoders based on LSTM network and corresponding decoders. Firstly, the boundary of initial proposals in MIL is extended to different positions according to well-designed sequential order. Then, CPE compares the extended proposal and the initial proposal by extracting the feature semantics of them using the encoders, and calculates the integrity of the initial proposal to optimize the score of the initial proposal. These contrastive contextual semantics will guide the basic WSOD to suppress bad proposals and improve the scores of good ones. In addition, a simple two-stream network is designed as the decoder to constrain the temporal coding of LSTM and improve the performance of WSOD further. Experiments on PASCAL VOC 2007, VOC 2012 and MS-COCO datasets show that our method has achieved the state-of-the-art results.
Towards Efficient NLP: A Standard Evaluation and A Strong Baseline

Supersized pre-trained language models have pushed the accuracy of various natural language processing (NLP) tasks to a new state-of-the-art (SOTA). Rather than pursuing the reachless SOTA accuracy, more and more researchers start paying attention to model efficiency and usability. Different from accuracy, the metric for efficiency varies across different studies, making them hard to be fairly compared. To that end, this work presents ELUE (Efficient Language Understanding Evaluation), a standard evaluation, and a public leaderboard for efficient NLP models. ELUE is dedicated to depicting the Pareto Frontier for various language understanding tasks, such that it can tell whether and how much a method achieves Pareto improvement. Along with the benchmark, we also release a strong baseline, ElasticBERT, which allows BERT to exit at any layer in both static and dynamic ways. We demonstrate the ElasticBERT, despite its simplicity, outperforms or performs on par with SOTA compressed and early exiting models. With ElasticBERT, the proposed ELUE has a strong Pareto Frontier and makes a better evaluation for efficient NLP models.
Same-diff ? Part II : A compendium of similarities between gauge transformations and diffeomorphisms

How should we understand gauge-(in)variant quantities and physical possibility? Does the redundancy present in gauge theory pose different interpretational issues than those present in general relativity? Here, I will assess new and old contrasts between general relativity and Yang-Mills theory, in particular, in relation to their symmetries. I will focus these comparisons on four topics: (i) non-locality, (ii) conserved charges, (iii) Aharonov-Bohm effect, and (iv) the choice of representational conventions of the field configuration. In a companion paper, I propose a new contrast and defend sophistication for both theories.
Interpretable AI forecasting for numerical relativity waveforms of quasi-circular, spinning, non-precessing binary black hole mergers

We present a deep-learning artificial intelligence model that is capable of learning and forecasting the late-inspiral, merger and ringdown of numerical relativity waveforms that describe quasi-circular, spinning, non-precessing binary black hole mergers. We used the NRHybSur3dq8 surrogate model to produce train, validation and test sets of $\ell=|m|=2$ waveforms that cover the parameter space of binary black hole mergers with mass-ratios $q\leq8$ and individual spins $|s^z_{\{1,2\}}| \leq 0.8$. These waveforms cover the time range $t\in[-5000\textrm{M}, 130\textrm{M}]$, where $t=0M$ marks the merger event, defined as the maximum value of the waveform amplitude. We harnessed the ThetaGPU supercomputer at the Argonne Leadership Computing Facility to train our AI model using a training set of 1.5 million waveforms. We used 16 NVIDIA DGX A100 nodes, each consisting of 8 NVIDIA A100 Tensor Core GPUs and 2 AMD Rome CPUs, to fully train our model within 3.5 hours. Our findings show that artificial intelligence can accurately forecast the dynamical evolution of numerical relativity waveforms in the time range $t\in[-100\textrm{M}, 130\textrm{M}]$. Sampling a test set of 190,000 waveforms, we find that the average overlap between target and predicted waveforms is $\gtrsim99\%$ over the entire parameter space under consideration. We also combined scientific visualization and accelerated computing to identify what components of our model take in knowledge from the early and late-time waveform evolution to accurately forecast the latter part of numerical relativity waveforms. This work aims to accelerate the creation of scalable, computationally efficient and interpretable artificial intelligence models for gravitational wave astrophysics.
ClonalNet: Classifying Better by Focusing on Confusing Categories

Existing neural classiﬁcation networks predominately adopt one-hot encoding due to its simplicity in representing categorical data. However, the one-hot representation neglects inter-category correlations, which may result in poor generalization. Herein, we observe that a pre-trained baseline network has paid attention to the target image region even though it incorrectly predicts the image, revealing which categories confuse the baseline. This observation motivates us to consider inter-category correlations. Therefore, we propose a clonal network, named ClonalNet, which learns to discriminate between confusing categories derived from the pre-trained baseline. The ClonalNet architecture can be identical or smaller than the baseline architecture. When identical, ClonalNet is a clonal version of the baseline but does not share weights. When smaller, the training process of ClonalNet resembles that of the standard knowledge distillation. The diﬀerence from knowledge distillation is that we design a focusing-picking loss to optimize ClonalNet. This novel loss enforces ClonalNet to concentrate on confusing categories and make more conﬁdent predictions on ground-truth labels with the baseline reference. Experiments show that ClonalNet signiﬁcantly
Zero-Shot Dense Retrieval with Momentum Adversarial Domain Invariant Representations

Dense retrieval (DR) methods conduct text retrieval by first encoding texts in the embedding space and then matching them by nearest neighbor search. This requires strong locality properties from the representation space, e.g., close allocations of each small group of relevant texts, which are hard to generalize to domains without sufficient training data. In this paper, we aim to improve the generalization ability of DR models from source training domains with rich supervision signals to target domains without any relevance label, in the zero-shot setting. To achieve that, we propose Momentum adversarial Domain Invariant Representation learning (MoDIR), which introduces a momentum method to train a domain classifier that distinguishes source versus target domains, and then adversarially updates the DR encoder to learn domain invariant representations. Our experiments show that MoDIR robustly outperforms its baselines on 10+ ranking datasets collected in the BEIR benchmark in the zero-shot setup, with more than 10% relative gains on datasets with enough sensitivity for DR models’ evaluation. Source code is available at https://github.com/ji-xin/modir.
Yang-Mills model for centrally extended 2D gravity

A Yang-Mills theory linear in the scalar curvature for 2d gravity with symmetry generated by the semidirect product formed with the Lie derivative of the algebra of diffeomorphisms with the two-dimensional Abelian algebra is formulated. As compared with dilaton models, the rôle of the dilaton is played by the dual field strength of a U(1) gauge field. All vacuum solutions are found. They are either black holes or have constant scalar curvature. Those with constant scalar curvature have constant dual field strength. In particular, solutions with vanishing cosmological constant but nonzero scalar curvature exist. In the conformal-Lorenz gauge, the model has a CFT interpretation whose residual symmetry combines holomorphic diffeomorphisms with a subclass of U(1) gauge transformations while preserving dS2 and AdS2 boundary conditions. This is the same symmetry as in Jackiw-Teitelboim-Maxwell gravity considered by Hartman and Strominger. It is argued that this is the only nontrivial Yang-Mills model linear in the scalar curvature that exists for real Lie algebras of dimension four.
The search for small association schemes with noncyclotomic eigenvalues

In this article we determine feasible parameter sets for (what could potentially be) commutative association schemes with noncyclotomic eigenvalues that are of smallest possible rank and order. A feasible parameter set for a commutative association scheme corresponds to a standard integral table algebra with integral multiplicities that satisfies all of the parameter restrictions known to hold for association schemes. For each rank and involution type, we generate an algebraic variety for which any suitable integral solution corresponds to a standard integral table algebra with integral multiplicities, and then try to find the smallest suitable solution. Our main results show the eigenvalues of commutative association schemes of rank 4 and nonsymmetric commutative association schemes of rank 5 will always be cyclotomic. In the rank 5 cases these our conclusions rely on calculations done by computer for Gr\"obner bases or for bases of rational vector spaces spanned by polynomials. We give several examples of feasible parameter sets for small symmetric association schemes of rank 5 that have noncyclotomic eigenvalues.