Miracle-dz/UniFAR-FTUs huggingface dataset:
Asynchronous speedup in decentralized optimization

In decentralized optimization, nodes of a communication network each possess a local objective function, and communicate using gossip-based methods in order to minimize the average of these per-node functions. While synchronous algorithms are heavily impacted by a few slow nodes or edges in the graph (the straggler problem ), their asynchronous counterparts are notoriously harder to parametrize. Indeed, their convergence properties for networks with heterogeneous communication and computation delays have deﬁed analysis so far. In this paper, we use a continuized framework to analyze asynchronous algorithms in networks with delays. Our approach yields a precise characterization of convergence time and of its dependency on heterogeneous delays in the network. Our continuized framework beneﬁts from the best of both continuous and discrete worlds: the algorithms it applies to are based on event-driven updates. They are thus essentially discrete and hence readily implementable. Yet their analysis is essentially in continuous time, relying in part on the theory of delayed ODEs. Our algorithms moreover achieve an asynchronous speedup : their rate of convergence is controlled by the eigengap of the network graph weighted by local delays, instead of the network- wide worst-case delay as in previous analyses. Our methods thus enjoy improved robustness to stragglers.
Coherence and realism in the Aharonov-Bohm effect

The Aharonov-Bohm effect is a fundamental topological phenomenon with a wide range of applications. It consists of a charge encircling a region with a magnetic flux in a superposition of wave packets having their relative phase affected by the flux. In this work, we analyze this effect using an entropic measure known as realism, originally introduced as a quantifier of a system's degree of reality and mathematically related to notions of global and local quantum coherence. More precisely, we look for observables that lead to gauge-invariant realism associated with the charge before it completes its loop. We find that the realism of these operators has a sudden change when the line connecting the center of both wave packets crosses the solenoid. Moreover, we consider the case of a quantized magnetic-field source, pointing out similarities and differences between the two cases. Finally, we discuss some consequences of these results.
Radio Localization and Sensing—Part II: State-of-the-Art and Challenges

This letter is part of a two-letter tutorial on radio localization and sensing, with a focus on mobile radio systems, i.e., 5G and beyond. Building on Part I, which focused on the fundamentals, here we go deeper into the state-of-the-art advances, as well as 6G, covering enablers and challenges related to modeling, coverage, and accuracy.
Systems of difference equations, symmetries and integrability conditions

We consider a class of systems of diﬀerence equations deﬁned on an elementary quadrilateral of the Z 2 lattice, deﬁne their eliminable and dynamical variables, and demonstrate their use. Using the existence of inﬁnite hierarchies of symmetries as integrability criterion, we derive necessary integrability conditions and employ them in the construction of the lowest order symmetries of a given system. These considerations are demonstrated with the help of three systems from the class of systems under consideration.
ContrastVAE: Contrastive Variational AutoEncoder for Sequential Recommendation

Aiming at exploiting the rich information in user behaviour sequences, sequential recommendation has been widely adopted in real-world recommender systems. However, current methods suffer from the following issues: 1) sparsity of user-item interactions, 2) uncertainty of sequential records, 3) long-tail items. In this paper, we propose to incorporate contrastive learning into the framework of Variational AutoEncoders to address these challenges simultaneously. Firstly, we introduce ContrastELBO, a novel training objective that extends the conventional single-view ELBO to two-view case and theoretically builds a connection between VAE and contrastive learning from a two-view perspective. Then we propose Contrastive Variational AutoEncoder (ContrastVAE in short), a two-branched VAE model with contrastive regularization as an embodiment of ContrastELBO for sequential recommendation. We further introduce two simple yet effective augmentation strategies named model augmentation and variational augmentation to create a second view of a sequence and thus making contrastive learning possible. Experiments on four benchmark datasets demonstrate the effectiveness of ContrastVAE and the proposed augmentation methods. Codes are available at https://github.com/YuWang-1024/ContrastVAE
Monodromy representations of $p$-adic differential equations in families

We derive a relative version of the local monodromy theorem for ordinary diﬀerential equations on an annulus over a mixed-characteristic nonarchimedean ﬁeld, and give several applications in p -adic cohomology and p -adic Hodge theory. These include a simpliﬁed proof of the semistable reduction theorem for overconvergent F -isocrystals, a relative version of Berger’s theorem that de Rham representations are potentially semistable, and a multivariate version of the local monodromy theorem in the style of Drinfeld’s lemma on fundamental groups.
A Heating Mechanism via Magnetic Pumping in the Intracluster Medium

Turbulence driven by active galactic nuclei activity, cluster mergers, and galaxy motion constitutes an attractive energy source for heating the intracluster medium (ICM). How this energy dissipates into the ICM plasma remains unclear, given its low collisionality and high magnetization (precluding viscous heating by Coulomb processes). Kunz et al. proposed a viable heating mechanism based on the anisotropy of the plasma pressure under ICM conditions. The present paper builds upon that work and shows that particles can be heated by large-scale turbulent fluctuations via magnetic pumping. We study how the anisotropy evolves under a range of forcing frequencies, what waves and instabilities are generated, and demonstrate that the particle distribution function acquires a high-energy tail. For this, we perform particle-in-cell simulations where we periodically vary the mean magnetic field B (t). When B (t) grows (dwindles), a pressure anisotropy P ⊥ > P ∥(P ⊥ < P ∥) builds up (P ⊥ and P ∥ are, respectively, the pressures perpendicular and parallel to B (t)). These pressure anisotropies excite mirror (P ⊥ > P ∥) and oblique firehose (P ∥ > P ⊥) instabilities, which trap and scatter the particles, limiting the anisotropy, and providing a channel to heat the plasma. The efficiency of this mechanism depends on the frequency of the large-scale turbulent fluctuations and the efficiency of the scattering the instabilities provide in their nonlinear stage. We provide a simplified analytical heating model that captures the phenomenology involved. Our results show that this process can be relevant in dissipating and distributing turbulent energy at kinetic scales in the ICM.
Unsupervised Simplification of Legal Texts

The processing of legal texts has been developing as an emerging field in natural language processing (NLP). Legal texts contain unique jargon and complex linguistic attributes in vocabulary, semantics, syntax, and morphology. Therefore, the development of text simplification (TS) methods specific to the legal domain is of paramount importance for facilitating comprehension of legal text by ordinary people and providing inputs to high-level models for mainstream legal NLP applications. While a recent study proposed a rule-based TS method for legal text, learning-based TS in the legal domain has not been considered previously. Here we introduce an unsupervised simplification method for legal texts (USLT). USLT performs domain-specific TS by replacing complex words and splitting long sentences. To this end, USLT detects complex words in a sentence, generates candidates via a masked-transformer model, and selects a candidate for substitution based on a rank score. Afterward, USLT recursively decomposes long sentences into a hierarchy of shorter core and context sentences while preserving semantic meaning. We demonstrate that USLT outperforms state-of-the-art domain-general TS methods in text simplicity while keeping the semantics intact.
Continuous-time Particle Filtering for Latent Stochastic Differential Equations

Particle filtering is a standard Monte-Carlo approach for a wide range of sequential inference tasks. The key component of a particle filter is a set of particles with importance weights that serve as a proxy of the true posterior distribution of some stochastic process. In this work, we propose continuous latent particle filters, an approach that extends particle filtering to the continuous-time domain. We demonstrate how continuous latent particle filters can be used as a generic plug-in replacement for inference techniques relying on a learned variational posterior. Our experiments with different model families based on latent neural stochastic differential equations demonstrate superior performance of continuous-time particle filtering in inference tasks like likelihood estimation and sequential prediction for a variety of stochastic processes.
Attack Tactic Identification by Transfer Learning of Language Model

Cybersecurity has become a primary global concern with the rapid increase in security attacks and data breaches. Artiﬁcial intelligence is promising to help humans analyzing and identifying attacks. However, labeling millions of packets for supervised learning is never easy. This study aims to leverage transfer learning technique that stores the knowledge gained from well-deﬁned attack lifecycle documents and applies it to hundred thousands of unlabeled attacks (packets) for identifying their attack tactics. We anticipate the knowledge of an attack is well-described in the documents, and the cutting edge transformer-based language model can embed the knowledge into a high-dimensional latent space. Then, reusing the information from the language model for the learning of attack tactic carried by packets to improve the learning e ﬃ ciency. We propose a system, PELAT, that ﬁne-tunes BERT model with 1,417 articles from MITRE ATT&CK lifecycle framework to enhance its attack knowledge (including syntax used and semantic meanings embedded). PELAT then transfers its knowledge to perform semi-supervised learning for unlabeled packets to generate their tactic labels. Further, when a new attack packet arrives, the packet payload will be processed by the PELAT language model with a downstream classiﬁer to predict its tactics. In this way, we can e ﬀ ectively reduce the burden of manually labeling big datasets. In a one-week honeypot attack dataset (227 thousand packets per day), PELAT performs 99% of precision, recall, and F1 on testing dataset. PELAT can infer over 99% of tactics on two other testing datasets (while nearly 90% of tactics are identiﬁed).
Bézier Gaussian Processes for Tall and Wide Data

Modern approximations to Gaussian processes are suitable for"tall data", with a cost that scales well in the number of observations, but under-performs on ``wide data'', scaling poorly in the number of input features. That is, as the number of input features grows, good predictive performance requires the number of summarising variables, and their associated cost, to grow rapidly. We introduce a kernel that allows the number of summarising variables to grow exponentially with the number of input features, but requires only linear cost in both number of observations and input features. This scaling is achieved through our introduction of the B\'ezier buttress, which allows approximate inference without computing matrix inverses or determinants. We show that our kernel has close similarities to some of the most used kernels in Gaussian process regression, and empirically demonstrate the kernel's ability to scale to both tall and wide datasets.
CPS Attack Detection under Limited Local Information in Cyber Security: A Multi-node Multi-class Classification Ensemble Approach

Cybersecurity breaches are the common anomalies for distributed cyber-physical systems (CPS). However, the cyber security breach classification is still a difficult problem, even using cutting-edge artificial intelligence (AI) approaches. In this paper, we study the multi-class classification problem in cyber security for attack detection. A challenging multi-node data-censoring case is considered. In such a case, data within each data center/node cannot be shared while the local data is incomplete. Particularly, local nodes contain only a part of the multiple classes. In order to train a global multi-class classifier without sharing the raw data across all nodes, the main result of our study is designing a multi-node multi-class classification ensemble approach. By gathering the estimated parameters of the binary classifiers and data densities from each local node, the missing information for each local node is completed to build the global multi-class classifier. Numerical experiments are given to validate the effectiveness of the proposed approach under the multi-node data-censoring case. Under such a case, we even show the out-performance of the proposed approach over the full-data approach. By we a multi-node multi-class
PointCLM: A Contrastive Learning-based Framework for Multi-instance Point Cloud Registration

Multi-instance point cloud registration is the problem of estimating multiple poses of source point cloud instances within a target point cloud. Solving this problem is challenging since inlier correspondences of one instance constitute outliers of all the other instances. Existing methods often rely on time-consuming hypothesis sampling or features leveraging spatial consistency, resulting in limited performance. In this paper, we propose PointCLM, a contrastive learning-based framework for mutli-instance point cloud registration. We first utilize contrastive learning to learn well-distributed deep representations for the input putative correspondences. Then based on these representations, we propose a outlier pruning strategy and a clustering strategy to efficiently remove outliers and assign the remaining correspondences to correct instances. Our method outperforms the state-of-the-art methods on both synthetic and real datasets by a large margin.
On Poisoned Wardrop Equilibrium in Congestion Games

Recent years have witnessed a growing number of attack vectors against increasingly interconnected traffic networks. Informational attacks have emerged as the prominent ones that aim to poison traffic data, misguide users, and manipulate traffic patterns. To study the impact of this class of attacks, we propose a game-theoretic framework where the attacker, as a Stackelberg leader, falsifies the traffic conditions to change the traffic pattern predicted by the Wardrop traffic equilibrium, achieved by the users, or the followers. The intended shift of the Wardrop equilibrium is a consequence of strategic informational poisoning. Leveraging game-theoretic and sensitivity analysis, we quantify the system-level impact of the attack by characterizing the concept of poisoned Price of Anarchy, which compares the poisoned Wardrop equilibrium and its non-poisoned system optimal counterpart. We use an evacuation case study to show that the Stackelberg equilibrium can be found through a two-time scale zeroth-order learning process and demonstrate the disruptive effects of informational poisoning, indicating a compelling need for defense policies to mitigate such security threats.
MSGNN: A Spectral Graph Neural Network Based on a Novel Magnetic Signed Laplacian

Signed and directed networks are ubiquitous in real-world applications. However, there has been relatively little work proposing spectral graph neural networks (GNNs) for such networks. Here we introduce a signed directed Laplacian matrix, which we call the magnetic signed Laplacian, as a natural generalization of both the signed Laplacian on signed graphs and the magnetic Laplacian on directed graphs. We then use this matrix to construct a novel efficient spectral GNN architecture and conduct extensive experiments on both node clustering and link prediction tasks. In these experiments, we consider tasks related to signed information, tasks related to directional information, and tasks related to both signed and directional information. We demonstrate that our proposed spectral GNN is effective for incorporating both signed and directional information, and attains leading performance on a wide range of data sets. Additionally, we provide a novel synthetic network model, which we refer to as the Signed Directed Stochastic Block Model, and a number of novel real-world data sets based on lead-lag relationships in financial time series.
The alignment problem from a deep learning perspective

In coming years or decades, artificial general intelligence (AGI) may surpass human capabilities at many critical tasks. We argue that, without substantial effort to prevent it, AGIs could learn to pursue goals that are in conflict (i.e. misaligned) with human interests. If trained like today's most capable models, AGIs could learn to act deceptively to receive higher reward, learn misaligned internally-represented goals which generalize beyond their fine-tuning distributions, and pursue those goals using power-seeking strategies. We review emerging evidence for these properties. AGIs with these properties would be difficult to align and may appear aligned even when they are not. Finally, we briefly outline how the deployment of misaligned AGIs might irreversibly undermine human control over the world, and we review research directions aimed at preventing this outcome.
What is missing in deep music generation? A study of repetition and structure in popular music

Structure is one of the most essential aspects of music, and music structure is commonly indicated through repetition. However, the nature of repetition and structure in music is still not well understood, especially in the context of music generation, and much remains to be explored with Music Information Retrieval (MIR) techniques. Analyses of two popular music datasets (Chinese and American) illustrate important music construction principles: (1) structure exists at multiple hierarchical levels, (2) songs use repetition and limited vocabulary so that individual songs do not follow general statistics of song collections, (3) structure interacts with rhythm, melody, harmony, and predictability, and (4) over the course of a song, repetition is not random, but follows a general trend as revealed by cross-entropy. These and other findings offer challenges as well as opportunities for deep-learning music generation and suggest new formal music criteria and evaluation methods. Music from recent music generation systems is analyzed and compared to human-composed music in our datasets, often revealing striking differences from a structural perspective.
Digraph Isomorphisms and Free Space Optical Networks

The de Bruijn digraph B(d,D) is usually defined by words of size D on an alphabet of cardinality d, through a cyclic left shift permutation on the words, after which the rightmost symbol is changed. In this paper, we show that any digraph defined on words and alphabets of the same size, through an arbitrary permutation on the alphabet and an arbitrary permutation on the word indices, is isomorphic to the de Bruijn, provided that this latter permutation is cyclic. As an application, we recall that it is known that the Optical Transpose Interconnection System from UCSD can implement the de Bruijn interconnections for n$ nodes, for a fixed d, with O(n) lenses. We show here how to improve this hardware requirement to $\Theta(\sqrtn)$. \\
Large-Scale Auto-Regressive Modeling Of Street Networks

We present a novel generative method for the creation of city-scale road layouts. While the output of recent methods is limited in both size of the covered area and diversity, our framework produces large traversable graphs of high quality consisting of vertices and edges representing complete street networks covering 400 square kilometers or more. While our framework can process general 2D embedded graphs, we focus on street networks due to the wide availability of training data. Our generative framework consists of a transformer decoder that is used in a sliding window manner to predict a field of indices, with each index encoding a representation of the local neighborhood. The semantics of each index is determined by a dictionary of context vectors. The index field is then input to a decoder to compute the street graph. Using data from OpenStreetMap, we train our system on whole cities and even across large countries such as the US, and finally compare it to the state of the art.
Evaluating generative audio systems and their metrics

Recent years have seen considerable advances in audio synthesis with deep generative models. However, the state-of-the-art is very difficult to quantify; different studies often use different evaluation methodologies and different metrics when reporting results, making a direct comparison to other systems difficult if not impossible. Furthermore, the perceptual relevance and meaning of the reported metrics in most cases unknown, prohibiting any conclusive insights with respect to practical usability and audio quality. This paper presents a study that investigates state-of-the-art approaches side-by-side with (i) a set of previously proposed objective metrics for audio reconstruction, and with (ii) a listening study. The results indicate that currently used objective metrics are insufficient to describe the perceptual quality of current systems.
Trading Off Privacy, Utility, and Efficiency in Federated Learning

Federated learning (FL) enables participating parties to collaboratively build a global model with boosted utility without disclosing private data information. Appropriate protection mechanisms have to be adopted to fulfill the opposing requirements in preserving privacy and maintaining high model utility. In addition, it is a mandate for a federated learning system to achieve high efficiency in order to enable large-scale model training and deployment. We propose a unified federated learning framework that reconciles horizontal and vertical federated learning. Based on this framework, we formulate and quantify the trade-offs between privacy leakage, utility loss, and efficiency reduction, which leads us to the No-Free-Lunch (NFL) theorem for the federated learning system. NFL indicates that it is unrealistic to expect an FL algorithm to simultaneously provide excellent privacy, utility, and efficiency in certain scenarios. We then analyze the lower bounds for the privacy leakage, utility loss, and efficiency reduction for several widely-adopted protection mechanisms, including Randomization, Homomorphic Encryption, Secret Sharing, and Compression. Our analysis could serve as a guide for selecting protection parameters to meet particular requirements.