OpenRaiser/Intern-Atlas huggingface dataset:
Rethinking Disclosure Prevention with Pointwise Maximal Leakage
Venue: arXiv cs.CR 2024
Year: 2024

This paper introduces a paradigm shift in the way privacy is defined, driven by a novel interpretation of the fundamental result of Dwork and Naor about the impossibility of absolute disclosure prevention. We propose a general model of utility and privacy in which utility is achieved by disclosing the value of low-entropy features of a secret $X$, while privacy is maintained by hiding the value of high-entropy features of $X$. Adopting this model, we prove that, contrary to popular opinion, it is possible to provide meaningful inferential privacy guarantees. These guarantees are given in terms of an operationally-meaningful information measure called pointwise maximal leakage (PML) and prevent privacy breaches against a large class of adversaries regardless of their prior beliefs about $X$. We show that PML-based privacy is compatible with and provides insights into existing notions such as differential privacy. We also argue that our new framework enables highly flexible mechanism designs, where the randomness of a mechanism can be adjusted to the entropy of the data, ultimately, leading to higher utility.
Optimized Control-Centric Communication in Cooperative Adaptive Cruise Control Systems
Venue: arXiv cs.MA 2024
Year: 2024

In this study, we explore an innovative approach to enhance cooperative driving in vehicle platooning systems through the use of vehicle-to-everything (V2X) communication technologies. As Connected and Autonomous Vehicles (CAVs) integrate into increasingly dense traffic networks, the challenge of efficiently managing communication resources becomes crucial. Our focus is on optimizing communication strategies to support the growing network of interconnected vehicles without compromising traffic safety and efficiency. We introduce a novel control-aware communication framework designed to reduce communication overhead while maintaining essential performance standards in vehicle platoons. This method pivots from traditional periodic communication to more adaptable aperiodic or event-triggered schemes. Additionally, we integrate Model-Based Communication (MBC) to enhance vehicle perception under suboptimal communication conditions. By merging control-aware communication with MBC, our approach effectively controls vehicle platoons, striking a balance between communication resource conservation and control performance. The results show a marked decrease in communication frequency by 47\%, with minimal impact on control accuracy, such as less than 1\% variation in speed. Extensive simulations validate the effectiveness of our combined approach in managing communication and control in vehicle platoons, offering a promising solution for future cooperative driving systems.
Haptics-Enabled Forceps with Multi-Modal Force Sensing: Towards Task-Autonomous Surgery
Venue: arXiv cs.RO 2024
Year: 2024

Many robotic surgical systems have been developed with micro-sized forceps for tissue manipulation. However, these systems often lack force sensing at the tool side and the manipulation forces are roughly estimated and controlled relying on the surgeon's visual perception. To address this challenge, we present a vision-based module to enable the micro-sized forceps' multi-modal force sensing. A miniature sensing module adaptive to common micro-sized forceps is proposed, consisting of a flexure, a camera, and a customised target. The deformation of the flexure is obtained by the camera estimating the pose variation of the top-mounted target. Then, the external force applied to the sensing module is calculated using the flexure's displacement and stiffness matrix. Integrating the sensing module into the forceps, in conjunction with a single-axial force sensor at the proximal end, we equip the forceps with haptic sensing capabilities. Mathematical equations are derived to estimate the multi-modal force sensing of the haptics-enabled forceps, including pushing/pulling forces (Mode-I) and grasping forces (Mode-II). A series of experiments on phantoms and ex vivo tissues are conducted to verify the feasibility of the proposed design and method. Results indicate that the haptics-enabled forceps can achieve multi-modal force estimation effectively and potentially realize autonomous robotic tissue grasping procedures with controlled forces. A video demonstrating the experiments can be found at https://youtu.be/pi9bqSkwCFQ.
Uncertainty-Aware Pedestrian Trajectory Prediction via Distributional Diffusion
Venue: arXiv cs.CV 2024
Year: 2024

Tremendous efforts have been put forth on predicting pedestrian trajectory with generative models to accommodate uncertainty and multi-modality in human behaviors. An individual's inherent uncertainty, e.g., change of destination, can be masked by complex patterns resulting from the movements of interacting pedestrians. However, latent variable-based generative models often entangle such uncertainty with complexity, leading to limited either latent expressivity or predictive diversity. In this work, we propose to separately model these two factors by implicitly deriving a flexible latent representation to capture intricate pedestrian movements, while integrating predictive uncertainty of individuals with explicit bivariate Gaussian mixture densities over their future locations. More specifically, we present a model-agnostic uncertainty-aware pedestrian trajectory prediction framework, parameterizing sufficient statistics for the mixture of Gaussians that jointly comprise the multi-modal trajectories. We further estimate these parameters of interest by approximating a denoising process that progressively recovers pedestrian movements from noise. Unlike previous studies, we translate the predictive stochasticity to explicit distributions, allowing it to readily generate plausible future trajectories indicating individuals' self-uncertainty. Moreover, our framework is compatible with different neural net architectures. We empirically show the performance gains over state-of-the-art even with lighter backbones, across most scenes on two public benchmarks.
Improving Out-of-Distribution Detection with Disentangled Foreground and Background Features
Venue: arXiv cs.CV 2024
Year: 2024

Detecting out-of-distribution (OOD) inputs is a principal task for ensuring the safety of deploying deep-neural-network classifiers in open-set scenarios. OOD samples can be drawn from arbitrary distributions and exhibit deviations from in-distribution (ID) data in various dimensions, such as foreground features (e.g., objects in CIFAR100 images vs. those in CIFAR10 images) and background features (e.g., textural images vs. objects in CIFAR10). Existing methods can confound foreground and background features in training, failing to utilize the background features for OOD detection. This paper considers the importance of feature disentanglement in out-of-distribution detection and proposes the simultaneous exploitation of both foreground and background features to support the detection of OOD inputs in in out-of-distribution detection. To this end, we propose a novel framework that first disentangles foreground and background features from ID training samples via a dense prediction approach, and then learns a new classifier that can evaluate the OOD scores of test images from both foreground and background features. It is a generic framework that allows for a seamless combination with various existing OOD detection methods. Extensive experiments show that our approach 1) can substantially enhance the performance of four different state-of-the-art (SotA) OOD detection methods on multiple widely-used OOD datasets with diverse background features, and 2) achieves new SotA performance on these benchmarks.
Nakamoto Consensus under Bounded Processing Capacity
Venue: arXiv cs.CR 2024
Year: 2024

For Nakamoto's longest-chain consensus protocol, whose proof-of-work (PoW) and proof-of-stake (PoS) variants power major blockchains such as Bitcoin and Cardano, we revisit the classic problem of the security-performance tradeoff: Given a network of nodes with finite communication- and computation-resources, against what fraction of adversary power is Nakamoto consensus (NC) secure for a given block production rate? State-of-the-art analyses of NC fail to answer this question, because their bounded-delay model does not capture the rate limits to nodes' processing of blocks, which cause congestion when blocks are released in quick succession. We develop a new analysis technique to prove a refined security-performance tradeoff for PoW NC in a bounded-capacity model. In this model, we show that, in contrast to the classic bounded-delay model, Nakamoto's private attack is no longer the worst attack, and a new attack we call the teasing strategy, that exploits congestion, is strictly worse. In PoS, equivocating blocks can exacerbate congestion, making traditional PoS NC insecure except at very low block production rates. To counter such equivocation spamming, we present a variant of PoS NC we call Blanking NC (BlaNC), which achieves the same resilience as PoW NC.
Integrating Temporality and Causality into Acyclic Argumentation Frameworks using a Transition System
Venue: arXiv cs.AI 2024
Year: 2024

In the context of abstract argumentation, we present the benefits of considering temporality, i.e. the order in which arguments are enunciated, as well as causality. We propose a formal method to rewrite the concepts of acyclic abstract argumentation frameworks into an action language, that allows us to model the evolution of the world, and to establish causal relationships between the enunciation of arguments and their consequences, whether direct or indirect. An Answer Set Programming implementation is also proposed, as well as perspectives towards explanations.
GADformer: A Transparent Transformer Model for Group Anomaly Detection on Trajectories
Venue: arXiv cs.LG 2024
Year: 2024

Group Anomaly Detection (GAD) identifies unusual pattern in groups where individual members might not be anomalous. This task is of major importance across multiple disciplines, in which also sequences like trajectories can be considered as a group. As groups become more diverse in heterogeneity and size, detecting group anomalies becomes challenging, especially without supervision. Though Recurrent Neural Networks are well established deep sequence models, their performance can decrease with increasing sequence lengths. Hence, this paper introduces GADformer, a BERT-based model for attention-driven GAD on trajectories in unsupervised and semi-supervised settings. We demonstrate how group anomalies can be detected by attention-based GAD. We also introduce the Block-Attention-anomaly-Score (BAS) to enhance model transparency by scoring attention patterns. In addition to that, synthetic trajectory generation allows various ablation studies. In extensive experiments we investigate our approach versus related works in their robustness for trajectory noise and novelties on synthetic data and three real world datasets.
Comprehensive Analysis of Maximum Power Association Policy for Cellular Networks Using Distance and Angular Coordinates
Venue: arXiv cs.IT 2024
Year: 2024

A novel stochastic geometry framework is proposed in this paper to study the downlink coverage performance in a millimeter wave (mmWave) cellular network by jointly considering the polar coordinates of the Base Stations (BSs) with respect to the typical user located at the origin. Specifically, both the Euclidean and the angular distances of the BSs in a maximum power-based association policy for the UE are considered to account for realistic beam management considerations, which have been largely ignored in the literature, especially in the cell association phase. For completeness, two other association schemes are considered and exact-form expressions for the coverage probability are derived. Subsequently, the key role of angular distances is highlighted by defining the dominant interferer using angular distance-based criteria instead of Euclidean distance-based, and conducting a dominant interferer-based coverage probability analysis. Among others, the numerical results revealed that considering angular distance-based criteria for determining both the serving and the dominant interfering BS, can approximate the coverage performance more accurately as compared to utilizing Euclidean distance-based criteria. To the best of the authors$'$ knowledge, this is the first work that rigorously explores the role of angular distances in the association policy and analysis of cellular networks.
Approximation of group explainers with coalition structure using Monte Carlo sampling on the product space of coalitions and features
Venue: arXiv cs.LG 2024
Year: 2024

In recent years, many Machine Learning (ML) explanation techniques have been designed using ideas from cooperative game theory. These game-theoretic explainers suffer from high complexity, hindering their exact computation in practical settings. In our work, we focus on a wide class of linear game values, as well as coalitional values, for the marginal game based on a given ML model and predictor vector. By viewing these explainers as expectations over appropriate sample spaces, we design a novel Monte Carlo sampling algorithm that estimates them at a reduced complexity that depends linearly on the size of the background dataset. We set up a rigorous framework for the statistical analysis and obtain error bounds for our sampling methods. The advantage of this approach is that it is fast, easily implementable, and model-agnostic. Furthermore, it has similar statistical accuracy as other known estimation techniques that are more complex and model-specific. We provide rigorous proofs of statistical convergence, as well as numerical experiments whose results agree with our theoretical findings.
Fixed Design Analysis of Regularization-Based Continual Learning
Venue: arXiv cs.LG 2024
Year: 2024

We consider a continual learning (CL) problem with two linear regression tasks in the fixed design setting, where the feature vectors are assumed fixed and the labels are assumed to be random variables. We consider an $\ell_2$-regularized CL algorithm, which computes an Ordinary Least Squares parameter to fit the first dataset, then computes another parameter that fits the second dataset under an $\ell_2$-regularization penalizing its deviation from the first parameter, and outputs the second parameter. For this algorithm, we provide tight bounds on the average risk over the two tasks. Our risk bounds reveal a provable trade-off between forgetting and intransigence of the $\ell_2$-regularized CL algorithm: with a large regularization parameter, the algorithm output forgets less information about the first task but is intransigent to extract new information from the second task; and vice versa. Our results suggest that catastrophic forgetting could happen for CL with dissimilar tasks (under a precise similarity measurement) and that a well-tuned $\ell_2$-regularization can partially mitigate this issue by introducing intransigence.
Unsupervised Learning for Solving the Travelling Salesman Problem
Venue: arXiv cs.AI 2024
Year: 2024

We propose UTSP, an unsupervised learning (UL) framework for solving the Travelling Salesman Problem (TSP). We train a Graph Neural Network (GNN) using a surrogate loss. The GNN outputs a heat map representing the probability for each edge to be part of the optimal path. We then apply local search to generate our final prediction based on the heat map. Our loss function consists of two parts: one pushes the model to find the shortest path and the other serves as a surrogate for the constraint that the route should form a Hamiltonian Cycle. Experimental results show that UTSP outperforms the existing data-driven TSP heuristics. Our approach is parameter efficient as well as data efficient: the model takes $\sim$ 10\% of the number of parameters and $\sim$ 0.2\% of training samples compared with reinforcement learning or supervised learning methods.
FedML-HE: An Efficient Homomorphic-Encryption-Based Privacy-Preserving Federated Learning System
Venue: arXiv cs.LG 2024
Year: 2024

Federated Learning trains machine learning models on distributed devices by aggregating local model updates instead of local data. However, privacy concerns arise as the aggregated local models on the server may reveal sensitive personal information by inversion attacks. Privacy-preserving methods, such as homomorphic encryption (HE), then become necessary for FL training. Despite HE's privacy advantages, its applications suffer from impractical overheads, especially for foundation models. In this paper, we present FedML-HE, the first practical federated learning system with efficient HE-based secure model aggregation. FedML-HE proposes to selectively encrypt sensitive parameters, significantly reducing both computation and communication overheads during training while providing customizable privacy preservation. Our optimized system demonstrates considerable overhead reduction, particularly for large foundation models (e.g., ~10x reduction for ResNet-50, and up to ~40x reduction for BERT), demonstrating the potential for scalable HE-based FL deployment.
DiffuMask: Synthesizing Images with Pixel-level Annotations for Semantic Segmentation Using Diffusion Models
Venue: arXiv cs.CV 2024
Year: 2024

Collecting and annotating images with pixel-wise labels is time-consuming and laborious. In contrast, synthetic data can be freely available using a generative model (e.g., DALL-E, Stable Diffusion). In this paper, we show that it is possible to automatically obtain accurate semantic masks of synthetic images generated by the Off-the-shelf Stable Diffusion model, which uses only text-image pairs during training. Our approach, called DiffuMask, exploits the potential of the cross-attention map between text and image, which is natural and seamless to extend the text-driven image synthesis to semantic mask generation. DiffuMask uses text-guided cross-attention information to localize class/word-specific regions, which are combined with practical techniques to create a novel high-resolution and class-discriminative pixel-wise mask. The methods help to reduce data collection and annotation costs obviously. Experiments demonstrate that the existing segmentation methods trained on synthetic data of DiffuMask can achieve a competitive performance over the counterpart of real data (VOC 2012, Cityscapes). For some classes (e.g., bird), DiffuMask presents promising performance, close to the stateof-the-art result of real data (within 3% mIoU gap). Moreover, in the open-vocabulary segmentation (zero-shot) setting, DiffuMask achieves a new SOTA result on Unseen class of VOC 2012. The project website can be found at https://weijiawu.github.io/DiffusionMask/.
Skeleton Regression: A Graph-Based Approach to Estimation with Manifold Structure
Venue: arXiv cs.LG 2024
Year: 2024

We introduce a new regression framework designed to deal with large-scale, complex data that lies around a low-dimensional manifold with noises. Our approach first constructs a graph representation, referred to as the skeleton, to capture the underlying geometric structure. We then define metrics on the skeleton graph and apply nonparametric regression techniques, along with feature transformations based on the graph, to estimate the regression function. We also discuss the limitations of some nonparametric regressors with respect to the general metric space such as the skeleton graph. The proposed regression framework suggests a novel way to deal with data with underlying geometric structures and provides additional advantages in handling the union of multiple manifolds, additive noises, and noisy observations. We provide statistical guarantees for the proposed method and demonstrate its effectiveness through simulations and real data examples.
Learning a Depth Covariance Function
Venue: arXiv cs.CV 2024
Year: 2024

We propose learning a depth covariance function with applications to geometric vision tasks. Given RGB images as input, the covariance function can be flexibly used to define priors over depth functions, predictive distributions given observations, and methods for active point selection. We leverage these techniques for a selection of downstream tasks: depth completion, bundle adjustment, and monocular dense visual odometry.
Robust Millimeter Beamforming via Self-Supervised Hybrid Deep Learning
Venue: arXiv cs.IT 2024
Year: 2024

Beamforming with large-scale antenna arrays has been widely used in recent years, which is acknowledged as an important part in 5G and incoming 6G. Thus, various techniques are leveraged to improve its performance, e.g., deep learning, advanced optimization algorithms, etc. Although its performance in many previous research scenarios with deep learning is quite attractive, usually it drops rapidly when the environment or dataset is changed. Therefore, designing effective beamforming network with strong robustness is an open issue for the intelligent wireless communications. In this paper, we propose a robust beamforming self-supervised network, and verify it in two kinds of different datasets with various scenarios. Simulation results show that the proposed self-supervised network with hybrid learning performs well in both classic DeepMIMO and new WAIR-D dataset with the strong robustness under the various environments. Also, we present the principle to explain the rationality of this kind of hybrid learning, which is instructive to apply with more kinds of datasets.
Almost-Optimal Sublinear Additive Spanners
Venue: arXiv cs.DS 2024
Year: 2024

Given an undirected unweighted graph $G = (V, E)$ on $n$ vertices and $m$ edges, a subgraph $H\subseteq G$ is a spanner of $G$ with stretch function $f: \mathbb{R}_+ \rightarrow \mathbb{R}_+$, if for every pair $s, t$ of vertices in $V$, $\text{dist}_{H}(s, t)\le f(\text{dist}_{G}(s, t))$. When $f(d) = d + o(d)$, $H$ is called a sublinear additive spanner; when $f(d) = d + o(n)$, $H$ is called an \emph{additive spanner}, and $f(d) - d$ is usually called the \emph{additive stretch} of $H$. As our primary result, we show that for any constant $δ>0$ and constant integer $k\geq 2$, every graph on $n$ vertices has a sublinear additive spanner with stretch function $f(d)=d+O(d^{1-1/k})$ and $O\big(n^{1+\frac{1+δ}{2^{k+1}-1}}\big)$ edges. When $k = 2$, this improves upon the previous spanner construction with stretch function $f(d) = d + O(d^{1/2})$ and $\tilde{O}(n^{1+3/17})$ edges; for any constant integer $k\geq 3$, this improves upon the previous spanner construction with stretch function $f(d) = d + O(d^{1-1/k})$ and $O\bigg(n^{1+\frac{(3/4)^{k-2}}{7 - 2\cdot (3/4)^{k-2}}}\bigg)$ edges. Most importantly, the size of our spanners almost matches the lower bound of $Ω\big(n^{1+\frac{1}{2^{k+1}-1}}\big)$, which holds for all compression schemes achieving the same stretch function. As our second result, we show a new construction of additive spanners with stretch $O(n^{0.403})$ and $\tilde{O}(n)$ edges, which slightly improves upon the previous stretch bound of $O(n^{3/7+\varepsilon})$ achieved by linear-size spanners. An additional advantage of our spanner is that it admits a subquadratic construction runtime of $\tilde{O}(m + n^{13/7})$, while the previous construction requires all-pairs shortest paths computation which takes $O(\min\{mn, n^{2.373}\})$ time.