OpenRaiser/Intern-Atlas huggingface dataset:
Non-strongly Stable Orders Also Define Interesting Simulation Relations
Venue: arXiv cs.LO 2024
Year: 2024

We present a study of the notion of coalgebraic simulation introduced by Hughes and Jacobs. Although in their original paper they allow any functorial order in their definition of coalgebraic simulation, for the simulation relations to have good properties they focus their attention on functors with orders which are strongly stable. This guarantees a so-called "composition-preserving" property from which all the desired good properties follow. We have noticed that the notion of strong stability not only ensures such good properties but also "distinguishes the direction" of the simulation. For example, the classic notion of simulation for labeled transition systems, the relation "p is simulated by q", can be defined as a coalgebraic simulation relation by means of a strongly stable order, whereas the opposite relation, "p simulates q", cannot. Our study was motivated by some interesting classes of simulations that illustrate the application of these results: covariant-contravariant simulations and conformance simulations.
On the specification of modal systems: A comparison of three frameworks
Venue: arXiv cs.LO 2024
Year: 2024

This paper studies the relationships between three notions of behavioural preorder that have been proposed in the literature: refinement over modal transition systems, and the covariant-contravariant simulation and the partial bisimulation preorders over labelled transition systems. It is shown that there are mutual translations between modal transition systems and labelled transition systems that preserve, and reflect, refinement and the covariant-contravariant simulation preorder. The translations are also shown to preserve the modal properties that can be expressed in the logics that characterize those preorders. A translation from labelled transition systems modulo the partial bisimulation preorder into the same model modulo the covariant-contravariant simulation preorder is also offered, together with some evidence that the former model is less expressive than the latter. In order to gain more insight into the relationships between modal transition systems modulo refinement and labelled transition systems modulo the covariant-contravariant simulation preorder, their connections are also phrased and studied in the context of institutions.
Rule Formats for Nominal Process Calculi
Venue: arXiv cs.LO 2024
Year: 2024

The nominal transition systems (NTSs) of Parrow et al. describe the operational semantics of nominal process calculi. We study NTSs in terms of the nominal residual transition systems (NRTSs) that we introduce. We provide rule formats for the specifications of NRTSs that ensure that the associated NRTS is an NTS and apply them to the operational specification of the early pi-calculus. Our study stems from the recent Nominal SOS of Cimini et al. and from earlier works in nominal sets and nominal logic by Gabbay, Pitts and their collaborators.
A generalized essentially non-hourglass total Lagrangian SPH solid dynamics
Venue: arXiv cs.CE 2024
Year: 2024

In this paper, we tackle a persistent numerical instability within the total Lagrangian smoothed particle hydrodynamics (TLSPH) solid dynamics. Specifically, we address the hourglass modes that may grow and eventually deteriorate the reliability of simulation, particularly in the scenarios characterized by large deformations. We propose a generalized essentially non-hourglass formulation based on volumetric-deviatoric stress decomposition, offering a general solution for elasticity, plasticity, anisotropy, and other material models. Comparing the standard SPH formulation with the original non-nested Laplacian operator applied in our previous work \cite{wu2023essentially} to handle the hourglass issues in standard elasticity, we introduce a correction for the discretization of shear stress that relies on the discrepancy produced by a tracing-back prediction of the initial inter-particle direction from the current deformation gradient. The present formulation, when applied to standard elastic materials, is able to recover the original Laplacian operator. Due to the dimensionless nature of the correction, this formulation handles complex material models in a very straightforward way. Furthermore, a magnitude limiter is introduced to minimize the correction in domains where the discrepancy is less pronounced. The present formulation is validated, with a single set of modeling parameters, through a series of benchmark cases, confirming good stability and accuracy across elastic, plastic, and anisotropic materials. To showcase its potential, the formulation is employed to simulate a complex problem involving viscous plastic Oobleck material, contacts, and very large deformation.
On the BER vs. Bandwidth-Efficiency Trade-offs in Windowed OTSM Dispensing with Zero-Padding
Venue: arXiv cs.IT 2024
Year: 2024

An orthogonal time sequency multiplexing (OTSM) scheme using practical signaling functions is proposed under strong phase noise (PHN) scenarios. By utilizing the transform relationships between the delay-sequency (DS), time-frequency (TF) and time-domains, we first conceive the DS-domain input-output relationship of our OTSM system, where the conventional zero-padding is discarded to increase the spectral efficiency. Then, the unconditional pairwise error probability is derived, followed by deriving the bit error ratio (BER) upper bound in closed-form. Moreover, we compare the BER performance of our OTSM system based on several practical signaling functions. Our simulation results demonstrate that the upper bound derived accurately predicts the BER performance in the case of moderate to high signal-to-noise ratios (SNRs), while harnessing practical window functions is capable of attaining an attractive out-of-band emission (OOBE) vs. BER trade-off.
Wireless Information Surveillance via STAR-RIS
Venue: arXiv cs.IT 2024
Year: 2024

We explore the potential of a simultaneously transmitting and reflecting reconfigurable intelligent surface (STAR-RIS) to enhance the performance of wireless surveillance systems. The STAR-RIS is deployed between a full-duplex (FD) multi-antenna legitimate eavesdropper (E) and a suspicious communication pair. It reflects the suspicious signal towards the suspicious receiver (SR), while simultaneously transmitting the same signal to E for interception purposes. Additionally, it enables the forwarding of a jamming signal from E to SR, which is located on the back side of the STAR-RIS. To enhance the eavesdropping non-outage probability, we formulate a non-convex joint optimization problem to design the beamforming vectors at E and reflection/transmission phase shift matrices at the STAR-RIS. We adopt the block coordinate descent (BCD) algorithm and propose an approach, mainly based on semi-definite relaxation (SDR) and successive convex approximation (SCA), for solving the resulting decoupled sub-problems. Finally, we compare the performance of the proposed design against low-complexity zero-forcing (ZF)-based beamforming designs.
IMUGPT 2.0: Language-Based Cross Modality Transfer for Sensor-Based Human Activity Recognition
Venue: arXiv cs.CV 2024
Year: 2024

One of the primary challenges in the field of human activity recognition (HAR) is the lack of large labeled datasets. This hinders the development of robust and generalizable models. Recently, cross modality transfer approaches have been explored that can alleviate the problem of data scarcity. These approaches convert existing datasets from a source modality, such as video, to a target modality (IMU). With the emergence of generative AI models such as large language models (LLMs) and text-driven motion synthesis models, language has become a promising source data modality as well as shown in proof of concepts such as IMUGPT. In this work, we conduct a large-scale evaluation of language-based cross modality transfer to determine their effectiveness for HAR. Based on this study, we introduce two new extensions for IMUGPT that enhance its use for practical HAR application scenarios: a motion filter capable of filtering out irrelevant motion sequences to ensure the relevance of the generated virtual IMU data, and a set of metrics that measure the diversity of the generated data facilitating the determination of when to stop generating virtual IMU data for both effective and efficient processing. We demonstrate that our diversity metrics can reduce the effort needed for the generation of virtual IMU data by at least 50%, which open up IMUGPT for practical use cases beyond a mere proof of concept.
Generation, Distillation and Evaluation of Motivational Interviewing-Style Reflections with a Foundational Language Model
Venue: arXiv cs.CL 2024
Year: 2024

Large Foundational Language Models are capable of performing many tasks at a high level but are difficult to deploy in many applications because of their size and proprietary ownership. Many will be motivated to distill specific capabilities of foundational models into smaller models that can be owned and controlled. In the development of a therapeutic chatbot, we wish to distill a capability known as reflective listening, in which a therapist produces reflections of client speech. These reflections either restate what a client has said, or connect what was said to a relevant observation, idea or guess that encourages and guides the client to continue contemplation. In this paper, we present a method for distilling the generation of reflections from a Foundational Language Model (GPT-4) into smaller models. We first show that GPT-4, using zero-shot prompting, can generate reflections at near 100% success rate, superior to all previous methods. Using reflections generated by GPT-4, we fine-tune different sizes of the GPT-2 family. The GPT-2-small model achieves 83% success on a hold-out test set and the GPT-2 XL achieves 90% success. We also show that GPT-4 can help in the labor-intensive task of evaluating the quality of the distilled models, using it as a zero-shot classifier. Using triple-human review as a guide, the classifier achieves a Cohen-Kappa of 0.66, a substantial inter-rater reliability figure.
Towards an Algebraic Framework For Approximating Functions Using Neural Network Polynomials
Venue: arXiv cs.LG 2024
Year: 2024

We make the case for neural network objects and extend an already existing neural network calculus explained in detail in Chapter 2 on \cite{bigbook}. Our aim will be to show that, yes, indeed, it makes sense to talk about neural network polynomials, neural network exponentials, sine, and cosines in the sense that they do indeed approximate their real number counterparts subject to limitations on certain of their parameters, $q$, and $\varepsilon$. While doing this, we show that the parameter and depth growth are only polynomial on their desired accuracy (defined as a 1-norm difference over $\mathbb{R}$), thereby showing that this approach to approximating, where a neural network in some sense has the structural properties of the function it is approximating is not entire intractable.
Salsa Fresca: Angular Embeddings and Pre-Training for ML Attacks on Learning With Errors
Venue: arXiv cs.CR 2024
Year: 2024

Learning with Errors (LWE) is a hard math problem underlying recently standardized post-quantum cryptography (PQC) systems for key exchange and digital signatures. Prior work proposed new machine learning (ML)-based attacks on LWE problems with small, sparse secrets, but these attacks require millions of LWE samples to train on and take days to recover secrets. We propose three key methods -- better preprocessing, angular embeddings and model pre-training -- to improve these attacks, speeding up preprocessing by $25\times$ and improving model sample efficiency by $10\times$. We demonstrate for the first time that pre-training improves and reduces the cost of ML attacks on LWE. Our architecture improvements enable scaling to larger-dimension LWE problems: this work is the first instance of ML attacks recovering sparse binary secrets in dimension $n=1024$, the smallest dimension used in practice for homomorphic encryption applications of LWE where sparse binary secrets are proposed.
Trustworthy Distributed AI Systems: Robustness, Privacy, and Governance
Venue: arXiv cs.LG 2024
Year: 2024

Emerging Distributed AI systems are revolutionizing big data computing and data processing capabilities with growing economic and societal impact. However, recent studies have identified new attack surfaces and risks caused by security, privacy, and fairness issues in AI systems. In this paper, we review representative techniques, algorithms, and theoretical foundations for trustworthy distributed AI through robustness guarantee, privacy protection, and fairness awareness in distributed learning. We first provide a brief overview of alternative architectures for distributed learning, discuss inherent vulnerabilities for security, privacy, and fairness of AI algorithms in distributed learning, and analyze why these problems are present in distributed learning regardless of specific architectures. Then we provide a unique taxonomy of countermeasures for trustworthy distributed AI, covering (1) robustness to evasion attacks and irregular queries at inference, and robustness to poisoning attacks, Byzantine attacks, and irregular data distribution during training; (2) privacy protection during distributed learning and model inference at deployment; and (3) AI fairness and governance with respect to both data and models. We conclude with a discussion on open challenges and future research directions toward trustworthy distributed AI, such as the need for trustworthy AI policy guidelines, the AI responsibility-utility co-design, and incentives and compliance.
Double-Dip: Thwarting Label-Only Membership Inference Attacks with Transfer Learning and Randomization
Venue: arXiv cs.LG 2024
Year: 2024

Transfer learning (TL) has been demonstrated to improve DNN model performance when faced with a scarcity of training samples. However, the suitability of TL as a solution to reduce vulnerability of overfitted DNNs to privacy attacks is unexplored. A class of privacy attacks called membership inference attacks (MIAs) aim to determine whether a given sample belongs to the training dataset (member) or not (nonmember). We introduce Double-Dip, a systematic empirical study investigating the use of TL (Stage-1) combined with randomization (Stage-2) to thwart MIAs on overfitted DNNs without degrading classification accuracy. Our study examines the roles of shared feature space and parameter values between source and target models, number of frozen layers, and complexity of pretrained models. We evaluate Double-Dip on three (Target, Source) dataset paris: (i) (CIFAR-10, ImageNet), (ii) (GTSRB, ImageNet), (iii) (CelebA, VGGFace2). We consider four publicly available pretrained DNNs: (a) VGG-19, (b) ResNet-18, (c) Swin-T, and (d) FaceNet. Our experiments demonstrate that Stage-1 reduces adversary success while also significantly increasing classification accuracy of nonmembers against an adversary with either white-box or black-box DNN model access, attempting to carry out SOTA label-only MIAs. After Stage-2, success of an adversary carrying out a label-only MIA is further reduced to near 50%, bringing it closer to a random guess and showing the effectiveness of Double-Dip. Stage-2 of Double-Dip also achieves lower ASR and higher classification accuracy than regularization and differential privacy-based methods.
Interpretation of Intracardiac Electrograms Through Textual Representations
Venue: arXiv cs.CL 2024
Year: 2024

Understanding the irregular electrical activity of atrial fibrillation (AFib) has been a key challenge in electrocardiography. For serious cases of AFib, catheter ablations are performed to collect intracardiac electrograms (EGMs). EGMs offer intricately detailed and localized electrical activity of the heart and are an ideal modality for interpretable cardiac studies. Recent advancements in artificial intelligence (AI) has allowed some works to utilize deep learning frameworks to interpret EGMs during AFib. Additionally, language models (LMs) have shown exceptional performance in being able to generalize to unseen domains, especially in healthcare. In this study, we are the first to leverage pretrained LMs for finetuning of EGM interpolation and AFib classification via masked language modeling. We formulate the EGM as a textual sequence and present competitive performances on AFib classification compared against other representations. Lastly, we provide a comprehensive interpretability study to provide a multi-perspective intuition of the model's behavior, which could greatly benefit the clinical use.
DeepAAT: Deep Automated Aerial Triangulation for Fast UAV-based Mapping
Venue: arXiv cs.CV 2024
Year: 2024

Automated Aerial Triangulation (AAT), aiming to restore image pose and reconstruct sparse points simultaneously, plays a pivotal role in earth observation. With its rich research heritage spanning several decades in photogrammetry, AAT has evolved into a fundamental process widely applied in large-scale Unmanned Aerial Vehicle (UAV) based mapping. Despite its advancements, classic AAT methods still face challenges like low efficiency and limited robustness. This paper introduces DeepAAT, a deep learning network designed specifically for AAT of UAV imagery. DeepAAT considers both spatial and spectral characteristics of imagery, enhancing its capability to resolve erroneous matching pairs and accurately predict image poses. DeepAAT marks a significant leap in AAT's efficiency, ensuring thorough scene coverage and precision. Its processing speed outpaces incremental AAT methods by hundreds of times and global AAT methods by tens of times while maintaining a comparable level of reconstruction accuracy. Additionally, DeepAAT's scene clustering and merging strategy facilitate rapid localization and pose determination for large-scale UAV images, even under constrained computing resources. The experimental results demonstrate DeepAAT's substantial improvements over conventional AAT methods, highlighting its potential in the efficiency and accuracy of UAV-based 3D reconstruction tasks. To benefit the photogrammetry society, the code of DeepAAT will be released at: https://github.com/WHU-USI3DV/DeepAAT.
A Multi-Agent Conversational Recommender System
Venue: arXiv cs.IR 2024
Year: 2024

Due to strong capabilities in conducting fluent, multi-turn conversations with users, Large Language Models (LLMs) have the potential to further improve the performance of Conversational Recommender System (CRS). Unlike the aimless chit-chat that LLM excels at, CRS has a clear target. So it is imperative to control the dialogue flow in the LLM to successfully recommend appropriate items to the users. Furthermore, user feedback in CRS can assist the system in better modeling user preferences, which has been ignored by existing studies. However, simply prompting LLM to conduct conversational recommendation cannot address the above two key challenges. In this paper, we propose Multi-Agent Conversational Recommender System (MACRS) which contains two essential modules. First, we design a multi-agent act planning framework, which can control the dialogue flow based on four LLM-based agents. This cooperative multi-agent framework will generate various candidate responses based on different dialogue acts and then choose the most appropriate response as the system response, which can help MACRS plan suitable dialogue acts. Second, we propose a user feedback-aware reflection mechanism which leverages user feedback to reason errors made in previous turns to adjust the dialogue act planning, and higher-level user information from implicit semantics. We conduct extensive experiments based on user simulator to demonstrate the effectiveness of MACRS in recommendation and user preferences collection. Experimental results illustrate that MACRS demonstrates an improvement in user interaction experience compared to directly using LLMs.
Efficient Reinforcement Learning for Routing Jobs in Heterogeneous Queueing Systems
Venue: arXiv cs.LG 2024
Year: 2024

We consider the problem of efficiently routing jobs that arrive into a central queue to a system of heterogeneous servers. Unlike homogeneous systems, a threshold policy, that routes jobs to the slow server(s) when the queue length exceeds a certain threshold, is known to be optimal for the one-fast-one-slow two-server system. But an optimal policy for the multi-server system is unknown and non-trivial to find. While Reinforcement Learning (RL) has been recognized to have great potential for learning policies in such cases, our problem has an exponentially large state space size, rendering standard RL inefficient. In this work, we propose ACHQ, an efficient policy gradient based algorithm with a low dimensional soft threshold policy parameterization that leverages the underlying queueing structure. We provide stationary-point convergence guarantees for the general case and despite the low-dimensional parameterization prove that ACHQ converges to an approximate global optimum for the special case of two servers. Simulations demonstrate an improvement in expected response time of up to ~30% over the greedy policy that routes to the fastest available server.
Scale Equalization for Multi-Level Feature Fusion
Venue: arXiv cs.CV 2024
Year: 2024

Deep neural networks have exhibited remarkable performance in a variety of computer vision fields, especially in semantic segmentation tasks. Their success is often attributed to multi-level feature fusion, which enables them to understand both global and local information from an image. However, we found that multi-level features from parallel branches are on different scales. The scale disequilibrium is a universal and unwanted flaw that leads to detrimental gradient descent, thereby degrading performance in semantic segmentation. We discover that scale disequilibrium is caused by bilinear upsampling, which is supported by both theoretical and empirical evidence. Based on this observation, we propose injecting scale equalizers to achieve scale equilibrium across multi-level features after bilinear upsampling. Our proposed scale equalizers are easy to implement, applicable to any architecture, hyperparameter-free, implementable without requiring extra computational cost, and guarantee scale equilibrium for any dataset. Experiments showed that adopting scale equalizers consistently improved the mIoU index across various target datasets, including ADE20K, PASCAL VOC 2012, and Cityscapes, as well as various decoder choices, including UPerHead, PSPHead, ASPPHead, SepASPPHead, and FCNHead.
Towards Quantum-Safe Federated Learning via Homomorphic Encryption: Learning with Gradients
Venue: arXiv cs.CR 2024
Year: 2024

This paper introduces a privacy-preserving distributed learning framework via private-key homomorphic encryption. Thanks to the randomness of the quantization of gradients, our learning with error (LWE) based encryption can eliminate the error terms, thus avoiding the issue of error expansion in conventional LWE-based homomorphic encryption. The proposed system allows a large number of learning participants to engage in neural network-based deep learning collaboratively over an honest-but-curious server, while ensuring the cryptographic security of participants' uploaded gradients.