OpenRaiser/Intern-Atlas huggingface dataset:
Exploiting Parallel Memory Write Requests for Covert Channel Attacks in Integrated CPU-GPU Systems
Venue: arXiv cs.CR 2023
Year: 2023

In heterogeneous SoCs, accelerators like integrated GPUs (iGPUs) are integrated on the same chip as CPUs, sharing the memory subsystem. In such systems, the massive memory requests from throughput-oriented accelerators significantly interfere with CPU memory requests. In addition to the large performance impact, this interference provides an attacker with a strong leakage vector for covert attacks across the processors, which is hard to achieve across the cores in a multi-core CPU. In this paper, we demonstrate that parallel memory write requests of the iGPU and more specifically, the management policy of the write buffer in the memory controller (MC) can lead to significantly stalling CPU memory read requests in heterogeneous SoCs. We characterize the slowdown on the shared read and write buffers in the memory controller and exploit it to build a cross-processor covert channel in Intel-based integrated CPU-GPU systems. We develop two attack variants that achieve a bandwidth of 1.65 kbps and 4.41 kbps and error rates of 0.49% and 4.32% respectively.
SEED-Bench: Benchmarking Multimodal LLMs with Generative Comprehension
Venue: arXiv cs.CL 2023
Year: 2023

Based on powerful Large Language Models (LLMs), recent generative Multimodal Large Language Models (MLLMs) have gained prominence as a pivotal research area, exhibiting remarkable capability for both comprehension and generation. In this work, we address the evaluation of generative comprehension in MLLMs as a preliminary step towards a comprehensive assessment of generative models, by introducing a benchmark named SEED-Bench. SEED-Bench consists of 19K multiple choice questions with accurate human annotations (x 6 larger than existing benchmarks), which spans 12 evaluation dimensions including the comprehension of both the image and video modality. We develop an advanced pipeline for generating multiple-choice questions that target specific evaluation dimensions, integrating both automatic filtering and manual verification processes. Multiple-choice questions with groundtruth options derived from human annotation enables an objective and efficient assessment of model performance, eliminating the need for human or GPT intervention during evaluation. We further evaluate the performance of 18 models across all 12 dimensions, covering both the spatial and temporal understanding. By revealing the limitations of existing MLLMs through evaluation results, we aim for SEED-Bench to provide insights for motivating future research. We will launch and consistently maintain a leaderboard to provide a platform for the community to assess and investigate model capability.
Publish for Public: Improving Access of Public Libraries Users to Research Findings through Plain Language Summaries
Venue: arXiv cs.IR 2023
Year: 2023

Public libraries play a crucial role in disseminating knowledge to society. However, most of their users do not have the specialized knowledge to understand the new research findings. Providing plain language summaries (PLSs) in public libraries is a way to make the new research findings more accessible and understandable for the public. This article proposes a framework for providing PLSs as a new service in public libraries. Drawing from the literature on science and society, PLSs, and public libraries, a theoretical framework is developed. The findings suggest that public libraries can collect PLSs through different methods, such as professional teams, researchers, crowdsourcing, etc. Library newsletters, special publications, brochures, independent online databases, and social networks are among the most effective for making PLSs accessible to users. By proposing a framework for providing PLSs in public libraries, this study helps to bridge the gap between scientific research and the public.
Robust Multi-Agent Reinforcement Learning with State Uncertainty
Venue: arXiv cs.LG 2023
Year: 2023

In real-world multi-agent reinforcement learning (MARL) applications, agents may not have perfect state information (e.g., due to inaccurate measurement or malicious attacks), which challenges the robustness of agents' policies. Though robustness is getting important in MARL deployment, little prior work has studied state uncertainties in MARL, neither in problem formulation nor algorithm design. Motivated by this robustness issue and the lack of corresponding studies, we study the problem of MARL with state uncertainty in this work. We provide the first attempt to the theoretical and empirical analysis of this challenging problem. We first model the problem as a Markov Game with state perturbation adversaries (MG-SPA) by introducing a set of state perturbation adversaries into a Markov Game. We then introduce robust equilibrium (RE) as the solution concept of an MG-SPA. We conduct a fundamental analysis regarding MG-SPA such as giving conditions under which such a robust equilibrium exists. Then we propose a robust multi-agent Q-learning (RMAQ) algorithm to find such an equilibrium, with convergence guarantees. To handle high-dimensional state-action space, we design a robust multi-agent actor-critic (RMAAC) algorithm based on an analytical expression of the policy gradient derived in the paper. Our experiments show that the proposed RMAQ algorithm converges to the optimal value function; our RMAAC algorithm outperforms several MARL and robust MARL methods in multiple multi-agent environments when state uncertainty is present. The source code is public on \url{https://github.com/sihongho/robust_marl_with_state_uncertainty}.
ScribbleVC: Scribble-supervised Medical Image Segmentation with Vision-Class Embedding
Venue: arXiv cs.CV 2023
Year: 2023

Medical image segmentation plays a critical role in clinical decision-making, treatment planning, and disease monitoring. However, accurate segmentation of medical images is challenging due to several factors, such as the lack of high-quality annotation, imaging noise, and anatomical differences across patients. In addition, there is still a considerable gap in performance between the existing label-efficient methods and fully-supervised methods. To address the above challenges, we propose ScribbleVC, a novel framework for scribble-supervised medical image segmentation that leverages vision and class embeddings via the multimodal information enhancement mechanism. In addition, ScribbleVC uniformly utilizes the CNN features and Transformer features to achieve better visual feature extraction. The proposed method combines a scribble-based approach with a segmentation network and a class-embedding module to produce accurate segmentation masks. We evaluate ScribbleVC on three benchmark datasets and compare it with state-of-the-art methods. The experimental results demonstrate that our method outperforms existing approaches in terms of accuracy, robustness, and efficiency. The datasets and code are released on GitHub.
Spiking Neural Networks and Bio-Inspired Supervised Deep Learning: A Survey
Venue: arXiv cs.NE 2023
Year: 2023

For a long time, biology and neuroscience fields have been a great source of inspiration for computer scientists, towards the development of Artificial Intelligence (AI) technologies. This survey aims at providing a comprehensive review of recent biologically-inspired approaches for AI. After introducing the main principles of computation and synaptic plasticity in biological neurons, we provide a thorough presentation of Spiking Neural Network (SNN) models, and we highlight the main challenges related to SNN training, where traditional backprop-based optimization is not directly applicable. Therefore, we discuss recent bio-inspired training methods, which pose themselves as alternatives to backprop, both for traditional and spiking networks. Bio-Inspired Deep Learning (BIDL) approaches towards advancing the computational capabilities and biological plausibility of current models.
Blockchain-based Decentralized Identity Management for Healthcare Systems
Venue: arXiv cs.CR 2023
Year: 2023

Blockchain-based decentralized identity management provides a promising solution to improve the security and privacy of healthcare systems and make them scalable. Traditional Identity Management Systems are centralized, which makes them single-point-of-failure, vulnerable to attacks and data breaches, and non-scalable. In contrast, decentralized identity management based on the blockchain can ensure secure and transparent access to patient data while preserving privacy. This approach enables patients to control their personal health data while granting permission for medical personnel to access specific information as needed. We propose a decentralized identity management system for healthcare systems named BDIMHS based on a permissioned blockchain with Hyperledger Indy and Hyperledger Aries. We develop further descriptions of required functionalities and provide high-level procedures for network initialization, enrollment, registration, issuance, verification and revocation functionalities. The proposed solution improves data security, privacy, immutability, interoperability, and patient autonomy by using selective disclosure, zero-knowledge proofs, Decentralized Identifiers, and Verifiable Credentials. Furthermore, we discuss the potential challenges associated with implementing this technology in healthcare and evaluate the performance and security of the proposed solution.
Count, Decode and Fetch: A New Approach to Handwritten Chinese Character Error Correction
Venue: arXiv cs.CV 2023
Year: 2023

Recently, handwritten Chinese character error correction has been greatly improved by employing encoder-decoder methods to decompose a Chinese character into an ideographic description sequence (IDS). However, existing methods implicitly capture and encode linguistic information inherent in IDS sequences, leading to a tendency to generate IDS sequences that match seen characters. This poses a challenge when dealing with an unseen misspelled character, as the decoder may generate an IDS sequence that matches a seen character instead. Therefore, we introduce Count, Decode and Fetch (CDF), a novel approach that exhibits better generalization towards unseen misspelled characters. CDF is mainly composed of three parts: the counter, the decoder, and the fetcher. In the first stage, the counter predicts the number of each radical class without the symbol-level position annotations. In the second stage, the decoder employs the counting information and generates the IDS sequence step by step. Moreover, by updating the counting information at each time step, the decoder becomes aware of the existence of each radical. With the decomposed IDS sequence, we can determine whether the given character is misspelled. If it is misspelled, the fetcher under the transductive transfer learning strategy predicts the ideal character that the user originally intended to write. We integrate our method into existing encoder-decoder models and significantly enhance their performance.
Communication-Sensing Region for Cell-Free Massive MIMO ISAC Systems
Venue: arXiv cs.IT 2023
Year: 2023

This paper investigates the system model and the transmit beamforming design for the Cell-Free massive multi-input multi-output (MIMO) integrated sensing and communication (ISAC) system. The impact of the uncertainty of the target locations on the propagation of wireless signals is considered during both uplink and downlink phases, and especially, the main statistics of the MIMO channel estimation error are theoretically derived in the closed-form fashion. A fundamental performance metric, termed communication-sensing (C-S) region, is defined for the considered system via three cases, i.e., the sensing-only case, the communication-only case and the ISAC case. The transmit beamforming design problems for the three cases are respectively carried out through different reformulations, e.g., the Lagrangian dual transform and the quadratic fractional transform, and some combinations of the block coordinate descent method and the successive convex approximation method. Numerical results present a 3-dimensional C-S region with a dynamic number of access points to illustrate the trade-off between communication and radar sensing. The advantage for radar sensing of the Cell-Free massive MIMO system is also studied via a comparison with the traditional cellular system. Finally, the efficacy of the proposed beamforming scheme is validated in comparison with zero-forcing and maximum ratio transmission schemes.
Recent Advances in Hierarchical Multi-label Text Classification: A Survey
Venue: arXiv cs.CL 2023
Year: 2023

Hierarchical multi-label text classification aims to classify the input text into multiple labels, among which the labels are structured and hierarchical. It is a vital task in many real world applications, e.g. scientific literature archiving. In this paper, we survey the recent progress of hierarchical multi-label text classification, including the open sourced data sets, the main methods, evaluation metrics, learning strategies and the current challenges. A few future research directions are also listed for community to further improve this field.
Representing and Reasoning with Multi-Stakeholder Qualitative Preference Queries
Venue: arXiv cs.AI 2023
Year: 2023

Many decision-making scenarios, e.g., public policy, healthcare, business, and disaster response, require accommodating the preferences of multiple stakeholders. We offer the first formal treatment of reasoning with multi-stakeholder qualitative preferences in a setting where stakeholders express their preferences in a qualitative preference language, e.g., CP-net, CI-net, TCP-net, CP-Theory. We introduce a query language for expressing queries against such preferences over sets of outcomes that satisfy specified criteria, e.g., $\mlangpref{ψ_1}{ψ_2}{A}$ (read loosely as the set of outcomes satisfying $ψ_1$ that are preferred over outcomes satisfying $ψ_2$ by a set of stakeholders $A$). Motivated by practical application scenarios, we introduce and analyze several alternative semantics for such queries, and examine their interrelationships. We provide a provably correct algorithm for answering multi-stakeholder qualitative preference queries using model checking in alternation-free $μ$-calculus. We present experimental results that demonstrate the feasibility of our approach.
Self-Supervised Learning of Gait-Based Biomarkers
Venue: arXiv cs.CV 2023
Year: 2023

Markerless motion capture (MMC) is revolutionizing gait analysis in clinical settings by making it more accessible, raising the question of how to extract the most clinically meaningful information from gait data. In multiple fields ranging from image processing to natural language processing, self-supervised learning (SSL) from large amounts of unannotated data produces very effective representations for downstream tasks. However, there has only been limited use of SSL to learn effective representations of gait and movement, and it has not been applied to gait analysis with MMC. One SSL objective that has not been applied to gait is contrastive learning, which finds representations that place similar samples closer together in the learned space. If the learned similarity metric captures clinically meaningful differences, this could produce a useful representation for many downstream clinical tasks. Contrastive learning can also be combined with causal masking to predict future timesteps, which is an appealing SSL objective given the dynamical nature of gait. We applied these techniques to gait analyses performed with MMC in a rehabilitation hospital from a diverse clinical population. We find that contrastive learning on unannotated gait data learns a representation that captures clinically meaningful information. We probe this learned representation using the framework of biomarkers and show it holds promise as both a diagnostic and response biomarker, by showing it can accurately classify diagnosis from gait and is responsive to inpatient therapy, respectively. We ultimately hope these learned representations will enable predictive and prognostic gait-based biomarkers that can facilitate precision rehabilitation through greater use of MMC to quantify movement in rehabilitation.
Causal-learn: Causal Discovery in Python
Venue: arXiv cs.LG 2023
Year: 2023

Causal discovery aims at revealing causal relations from observational data, which is a fundamental task in science and engineering. We describe $\textit{causal-learn}$, an open-source Python library for causal discovery. This library focuses on bringing a comprehensive collection of causal discovery methods to both practitioners and researchers. It provides easy-to-use APIs for non-specialists, modular building blocks for developers, detailed documentation for learners, and comprehensive methods for all. Different from previous packages in R or Java, $\textit{causal-learn}$ is fully developed in Python, which could be more in tune with the recent preference shift in programming languages within related communities. The library is available at https://github.com/py-why/causal-learn.
Traffic Offloading Probability for Integrated LEO Satellite-Terrestrial Networks
Venue: arXiv cs.IT 2023
Year: 2023

In this paper, we consider traffic offloading of integrated low earth orbit (LEO) satellite-terrestrial network. We first derive traffic offloading probability from the terrestrial network to the LEO satellite network based on the instantaneous radio signal strength. Then to overcome limited coverage and also traffic congestion of the terrestrial network, we design an optimal satellite network in which the minimum number of LEO satellites maximizes the traffic offloading probability, while probability of a generic LEO satellite being idle is close to zero. Since the satellite network is optimized regarding the intensities of the base stations (BSs) and the users, it can control the terrestrial traffic. Numerical results show that an appropriate number of LEO satellites overcomes the limited coverage in a region with low intensity of the BSs and also the traffic congestion by controlling the traffic in a region that the intensity of the users increases.
Two-dimensional Dyck words
Venue: arXiv cs.FL 2023
Year: 2023

We propose different ways of lifting the notion of Dyck language from words to 2-dimensional (2D) pictures, by means of new definitions of increasing comprehensiveness. Two of the proposals are based on alternative definitions of a Dyck language, which are equivalent over words but not on pictures. First, the property that any two pairs of matching parentheses are well-nested or disjoint is rephrased for rectangular boxes and leads to the well-nested Dyck, $DW_k$. This is a generalization of the known Chinese box language, but, unlike the Chinese boxes, $DW_k$ is not recognizable by a tiling system. Second, the Dyck cancellation rule is rephrased as a neutralization rule, mapping a quadruple of symbols representing the corners of a subpicture onto neutral symbols.The neutralizable Dyck language $DN_k$ is obtained by iterating neutralizations, starting from 2-by-2 subpictures, until the picture is wholly neutralized. Third, we define the Dyck crossword $DC_k$ as the row-column combination of Dyck word languages, which prescribes that each column and row is a Dyck word. The relation between matching parentheses is represented in $DC_k$ by an edge of a graph situated on the picture grid. Such edges form a circuit, of path length multiple of four, of alternating row and column matches. Length-four circuits have rectangular shape, while longer ones exhibit a large variety of forms. A proper subset of $DC_k$, called quaternate, is also introduced by excluding all circuits of length greater than 4. We prove that $DN_k$ properly includes $DW_k$, and that it coincides with the quaternate $DC_k$ such that the neutralizability relation between subpictures induces a partial order. The 2D languages well-nested, neutralizable, quaternate and Dyck crossword are ordered by strict inclusions. This work can be also seen as a first step towards the definition of context-free picture languages.
Human Preferences and Robot Constraints Aware Shared Control for Smooth Follower Motion Execution
Venue: arXiv cs.RO 2023
Year: 2023

With the continuous advancement of robot teleoperation technology, shared control is used to reduce the physical and mental load of the operator in teleoperation system. This paper proposes an alternating shared control framework for object grasping that considers both operator's preferences through their manual manipulation and the constraints of the follower robot. The switching between manual mode and automatic mode enables the operator to intervene the task according to their wishes. The generation of the grasping pose takes into account the current state of the operator's hand pose, as well as the manipulability of the robot. The object grasping experiment indicates that the use of the proposed grasping pose selection strategy leads to smoother follower movements when switching from manual mode to automatic mode.
No Fair Lunch: A Causal Perspective on Dataset Bias in Machine Learning for Medical Imaging
Venue: arXiv cs.LG 2023
Year: 2023

As machine learning methods gain prominence within clinical decision-making, addressing fairness concerns becomes increasingly urgent. Despite considerable work dedicated to detecting and ameliorating algorithmic bias, today's methods are deficient with potentially harmful consequences. Our causal perspective sheds new light on algorithmic bias, highlighting how different sources of dataset bias may appear indistinguishable yet require substantially different mitigation strategies. We introduce three families of causal bias mechanisms stemming from disparities in prevalence, presentation, and annotation. Our causal analysis underscores how current mitigation methods tackle only a narrow and often unrealistic subset of scenarios. We provide a practical three-step framework for reasoning about fairness in medical imaging, supporting the development of safe and equitable AI prediction models.
Compositional Verification in Rewriting Logic
Venue: arXiv cs.LO 2023
Year: 2023

In previous work, summarized in this paper, we proposed an operation of parallel composition for rewriting-logic theories, allowing compositional specification of systems and reusability of components. The present paper focuses on compositional verification. We show how the assume/guarantee technique can be transposed to our setting, by giving appropriate definitions of satisfaction based on transition structures and path semantics. We also show that simulation and equational abstraction can be done componentwise. Appropriate concepts of fairness and deadlock for our composition operation are discussed, as they affect satisfaction of temporal formulas. We keep in parallel a distributed and a global view of composed systems. We show that these views are equivalent and interchangeable, which may help our intuition and also has practical uses as, for example, it allows global-style verification of a modularly specified system.