OpenRaiser/Intern-Atlas huggingface dataset:
Key Safety Design Overview in AI-driven Autonomous Vehicles
Venue: arXiv cs.SE 2024
Year: 2024

With the increasing presence of autonomous SAE level 3 and level 4, which incorporate artificial intelligence software, along with the complex technical challenges they present, it is essential to maintain a high level of functional safety and robust software design. This paper explores the necessary safety architecture and systematic approach for automotive software and hardware, including fail soft handling of automotive safety integrity level (ASIL) D (highest level of safety integrity), integration of artificial intelligence (AI), and machine learning (ML) in automotive safety architecture. By addressing the unique challenges presented by increasing AI-based automotive software, we proposed various techniques, such as mitigation strategies and safety failure analysis, to ensure the safety and reliability of automotive software, as well as the role of AI in software reliability throughout the data lifecycle. Index Terms Safety Design, Automotive Software, Performance Evaluation, Advanced Driver Assistance Systems (ADAS) Applications, Automotive Software Systems, Electronic Control Units.
AI-assisted Knowledge Discovery in Biomedical Literature to Support Decision-making in Precision Oncology
Venue: arXiv cs.CL 2024
Year: 2024

The delivery of appropriate targeted therapies to cancer patients requires the complete analysis of the molecular profiling of tumors and the patient's clinical characteristics in the context of existing knowledge and recent findings described in biomedical literature and several other sources. We evaluated the potential contributions of specific natural language processing solutions to support knowledge discovery from biomedical literature. Two models from the Bidirectional Encoder Representations from Transformers (BERT) family, two Large Language Models, and PubTator 3.0 were tested for their ability to support the named entity recognition (NER) and the relation extraction (RE) tasks. PubTator 3.0 and the BioBERT model performed best in the NER task (best F1-score equal to 0.93 and 0.89, respectively), while BioBERT outperformed all other solutions in the RE task (best F1-score 0.79) and a specific use case it was applied to by recognizing nearly all entity mentions and most of the relations.
Self-regulated Learning Processes in Secondary Education: A Network Analysis of Trace-based Measures
Venue: arXiv cs.HC 2024
Year: 2024

While the capacity to self-regulate has been found to be crucial for secondary school students, prior studies often rely on self-report surveys and think-aloud protocols that present notable limitations in capturing self-regulated learning (SRL) processes. This study advances the understanding of SRL in secondary education by using trace data to examine SRL processes during multi-source writing tasks, with higher education participants included for comparison. We collected fine-grained trace data from 66 secondary school students and 59 university students working on the same writing tasks within a shared SRL-oriented learning environment. The data were labelled using Bannert's validated SRL coding scheme to reflect specific SRL processes, and we examined the relationship between these processes, essay performance, and educational levels. Using epistemic network analysis (ENA) to model and visualise the interconnected SRL processes in Bannert's coding scheme, we found that: (a) secondary school students predominantly engaged in three SRL processes -- Orientation, Re-reading, and Elaboration/Organisation; (b) high-performing secondary students engaged more in Re-reading, while low-performing students showed more Orientation process; and (c) higher education students exhibited more diverse SRL processes such as Monitoring and Evaluation than their secondary education counterparts, who heavily relied on following task instructions and rubrics to guide their writing. These findings highlight the necessity of designing scaffolding tools and developing teacher training programs to enhance awareness and development of SRL skills for secondary school learners.
Align, Generate, Learn: A Novel Closed-Loop Framework for Cross-Lingual In-Context Learning
Venue: arXiv cs.CL 2024
Year: 2024

Cross-lingual in-context learning (XICL) has emerged as a transformative paradigm for leveraging large language models (LLMs) to tackle multilingual tasks, especially for low-resource languages. However, existing approaches often rely on external retrievers or task-specific fine-tuning, limiting their scalability and generalizability. In this paper, we propose a novel self-supervised framework that harnesses the generative capabilities of LLMs to internally select and utilize task-relevant examples. Our method introduces two key objectives: a retrieval-generation alignment loss to optimize the quality of selected examples and a semantic coherence loss to ensure cross-lingual consistency. Through extensive experiments on multilingual benchmarks, our approach achieves state-of-the-art performance, significantly outperforming existing baselines. Further analysis highlights its robustness across diverse language families and its ability to generalize to unseen tasks. Human evaluations confirm the superior fluency, relevance, and semantic correctness of outputs generated by our method. This work provides a scalable, effective, and generalizable solution for cross-lingual in-context learning.
AFFAKT: A Hierarchical Optimal Transport based Method for Affective Facial Knowledge Transfer in Video Deception Detection
Venue: arXiv cs.CV 2024
Year: 2024

The scarcity of high-quality large-scale labeled datasets poses a huge challenge for employing deep learning models in video deception detection. To address this issue, inspired by the psychological theory on the relation between deception and expressions, we propose a novel method called AFFAKT in this paper, which enhances the classification performance by transferring useful and correlated knowledge from a large facial expression dataset. Two key challenges in knowledge transfer arise: 1) \textit{how much} knowledge of facial expression data should be transferred and 2) \textit{how to} effectively leverage transferred knowledge for the deception classification model during inference. Specifically, the optimal relation mapping between facial expression classes and deception samples is firstly quantified using proposed H-OTKT module and then transfers knowledge from the facial expression dataset to deception samples. Moreover, a correlation prototype within another proposed module SRKB is well designed to retain the invariant correlations between facial expression classes and deception classes through momentum updating. During inference, the transferred knowledge is fine-tuned with the correlation prototype using a sample-specific re-weighting strategy. Experimental results on two deception detection datasets demonstrate the superior performance of our proposed method. The interpretability study reveals high associations between deception and negative affections, which coincides with the theory in psychology.
Deep Learning Model Security: Threats and Defenses
Venue: arXiv cs.CR 2024
Year: 2024

Deep learning has transformed AI applications but faces critical security challenges, including adversarial attacks, data poisoning, model theft, and privacy leakage. This survey examines these vulnerabilities, detailing their mechanisms and impact on model integrity and confidentiality. Practical implementations, including adversarial examples, label flipping, and backdoor attacks, are explored alongside defenses such as adversarial training, differential privacy, and federated learning, highlighting their strengths and limitations. Advanced methods like contrastive and self-supervised learning are presented for enhancing robustness. The survey concludes with future directions, emphasizing automated defenses, zero-trust architectures, and the security challenges of large AI models. A balanced approach to performance and security is essential for developing reliable deep learning systems.
Motor Imagery Teleoperation of a Mobile Robot Using a Low-Cost Brain-Computer Interface for Multi-Day Validation
Venue: arXiv cs.RO 2024
Year: 2024

Brain-computer interfaces (BCI) have the potential to provide transformative control in prosthetics, assistive technologies (wheelchairs), robotics, and human-computer interfaces. While Motor Imagery (MI) offers an intuitive approach to BCI control, its practical implementation is often limited by the requirement for expensive devices, extensive training data, and complex algorithms, leading to user fatigue and reduced accessibility. In this paper, we demonstrate that effective MI-BCI control of a mobile robot in real-world settings can be achieved using a fine-tuned Deep Neural Network (DNN) with a sliding window, eliminating the need for complex feature extractions for real-time robot control. The fine-tuning process optimizes the convolutional and attention layers of the DNN to adapt to each user's daily MI data streams, reducing training data by 70% and minimizing user fatigue from extended data collection. Using a low-cost (~$3k), 16-channel, non-invasive, open-source electroencephalogram (EEG) device, four users teleoperated a quadruped robot over three days. The system achieved 78% accuracy on a single-day validation dataset and maintained a 75% validation accuracy over three days without extensive retraining from day-to-day. For real-world robot command classification, we achieved an average of 62% accuracy. By providing empirical evidence that MI-BCI systems can maintain performance over multiple days with reduced training data to DNN and a low-cost EEG device, our work enhances the practicality and accessibility of BCI technology. This advancement makes BCI applications more feasible for real-world scenarios, particularly in controlling robotic systems.
CLEAR: Channel Learning and Enhanced Adaptive Reconstruction for Semantic Communication in Complex Time-Varying Environments
Venue: arXiv cs.NI 2024
Year: 2024

To address the challenges of robust data transmission over complex time-varying channels, this paper introduces channel learning and enhanced adaptive reconstruction (CLEAR) strategy for semantic communications. CLEAR integrates deep joint source-channel coding (DeepJSCC) with an adaptive diffusion denoising model (ADDM) to form a unique framework. It leverages a trainable encoder-decoder architecture to encode data into complex semantic codes, which are then transmitted and reconstructed while minimizing distortion, ensuring high semantic fidelity. By addressing multipath effects, frequency-selective fading, phase noise, and Doppler shifts, CLEAR achieves high semantic fidelity and reliable transmission across diverse signal-to-noise ratios (SNRs) and channel conditions. Extensive experiments demonstrate that CLEAR achieves a 2.3 dB gain on peak signal-to-noise ratio (PSNR) over the existing state-of-the-art method, DeepJSCC-V. Furthermore, the results verify that CLEAR is robust against varying channel conditions, particularly in scenarios characterized by high Doppler shifts and strong phase noise.
A Wander Through the Multimodal Landscape: Efficient Transfer Learning via Low-rank Sequence Multimodal Adapter
Venue: arXiv cs.LG 2024
Year: 2024

Efficient transfer learning methods such as adapter-based methods have shown great success in unimodal models and vision-language models. However, existing methods have two main challenges in fine-tuning multimodal models. Firstly, they are designed for vision-language tasks and fail to extend to situations where there are more than two modalities. Secondly, they exhibit limited exploitation of interactions between modalities and lack efficiency. To address these issues, in this paper, we propose the loW-rank sequence multimodal adapter (Wander). We first use the outer product to fuse the information from different modalities in an element-wise way effectively. For efficiency, we use CP decomposition to factorize tensors into rank-one components and achieve substantial parameter reduction. Furthermore, we implement a token-level low-rank decomposition to extract more fine-grained features and sequence relationships between modalities. With these designs, Wander enables token-level interactions between sequences of different modalities in a parameter-efficient way. We conduct extensive experiments on datasets with different numbers of modalities, where Wander outperforms state-of-the-art efficient transfer learning methods consistently. The results fully demonstrate the effectiveness, efficiency and universality of Wander.
An Event-Triggered Framework for Trust-Mediated Human-Autonomy Interaction
Venue: arXiv cs.RO 2024
Year: 2024

Inspired by the increased cooperation between humans and autonomous systems, we present a new hybrid systems framework capturing the interconnected dynamics underlying these interactions. The framework accommodates models arising from both the autonomous systems and cognitive psychology literature in order to represent key elements such as human trust in the autonomous system. The intermittent nature of human interactions are incorporated by asynchronous event-triggered sampling at the framework's human-autonomous system interfaces. We illustrate important considerations for tuning framework parameters by investigating a practical application to an autonomous robotic swarm search and rescue scenario. In this way, we demonstrate how the proposed framework may assist in designing more efficient and effective interactions between humans and autonomous systems.
CBCMS: A Compliance Management System for Cross-Border Data Transfer
Venue: arXiv cs.CR 2024
Year: 2024

Cross-border data transfer is vital for the digital economy by enabling data flow across different countries or regions. However, ensuring compliance with diverse data protection regulations during the transfer introduces significant complexities. Existing solutions either focus on a single legal framework or neglect real-time and concurrent processing demands, resulting in incomplete and inconsistent compliance management. To address this issue, we propose Cross-Border Compliance Management System (CBCMS), which not only enables the unified management of data processing policies across multiple jurisdictions to ensure compliance with various legal frameworks involved in cross-border data transfer, but also supports real-time and high-concurrency processing capabilities. We design Policy Definition Language (PDL) that supports the unified management of data processing policies, bridging the gap between natural language policies and machine-processable expressions, thereby allowing various legal frameworks to be seamlessly integrated into CBCMS. We present Compliance Policy Generation Model (CPGM), the core component of CBCMS, which generates compliant data processing policies with high accuracy, achieving up to 25.16% improvement in F1 score (reaching 97.32%) compared to rule-based baseline. CPGM achieves inference time in the order of milliseconds (6 to 13 ms), and keeps low latency even under high-load scenarios, demonstrating high real-time and concurrent performance. To our knowledge, CBCMS is the first system to support unified compliance management across jurisdictions while ensuring real-time and concurrent processing capabilities.
MS2Mesh-XR: Multi-modal Sketch-to-Mesh Generation in XR Environments
Venue: arXiv cs.CV 2024
Year: 2024

We present MS2Mesh-XR, a novel multi-modal sketch-to-mesh generation pipeline that enables users to create realistic 3D objects in extended reality (XR) environments using hand-drawn sketches assisted by voice inputs. In specific, users can intuitively sketch objects using natural hand movements in mid-air within a virtual environment. By integrating voice inputs, we devise ControlNet to infer realistic images based on the drawn sketches and interpreted text prompts. Users can then review and select their preferred image, which is subsequently reconstructed into a detailed 3D mesh using the Convolutional Reconstruction Model. In particular, our proposed pipeline can generate a high-quality 3D mesh in less than 20 seconds, allowing for immersive visualization and manipulation in run-time XR scenes. We demonstrate the practicability of our pipeline through two use cases in XR settings. By leveraging natural user inputs and cutting-edge generative AI capabilities, our approach can significantly facilitate XR-based creative production and enhance user experiences. Our code and demo will be available at: https://yueqiu0911.github.io/MS2Mesh-XR/
Building a Mastodon Compatible Java Server for ActivityPub
Venue: arXiv cs.SI 2024
Year: 2024

ActivityPub is a decentralized social networking protocol that has gained significant attention from the media for its ability to communicate through the Fediverse, short for the federated web. Servers such as Mastodon implement the ActivityPub protocol to communicate over the Fediverse. In this paper, we deconstruct the core protocols used to build the distributed servers of the Fediverse. We explore Mastodon's complex implementation of ActivityPub and created our own Mastodon instance using Java Spring Boot and ActivityPub to interoperate with Mastodon servers.
Hyperbolic-constraint Point Cloud Reconstruction from Single RGB-D Images
Venue: arXiv cs.CV 2024
Year: 2024

Reconstructing desired objects and scenes has long been a primary goal in 3D computer vision. Single-view point cloud reconstruction has become a popular technique due to its low cost and accurate results. However, single-view reconstruction methods often rely on expensive CAD models and complex geometric priors. Effectively utilizing prior knowledge about the data remains a challenge. In this paper, we introduce hyperbolic space to 3D point cloud reconstruction, enabling the model to represent and understand complex hierarchical structures in point clouds with low distortion. We build upon previous methods by proposing a hyperbolic Chamfer distance and a regularized triplet loss to enhance the relationship between partial and complete point clouds. Additionally, we design adaptive boundary conditions to improve the model's understanding and reconstruction of 3D structures. Our model outperforms most existing models, and ablation studies demonstrate the significance of our model and its components. Experimental results show that our method significantly improves feature extraction capabilities. Our model achieves outstanding performance in 3D reconstruction tasks.
Multi-view Clustering via Unified Multi-kernel Learning and Matrix Factorization
Venue: arXiv cs.LG 2024
Year: 2024

Multi-view clustering has become increasingly important due to the multi-source character of real-world data. Among existing multi-view clustering methods, multi-kernel clustering and matrix factorization-based multi-view clustering have gained widespread attention as mainstream approaches. However, multi-kernel clustering tends to learn an optimal kernel and then perform eigenvalue decomposition on it, which leads to high computational complexity. Matrix factorization-based multi-view clustering methods impose orthogonal constraints on individual views. This overly emphasizes the accuracy of clustering structures within single views and restricts the learning of individual views. Based on this analysis, we propose a multi-view clustering method that integrates multi-kernel learning with matrix factorization. This approach combines the advantages of both multi-kernel learning and matrix factorization. It removes the orthogonal constraints on individual views and imposes orthogonal constraints on the consensus matrix, resulting in an accurate final clustering structure. Ultimately, the method is unified into a simple form of multi-kernel clustering, but avoids learning an optimal kernel, thus reducing the time complexity. Furthermore, we propose an efficient three-step optimization algorithm to achieve a locally optimal solution. Experiments on widely-used real-world datasets demonstrate the effectiveness of our proposed method.
SVasP: Self-Versatility Adversarial Style Perturbation for Cross-Domain Few-Shot Learning
Venue: arXiv cs.CV 2024
Year: 2024

Cross-Domain Few-Shot Learning (CD-FSL) aims to transfer knowledge from seen source domains to unseen target domains, which is crucial for evaluating the generalization and robustness of models. Recent studies focus on utilizing visual styles to bridge the domain gap between different domains. However, the serious dilemma of gradient instability and local optimization problem occurs in those style-based CD-FSL methods. This paper addresses these issues and proposes a novel crop-global style perturbation method, called \underline{\textbf{S}}elf-\underline{\textbf{V}}ersatility \underline{\textbf{A}}dversarial \underline{\textbf{S}}tyle \underline{\textbf{P}}erturbation (\textbf{SVasP}), which enhances the gradient stability and escapes from poor sharp minima jointly. Specifically, SVasP simulates more diverse potential target domain adversarial styles via diversifying input patterns and aggregating localized crop style gradients, to serve as global style perturbation stabilizers within one image, a concept we refer to as self-versatility. Then a novel objective function is proposed to maximize visual discrepancy while maintaining semantic consistency between global, crop, and adversarial features. Having the stabilized global style perturbation in the training phase, one can obtain a flattened minima in the loss landscape, boosting the transferability of the model to the target domains. Extensive experiments on multiple benchmark datasets demonstrate that our method significantly outperforms existing state-of-the-art methods. Our codes are available at https://github.com/liwenqianSEU/SVasP.
OriginPruner: Leveraging Method Origins for Guided Call Graph Pruning
Venue: arXiv cs.CR 2024
Year: 2024

Most static program analyses depend on Call Graphs (CGs), including reachability of security vulnerabilities. Static CGs ensure soundness through over-approximation, which results in inflated sizes and imprecision. Recent research has employed machine learning (ML) models to prune false edges and enhance CG precision. However, these models require real-world programs with high test coverage to generalize effectively and the inference is expensive. In this paper, we present OriginPruner, a novel call graph pruning technique that leverages the method origin, which is where a method signature is first introduced within a class hierarchy. By incorporating insights from a localness analysis that investigated the scope of method interactions into our approach, OriginPruner confidently identifies and prunes edges related to these origin methods. Our key findings reveal that (1) dominant origin methods, such as Iterator.next, significantly impact CG sizes; (2) derivatives of these origin methods are primarily local, enabling safe pruning without affecting downstream inter-procedural analyses; (3) OriginPruner achieves a significant reduction in CG size while maintaining the soundness of CGs for security applications like vulnerability propagation analysis; and (4) OriginPruner introduces minimal computational overhead. These findings underscore the potential of leveraging domain knowledge about the type system for more effective CG pruning, offering a promising direction for future work in static program analysis.