OpenRaiser/Intern-Atlas huggingface dataset:
An Approach to Optimizing the VABA Protocol Using $κ$-size Committee
Venue: arXiv cs.DC 2024
Year: 2024

Byzantine agreement protocols in asynchronous networks have gained renewed attention due to their independence from network timing assumptions to ensure termination. Traditional asynchronous Byzantine agreement protocols require every party to broadcast its requests (e.g., transactions), leading to high communication costs as parties ultimately agree on one party's request. This inefficiency is particularly significant in multi-valued Byzantine agreement protocols, where parties aim to agree on one party's requests under the assumption $n=3f+1$, where $n$ is the total number of parties, and $f$ is the number of Byzantine parties. To address these inefficiencies, we propose Efficient-VABA (eVABA), an optimized protocol for the asynchronous Byzantine agreement (ABA) problem. By limiting broadcasts to a selected subset of parties, the protocol reduces the number of messages and computation overhead.
Towards an Autonomous Test Driver: High-Performance Driver Modeling via Reinforcement Learning
Venue: arXiv cs.RO 2024
Year: 2024

Success in racing requires a unique combination of vehicle setup, understanding of the racetrack, and human expertise. Since building and testing many different vehicle configurations in the real world is prohibitively expensive, high-fidelity simulation is a critical part of racecar development. However, testing different vehicle configurations still requires expert human input in order to evaluate their performance on different racetracks. In this work, we present the first steps towards an autonomous test driver, trained using deep reinforcement learning, capable of evaluating changes in vehicle setup on racing performance while driving at the level of the best human drivers. In addition, the autonomous driver model can be tuned to exhibit more human-like behavioral patterns by incorporating imitation learning into the RL training process. This extension permits the possibility of driver-specific vehicle setup optimization.
EditScout: Locating Forged Regions from Diffusion-based Edited Images with Multimodal LLM
Venue: arXiv cs.CV 2024
Year: 2024

Image editing technologies are tools used to transform, adjust, remove, or otherwise alter images. Recent research has significantly improved the capabilities of image editing tools, enabling the creation of photorealistic and semantically informed forged regions that are nearly indistinguishable from authentic imagery, presenting new challenges in digital forensics and media credibility. While current image forensic techniques are adept at localizing forged regions produced by traditional image manipulation methods, current capabilities struggle to localize regions created by diffusion-based techniques. To bridge this gap, we present a novel framework that integrates a multimodal Large Language Model (LLM) for enhanced reasoning capabilities to localize tampered regions in images produced by diffusion model-based editing methods. By leveraging the contextual and semantic strengths of LLMs, our framework achieves promising results on MagicBrush, AutoSplice, and PerfBrush (novel diffusion-based dataset) datasets, outperforming previous approaches in mIoU and F1-score metrics. Notably, our method excels on the PerfBrush dataset, a self-constructed test set featuring previously unseen types of edits. Here, where traditional methods typically falter, achieving markedly low scores, our approach demonstrates promising performance.
Detecting Redundant Health Survey Questions Using Language-agnostic BERT Sentence Embedding (LaBSE)
Venue: arXiv cs.CL 2024
Year: 2024

The goal of this work was to compute the semantic similarity among publicly available health survey questions in order to facilitate the standardization of survey-based Person-Generated Health Data (PGHD). We compiled various health survey questions authored in both English and Korean from the NIH CDE Repository, PROMIS, Korean public health agencies, and academic publications. Questions were drawn from various health lifelog domains. A randomized question pairing scheme was used to generate a Semantic Text Similarity (STS) dataset consisting of 1758 question pairs. Similarity scores between each question pair were assigned by two human experts. The tagged dataset was then used to build three classifiers featuring: Bag-of-Words, SBERT with BERT-based embeddings, and SBRET with LaBSE embeddings. The algorithms were evaluated using traditional contingency statistics. Among the three algorithms, SBERT-LaBSE demonstrated the highest performance in assessing question similarity across both languages, achieving an Area Under the Receiver Operating Characteristic (ROC) and Precision-Recall Curves of over 0.99. Additionally, it proved effective in identifying cross-lingual semantic similarities.The SBERT-LaBSE algorithm excelled at aligning semantically equivalent sentences across both languages but encountered challenges in capturing subtle nuances and maintaining computational efficiency. Future research should focus on testing with larger multilingual datasets and on calibrating and normalizing scores across the health lifelog domains to improve consistency. This study introduces the SBERT-LaBSE algorithm for calculating semantic similarity across two languages, showing it outperforms BERT-based models and the Bag of Words approach, highlighting its potential to improve semantic interoperability of survey-based PGHD across language barriers.
LL-ICM: Image Compression for Low-level Machine Vision via Large Vision-Language Model
Venue: arXiv cs.CV 2024
Year: 2024

Image Compression for Machines (ICM) aims to compress images for machine vision tasks rather than human viewing. Current works predominantly concentrate on high-level tasks like object detection and semantic segmentation. However, the quality of original images is usually not guaranteed in the real world, leading to even worse perceptual quality or downstream task performance after compression. Low-level (LL) machine vision models, like image restoration models, can help improve such quality, and thereby their compression requirements should also be considered. In this paper, we propose a pioneered ICM framework for LL machine vision tasks, namely LL-ICM. By jointly optimizing compression and LL tasks, the proposed LL-ICM not only enriches its encoding ability in generalizing to versatile LL tasks but also optimizes the processing ability of down-stream LL task models, achieving mutual adaptation for image codecs and LL task models. Furthermore, we integrate large-scale vision-language models into the LL-ICM framework to generate more universal and distortion-robust feature embeddings for LL vision tasks. Therefore, one LL-ICM codec can generalize to multiple tasks. We establish a solid benchmark to evaluate LL-ICM, which includes extensive objective experiments by using both full and no-reference image quality assessments. Experimental results show that LL-ICM can achieve 22.65% BD-rate reductions over the state-of-the-art methods.
How Good is ChatGPT in Giving Adaptive Guidance Using Knowledge Graphs in E-Learning Environments?
Venue: arXiv cs.AI 2024
Year: 2024

E-learning environments are increasingly harnessing large language models (LLMs) like GPT-3.5 and GPT-4 for tailored educational support. This study introduces an approach that integrates dynamic knowledge graphs with LLMs to offer nuanced student assistance. By evaluating past and ongoing student interactions, the system identifies and appends the most salient learning context to prompts directed at the LLM. Central to this method is the knowledge graph's role in assessing a student's comprehension of topic prerequisites. Depending on the categorized understanding (good, average, or poor), the LLM adjusts its guidance, offering advanced assistance, foundational reviews, or in-depth prerequisite explanations, respectively. Preliminary findings suggest students could benefit from this tiered support, achieving enhanced comprehension and improved task outcomes. However, several issues related to potential errors arising from LLMs were identified, which can potentially mislead students. This highlights the need for human intervention to mitigate these risks. This research aims to advance AI-driven personalized learning while acknowledging the limitations and potential pitfalls, thus guiding future research in technology and data-driven education.
Safeguarding Text-to-Image Generation via Inference-Time Prompt-Noise Optimization
Venue: arXiv cs.CV 2024
Year: 2024

Text-to-Image (T2I) diffusion models are widely recognized for their ability to generate high-quality and diverse images based on text prompts. However, despite recent advances, these models are still prone to generating unsafe images containing sensitive or inappropriate content, which can be harmful to users. Current efforts to prevent inappropriate image generation for diffusion models are easy to bypass and vulnerable to adversarial attacks. How to ensure that T2I models align with specific safety goals remains a significant challenge. In this work, we propose a novel, training-free approach, called Prompt-Noise Optimization (PNO), to mitigate unsafe image generation. Our method introduces a novel optimization framework that leverages both the continuous prompt embedding and the injected noise trajectory in the sampling process to generate safe images. Extensive numerical results demonstrate that our framework achieves state-of-the-art performance in suppressing toxic image generations and demonstrates robustness to adversarial attacks, without needing to tune the model parameters. Furthermore, compared with existing methods, PNO uses comparable generation time while offering the best tradeoff between the conflicting goals of safe generation and prompt-image alignment.
Offloading Revenue Maximization in Multi-UAV-Assisted Mobile Edge Computing for Video Stream
Venue: arXiv cs.ET 2024
Year: 2024

Traditional video transmission systems assisted by multiple Unmanned Aerial Vehicles (UAVs) are often limited by computing resources, making it challenging to meet the demands for efficient video processing. To solve this challenge, this paper presents a multi-UAV-assisted Device-to-Device (D2D) mobile edge computing system for the maximization of task offloading profits in video stream transmission. In particular, the system enables UAVs to collaborate with idle user devices to process video computing tasks by introducing D2D communications. To maximize the system efficiency, the paper jointly optimizes power allocation, video transcoding strategies, computing resource allocation, and UAV trajectory. The resulting non-convex optimization problem is formulated as a Markov decision process and solved relying on the Twin Delayed Deep Deterministic policy gradient (TD3) algorithm. Numerical results indicate that the proposed TD3 algorithm performs a significant advantage over other traditional algorithms in enhancing the overall system efficiency.
HyperDefect-YOLO: Enhance YOLO with HyperGraph Computation for Industrial Defect Detection
Venue: arXiv cs.CV 2024
Year: 2024

In the manufacturing industry, defect detection is an essential but challenging task aiming to detect defects generated in the process of production. Though traditional YOLO models presents a good performance in defect detection, they still have limitations in capturing high-order feature interrelationships, which hurdles defect detection in the complex scenarios and across the scales. To this end, we introduce hypergraph computation into YOLO framework, dubbed HyperDefect-YOLO (HD-YOLO), to improve representative ability and semantic exploitation. HD-YOLO consists of Defect Aware Module (DAM) and Mixed Graph Network (MGNet) in the backbone, which specialize for perception and extraction of defect features. To effectively aggregate multi-scale features, we propose HyperGraph Aggregation Network (HGANet) which combines hypergraph and attention mechanism to aggregate multi-scale features. Cross-Scale Fusion (CSF) is proposed to adaptively fuse and handle features instead of simple concatenation and convolution. Finally, we propose Semantic Aware Module (SAM) in the neck to enhance semantic exploitation for accurately localizing defects with different sizes in the disturbed background. HD-YOLO undergoes rigorous evaluation on public HRIPCB and NEU-DET datasets with significant improvements compared to state-of-the-art methods. We also evaluate HD-YOLO on self-built MINILED dataset collected in real industrial scenarios to demonstrate the effectiveness of the proposed method. The source codes are at https://github.com/Jay-zzcoder/HD-YOLO.
Digital Twin for Evaluating Detective Countermeasures in Smart Grid Cybersecurity
Venue: arXiv cs.CR 2024
Year: 2024

As the integration of digital technologies and communication systems continues within distribution grids, new avenues emerge to tackle energy transition challenges. Nevertheless, this deeper technological immersion amplifies the necessity for resilience against threats, encompassing both systemic outages and targeted cyberattacks. To ensure the robustness and safeguarding of vital infrastructure, a thorough examination of potential smart grid vulnerabilities and subsequent countermeasure development is essential. This study delves into the potential of digital twins, replicating a smart grid's cyber-physical laboratory environment, thereby enabling focused cybersecurity assessments. Merging the nuances of communication network emulation and power network simulation, we introduce a flexible, comprehensive digital twin model equipped for hardware-in-the-loop evaluations. Through this innovative framework, we not only verify and refine security countermeasures but also underscore their role in maintaining grid stability and trustworthiness.
UNCOVER: Unknown Class Object Detection for Autonomous Vehicles in Real-time
Venue: arXiv cs.CV 2024
Year: 2024

Autonomous driving (AD) operates in open-world scenarios, where encountering unknown objects is inevitable. However, standard object detectors trained on a limited number of base classes tend to ignore any unknown objects, posing potential risks on the road. To address this, it is important to learn a generic rather than a class specific objectness from objects seen during training. We therefore introduce an occupancy prediction together with bounding box regression. It learns to score the objectness by calculating the ratio of the predicted area occupied by actual objects. To enhance its generalizability, we increase the object diversity by exploiting data from other domains via Mosaic and Mixup augmentation. The objects outside the AD training classes are classified as a newly added out-of-distribution (OOD) class. Our solution UNCOVER, for UNknown Class Object detection for autonomous VEhicles in Real-time, excels at achieving both real-time detection and high recall of unknown objects on challenging AD benchmarks. To further attain very low false positive rates, particularly for close objects, we introduce a post-hoc filtering step that utilizes geometric cues extracted from the depth map, typically available within the AD system.
Enabling Sustainable Urban Mobility: The Role of 5G Communication in the Mobilities for EU Project
Venue: arXiv cs.NI 2024
Year: 2024

This paper examines the role of 5G communication in the Mobilities for EU project, a collaborative initiative involving 29 partners and 11 pilots aimed at revolutionizing urban mobility through electrification, automation, and connectivity. Focusing on Dresden as a Lead City, we explore the integration of 27 innovative solutions, including autonomous freight transport, eBuses, and charging robots, using a 5G communication network as the central framework. We analyze how 5G enables seamless connectivity and real-time data processing across diverse technologies, fostering interdependencies and synergies. This approach not only provides a cohesive understanding of the project's scope but also demonstrates 5G's critical role in smart city infrastructure. We evaluate the anticipated impact on sustainability metrics such as air quality, noise levels, CO2 emissions, and traffic congestion. The paper concludes by discussing challenges and strategies in leveraging 5G for comprehensive urban mobility solutions and its potential impact on future smart city developments.
Exploring the Influence of Label Aggregation on Minority Voices: Implications for Dataset Bias and Model Training
Venue: arXiv cs.CL 2024
Year: 2024

Resolving disagreement in manual annotation typically consists of removing unreliable annotators and using a label aggregation strategy such as majority vote or expert opinion to resolve disagreement. These may have the side-effect of silencing or under-representing minority but equally valid opinions. In this paper, we study the impact of standard label aggregation strategies on minority opinion representation in sexism detection. We investigate the quality and value of minority annotations, and then examine their effect on the class distributions in gold labels, as well as how this affects the behaviour of models trained on the resulting datasets. Finally, we discuss the potential biases introduced by each method and how they can be amplified by the models.
Automated Medical Report Generation for ECG Data: Bridging Medical Text and Signal Processing with Deep Learning
Venue: arXiv cs.CL 2024
Year: 2024

Recent advances in deep learning and natural language generation have significantly improved image captioning, enabling automated, human-like descriptions for visual content. In this work, we apply these captioning techniques to generate clinician-like interpretations of ECG data. This study leverages existing ECG datasets accompanied by free-text reports authored by healthcare professionals (HCPs) as training data. These reports, while often inconsistent, provide a valuable foundation for automated learning. We introduce an encoder-decoder-based method that uses these reports to train models to generate detailed descriptions of ECG episodes. This represents a significant advancement in ECG analysis automation, with potential applications in zero-shot classification and automated clinical decision support. The model is tested on various datasets, including both 1- and 12-lead ECGs. It significantly outperforms the state-of-the-art reference model by Qiu et al., achieving a METEOR score of 55.53% compared to 24.51% achieved by the reference model. Furthermore, several key design choices are discussed, providing a comprehensive overview of current challenges and innovations in this domain. The source codes for this research are publicly available in our Git repository https://git.zib.de/ableich/ecg-comment-generation-public
Boundary-Guided Learning for Gene Expression Prediction in Spatial Transcriptomics
Venue: arXiv cs.LG 2024
Year: 2024

Spatial transcriptomics (ST) has emerged as an advanced technology that provides spatial context to gene expression. Recently, deep learning-based methods have shown the capability to predict gene expression from WSI data using ST data. Existing approaches typically extract features from images and the neighboring regions using pretrained models, and then develop methods to fuse this information to generate the final output. However, these methods often fail to account for the cellular structure similarity, cellular density and the interactions within the microenvironment. In this paper, we propose a framework named BG-TRIPLEX, which leverages boundary information extracted from pathological images as guiding features to enhance gene expression prediction from WSIs. Specifically, our model consists of three branches: the spot, in-context and global branches. In the spot and in-context branches, boundary information, including edge and nuclei characteristics, is extracted using pretrained models. These boundary features guide the learning of cellular morphology and the characteristics of microenvironment through Multi-Head Cross-Attention. Finally, these features are integrated with global features to predict the final output. Extensive experiments were conducted on three public ST datasets. The results demonstrate that our BG-TRIPLEX consistently outperforms existing methods in terms of Pearson Correlation Coefficient (PCC). This method highlights the crucial role of boundary features in understanding the complex interactions between WSI and gene expression, offering a promising direction for future research.
Practical Considerations for Agentic LLM Systems
Venue: arXiv cs.AI 2024
Year: 2024

As the strength of Large Language Models (LLMs) has grown over recent years, so too has interest in their use as the underlying models for autonomous agents. Although LLMs demonstrate emergent abilities and broad expertise across natural language domains, their inherent unpredictability makes the implementation of LLM agents challenging, resulting in a gap between related research and the real-world implementation of such systems. To bridge this gap, this paper frames actionable insights and considerations from the research community in the context of established application paradigms to enable the construction and facilitate the informed deployment of robust LLM agents. Namely, we position relevant research findings into four broad categories--Planning, Memory, Tools, and Control Flow--based on common practices in application-focused literature and highlight practical considerations to make when designing agentic LLMs for real-world applications, such as handling stochasticity and managing resources efficiently. While we do not conduct empirical evaluations, we do provide the necessary background for discussing critical aspects of agentic LLM designs, both in academia and industry.
SRAM-Based PUF Reliability Prediction Using Cell-Imbalance Characterization in the State Space Diagram
Venue: arXiv cs.CR 2024
Year: 2024

This work proposes a methodology to estimate the statistical distribution of the probability that a 6T bit-cell starts up to a given logic value in SRAM memories for PUF applications. First, the distribution is obtained experimentally in a 65-nm CMOS device. As this distribution cannot be reproduced by electrical simulation, we explore the use of an alternative parameter defined as the distance between the origin and the separatrix in the bit-cell state space to quantify the mismatch of the cell. The resulting distribution of this parameter obtained from Monte Carlo simulations is then related to the start-up probability distribution using a two-component logistic function. The reported results show that the proposed imbalance factor is a good predictor for PUF-related reliability estimation with the advantage that can be applied at the early design stages.