OpenRaiser/Intern-Atlas huggingface dataset:
POMDP-Based Trajectory Planning for On-Ramp Highway Merging
Venue: arXiv cs.RO 2024
Year: 2024

This paper addresses the trajectory planning problem for automated vehicle on-ramp highway merging. To tackle this challenge, we extend our previous work on trajectory planning at unsignalized intersections using Partially Observable Markov Decision Processes (POMDPs). The method utilizes the Adaptive Belief Tree (ABT) algorithm, an approximate sampling-based approach to solve POMDPs efficiently. We outline the POMDP formulation process, beginning with discretizing the highway topology to reduce problem complexity. Additionally, we describe the dynamics and measurement models used to predict future states and establish the relationship between available noisy measurements and predictions. Building on our previous work, the dynamics model is expanded to account for lateral movements necessary for lane changes during the merging process. We also define the reward function, which serves as the primary mechanism for specifying the desired behavior of the automated vehicle, combining multiple goals such as avoiding collisions or maintaining appropriate velocity. Our simulation results, conducted on three scenarios based on real-life traffic data from German highways, demonstrate the method's ability to generate safe, collision-free, and efficient merging trajectories. This work shows the versatility of this POMDP-based approach in tackling various automated driving problems.
Paired Wasserstein Autoencoders for Conditional Sampling
Venue: arXiv cs.LG 2024
Year: 2024

Wasserstein distances greatly influenced and coined various types of generative neural network models. Wasserstein autoencoders are particularly notable for their mathematical simplicity and straight-forward implementation. However, their adaptation to the conditional case displays theoretical difficulties. As a remedy, we propose the use of two paired autoencoders. Under the assumption of an optimal autoencoder pair, we leverage the pairwise independence condition of our prescribed Gaussian latent distribution to overcome this theoretical hurdle. We conduct several experiments to showcase the practical applicability of the resulting paired Wasserstein autoencoders. Here, we consider imaging tasks and enable conditional sampling for denoising, inpainting, and unsupervised image translation. Moreover, we connect our image translation model to the Monge map behind Wasserstein-2 distances.
TrojanWhisper: Evaluating Pre-trained LLMs to Detect and Localize Hardware Trojans
Venue: arXiv cs.CR 2024
Year: 2024

Existing Hardware Trojans (HT) detection methods face several critical limitations: logic testing struggles with scalability and coverage for large designs, side-channel analysis requires golden reference chips, and formal verification methods suffer from state-space explosion. The emergence of Large Language Models (LLMs) offers a promising new direction for HT detection by leveraging their natural language understanding and reasoning capabilities. For the first time, this paper explores the potential of general-purpose LLMs in detecting various HTs inserted in Register Transfer Level (RTL) designs, including SRAM, AES, and UART modules. We propose a novel tool for this goal that systematically assesses state-of-the-art LLMs (GPT-4o, Gemini 1.5 pro, and Llama 3.1) in detecting HTs without prior fine-tuning. To address potential training data bias, the tool implements perturbation techniques, i.e., variable name obfuscation, and design restructuring, that make the cases more sophisticated for the used LLMs. Our experimental evaluation demonstrates perfect detection rates by GPT-4o and Gemini 1.5 pro in baseline scenarios (100%/100% precision/recall), with both models achieving better trigger line coverage (TLC: 0.82-0.98) than payload line coverage (PLC: 0.32-0.46). Under code perturbation, while Gemini 1.5 pro maintains perfect detection performance (100%/100%), GPT-4o (100%/85.7%) and Llama 3.1 (66.7%/85.7%) show some degradation in detection rates, and all models experience decreased accuracy in localizing both triggers and payloads. This paper validates the potential of LLM approaches for hardware security applications, highlighting areas for future improvement.
Analytical-Heuristic Modeling and Optimization for Low-Light Image Enhancement
Venue: arXiv cs.CV 2024
Year: 2024

Low-light image enhancement remains an open problem, and the new wave of artificial intelligence is at the center of this problem. This work describes the use of genetic algorithms for optimizing analytical models that can improve the visualization of images with poor light. Genetic algorithms are part of metaheuristic approaches, which proved helpful in solving challenging optimization tasks. We propose two analytical methods combined with optimization reasoning to approach a solution to the physical and computational aspects of transforming dark images into visible ones. The experiments demonstrate that the proposed approach ranks at the top among 26 state-of-the-art algorithms in the LOL benchmark. The results show evidence that a simple genetic algorithm combined with analytical reasoning can defeat the current mainstream in a challenging computer vision task through controlled experiments and objective comparisons. This work opens interesting new research avenues for the swarm and evolutionary computation community and others interested in analytical and heuristic reasoning.
Granite Guardian
Venue: arXiv cs.CL 2024
Year: 2024

We introduce the Granite Guardian models, a suite of safeguards designed to provide risk detection for prompts and responses, enabling safe and responsible use in combination with any large language model (LLM). These models offer comprehensive coverage across multiple risk dimensions, including social bias, profanity, violence, sexual content, unethical behavior, jailbreaking, and hallucination-related risks such as context relevance, groundedness, and answer relevance for retrieval-augmented generation (RAG). Trained on a unique dataset combining human annotations from diverse sources and synthetic data, Granite Guardian models address risks typically overlooked by traditional risk detection models, such as jailbreaks and RAG-specific issues. With AUC scores of 0.871 and 0.854 on harmful content and RAG-hallucination-related benchmarks respectively, Granite Guardian is the most generalizable and competitive model available in the space. Released as open-source, Granite Guardian aims to promote responsible AI development across the community. https://github.com/ibm-granite/granite-guardian
StyleMaster: Stylize Your Video with Artistic Generation and Translation
Venue: arXiv cs.CV 2024
Year: 2024

Style control has been popular in video generation models. Existing methods often generate videos far from the given style, cause content leakage, and struggle to transfer one video to the desired style. Our first observation is that the style extraction stage matters, whereas existing methods emphasize global style but ignore local textures. In order to bring texture features while preventing content leakage, we filter content-related patches while retaining style ones based on prompt-patch similarity; for global style extraction, we generate a paired style dataset through model illusion to facilitate contrastive learning, which greatly enhances the absolute style consistency. Moreover, to fill in the image-to-video gap, we train a lightweight motion adapter on still videos, which implicitly enhances stylization extent, and enables our image-trained model to be seamlessly applied to videos. Benefited from these efforts, our approach, StyleMaster, not only achieves significant improvement in both style resemblance and temporal coherence, but also can easily generalize to video style transfer with a gray tile ControlNet. Extensive experiments and visualizations demonstrate that StyleMaster significantly outperforms competitors, effectively generating high-quality stylized videos that align with textual content and closely resemble the style of reference images. Our project page is at https://zixuan-ye.github.io/stylemaster
LoRA3D: Low-Rank Self-Calibration of 3D Geometric Foundation Models
Venue: arXiv cs.CV 2024
Year: 2024

Emerging 3D geometric foundation models, such as DUSt3R, offer a promising approach for in-the-wild 3D vision tasks. However, due to the high-dimensional nature of the problem space and scarcity of high-quality 3D data, these pre-trained models still struggle to generalize to many challenging circumstances, such as limited view overlap or low lighting. To address this, we propose LoRA3D, an efficient self-calibration pipeline to $\textit{specialize}$ the pre-trained models to target scenes using their own multi-view predictions. Taking sparse RGB images as input, we leverage robust optimization techniques to refine multi-view predictions and align them into a global coordinate frame. In particular, we incorporate prediction confidence into the geometric optimization process, automatically re-weighting the confidence to better reflect point estimation accuracy. We use the calibrated confidence to generate high-quality pseudo labels for the calibrating views and use low-rank adaptation (LoRA) to fine-tune the models on the pseudo-labeled data. Our method does not require any external priors or manual labels. It completes the self-calibration process on a $\textbf{single standard GPU within just 5 minutes}$. Each low-rank adapter requires only $\textbf{18MB}$ of storage. We evaluated our method on $\textbf{more than 160 scenes}$ from the Replica, TUM and Waymo Open datasets, achieving up to $\textbf{88% performance improvement}$ on 3D reconstruction, multi-view pose estimation and novel-view rendering.
On Motion Blur and Deblurring in Visual Place Recognition
Venue: arXiv cs.CV 2024
Year: 2024

Visual Place Recognition (VPR) in mobile robotics enables robots to localize themselves by recognizing previously visited locations using visual data. While the reliability of VPR methods has been extensively studied under conditions such as changes in illumination, season, weather and viewpoint, the impact of motion blur is relatively unexplored despite its relevance not only in rapid motion scenarios but also in low-light conditions where longer exposure times are necessary. Similarly, the role of image deblurring in enhancing VPR performance under motion blur has received limited attention so far. This paper bridges these gaps by introducing a new benchmark designed to evaluate VPR performance under the influence of motion blur and image deblurring. The benchmark includes three datasets that encompass a wide range of motion blur intensities, providing a comprehensive platform for analysis. Experimental results with several well-established VPR and image deblurring methods provide new insights into the effects of motion blur and the potential improvements achieved through deblurring. Building on these findings, the paper proposes adaptive deblurring strategies for VPR, designed to effectively manage motion blur in dynamic, real-world scenarios.
Can LLMs faithfully generate their layperson-understandable 'self'?: A Case Study in High-Stakes Domains
Venue: arXiv cs.HC 2024
Year: 2024

Large Language Models (LLMs) have significantly impacted nearly every domain of human knowledge. However, the explainability of these models esp. to laypersons, which are crucial for instilling trust, have been examined through various skeptical lenses. In this paper, we introduce a novel notion of LLM explainability to laypersons, termed $\textit{ReQuesting}$, across three high-priority application domains -- law, health and finance, using multiple state-of-the-art LLMs. The proposed notion exhibits faithful generation of explainable layman-understandable algorithms on multiple tasks through high degree of reproducibility. Furthermore, we observe a notable alignment of the explainable algorithms with intrinsic reasoning of the LLMs.
Digital Democracy in the Age of Artificial Intelligence
Venue: arXiv cs.CY 2024
Year: 2024

This chapter explores the influence of Artificial Intelligence (AI) on digital democracy, focusing on four main areas: citizenship, participation, representation, and the public sphere. It traces the evolution from electronic to virtual and network democracy, underscoring how each stage has broadened democratic engagement through technology. Focusing on digital citizenship, the chapter examines how AI can improve online engagement and promote ethical behaviour while posing privacy risks and fostering identity stereotyping. Regarding political participation, it highlights AI's dual role in mobilising civic actions and spreading misinformation. Regarding representation, AI's involvement in electoral processes can enhance voter registration, e-voting, and the efficiency of result tabulation but raises concerns regarding privacy and public trust. Also, AI's predictive capabilities shift the dynamics of political competition, posing ethical questions about manipulation and the legitimacy of democracy. Finally, the chapter examines how integrating AI and digital technologies can facilitate democratic political advocacy and personalised communication. However, this also comes with higher risks of misinformation and targeted propaganda.
SALON: Self-supervised Adaptive Learning for Off-road Navigation
Venue: arXiv cs.RO 2024
Year: 2024

Autonomous robot navigation in off-road environments presents a number of challenges due to its lack of structure, making it difficult to handcraft robust heuristics for diverse scenarios. While learned methods using hand labels or self-supervised data improve generalizability, they often require a tremendous amount of data and can be vulnerable to domain shifts. To improve generalization in novel environments, recent works have incorporated adaptation and self-supervision to develop autonomous systems that can learn from their own experiences online. However, current works often rely on significant prior data, for example minutes of human teleoperation data for each terrain type, which is difficult to scale with more environments and robots. To address these limitations, we propose SALON, a perception-action framework for fast adaptation of traversability estimates with minimal human input. SALON rapidly learns online from experience while avoiding out of distribution terrains to produce adaptive and risk-aware cost and speed maps. Within seconds of collected experience, our results demonstrate comparable navigation performance over kilometer-scale courses in diverse off-road terrain as methods trained on 100-1000x more data. We additionally show promising results on significantly different robots in different environments. Our code is available at https://theairlab.org/SALON.
RUMC: A Rule-based Classifier Inspired by Evolutionary Methods
Venue: arXiv cs.LG 2024
Year: 2024

As the field of data analysis grows rapidly due to the large amounts of data being generated, effective data classification has become increasingly important. This paper introduces the RUle Mutation Classifier (RUMC), which represents a significant improvement over the Rule Aggregation ClassifiER (RACER). RUMC uses innovative rule mutation techniques based on evolutionary methods to improve classification accuracy. In tests with forty datasets from OpenML and the UCI Machine Learning Repository, RUMC consistently outperformed twenty other well-known classifiers, demonstrating its ability to uncover valuable insights from complex data.
Rethinking Emotion Annotations in the Era of Large Language Models
Venue: arXiv cs.CL 2024
Year: 2024

Modern affective computing systems rely heavily on datasets with human-annotated emotion labels, for training and evaluation. However, human annotations are expensive to obtain, sensitive to study design, and difficult to quality control, because of the subjective nature of emotions. Meanwhile, Large Language Models (LLMs) have shown remarkable performance on many Natural Language Understanding tasks, emerging as a promising tool for text annotation. In this work, we analyze the complexities of emotion annotation in the context of LLMs, focusing on GPT-4 as a leading model. In our experiments, GPT-4 achieves high ratings in a human evaluation study, painting a more positive picture than previous work, in which human labels served as the only ground truth. On the other hand, we observe differences between human and GPT-4 emotion perception, underscoring the importance of human input in annotation studies. To harness GPT-4's strength while preserving human perspective, we explore two ways of integrating GPT-4 into emotion annotation pipelines, showing its potential to flag low-quality labels, reduce the workload of human annotators, and improve downstream model learning performance and efficiency. Together, our findings highlight opportunities for new emotion labeling practices and suggest the use of LLMs as a promising tool to aid human annotation.
How Can I Assist You Today?: A Comparative Analysis of a Humanoid Robot and a Virtual Human Avatar in Human Perception
Venue: arXiv cs.HC 2024
Year: 2024

This study explores human perceptions of intelligent agents by comparing interactions with a humanoid robot and a virtual human avatar, both utilizing GPT-3 for response generation. The study aims to understand how physical and virtual embodiments influence perceptions of anthropomorphism, animacy, likeability, and perceived intelligence. The uncanny valley effect was also investigated in the scope of this study based on the two agents' human-likeness and affinity. Conducted with ten participants from Sabanci University, the experiment involved tasks that sought advice, followed by assessments using the Godspeed Questionnaire Series and structured interviews. Results revealed no significant difference in anthropomorphism between the humanoid robot and the virtual human avatar, but the humanoid robot was perceived as more likable and slightly more intelligent, highlighting the importance of physical presence and interactive gestures. These findings suggest that while virtual avatars can achieve high human-likeness, physical embodiment enhances likeability and perceived intelligence. However, the study's scope was insufficient to claim the existence of the uncanny valley effect in the participants' interactions. The study offers practical insights for designing future intelligent assistants, emphasizing the need for integrating physical elements and sophisticated communicative behaviors to improve user experience and acceptance.
Robust Multiple Description Neural Video Codec with Masked Transformer for Dynamic and Noisy Networks
Venue: arXiv cs.CV 2024
Year: 2024

Multiple Description Coding (MDC) is a promising error-resilient source coding method that is particularly suitable for dynamic networks with multiple (yet noisy and unreliable) paths. However, conventional MDC video codecs suffer from cumbersome architectures, poor scalability, limited loss resilience, and lower compression efficiency. As a result, MDC has never been widely adopted. Inspired by the potential of neural video codecs, this paper rethinks MDC design. We propose a novel MDC video codec, NeuralMDC, demonstrating how bidirectional transformers trained for masked token prediction can vastly simplify the design of MDC video codec. To compress a video, NeuralMDC starts by tokenizing each frame into its latent representation and then splits the latent tokens to create multiple descriptions containing correlated information. Instead of using motion prediction and warping operations, NeuralMDC trains a bidirectional masked transformer to model the spatial-temporal dependencies of latent representations and predict the distribution of the current representation based on the past. The predicted distribution is used to independently entropy code each description and infer any potentially lost tokens. Extensive experiments demonstrate NeuralMDC achieves state-of-the-art loss resilience with minimal sacrifices in compression efficiency, significantly outperforming the best existing residual-coding-based error-resilient neural video codec.
SDPERL: A Framework for Software Defect Prediction Using Ensemble Feature Extraction and Reinforcement Learning
Venue: arXiv cs.SE 2024
Year: 2024

Ensuring software quality remains a critical challenge in complex and dynamic development environments, where software defects can result in significant operational and financial risks. This paper proposes an innovative framework for software defect prediction that combines ensemble feature extraction with reinforcement learning (RL)--based feature selection. We claim that this work is among the first in recent efforts to address this challenge at the file-level granularity. The framework extracts diverse semantic and structural features from source code using five code-specific pre-trained models. Feature selection is enhanced through a custom-defined embedding space tailored to represent feature interactions, coupled with a pheromone table mechanism inspired by Ant Colony Optimization (ACO) to guide the RL agent effectively. Using the Proximal Policy Optimization (PPO) algorithm, the proposed method dynamically identifies the most predictive features for defect detection. Experimental evaluations conducted on the PROMISE dataset highlight the framework's superior performance on the F1-Score metric, achieving an average improvement of $6.25\%$ over traditional methods and baseline models across diverse datasets. This study underscores the potential for integrating ensemble learning and RL for adaptive and scalable defect prediction in modern software systems.
Machines of Meaning
Venue: arXiv cs.AI 2024
Year: 2024

One goal of Artificial Intelligence is to learn meaningful representations for natural language expressions, but what this entails is not always clear. A variety of new linguistic behaviours present themselves embodied as computers, enhanced humans, and collectives with various kinds of integration and communication. But to measure and understand the behaviours generated by such systems, we must clarify the language we use to talk about them. Computational models are often confused with the phenomena they try to model and shallow metaphors are used as justifications for (or to hype) the success of computational techniques on many tasks related to natural language; thus implying their progress toward human-level machine intelligence without ever clarifying what that means. This paper discusses the challenges in the specification of "machines of meaning", machines capable of acquiring meaningful semantics from natural language in order to achieve their goals. We characterize "meaning" in a computational setting, while highlighting the need for detachment from anthropocentrism in the study of the behaviour of machines of meaning. The pressing need to analyse AI risks and ethics requires a proper measurement of its capabilities which cannot be productively studied and explained while using ambiguous language. We propose a view of "meaning" to facilitate the discourse around approaches such as neural language models and help broaden the research perspectives for technology that facilitates dialogues between humans and machines.