pwc-archive/papers-with-abstracts huggingface dataset:
Open6DOR: Benchmarking Open-instruction 6-DoF Object Rearrangement and A VLM-based Approach
Authors: Yufei Ding, Haoran Geng, Chaoyi Xu, Xiaomeng Fang, Jiazhao Zhang, Songlin Wei, Qiyu Dai, Zhizheng Zhang, He Wang
Date: 2024-10-24 00:00:00
Proceeding: IROS2024 2024 10
Tasks: Benchmarking, Instruction Following, Object Rearrangement

In this work, we propel the pioneer construction of the benchmark and approach for table-top Open-instruction 6-DoF Object Rearrangement (Open6DOR). Specifically, we collect a synthetic dataset of 200+ objects and carefully design 2400+ Open6DOR tasks. These tasks are divided into the Position-track, Rotation-track, and 6-DoF-track for evaluating different embodied agents in predicting the positions and rotations of target objects. Besides, we also propose a VLM-based approach for Open6DOR, named Open6DOR-GPT, which empowers GPT-4V with 3D-awareness and simulation-assistance while exploiting its strengths in generalizability and instruction-following for this task. We compare the existing embodied agents with our Open6DOR-GPT on the proposed Open6DOR benchmark and find that Open6DOR-GPT achieves the state-of-the-art performance. We further show the impressive performance of Open6DOR-GPT in diverse real-world experiments. We plan to release the final version of the benchmark, along with our refined method, in early September, and we recommend waiting until then to download the dataset.
KnowZRel: Common Sense Knowledge-based Zero-Shot Relationship Retrieval for Generalised Scene Graph Generation
Authors: M. Jaleed Khan, John G. Breslin, Edward Curry
Date: 2025-02-21 00:00:00
Proceeding: IEEE Transactions on Artificial Intelligence 2025 2
Tasks: Common Sense Reasoning, Graph Generation, object-detection, Object Detection, Retrieval, Scene Graph Generation, Visual Reasoning, Zero-Shot Scene Graph Generation

A scene graph is a key image representation in visual reasoning. The generalisability of Scene Graph Generation (SGG) methods is crucial for reliable reasoning and real-world applicability. However, imbalanced training datasets limit this, underrepresenting meaningful visual relationships. Current SGG methods using external knowledge sources face limitations due to these imbalances or restricted relationship coverage, impacting their reasoning and generalisation capabilities. We propose a novel neurosymbolic approach that integrates data-driven object detection with heterogeneous knowledge graph-based object refinement and zero-shot relationship retrieval, highlighting the loosely coupled synergy between neural and symbolic components. This combination addresses the limitations of imbalanced training datasets in scene graph generation and enables effective prediction of unseen visual relationships. Objects are detected using a region-based deep neural network and refined based on their positional and structural similarity, followed by retrieval of pairwise visual relationships using a heterogeneous knowledge graph. The redundant and irrelevant visual relationships are discarded based on the similarity of relationship labels and node embeddings. Finally, the visual relationships are interlinked to generate the scene graph. The employed heterogeneous knowledge graph combines diverse knowledge sources, offering rich common sense knowledge about objects and their interactions in the world. Our method, evaluated using the benchmark Visual Genome dataset and zero-shot recall (zR@K) metric, shows a 59.96% improvement over existing state-of-the-art methods, highlighting its effectiveness in generalised SGG. The object refinement step effectively improved the object detection performance by 57.1%. Additional evaluation using the GQA dataset confirms the cross-dataset generalisability of our method. We also compared various knowledge sources and embedding models to determine an optimal combination for zero-shot SGG. The source code is available at https://github.com/jaleedkhan/zsrr-sgg.
R1-Onevision：An Open-Source Multimodal Large Language Model Capable of Deep Reasoning
Authors: Yi Yang*, Xiaoxuan He*, Hongkun Pan*, Xiyan Jiang, Yan Deng, Xingtao Yang, Haoyu Lu, Minfeng Zhu†, Bo Zhang†, Wei Chen†
Date: 2025-02-24 00:00:00
Proceeding: ongoing 2025 2
Tasks: Language Modeling, Language Modelling, Large Language Model, Logical Reasoning, Multimodal Large Language Model, Multimodal Reasoning, Visual Reasoning

R1-OneVision is a versatile multimodal reasoning large model, designed to tackle complex visual reasoning tasks. It seamlessly integrates visual and textual data to offer precise interpretations of multimodal information, excelling in areas such as mathematics, science, deep image understanding, and logical reasoning. With its robust ability to perform multimodal reasoning, R1-OneVision emerges as a powerful AI assistant capable of addressing a wide range of problem-solving challenges across different domains.
SafeInt: Shielding Large Language Models from Jailbreak Attacks via Safety-Aware Representation Intervention
Authors: Jiaqi Wu, Chen Chen, Chunyan Hou, Xiaojie Yuan
Date: 2025-02-21 00:00:00

With the widespread real-world deployment of large language models (LLMs), ensuring their behavior complies with safety standards has become crucial. Jailbreak attacks exploit vulnerabilities in LLMs to induce undesirable behavior, posing a significant threat to LLM safety. Previous defenses often fail to achieve both effectiveness and efficiency simultaneously. Defenses from a representation perspective offer new insights, but existing interventions cannot dynamically adjust representations based on the harmfulness of the queries. To address this limitation while ensuring both effectiveness and efficiency, we propose SafeIntervention (SafeInt), a novel defense method that shields LLMs from jailbreak attacks through safety-aware representation intervention. SafeInt is built on our analysis of the representations of jailbreak samples. It adjusts representation distributions of jailbreak samples through intervention to align them with the representations of unsafe samples while minimizing unnecessary perturbations to jailbreak-irrelevant representations. We conduct comprehensive experiments covering six jailbreak attacks, two jailbreak datasets, and two utility benchmarks. Experimental results demonstrate that SafeInt outperforms all baselines in defending LLMs against jailbreak attacks while largely maintaining utility. Additionally, we evaluate SafeInt against adaptive attacks and verify its effectiveness in mitigating real-time attacks.
An ocean front detection and tracking algorithm
Authors: Yishuo Wang, Feng Zhou, Qicheng Meng, Muping Zhou, Zhijun Hu, Chengqing Zhang, Tianhao Zhao
Date: 2025-02-21 00:00:00
Tasks: Edge Detection

Existing ocean front detection methods--including histogram-based variance analysis, Lyapunov exponent, gradient thresholding, and machine learning--suffer from critical limitations: discontinuous outputs, over-detection, reliance on single-threshold decisions, and lack of open-source implementations. To address these challenges, this paper proposes the Bayesian Front Detection and Tracking framework with Metric Space Analysis (BFDT-MSA). The framework introduces three innovations: (1) a Bayesian decision mechanism that integrates gradient priors and field operators to eliminate manual threshold sensitivity; (2) morphological refinement algorithms for merging fragmented fronts, deleting spurious rings, and thinning frontal zones to pixel-level accuracy; and (3) a novel metric space definition for temporal front tracking, enabling systematic analysis of front evolution. Validated on global SST data (2022--2024), BFDT-MSA reduces over-detection by $73\%$ compared to histogram-based methods while achieving superior intensity ($0.16^\circ$C/km), continuity, and spatiotemporal coherence. The open-source release bridges a critical gap in reproducible oceanographic research.
VisCon-100K: Leveraging Contextual Web Data for Fine-tuning Vision Language Models
Authors: Gokul Karthik Kumar, Iheb Chaabane, Kebin Wu
Date: 2025-02-14 00:00:00
Tasks: Image Captioning, Large Language Model, Multiple-choice, Visual Question Answering (VQA)

Vision-language models (VLMs) excel in various visual benchmarks but are often constrained by the lack of high-quality visual fine-tuning data. To address this challenge, we introduce VisCon-100K, a novel dataset derived from interleaved image-text web documents. Our approach transforms 45K web documents from the OBELICS dataset into 100K image conversation samples. We utilize GPT-4V to generate image-contextual captions and OpenChat 3.5 model to convert these captions into diverse free-form and multiple-choice question-answer pairs. Integrating this dataset for fine-tuning considerably enhances VLM performance across multiple benchmarks. Unlike methods that focus solely on fine-grained visual content, our approach leverages accompanying web context, yielding superior results. We also discover that a `leaky modality mix,' where conversation samples contain questions answerable from both the image and its contextual caption, outperforms non-leaky combinations of captions and Q\&A pairs. VisCon-100k dataset shows strong performance with two popular VLM approaches: text-only large language model (LLM) aligned with a vision encoder using image captions data (ShareGPT4V-7b) and multimodally pretrained LLM (IDEFICS2-8b) using interleaved image-text data. In addition to releasing the VisCon-100K dataset, we provide a contextual captioner trained on this dataset, facilitating scalable fine-tuning data generation for future research and open-source applications. Using the same pipeline, but substituting our trained contextual captioner for GPT-4V, we also release the larger VisCon-1M dataset.
VaViM and VaVAM: Autonomous Driving through Video Generative Modeling
Authors: Florent Bartoccioni, Elias Ramzi, Victor Besnier, Shashanka Venkataramanan, Tuan-Hung Vu, Yihong Xu, Loick Chambon, Spyros Gidaris, Serkan Odabas, David Hurych, Renaud Marlet, Alexandre Boulch, Mickael Chen, Éloi Zablocki, Andrei Bursuc, Eduardo Valle, Matthieu Cord
Date: 2025-02-21 00:00:00
Tasks: Autonomous Driving, Imitation Learning

We explore the potential of large-scale generative video models for autonomous driving, introducing an open-source auto-regressive video model (VaViM) and its companion video-action model (VaVAM) to investigate how video pre-training transfers to real-world driving. VaViM is a simple auto-regressive video model that predicts frames using spatio-temporal token sequences. We show that it captures the semantics and dynamics of driving scenes. VaVAM, the video-action model, leverages the learned representations of VaViM to generate driving trajectories through imitation learning. Together, the models form a complete perception-to-action pipeline. We evaluate our models in open- and closed-loop driving scenarios, revealing that video-based pre-training holds promise for autonomous driving. Key insights include the semantic richness of the learned representations, the benefits of scaling for video synthesis, and the complex relationship between model size, data, and safety metrics in closed-loop evaluations. We release code and model weights at https://github.com/valeoai/VideoActionModel
Pub-Guard-LLM: Detecting Fraudulent Biomedical Articles with Reliable Explanations
Authors: Lihu Chen, Shuojie Fu, Gabriel Freedman, Cemre Zor, Guy Martin, James Kinross, Uddhav Vaghela, Ovidiu Serban, Francesca Toni
Date: 2025-02-21 00:00:00
Tasks: Articles, Fraud Detection, Language Modeling, Language Modelling, Large Language Model, Retrieval-augmented Generation

A significant and growing number of published scientific articles is found to involve fraudulent practices, posing a serious threat to the credibility and safety of research in fields such as medicine. We propose Pub-Guard-LLM, the first large language model-based system tailored to fraud detection of biomedical scientific articles. We provide three application modes for deploying Pub-Guard-LLM: vanilla reasoning, retrieval-augmented generation, and multi-agent debate. Each mode allows for textual explanations of predictions. To assess the performance of our system, we introduce an open-source benchmark, PubMed Retraction, comprising over 11K real-world biomedical articles, including metadata and retraction labels. We show that, across all modes, Pub-Guard-LLM consistently surpasses the performance of various baselines and provides more reliable explanations, namely explanations which are deemed more relevant and coherent than those generated by the baselines when evaluated by multiple assessment methods. By enhancing both detection performance and explainability in scientific fraud detection, Pub-Guard-LLM contributes to safeguarding research integrity with a novel, effective, open-source tool.
The Role of Background Information in Reducing Object Hallucination in Vision-Language Models: Insights from Cutoff API Prompting
Authors: Masayo Tomita, Katsuhiko Hayashi, Tomoyuki Kaneko
Date: 2025-02-21 00:00:00
Tasks: Hallucination, Object, Object Hallucination, Visual Prompting

Vision-Language Models (VLMs) occasionally generate outputs that contradict input images, constraining their reliability in real-world applications. While visual prompting is reported to suppress hallucinations by augmenting prompts with relevant area inside an image, the effectiveness in terms of the area remains uncertain. This study analyzes success and failure cases of Attention-driven visual prompting in object hallucination, revealing that preserving background context is crucial for mitigating object hallucination.
DEFT: Differentiable Branched Discrete Elastic Rods for Modeling Furcated DLOs in Real-Time
Authors: Yizhou Chen, Xiaoyue Wu, Yeheng Zong, Yuzhen Chen, Anran Li, Bohao Zhang, Ram Vasudevan
Date: 2025-02-20 00:00:00

Autonomous wire harness assembly requires robots to manipulate complex branched cables with high precision and reliability. A key challenge in automating this process is predicting how these flexible and branched structures behave under manipulation. Without accurate predictions, it is difficult for robots to reliably plan or execute assembly operations. While existing research has made progress in modeling single-threaded Deformable Linear Objects (DLOs), extending these approaches to Branched Deformable Linear Objects (BDLOs) presents fundamental challenges. The junction points in BDLOs create complex force interactions and strain propagation patterns that cannot be adequately captured by simply connecting multiple single-DLO models. To address these challenges, this paper presents Differentiable discrete branched Elastic rods for modeling Furcated DLOs in real-Time (DEFT), a novel framework that combines a differentiable physics-based model with a learning framework to: 1) accurately model BDLO dynamics, including dynamic propagation at junction points and grasping in the middle of a BDLO, 2) achieve efficient computation for real-time inference, and 3) enable planning to demonstrate dexterous BDLO manipulation. A comprehensive series of real-world experiments demonstrates DEFT's efficacy in terms of accuracy, computational speed, and generalizability compared to state-of-the-art alternatives. Project page:https://roahmlab.github.io/DEFT/.
Auxiliary-Variable Adaptive Control Barrier Functions
Authors: Shuo Liu, Wei Xiao, Calin A. Belta
Date: 2025-02-20 00:00:00

This paper addresses the challenge of ensuring safety and feasibility in control systems using Control Barrier Functions (CBFs). Existing CBF-based Quadratic Programs (CBF-QPs) often encounter feasibility issues due to mixed relative degree constraints, input nullification problems, and the presence of tight or time-varying control bounds, which can lead to infeasible solutions and compromised safety. To address these challenges, we propose Auxiliary-Variable Adaptive Control Barrier Functions (AVCBFs), a novel framework that introduces auxiliary variables in auxiliary functions to dynamically adjust CBF constraints without the need of excessive additional constraints. The AVCBF method ensures that all components of the control input explicitly appear in the desired-order safety constraint, thereby improving feasibility while maintaining safety guarantees. Additionally, we introduce an automatic tuning method that iteratively adjusts AVCBF hyperparameters to ensure feasibility and safety with less conservatism. We demonstrate the effectiveness of the proposed approach in adaptive cruise control and obstacle avoidance scenarios, showing that AVCBFs outperform existing CBF methods by reducing infeasibility and enhancing adaptive safety control under tight or time-varying control bounds.
Synchronization of oscillators through time-shifted common inputs
Authors: Ehsan Bolhasani, Yousef Azizi, Alireza Valizadeh, Matjaz Perc
Date: 2016-09-27 00:00:00

Shared upstream dynamical processes are frequently the source of common inputs in various physical and biological systems. However, due to finite signal transmission speeds and differences in the distance to the source, time shifts between otherwise common inputs are unavoidable. Since common inputs can be a source of correlation between the elements of multi-unit dynamical systems, regardless of whether these elements are directly connected with one another or not, it is of importance to understand their impact on synchronization. As a canonical model that is representative for a variety of different dynamical systems, we study limit-cycle oscillators that are driven by stochastic time-shifted common inputs. We show that if the oscillators are coupled, time shifts in stochastic common inputs do not simply shift the distribution of the phase differences, but rather the distribution actually changes as a result. The best synchronization is therefore achieved at a precise intermediate value of the time shift, which is due to a resonance-like effect with the most probable phase difference that is determined by the deterministic dynamics.
Estimating Neural Representation Alignment from Sparsely Sampled Inputs and Features
Authors: Chanwoo Chun, Abdulkadir Canatar, SueYeon Chung, Daniel D. Lee
Date: 2025-02-20 00:00:00

In both artificial and biological systems, the centered kernel alignment (CKA) has become a widely used tool for quantifying neural representation similarity. While current CKA estimators typically correct for the effects of finite stimuli sampling, the effects of sampling a subset of neurons are overlooked, introducing notable bias in standard experimental scenarios. Here, we provide a theoretical analysis showing how this bias is affected by the representation geometry. We then introduce a novel estimator that corrects for both input and feature sampling. We use our method for evaluating both brain-to-brain and model-to-brain alignments and show that it delivers reliable comparisons even with very sparsely sampled neurons. We perform within-animal and across-animal comparisons on electrophysiological data from visual cortical areas V1, V4, and IT data, and use these as benchmarks to evaluate model-to-brain alignment. We also apply our method to reveal how object representations become progressively disentangled across layers in both biological and artificial systems. These findings underscore the importance of correcting feature-sampling biases in CKA and demonstrate that our bias-corrected estimator provides a more faithful measure of representation alignment. The improved estimates increase our understanding of how neural activity is structured across both biological and artificial systems.
FormalSpecCpp: A Dataset of C++ Formal Specifications created using LLMs
Authors: Madhurima Chakraborty, Peter Pirkelbauer, Qing Yi
Date: 2025-02-21 00:00:00

FormalSpecCpp is a dataset designed to fill the gap in standardized benchmarks for verifying formal specifications in C++ programs. To the best of our knowledge, this is the first comprehensive collection of C++ programs with well-defined preconditions and postconditions. It provides a structured benchmark for evaluating specification inference tools and testing theaccuracy of generated specifications. Researchers and developers can use this dataset to benchmark specification inference tools,fine-tune Large Language Models (LLMs) for automated specification generation, and analyze the role of formal specifications in improving program verification and automated testing. By making this dataset publicly available, we aim to advance research in program verification, specification inference, and AI-assisted software development. The dataset and the code are available at https://github.com/MadhuNimmo/FormalSpecCpp.
Drug-Target Interaction/Affinity Prediction: Deep Learning Models and Advances Review
Authors: Ali Vefghi, Zahed Rahmati, Mohammad Akbari
Date: 2025-02-21 00:00:00
Tasks: Drug Discovery, Prediction

Drug discovery remains a slow and expensive process that involves many steps, from detecting the target structure to obtaining approval from the Food and Drug Administration (FDA), and is often riddled with safety concerns. Accurate prediction of how drugs interact with their targets and the development of new drugs by using better methods and technologies have immense potential to speed up this process, ultimately leading to faster delivery of life-saving medications. Traditional methods used for drug-target interaction prediction show limitations, particularly in capturing complex relationships between drugs and their targets. As an outcome, deep learning models have been presented to overcome the challenges of interaction prediction through their precise and efficient end results. By outlining promising research avenues and models, each with a different solution but similar to the problem, this paper aims to give researchers a better idea of methods for even more accurate and efficient prediction of drug-target interaction, ultimately accelerating the development of more effective drugs. A total of 180 prediction methods for drug-target interactions were analyzed throughout the period spanning 2016 to 2025 using different frameworks based on machine learning, mainly deep learning and graph neural networks. Additionally, this paper discusses the novelty, architecture, and input representation of these models.
Intermediate triple table: A general architecture for virtual knowledge graphs
Authors: Julián Arenas-Guerrero, Oscar Corcho, María S. Pérez
Date: 2025-02-24 00:00:00
Proceeding: Knowledge-Based Systems 2025 2
Tasks: Data Integration, Knowledge Graphs

Virtual knowledge graphs (VKGs) have been widely applied to access relational data with a semantic layer by using an ontology in use cases that are dynamic in nature. However, current VKG techniques focus mainly on accessing a single relational database and remain largely unstudied for data integration with several heterogeneous data sources. To overcome this limitation, we propose intermediate triple table (ITT), a general VKG architecture to access multiple and diverse data sources. Our proposal is based on data shipping and addresses heterogeneity by adopting a schema-oblivious graph representation that intervenes between the sources and the queries. We minimize data computation by just materializing a relevant subgraph for a specific query. We employ star-shaped query processing and extend this technique to mapping candidate selection. For rapid materialization of the ITT, we apply a mapping partitioning technique to parallelize mapping execution, which also guarantees duplicate-free subgraphs and reduces memory consumption. We use SPARQL-to-SQL query translation to homogeneously evaluate queries over the ITT and execute them with an in-process analytical store. We implemented ITT on top of a knowledge graph materialization engine and evaluated it with two VKG benchmarks. The experimental results show that our proposal outperforms state-of-the-art techniques for complex graph queries in terms of execution time. It also decreases the number of timeouts although it uses more memory as a trade-off. The experiments also demonstrate the source independence of the architecture on a mixed distribution of data with SQL and document stores together with various file formats.