GenAI4ELab/papercli-papers huggingface dataset:
Auto-BenchmarkCard: Automated Synthesis of Benchmark Documentation
Authors: Aris Hofmann; Inge Vejsbjerg; Dhaval Salwala; Elizabeth M. Daly
Venue: AAAI 2026 (main)

Abstract: We present Auto-BenchmarkCard, a workflow for generating validated descriptions of AI benchmarks. Benchmark documentation is often incomplete or inconsistent, making it difficult to interpret and compare benchmarks across tasks or domains. Auto-BenchmarkCard addresses this gap by combining multi-agent data extraction from heterogeneous sources (e.g., Hugging Face, Unitxt, academic papers) with LLM-driven synthesis. A validation phase evaluates factual accuracy through atomic entailment scoring using the FactReasoner tool. This workflow has the potential to promote transparency, comparability, and reusability in AI benchmark reporting, enabling researchers and practitioners to better navigate and evaluate benchmark choices.
RAPID: A Rapid Prototyping Platform for Industrial Automation
Authors: Sunghoon Hong; Junseok Park; Whiyoung Jung; Deunsol Yoon; Woohyung Lim; Soonyoung Lee; Kanghoon Lee
Venue: AAAI 2026 (main)

Abstract: Industrial automation in smart logistics and factories requires simulation platforms that support rapid environment building before costly physical deployment. Yet existing tools often require substantial expertise, complex setup, and long configuration times, hindering agile prototyping. We present RAPID, a simulation platform with two components: layout design, which enables intuitive visual configuration of factory layouts, and behavior simulation and validation, which allows users to attach behavior models and evaluate system performance. RAPID lowers the entry barrier to industrial simulation, letting users apply existing behavior models or trained reinforcement learning (RL) agents to new layouts with minimal effort. This approach lets practitioners prototype facilities in minutes rather than weeks and gives researchers a standardized environment for benchmarking multi-agent RL and coordination algorithms. By combining rapid design with simulation-based validation, RAPID accelerates automation development from concept to implementation.
GPTKB v1.5: A Massive Knowledge Base for Exploring Factual LLM Knowledge
Authors: Yujia Hu; Tuan-Phong Nguyen; Shrestha Ghosh; Moritz Müller; Simon Razniewski
Venue: AAAI 2026 (main)

Abstract: Language models are powerful artifacts, yet their factual knowledge is still poorly understood, and inaccessible to ad-hoc browsing and scalable statistical analysis. This demonstration introduces GPTKB v1.5, a densely interlinked 100-million-triple knowledge base (KB) built for $14,000 from GPT-4.1, using the GPTKB methodology for massive-recursive LLM knowledge materialization. This demo focuses on three use cases: (1) link-traversal-based LLM knowledge exploration, (2) SPARQL-based structured LLM knowledge querying, (3) comparative exploration of the strengths and weaknesses of LLM knowledge. Massive-recursive LLM knowledge materialization is a groundbreaking opportunity both for the systematic analysis of LLM knowledge, as well as for automated KB construction.
Evaluating the Factuality of Large Language Models Using Multiple Plug-and-Play Fact Sources
Authors: Zhaoheng Huang; Yutao Zhu; Jirong Wen; Zhicheng Dou
Venue: AAAI 2026 (main)

Abstract: Large language models (LLMs) often produce factually inaccurate content, or hallucinations, which undermines their reliability. Existing factuality evaluation systems usually rely on a single predefined fact source, making them task-specific and hard to extend. We present UFO, a unified framework for factuality evaluation that supports multiple plug-and-play fact sources. UFO integrates human-written evidence, web search results, and LLM knowledge within a single evaluation pipeline, and allows users to flexibly select, reorder, and even define customized sources. The system is accessible through both a Python interface and a web-based demo, offering interactive claim-level verification and visualization. Experiments show that UFO system achieves moderate consistency with human annotations. Overall, UFO serves as a transparent and extensible platform for benchmarking fact sources, comparing LLMs, and enabling real-world fact-checking applications across diverse domains.
AirNavigation: Let UAV Navigation Tell Its Own Story
Authors: Jianyu Jiang; Zequan Wang; Liang Yao; Shengxiang Xu; Fan Liu
Venue: AAAI 2026 (main)

Abstract: Testing autonomous navigation algorithms of Unmanned Aerial Vehicles (UAVs) in real-world scenarios often entails significant safety risks. In this paper, we aim to build a flexible yet user-friendly UAV autonomous navigation simulator. Ideally, it should closely emulate real-world environments, support diverse UAV models and algorithms, and provide a flexible evaluation framework. Existing frameworks fail to satisfy all three requirements simultaneously. To this end, we present AirNavigation, an integrated simulation platform designed to support the end-to-end workflow of UAV navigation research. Specifically, our system leverages Unreal Engine to simulate highly realistic environments and diverse UAV models. It further facilitates semi-automated scene generation and multi-modal synthetic training data production. To lower the barrier of adoption, we develop a suite of user-friendly interfaces to enable seamless integration of diverse navigation algorithms. Moreover, we introduce a novel evaluation system powered by large language models to deliver personalized and fine-grained performance analysis.
GeoProblem Factory: A Visual Interaction System for Solvable and Controllable Geometric Problem Generation by Leveraging Symbolic Deduction Engine
Authors: Zhuoxuan Jiang; Yanpeng Li; Tianyang Zhang; Jing Chen; Yong Li; Mo Guang; Wen Si; Shaohua Zhang
Venue: AAAI 2026 (main)

Abstract: We propose a novel system, GeoProblem Factory, designed to effectively generate high-quality geometry problems for intelligent education. The system enables to efficiently produce batches of geometry problems for teachers and students, either to save time and manual effort or to support personalized learning. Generating geometry problems is particularly challenging, as it requires ensuring both solvability and controllability from a pedagogical perspective. To address these issues, we adopt a state-of-the-art pipeline method based on a symbolic deduction engine and develop a visual interaction demo. This demo allows users to easily refine the generated problems through visual operations. It provides two modes for inputting controllable information: specifying knowledge points or supplying a reference problem. Moreover, the system can automatically generate a preliminary geometric diagram corresponding to each problem for further refinement. Through human–machine interaction, the system can more efficiently produce high-quality geometry problems than ever.
RL-Studio: A System for Multi-Phase Reinforcement Learning Experimentation
Authors: Whiyoung Jung; Sunghoon Hong; Deunsol Yoon; Jeonghye Kim; Yongjae Shin; Suhyun Jung; Hyundam Yoo; Youngjin Kim; Chanwoo Moon; Woohyung Lim; Soonyoung Lee; Kanghoon Lee
Venue: AAAI 2026 (main)

Abstract: Reinforcement learning (RL) has evolved beyond monolithic training, yet existing frameworks remain limited to single algorithms or simple offline-to-online transitions. We present multi-phase RL, a framework that orchestrates multiple learning phases for continual policy improvement. It enables efficient fine-tuning of pretrained policies with new data and smooth adaptation from simulation to real-world environments. To support this paradigm, we introduce RL-Studio, a platform that addresses key implementation barriers, including neural architecture mismatches, parameter transfer complexities, and experiment management overhead. It provides phase orchestration, transition-point monitoring, and full experiment lineage tracking. We demonstrate the effectiveness of multi-phase RL through representative scenarios and highlight RL-Studio’s capabilities.
AEGIS: Toward Expert-in-the-loop Industrial Anomaly Detection
Authors: Dongmin Kim; Ye Seul Sim; Suhee Yoon; Sanghyu Yoon; Seungdong Yoa; Soonyoung Lee; Woohyung Lim
Venue: AAAI 2026 (main)

Abstract: Anomaly detection platforms in real-world environments require continuous interaction between automated systems and domain experts, as anomalies evolve dynamically and their definitions vary across contexts.
Therefore, an effective platform must collaborate with experts and incorporate their feedback to update the system.
This paper introduces AEGIS, an anomaly detection platform that aims to support interaction between domain experts and data-driven agents through three core capabilities:
(1) data-driven insights through real-time monitoring, explanations, and distribution shift detection, which invoke customized tools to generate appropriate responses,
(2) an expert feedback interface for labeling and direct updates via chat-based interaction, and
(3) autonomous model construction that leverages expert-labeled data with LLM-driven hyperparameter optimization.
Through this design, AEGIS fosters continuous interaction in which the platform provides insights while experts guide model improvement, ensuring user intent is reflected and robustness is maintained under evolving data distributions.
GAICo: Demonstrating a Unified Framework for Multi-Modal GenAI Evaluation
Authors: Pallav Koppisetti; Nitin Gupta; Kausik Lakkaraju; Biplav Srivastava
Venue: AAAI 2026 (main)

Abstract: The rapid evolution of Generative AI, yielding outputs across text, structured data, images, and audio, has outpaced the development of standardized evaluation tools, leading to fragmented and non-reproducible practices. GAICo (Generative AI Comparator) offers a solution: a deployed, open-source Python library that provides a unified, extensible, and reproducible framework for multi-modal GenAI evaluation. Our demonstration highlights GAICo’s utility through a practical case study: evaluating and debugging composite AI Travel Assistant pipelines. We show how GAICo facilitates isolating performance issues, for instance, distinguishing orchestrator LLM planning deficiencies from specialist image model generation flaws, by consistently comparing diverse outputs against tailored references. This framework streamlines development, improves system reliability, and promotes reproducible evaluation, making it a critical tool for building safer and more effective AI. Its rapid adoption, evidenced by over 16,000 downloads in the first 6 months, underscores its relevance and impact within the AI community.
RefLens: End-to-End Evidence-Grounded Citation Verification with LLM Agents
Authors: SeungHoo Lee; JuneHyoung Kwon; Jooweon Choi; JungMin Yun; Seunguk Yu; Yoonji Lee; Jinhee Jang; YoungBin Kim
Venue: AAAI 2026 (main)

Abstract: Accurate citation is critical, yet error rates remain high across scientific literature. We present RefLens, an end-to-end system that automates citation verification from PDF parsing to interactive report generation. Unlike summary- or embedding-based approaches, RefLens performs evidence-grounded verification by extracting verbatim spans from original sources and displaying citation-level cards and a paper-level dashboard. In a 35-participant study, users rated value (M=4.34), trust (M=4.15), and usability (M=4.19) highly, with strong adoption intention (M=4.28).
Risk-Aware Bilingual Spoken Dialogue for Campus Mental Health Support
Authors: You-Teng Lin; Li-Yang Zhang; Yi-Tang Chen; Jen-Tzung Chien
Venue: AAAI 2026 (main)

Abstract: This work presented a web-based system which introduces an active-listening strategy in a spoken dialogue for self-disclosure to support mental health of a campus user. To enhance the system usability and safety, this demo is developed to conduct the bilingual (Mandarin/English) spoken dialogue where a high-risk dialogue detection during speech interaction is reliably augmented. In particular, a prompt-driven GPT classifier identifies the utterances indicating self-harm or suicide intent and triggers safety alerts with help center and counselor notification. We also integrate a TTS module for Taiwanese Mandarin and standard English, and redesign the user interface to automatically pop up alert messages when high-risk dialogue is detected. In addition, we collect speech data under diverse mental dialogue scenarios with bilingual speech to enable system analysis, evaluation and refinement. Overall, these extensions build a framework that promotes empathetic interactions, enables timely alert in critical cases, and improves the accessibility for diverse users.
DS SERVE: A Framework for Efficient and Scalable Neural Retrieval
Authors: Jinjian Liu; Yichuan Wang; Xinxi Lyu; Rulin Shao; Joseph E. Gonzalez; Matei Zaharia; Sewon Min
Venue: AAAI 2026 (main)

Abstract: We present DS SERVE, a framework that transforms large-scale text datasets—comprising half a trillion tokens—into a high-performance neural retrieval system. DS SERVE offers both a web interface and API endpoints, achieving low latency with modest memory overhead on a single node. The framework also supports inference-time tradeoffs between latency, accuracy, and result diversity. We anticipate that DS SERVE will be broadly useful for a range of applications such as large-scale retrieval-augmented generation (RAG), training data attribution, training a search agent, and beyond.
Framework GNN-AID: Graph Neural Network Analysis, Interpretation and Defense
Authors: Kirill Lukianov; Mikhail Drobyshevskiy; Georgii Sazonov; Mikhail Soloviov; Ilya Makarov
Venue: AAAI 2026 (main)

Abstract: The rising demand for Trusted AI (TAI) underscores the need for interpretable and robust models, yet existing tools rarely support graph-structured data or integrate interpretability with security. At the same time, Graph Neural Networks (GNNs) deliver state-of-the-art performance on numerous graph tasks.

We present GNN-AID (Graph Neural Network Analysis, Interpretation, and Defense), an open-source Python framework for analyzing, interpreting, and defending GNNs, addressing this critical gap. Built on PyTorch-Geometric, GNN-AID offers preloaded datasets, model libraries, flexible APIs, and a web interface for visualization and no-code model design. MLOps features further support reproducibility and experiment tracking.
Chatsparent: An Interactive System for Detecting and Mitigating Cognitive Fatigue in LLMs
Authors: Riju Marwah; Vishal Pallagani; Ritvik Garimella; Amit Sheth
Venue: AAAI 2026 (main)

Abstract: LLMs are increasingly being deployed as chatbots, but today’s interfaces offer little to no friction: users interact through seamless conversations that conceal when the model is drifting, hallucinating or failing. This lack of transparency fosters blind trust, even as models produce unstable or repetitive outputs. We introduce an interactive demo that surfaces and mitigates cognitive fatigue, a failure mode where LLMs gradually lose coherence during auto-regressive generation. Our system, Chatsparent, instruments real-time, token-level signals of fatigue, including attention-to-prompt decay, embedding drift, and entropy collapse, and visualizes them as a unified fatigue index. When fatigue thresholds are crossed, the interface allows users to activate lightweight interventions such as attention resets, entropy-regularized decoding, and self-reflection checkpoints. The demo streams live text and fatigue signals, allowing users to observe when fatigue arises, how it affects output quality, and how interventions restore stability. By turning passive chatbot interaction into an interactive diagnostic experience, our system empowers users to better understand LLM behavior while improving reliability at inference time.
PANSim: Visualization Tool for Planning and Acting against Nature
Authors: Erol Medenčević; Jakub Med; Lukáš Chrpa
Venue: AAAI 2026 (main)

Abstract: The demo presents a tool that visualizes the acting of planning agents in dynamic environments that might be modified by "acts of nature'', The purpose of this tool is to better understand the behavior of the agent, debug agent's behavior, and for making the underlying planning concepts accessible to wider audience.
OMEGA: An Ontology-Driven Tool for Explaining Multi-Agent Path Finding
Authors: Bharath Muppasani; Ritirupa Dey; Biplav Srivastava; Vignesh Narayanan
Venue: AAAI 2026 (main)

Abstract: Multi-Agent Path Finding (MAPF) algorithms provide highly optimized solutions for coordinating multiple agents in shared environments, yet their outputs lack explainability to human stakeholders. Existing explanation approaches, such as visual trace segmentation or logic-based reasoning, remain fragmented. In this demo, we present OMEGA, an interactive explanation platform that generates Natural Language (NL) explanations using the novel Multi-Agent Planning Ontology (maPO). Our framework transforms raw MAPF planner execution logs into a semantic knowledge graph, enabling SPARQL-based explanations of collision events, replanning strategies, and efficiency trade-offs. A lightweight web interface allows users to query, visualize, and interpret planner decisions, thereby making MAPF solutions transparent and auditable. We conducted a user study that confirms the ontology-driven explanations are significantly clearer and more preferred than raw logs, underscoring the potential of semantic technologies for explainable multi-agent systems.
MemoVision: A Digital Catalog for Everyday Interactions
Authors: Lai Xing Ng; Keith Tien Wei Tang; Jacky Jie Wei Tan
Venue: AAAI 2026 (main)

Abstract: We present MemoVision, a digital catalog system that captures semantic, spatial, temporal and interaction information as users move around physical environments using client devices such as smart glasses. The system utilizes open-vocabulary semantic segmentation and 3D scans to store objects-of-interest with comprehensive semantic, spatial, temporal and interaction labels. Our demonstration shows multimodal information query and retrieval capabilities, supporting specific queries about object locations, temporal events and user interactions including eye gaze and hand poses, enabling more contextualized responses compared to current multimodal large language models.
VulnBench: A Comprehensive Benchmark for Transformer-Based Vulnerability Detection
Authors: Jake Norton; David Eyers; Veronica Liesaputra
Venue: AAAI 2026 (main)

Abstract: Reproducible benchmarking of tools that automatically detect vulnerabilities in source code remains challenging due to inconsistent implementations, varying data preprocessing, and methodological flaws that compromise fair model comparison. In a recent study, 9 in 10 vulnerability detection studies were found to use inappropriate evaluation approaches, with models achieving high scores through spurious correlations rather than actual vulnerability detection. We present VulnBench, an extensible, open-source benchmarking tool that enables fair comparison across models and datasets. Our systematic evaluation of CodeBERT, GraphCodeBERT, CodeT5 (encoder-only and full), and NatGen across eight mostly C/C++ source code datasets reveals that proper threshold optimization can improve F1-scores by up to 54%, as well as wide variation in F1-scores showing the large gap in the difficulty of the vulnerability dataset field. By standardising evaluation protocols, VulnBench enables researchers to distinguish between genuine model improvements and methodological artifacts as well as reducing wasteful duplication of effort spent on reproducing results.
Market-Aware Event Timeline Summarization: Integrating Price Signals to Improve Financial News Understanding
Authors: Heesoo Park; Sujin Park; Dongjun Lee
Venue: AAAI 2026 (main)

Abstract: We present a practical system that supports in-depth analysis of cryptocurrency markets through timeline-based event detection and contextual summarization. Our framework processes continuous news streams, identifies price-relevant events, and organizes them into semantic timelines with concise background summaries generated by large language models (LLMs). This design allows traders and analysts to retrospectively explore events alongside price charts, facilitating a deeper understanding of how news developments relate to market fluctuations. By transforming unstructured news data into structured insights, the system provides a valuable tool for market analysis, risk evaluation, and behavioral studies in volatile trading environments.
MulTiCast: A Multimodal Time Series Forecasting System
Authors: Sehyuk Park; Caren Han; Hyunsuk Chung
Venue: AAAI 2026 (main)

Abstract: Time-series forecasting plays an essential role in domains such as finance, healthcare, and energy. Yet most existing systems operate in a unimodal setting, overlooking complementary information available from visual and textual modalities. There are surprisingly few time-series forecasting demos, and multimodal, interpretable demos are rarer still. In practice, it is difficult for users to experiment with foundation models on their own time-series data due to strict input requirements, heavy setup burdens, and limited interpretability support. We present MulTiCast, an interactive Multimodal Time series foreCasting system that enables users to combine numerical signals with visual and textual context to improve predictions. The system builds on pretrained models with lightweight adaptation, but its central contribution lies in the interactive demonstration platform. Through a web interface via Hugging Face Spaces, users can load datasets, toggle modality inclusion, and visualize forecasts together with the attention maps of each modality, providing insights into the reasoning path behind the predictions.
AssetOpsBench-Live: Privacy-Aware Online Evaluation of Multi-Agent Performance in Industrial Operations
Authors: Dhaval Patel; Nianjun Zhou; Shuxin Lin; James Rayfield; Chathurangi Shyalika; Suryanarayana Reddy Yarrabothula
Venue: AAAI 2026 (main)

Abstract: Industrial automation increasingly relies on multi-agent AI, yet evaluation remains difficult due to task complexity and data confidentiality. We present AssetOpsBench-Live, a demo of a competition-ready platform for real-time, privacy-preserving evaluation of multi-agent AI in industrial contexts. The platform integrates AssetOpsBench, which measures six dimensions of multi-agent performance and performs automated failure-mode discovery, with Codabench, which supports reproducible, code-oriented competitions. End users first validate agents locally, then submit containerized code for execution on hidden industrial scenarios. Instead of raw trajectories, the system provides quantitative scores and clustered failure modes (e.g., reasoning--action mismatch, step repetition), enabling participants to identify failures, apply targeted improvements, and iteratively resubmit. By combining competition-based engagement with actionable diagnostics, AssetOpsBench-Live delivers reproducible, real-time insights reflecting real-world industrial constraints.
KnowThyself: An Agentic Assistant for LLM Interpretability
Authors: Suraj Prasai; Mengnan Du; Ying Zhang; Fan Yang
Venue: AAAI 2026 (main)

Abstract: We develop KnowThyself, an agentic assistant that advances large language model (LLM) interpretability. Existing tools provide useful insights but remain fragmented and code-intensive. KnowThyself consolidates these capabilities into a chat-based interface, where users can upload models, pose natural language questions, and obtain interactive visualizations with guided explanations. At its core, an orchestrator LLM first reformulates user queries, an agent router further directs them to specialized modules, and the outputs are finally contextualized into coherent explanations. This design lowers technical barriers and provides an extensible platform for LLM inspection. By embedding the whole process into a conversational workflow, KnowThyself offers a robust foundation for accessible LLM interpretability.