Executable Knowledge Graphs for Replicating AI Research
Authors: name: Yujie Luo, name: Zhuoyun Yu, name: Xuehai Wang, name: Yuqi Zhu, name: Ningyu Zhang, name: Lanning Wei, name: Lun Du, name: Da Zheng, name: Huajun Chen
Organization: Ant Group

Replicating AI research is a crucial yet challenging task for large language
model (LLM) agents. Existing approaches often struggle to generate executable
code, primarily due to insufficient background knowledge and the limitations of
retrieval-augmented generation (RAG) methods, which fail to capture latent
technical details hidden in referenced papers. Furthermore, previous approaches
tend to overlook valuable implementation-level code signals and lack structured
knowledge representations that support multi-granular retrieval and reuse. To
overcome these challenges, we propose Executable Knowledge Graphs (xKG), a
modular and pluggable knowledge base that automatically integrates technical
insights, code snippets, and domain-specific knowledge extracted from
scientific literature. When integrated into three agent frameworks with two
different LLMs, xKG shows substantial performance gains (10.9% with o3-mini) on
PaperBench, demonstrating its effectiveness as a general and extensible
solution for automated AI research replication. Code will released at
https://github.com/zjunlp/xKG.