Spatial Knowledge Graph-Guided Multimodal Synthesis
Authors: name: Yida Xue; name: Zhen Bi; name: Jinnan Yang; name: Jungang Lou; name: Huajun Chen; name: Ningyu Zhang
Published: 2025-05-28 17:50:21

Abstract: Recent advances in multimodal large language models (MLLMs) have
significantly enhanced their capabilities; however, their spatial perception
abilities remain a notable limitation. To address this challenge, multimodal
data synthesis offers a promising solution. Yet, ensuring that synthesized data
adhere to spatial common sense is a non-trivial task. In this work, we
introduce SKG2Data, a novel multimodal synthesis approach guided by spatial
knowledge graphs, grounded in the concept of knowledge-to-data generation.
SKG2Data automatically constructs a Spatial Knowledge Graph (SKG) to emulate
human-like perception of spatial directions and distances, which is
subsequently utilized to guide multimodal data synthesis. Extensive experiments
demonstrate that data synthesized from diverse types of spatial knowledge,
including direction and distance, not only enhance the spatial perception and
reasoning abilities of MLLMs but also exhibit strong generalization
capabilities. We hope that the idea of knowledge-based data synthesis can
advance the development of spatial intelligence.