RuleMaze

RuleMaze is a benchmark and training framework for rule-compliant visual spatial planning with Multimodal Large Language Models (MLLMs).

Given a visual maze and a set of natural-language rules, the model is required to understand the environment, follow the rules, and generate a valid multi-step trajectory.

This repository provides LoRA adapters fine-tuned from Qwen2.5-VL-3B-Instruct on the RuleMaze training data.

Checkpoints

Two scene types are provided:

• RuleMaze/regular/checkpoint
• RuleMaze/quest/checkpoint

The checkpoints are PEFT/LoRA adapters and should be loaded together with the base model:

Qwen/Qwen2.5-VL-3B-Instruct

Resources

• Code: https://github.com/oceanflowlab/RuleMaze
• Dataset: https://huggingface.co/datasets/Fish-03/RuleMaze
• Base Model: https://huggingface.co/Qwen/Qwen2.5-VL-3B-Instruct

Training and Evaluation

The models are trained using the RuleMaze DMP training pipeline with LLaMA-Factory.

RuleMaze evaluates visual planning under both seen-rule and unseen-rule settings and different rule difficulties.

For training and evaluation details, please refer to the official code repository.

Intended Use

The models are intended for research on:

• multimodal reasoning
• visual spatial planning
• rule following
• compositional generalization

Citation

If you find RuleMaze useful, please cite:

@misc{rulemaze,
  title  = {Rule-Compliant Visual Spatial Planning for Multimodal Large Language Models},
  author = {Yu Chen, Ting Lei, Yaoyi Li, Jia Cai, Zhecen Wu and Yang Liu},
  year   = {2026},
  note   = {Code and dataset release}
}