YAML Metadata Warning:empty or missing yaml metadata in repo card
Check out the documentation for more information.
Learning to Interact and Direct: Co-Evolving Video Director Agents
Overview
LIT (Learning to Interact and DirecT) pairs a video-director agent with a simulated user to produce videos and learn from dialogue, production traces, and generated media. A Meta Engineer uses external references to identify problems, develop and validate revisions, and improve the director's production and interaction Skills alongside the simulated user, evaluator, and its own methods.
Repository Layout
.
├── evo-project/
│ ├── ENTRY.md # Controller entrypoint
│ ├── config.json # Number of evolution rounds
│ ├── interfaces.md # Role communication and submission interfaces
│ ├── EXTERNAL_DATA.md # External resources and their sources
│ ├── .codex/ # Controller configuration
│ ├── .agents/skills/ # Shared Skill discovery links
│ ├── global/
│ │ ├── common/ # Shared Skills, tools, and service configuration
│ │ ├── controller/ # Task scheduling and orchestration
│ │ ├── worker/ # Reusable production resources
│ │ ├── evaluator/ # Shared evaluation tools
│ │ ├── meta/ # Reference data and retrieval tools
│ │ └── user/ # Shared simulated-user resources
│ └── versions/
│ ├── current.json # Selected role versions
│ ├── worker/initial/ # Creator and Director-building Skills
│ ├── evaluator/initial/ # Development evaluator
│ ├── user/initial/ # Simulated user
│ └── meta/initial/ # Role optimizers and local Skill development
└── privilege-eval/
├── user/ # Test-time Privileged User
└── evaluator/ # Fixed test evaluator
The production, evaluation, and evolution workflows live in evo-project/. The test-time Privileged User and Privileged Evaluator live separately in privilege-eval/.
Setup
Prepare the task data, Web dataset, media libraries, and model weights listed in EXTERNAL_DATA.md. Configure the qwen38 profile and the model and media services required by your tasks, and follow each resource's license terms.
Service configuration is in local-services.yaml and codex.toml. Use .env.example to prepare private credentials and keep them outside the repository.
Running
From the repository root, enter evo-project/ and start the Controller:
cd evo-project
codex --profile qwen38
Send this message in the session:
Read ENTRY.md, then start working.
Give the Controller the objective, case scope, concurrency limit, and production and validation budgets. It coordinates User–Worker production and independent evaluation, then runs Worker and Evaluator optimization in parallel, followed by User and Meta optimization.
The evolution_rounds setting in config.json controls the number of improvement rounds; the default is 4. See the usage guide and role interfaces for details.
Citation
TODO