ryzerrr/unity-game-agent

Unity Agent

A Python package that generates complete, playable Unity 2022.3 LTS projects on disk. The agent exposes a registry of code-generation tools that emit C# scripts (MonoBehaviours), .unity scene files, project configuration (manifest, ProjectSettings, .meta) and a full open-world city example game. The output is a real Unity project you can open in Unity Hub and press Play.

What it does

• Writes complete, compilable C# MonoBehaviours (player, vehicle, camera, AI, weapons, inventory, quests, save system, ...).
• Scaffolds a Unity project structure (Assets/, Packages/manifest.json, ProjectSettings/, .meta files) exactly the way Unity expects.
• Provides a system prompt that teaches an LLM to act like a senior Unity engineer and to call the right tool in the right order.
• Ships a knowledge base of focused markdown references (physics, navigation, input, UI, audio, rendering, open-world design).
• Runs static QA against generated projects to catch unbalanced braces, missing meta files, missing manifest, etc.
• Comes with a fully working open-world city example generated end-to-end from the tool layer.

Installation

git clone <this-repo> unity-agent
cd unity-agent
pip install -e .            # optional; the package also works with PYTHONPATH=.

The package has no third-party Python dependencies -- it only uses the standard library. Python 3.9+ is required.

Quick start

Generate the bundled example

# Generate the full open-world city example into examples/open_world_city/
python -m unity_agent generate --game open_world_city --preset open_world_city --output-dir examples

# Run static QA on it
python -m unity_agent qa examples/open_world_city

Open examples/open_world_city/ in Unity Hub (2022.3 LTS or newer) and press Play.

Use the package from Python

from unity_agent.config import Settings
from unity_agent.transport.unity_transport import UnityTransport
from unity_agent.tools.unity_tools import (
    SetupUnityProjectTool,
    CreatePlayerControllerTool,
    CreateProceduralCityTool,
    WriteSceneFileTool,
)

settings = Settings(output_dir="my_games")
transport = UnityTransport(settings)

SetupUnityProjectTool(settings, transport).run(project_name="MyGame")
CreatePlayerControllerTool(settings, transport).run(project_name="MyGame")
CreateProceduralCityTool(settings, transport).run(project_name="MyGame")
WriteSceneFileTool(settings, transport).run(project_name="MyGame", scene_name="MainScene")

print(transport.summary())

CLI

python -m unity_agent --help

# Commands
python -m unity_agent list-tools
python -m unity_agent setup --project-name MyGame --output-dir my_games
python -m unity_agent generate --game MyGame --preset open_world_city
python -m unity_agent generate --game MyFPS --preset fps_arena
python -m unity_agent generate --game Minimal --preset minimal
python -m unity_agent qa path/to/MyGame

Project layout

unity-agent/
  unity_agent/
    __init__.py              # Package entry point
    __main__.py              # CLI: `python -m unity_agent ...`
    config.py                # Settings (Unity path, LLM keys, output dirs)
    tests.py                 # Self-test suite (run with `python -m unity_agent.tests`)
    transport/
      __init__.py
      unity_transport.py     # Writes .cs / .unity / .meta / config files
    tools/
      __init__.py
      base.py                # ToolBase + ToolRegistry + @register_tool
      unity_tools.py         # 24 tools (see below)
    orchestrator/
      __init__.py
      prompts.py             # SYSTEM_PROMPT + build_user_prompt + tool_catalog
    knowledge/
      README.md
      __init__.py            # load_entry / list_entries / load_all
      entries/
        unity_fundamentals.md
        unity_physics.md
        unity_rendering.md
        unity_navigation.md
        unity_input_system.md
        unity_ui.md
        unity_audio.md
        open_world_design.md
    qa/
      __init__.py            # ProjectQA + Issue + QAReport + CLI
  examples/
    generate_example.py      # Regenerates examples/open_world_city/
    open_world_city/         # A complete generated Unity project
      Assets/Scripts/*.cs
      Assets/Scenes/MainScene.unity
      Packages/manifest.json
      ProjectSettings/*
      README.md
  pyproject.toml
  README.md

Tools (24 total)

Tool	Output
setup_unity_project	Project scaffold (manifest, ProjectSettings, asmdef)
create_player_controller	PlayerController.cs (WASD + jump + physics)
create_vehicle_controller	VehicleController.cs (arcade car physics)
create_procedural_city	CityGenerator.cs + BuildingGenerator.cs (50+ buildings)
create_third_person_camera	ThirdPersonCamera.cs (smooth follow + collision)
create_fps_controller	FPSController.cs (mouse-look + WASD + jump)
create_day_night_cycle	DayNightCycle.cs (sun rotation + sky/ambient lerp)
create_ai_npc	NPCController.cs (NavMesh wander/chase FSM)
create_pickup_system	PickupSystem.cs + Pickup.cs (score + collectibles)
create_health_system	HealthSystem.cs + HealthBar.cs
create_weapon_system	WeaponSystem.cs (raycast hitscan + ammo + reload)
create_audio_manager	AudioManager.cs (SFX + music singleton)
create_ui_manager	UIManager.cs (TMP HUD: score / health / messages)
create_terrain_generator	TerrainGenerator.cs (Perlin heightmap + splat)
create_water_shader	WaterMaterial.cs (animated standard-material water)
create_particle_effects	ParticleSpawner.cs (bursts + explosion helper)
create_save_system	SaveSystem.cs (JSON save/load to persistentDataPath)
create_inventory_system	InventorySystem.cs (stacks + capacity + events)
create_quest_system	QuestSystem.cs (state machine + progress + completion)
create_building_generator	BuildingGenerator.cs (walls + emissive windows + collider)
create_road_network	RoadNetwork.cs (grid layout + intersections)
write_csharp_script	Any C# script to Assets/Scripts/
write_scene_file	.unity scene file to Assets/Scenes/
generate_complete_game	Full game in one call (presets: open_world_city, fps_arena, minimal)

The system prompt

unity_agent.orchestrator.prompts.SYSTEM_PROMPT is a long, prescriptive prompt that teaches the LLM to:

• Think like a senior Unity engineer (lifecycle, physics step, idiomatic C#).
• Know the tool catalog and the recommended workflow (scaffold -> components -> bootstrap script -> scene).
• Always emit code that compiles on the first try in Unity 2022.3 LTS.
• Prefer procedural content (runtime Start() generation) over inspector-wired prefabs so the example "just plays" on open.
• Use the UnityAgent namespace and the UnityAgent.Scripts asmdef.

Use it like this from your own LLM driver:

from unity_agent.orchestrator import SYSTEM_PROMPT, build_user_prompt

prompt = build_user_prompt("Build me an open-world city game I can drive around in")
# Send SYSTEM_PROMPT + prompt to your LLM, then dispatch tool calls.

Knowledge base

Eight focused markdown references in unity_agent/knowledge/entries/:

1. unity_fundamentals.md -- MonoBehaviour lifecycle, GameObjects, components, coroutines, inspector best practices.
2. unity_physics.md -- Rigidbody, colliders, raycasting, ForceModes, physics vs frame step.
3. unity_rendering.md -- Materials, Standard shader, lighting, post- processing, LOD, fog.
4. unity_navigation.md -- NavMesh, NavMeshAgent, AI state machines, NavMeshObstacle, off-mesh links.
5. unity_input_system.md -- Legacy Input Manager, mouse-look recipes, new Input System, touch.
6. unity_ui.md -- Canvas, anchoring, HUD pattern, TMP, performance.
7. unity_audio.md -- AudioSource, 3D audio, AudioMixer, AudioManager singleton.
8. open_world_design.md -- Procedural generation, LOD, streaming, day/night cycle, save systems.

Load them programmatically:

from unity_agent.knowledge import list_entries, load_entry, load_all

for slug in list_entries():
    print(slug, len(load_entry(slug)))

QA

unity_agent.qa.ProjectQA runs static checks against a generated project:

• Required directories exist (Assets/, Assets/Scripts/, Packages/, ProjectSettings/).
• Required files exist (Packages/manifest.json, ProjectSettings/ProjectVersion.txt, ProjectSettings/ProjectSettings.asset).
• Every .cs file has balanced braces and at least one type declaration.
• Every .cs and .unity file has a sibling .meta.
• Scene files start with the %YAML header.
• manifest.json parses as JSON and has a dependencies object.

from unity_agent.qa import ProjectQA
report = ProjectQA("examples/open_world_city").run()
print(report.summary())
print("PASS" if report.passed else "FAIL")

Tests

python -m unity_agent.tests

Verifies the tool registry has 24 entries, the project scaffolder writes the expected files, generated C# has balanced braces, the full game generator produces 12+ scripts, and QA passes on a freshly generated project.

Requirements

• Python 3.9+ (no third-party dependencies).
• Unity Hub + Unity 2022.3 LTS (or newer) to actually open and play the generated projects. The Python layer works without Unity installed.

License

MIT.