Tiny Scene

Text → animated 3D scene in three.js with generated music, driven by a tiny Transformer trained from scratch that runs entirely in the browser.

Example scene built from one prompt and two edits

Inspired by WVY-Blender-5m: same idea (a small model translates a sentence into a compact scene language, and procedural code builds the result), but the builder is three.js instead of Blender. No code or weights from that repository are reused.

"a red fox jumping over a blue car in the desert at sunset"
        │  model (4.85 M parameters, ~0.3 s on CPU, plain JavaScript)
        ▼
<new> loc=desert time=sunset <ent> type=fox color=red act=jump rel=over <ent> type=car color=blue
        │  interpreter + procedural builder (three.js, no custom shaders)
        │  + procedural band (Tone.js): chords, bass, drums and melody chosen from the scene
        ▼
real-time animated scene with music, editable with more sentences

Try it

Download the repository and serve the web/ folder with any static server, for example:

python3 -m http.server 8080 --directory web

Open http://localhost:8080. Type in English or Spanish:

Kind Examples
New scene an alien riding a horse on the moon · tres pingüinos caminando en la nieve mientras nieva
Edit make the fox green · add a giant tree · remove the car · make it rain · zoom out · orbit the camera
Music a robot dancing in the city with fast electronic music · play sad jazz music · make the music faster · no drums · stop the music
Ask what can you do? · how do i enable shadows · what objects do you know

Music and sound

Every scene gets a soundtrack, generated live in the browser. Nothing is recorded: there are no audio files.

  • Chosen from your words: mood (happy, sad, calm, tense, epic, spooky, playful, dreamy), style (chiptune, electro, ambient, rock, jazz, orchestral, lofi) and tempo (slow, medium, fast).
  • Or from the scene when you say nothing: a storm is tense, a ghost is spooky, space is dreamy and ambient, a robot gets electro, a knight gets an orchestra, running speeds it up.
  • Editable: make the music faster, louder, no drums, add the bass back, stop the music, or the Música section of the Editar tab.
  • In sync: dancing, bouncing and jumping follow the beat (slowly is half time, fast double time).
  • Ambience: rain, wind, waves, night crickets, thunder timed to the lightning, and engine hums that are positioned in 3D for moving vehicles.
  • Sonido mutes everything. Grabar vídeo records the view with its sound to a .webm file (up to 60 s).

Browsers only allow sound after a click or key press, so the music starts with your first interaction.

Hand editing

Click an object in the view (or in the list under the Editar tab) to select it, then:

  • Drag it in the view to move it. Riders, and objects that orbit, jump over or chase it, move with it.
  • Change its type, action, size, count, speed, relation and colour in the Editar tab, or Duplicar / Borrar it.
  • Add new objects, and change place, time, weather, shot and camera move, from the same tab.
  • Typed edits keep working, and "it" refers to the selection: select the car and type make it yellow.
  • Ctrl+Z / Ctrl+Shift+Z undo and redo every change, typed or by hand. Delete removes the selection, Esc clears it.

Hand edits change the same scene description the model writes, so both can be mixed freely.

Drag the background to orbit the view. Export .glb saves the current frame (opens in Blender); animations are procedural and are not exported.

The page also accepts ?q=first prompt;second prompt&t=3 to run prompts on load and freeze time at that second.

The interface labels are in Spanish; the model understands both languages.

What it can do

Objects (54) animals, characters, vehicles and props: dog, dragon, robot, astronaut, car, rocket, UFO, tree, tower…
Actions (17) idle, walk, run, jump, spin, dance, fly, float, sit, sleep, wave, bounce, roll, circle, shake, swim, drive
Relations (8) on, over, near, behind, front, around, ride, chase
Places (15) field, desert, snow, beach, forest, city, space, moon, ocean, room, mountain, farm, road, park, studio
Time and weather day, night, sunset, dawn · clear, rain, snow, fog, storm
Camera shots close, medium, wide, top, low · moves static, orbit, zoomin, zoomout, pan, follow
Attributes 14 colours, 4 sizes (plus "bigger / smaller"), count 2–6, speed
Music 8 moods, 7 styles, 3 tempos, faster/slower, louder/quieter, drums/bass/melody on or off
Questions 25 topics about three.js and the app itself; the model picks language and topic, and the answer is stored in web/model/qa.json

How it works

ml/      lexicon.py   scene-language vocabulary and the English and Spanish words for it
         gen.py       generator of (sentence → tokens) pairs, with typos and dropped accents
         model.py     decoder-only Transformer (6 layers, d=256, 8 heads, context 208)
         train.py     trains on pairs generated on the fly and exports to web/model/
         infer.py     command-line inference
web/     js/nn/model.js         the same Transformer in plain JavaScript, with a KV cache
         js/audio/theory.js     scene → mood/style/tempo, and the notes of an 8-bar loop (pure JS)
         js/audio/engine.js     Tone.js playback, ambience, engine sounds and video recording
         js/editor.js           the hand-editing panel
         js/lang/scenelang.js   tokens → scene JSON, and applying edits
         js/scene/              creatures.js, things.js (objects), animate.js (poses),
                                layout.js (placement and relations), environment.js, stage.js
tests/   check_scene.mjs   builds every object × action and place × time × weather combination
         check_model.mjs   JS ↔ PyTorch parity and hand-written prompts
         check_export.mjs  .glb export in a real browser
         check_editor.mjs  click, drag, panel edits, undo/redo and delete in a real browser
         check_music.mjs   note generation, scene rules, and real playback, beat clock and video export in a browser
  • Input: the sentence is split into characters (50 symbols), which makes it tolerant to typos. Output: fixed scene-language tokens. Total vocabulary: 267.
  • Music: the model only picks labels (<mus> mood=sad genre=jazz tempo=fast). theory.js turns them into a scale, a four-chord progression and seeded patterns for four layers, so the same scene always gets the same tune.
  • Data: there is no dataset on disk. gen.py produces endless random pairs from templates; every batch is new.
  • Weights: web/model/weights.bin is 9.7 MB in float16 (used by the browser). ml/model.pt is the float32 PyTorch checkpoint.

Results

Included model: 4,860,672 parameters, 16,000 steps with batches of 256 (4.1 M pairs), 22 minutes on an RTX 5060 Ti.

Test Result
New generator sentences, clean (3,000) 99.90 % exact outputs
New generator sentences, with typos (3,000) 99.50 %
Hand-written sentences outside the templates (76, tests/natural.json) 75 exact
JavaScript vs PyTorch (300 sentences) 300 identical
Time per sentence in JavaScript (Node, one thread) ~245 ms
Scenes built without errors (object × action, place × time × weather, relations) 1,478 of 1,478
Music loops generated and validated (mood × style × seed) 168 of 168
Styles that produce sound in a real browser 7 of 7

The first two rows measure sentences with the same grammar as training, so they are optimistic. The third is the most honest, but it is small, and two early failures (a wide shot of…, an object mentioned after the place) led to adding templates and retraining. The 16 music sentences in it were written before the music model was trained; 15 were exact on the first run.

The audio tests check that sound comes out, that the beat clock matches the tempo and that the video has an audio track. They do not judge how good the music sounds.

Retrain

uv venv .venv && uv pip install --python .venv/bin/python torch numpy
npm install
npm run train
npm test

To add an object: an entry in _TYPES in ml/lexicon.py, a builder in web/js/scene/things.js or creatures.js, then retrain. npm test fails if the vocabulary names something three.js cannot build.

Limitations

  • It only understands the vocabulary above. Anything unknown is replaced by the closest match or ignored (a little girl flying a kite gives a small woman flying).
  • It does not generate new geometry: every object is hand-modelled from primitives.
  • It does not compose: the music is rule-based loops of eight bars with synthesised instruments. It is coherent but simple and repetitive, and there are no named instruments (piano music is not understood).
  • The exported video is recorded in real time from the screen, so its quality depends on the machine.
  • Typed edits refer to objects by type (the dog), not by colour or position; with two different dogs, both change. Select one and use it, or edit it by hand.
  • Hand-moved objects keep an offset from where the automatic layout puts them, so adding or removing other objects can still shift them.
  • A place has no colour (in a blue house is read as a dog and a blue house).
  • Known miss: something calmer, maybe ambient sets the music correctly but also adds a man.
  • Long sentences with three objects and many modifiers are where it fails most.

Credits

three.js and Tone.js (both MIT license) are bundled in web/vendor/; their licenses are in web/vendor/LICENSE and web/vendor/LICENSE-tone.md.

Downloads last month

-

Downloads are not tracked for this model. How to track
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support