ALWM

Accessible Latent World Modeling for Predictive Video Reasoning

ALWM is a trained checkpoint based on Qwen3-VL-8B-Instruct. It is designed to reason about what may happen next in a video, using the observed video context and a question. The unobserved future video is not an input.

ALWM represents a possible future world state with continuous latent states, then uses those states together with the existing context to generate a future-event description and answer. It predicts in latent space; it does not generate future video frames.

Release status: The complete checkpoint weights and inference configuration are available in this repository. The continuous-latent decoding requirement below still applies.

Intended use

• Future-event prediction: describe a plausible next action, interaction, or scene change from an observed video prefix.
• Predictive video question answering: answer questions about an unobserved continuation, including questions with multiple-choice options.
• Latent-mediated reasoning: use a predicted continuous representation as an intermediate state before producing the final textual prediction and answer.

How the latent interval works

The ALWM decoding path alternates text generation with a continuous latent interval:

Observed video + question (+ answer options)
                    |
                    v
       Observation and textual reasoning
                    |
                    v
               <latent>
                    |
                    v
         z1 -> z2 -> ... -> z8
         continuous hidden states
                    |
                    v
              </latent>
                    |
                    v
       Future-event prediction + answer

Each z is a continuous hidden-state vector, not a word, vocabulary token ID, or image. The decoder feeds a latent state back as an input embedding to produce the next state, retaining the existing causal context. After eight latent states, text generation resumes.

The <latent> and </latent> tags mark the boundary in the textual format. The vectors themselves are handled by the inference runtime and are not represented by printable text. “Accessible” refers to using the representation for downstream language output, not to individual latent dimensions having guaranteed human-readable meanings.

Illustrated example

This is an illustrative example, not a benchmark sample or a measured model response.

Observed video: A ball rolls across a table toward its edge. The clip ends before the ball reaches the edge.

Question: What is most likely to happen next?

A. The ball falls off the table.
B. The ball stops just before reaching the edge.
C. The ball rolls back toward the center of the table.
D. Someone catches the ball before it reaches the edge.

+------------------------------+
| Observed: ball rolls toward   |
| the edge of the table         |
+------------------------------+
               |
               v
+------------------------------+
| Reason about the visible     |
| motion and loss of support   |
+------------------------------+
               |
               v
+------------------------------+
| Latent future representation |
| [z1] [z2] [z3] ... [z8]       |
| (continuous vectors)         |
+------------------------------+
               |
               v
+------------------------------+
| Prediction: the ball falls   |
| off the table.               |
| Answer: A                    |
+------------------------------+

An illustrative text-only view of the response is:

<observation>A ball is rolling toward the edge of a table.</observation>
<reasoning>If it continues past the edge, it will no longer be supported by the table.</reasoning>
<latent></latent>
<prediction>The ball is likely to roll off the edge and fall.</prediction><answer>A</answer>

The empty text between <latent> and </latent> does not mean that the latent computation is empty: the eight continuous vectors are carried separately by the runtime. The diagram's z1–z8 labels are explanatory notation, not strings to insert into a prompt.

Checkpoint file structure

The published checkpoint package has the following structure:

ALWM/
├── README.md
├── config.json                    # Qwen3-VL model architecture
├── alwm_config.json               # Continuous-latent inference contract
├── generation_config.json         # Text-generation defaults
├── model.safetensors.index.json   # Tensor-to-shard mapping
├── model-*.safetensors            # Complete model weights
├── processor_config.json          # Image/video processor settings
├── tokenizer.json                 # Tokenizer vocabulary and encoding
├── tokenizer_config.json          # Tokenizer settings and special tokens
└── chat_template.jinja            # Conversation formatting

The weights include the visual encoder, language model, and language-model output head. The package is a full checkpoint, not an adapter-only release. Weight shards use the Safetensors format.

Inference compatibility

The weight files and model configuration follow the Qwen3-VL architecture. The complete ALWM behavior additionally requires an ALWM-aware decoding loop that performs hidden-state feedback during the latent interval and then resumes text decoding.

An ordinary text-only generate() call, or a prompt containing literal z1–z8 strings, does not implement this continuous-latent computation. This README does not claim a ready-to-run generic Transformers or hosted-inference pipeline for the complete ALWM path.

Limitations

Future predictions are plausible hypotheses, not observations or guaranteed outcomes. The model can miss relevant motion, confuse objects, or produce incorrect descriptions and answers. Continuous latent states are not a calibrated physical simulator. Use predictions for research and exploratory applications, with independent validation where correctness matters.