covt1028

covt1028 is a Dynamics-ER physical-evidence conditioned vision-language model based on Qwen3.5-4B. This release contains the final policy after 2,099 optimizer updates over one pass through 268,571 training rows, a precision-preserving Safetensors export, a self-contained model implementation, and numeric training diagnostics.

The model combines the trained Qwen native vision encoder, language model and language head with a Physical Adapter. Causal perceptual relation tokens are injected after the decoder token mixer and before the feed-forward block. The Adapter is present in decoder layers 0–23, uses 256-dimensional relation features, an attention width of 256, eight heads, and learned scalar gates.

The intended tasks are object memory, physical-state recognition, task-stage recognition, state-change recognition, and task-outcome recognition. This is an inference release of the trained policy. It is not a complete autonomous robot policy or a training-resumption checkpoint.

Release contents

• weights-*.safetensors and model.safetensors.index.json: the trained native vision, language, language head, and Physical Adapter tensors.
• configuration_dynamics_er.py, modeling_dynamics_er.py, and physical_evidence.py: the model configuration, Adapter-preserving decoder, and public evidence schema.
• config.json, generation_config.json, and requirements.txt: generic inference configuration and dependency versions.
• training_config.json, training_metrics.csv, and metrics_description.json: a reviewed training recipe and the documented numeric metrics from all formal updates.
• figures/training_diagnostics.png: a visualization regenerated from the distributed numeric table.
• SHA256SUMS and release_manifest.json: the exact release-file inventory and integrity checks.

The pretrained tokenizer and image processor are obtained from the official base model at revision 851bf6e806efd8d0a36b00ddf55e13ccb7b8cd0a. Third-party DINO and PSM weights, training examples, videos, optimizer state, random-number-generator state, and data-loader state are not included.

Training recipe

The policy was initialized from the official Qwen3.5-4B model with fresh zero-gated Physical Adapters. It was not initialized from the historical 200k SFT model, and no additional CoT-SFT phase was inserted.

Setting	Value
Method	Single-step GRPO; one PPO epoch per batch
Training rows / epochs	268,571 / 1
Optimizer updates	2,099
Global prompt batch	128
Responses per prompt	4
Global response batch	512
Gradient accumulation	1
Native vision / language learning rate	1e-6
Physical Adapter learning rate	1e-5
Optimizer	AdamW, betas=(0.9, 0.999), weight decay=0
Learning-rate schedule	Constant; no warmup
Gradient clipping	L2 norm 1.0
PPO clipping / KL coefficient	0.2 / 0.02
Reference	Fixed official model with zero-gated Adapter
Format / task reward maximum	0.3 / 1.7; total range 0–2
Prompt / response / context limits	7,168 / 1,024 / 8,192 tokens
Training sampling	Temperature=1.0, top-p=1.0, unrestricted top-k
Precision	BF16 native parameters and native optimizer states; FP32 Adapter
Trainable components	Native vision, language, LM head, Physical Adapter
Frozen components	DINO and PSM
LoRA / gradient checkpointing	No LoRA / enabled
Random seed	20261015

The final transport batch uses 101 padding positions with zero policy, KL, and reward weight. All real training rows retain their original order and weights. There are 27 real prompts in the final batch. Increasing the global batch changes the optimization trajectory; no equivalence to smaller-batch historical runs is claimed.

Software used for training: Python 3.12.13, PyTorch 2.11.0+cu129, Transformers 5.16.1, vLLM 0.20.0+cu129, CUDA 12.9, and Triton 3.7.1. Training used verl with FSDP2 and an Adapter-preserving rollout implementation. The inference implementation here uses Transformers; an unmodified native Qwen or stock rollout loader must not be used to silently discard the Adapter.

Training data and boundaries

Family	Training rows
Object memory	150,000
Physical state	50,000
Subtask stage	38,571
State change	20,000
Task outcome	10,000
Total	268,571

These are row counts, not counts of independent videos or events. The mixture retains a known source relationship: 248 training rows share a parent trajectory with 30 r09 evaluation episodes. Strict zero source overlap is not claimed. The 13,950 memory rows carrying the original eligibility=false marker were retained, without changing their original weights. Raw examples and source-specific identifiers are not distributed in this model package.

Training is single-step GRPO. A later multi-step memory evaluation does not turn it into multi-step reinforcement learning. The <think> / <answer> output convention is not evidence that a separate CoT-supervised corpus was used.

Training diagnostics

The figure shows raw values and a trailing 50-update arithmetic mean from all 2,099 formal updates. Exact values are in training_metrics.csv. The last batch includes zero-weight padding; the logged batch mean reward should be interpreted with that boundary in mind. Reward includes a format component and must not be reported as accuracy. No sample prompts, responses, images, timestamps, or execution identifiers are included in this table.

Loading and inference

This model requires physical evidence in addition to the ordinary Qwen inputs. A generic text-only or native-Qwen loader is not an equivalent replacement. Review the included custom model code before enabling trust_remote_code.

python -m pip install -r requirements.txt

import torch
from transformers import AutoModelForImageTextToText, AutoProcessor

model_id = "jeix/covt1028"
processor = AutoProcessor.from_pretrained(
    "Qwen/Qwen3.5-4B",
    revision="851bf6e806efd8d0a36b00ddf55e13ccb7b8cd0a",
)
model = AutoModelForImageTextToText.from_pretrained(
    model_id,
    trust_remote_code=True,
    dtype="auto",
    device_map="auto",
    attn_implementation="sdpa",
).eval()

# Prepare `messages` from your permitted causal image history and question.
inputs = processor.apply_chat_template(
    messages, tokenize=True, add_generation_prompt=True,
    return_dict=True, return_tensors="pt",
).to(model.device)

# Obtain these tensors from the matching frozen perceptual evidence pipeline.
# Do not substitute random or zero tensors for a valid observation history.
# physical_tokens: float tensor [batch, relation_length, 256]
# physical_valid: bool tensor [batch]
# physical_lengths: int64 tensor [batch]
with torch.inference_mode():
    generated = model.generate(
        **inputs,
        physical_tokens=physical_tokens.to(model.device),
        physical_valid=physical_valid.to(model.device),
        physical_lengths=physical_lengths.to(model.device),
        do_sample=False,
        max_new_tokens=1024,
    )
answer = processor.batch_decode(
    generated[:, inputs["input_ids"].shape[1]:],
    skip_special_tokens=True,
)

The example uses variables supplied by the application: messages, physical_tokens, physical_valid, and physical_lengths. It does not construct the perceptual evidence for you. The release deliberately fails when evidence fields are absent. Adapter parameters and arithmetic remain FP32; do not convert the exported weights wholesale to BF16 or FP16.

Physical evidence contract

• physical_tokens: finite floating-point tensor [B, K, 256], containing causal relation features from the matching frozen DINO/PSM pipeline.
• physical_valid: boolean tensor [B]; false represents unavailable initialization history and yields zero Adapter contribution.
• physical_lengths: int64 tensor [B]; each length is 8 or a multiple of 40, up to 560 and no greater than K. Batches may be padded to their maximum K.
• Ordinary padded input_ids and attention_mask determine the image-token positions. Evidence is attached to those positions during prefill and retained as input during cached generation.
• Only permitted causal observations may be used. Future frames, answer labels, and evaluation truth must not enter the evidence pipeline.

External perceptual weights and their redistribution rights are separate from this policy release. Users must provide the matching licensed assets. This package does not claim raw-video-to-prediction deployment readiness without that pipeline.

Evaluation status and intended use

Export validation covered all 1,036 named tensors, their shapes and original dtypes, exact equality after loading, the 312 FP32 Adapter tensors, and tied embeddings. Full-size CPU visual forward and cached generation checks passed on synthetic interface inputs using the upstream reference PyTorch kernels. These checks validate export and loading behavior; they are not quality measurements.

Full final-checkpoint r09 typed-state, closed-loop memory, and robot-rollout evaluation results are not available in this release. Historical scores from other models and output protocols are not scores for covt1028.

The planned r09 state and memory evaluations each cover 1,497 episodes and 52,366 state points, with separate metrics and fixed failure handling. Closed-loop memory must carry only the model's valid updates into the next time step; truth is for offline scoring. Robot task success requires an independent environment rollout evaluation.

This is a research model for physical-state and memory reasoning. No validated robot-success improvement, anticipation capability, benchmark superiority, or suitability for safety-critical decisions is claimed. A 1,024-token response limit can truncate outputs. Outputs and proposed memory changes need application-level validation.

License and attribution

The policy weights and included Dynamics-ER-derived inference code are distributed under the Apache License, Version 2.0; see LICENSE and NOTICE. The official base model is Qwen3.5-4B, under Apache-2.0. Third-party perceptual encoders, data, and assets retain their own terms and are not relicensed or redistributed by this package.

Artifact privacy review

The release uses a positive file and field allowlist. Only named model tensors are reserialized, with minimal Safetensors metadata. Documentation and configuration are rebuilt from reviewed generic settings; metrics are numeric-only and plots are regenerated. The file inventory and checksums cover the exact published artifacts. This artifact review does not certify that learned weights cannot memorize training data; no memorization evaluation is claimed.