jacob-valdez/tensorcode-scene-vsr-experimental-001

TensorCode Scene: CLIP/VSR negative experiment

This checkpoint does not demonstrate useful visual spatial reasoning. It is a reproducible negative experiment for an owned trainable multimodal workspace, not a ready-to-go vision assistant.

The complete frozen CLIP model, tokenizer, preprocessing configuration, trainable image/text projections, recurrent learned-slot Workspace and ranking head are included. The model accepts an image, a question/caption and explicitly supplied text candidates. It returns rankings, source patch coordinates, routing attention and slot relations. These are fallible interpretations; attention is not factual proof. It generates no image claims and includes no object/spatial truth rules.

Provenance and preprocessing

Perception initializes from openai/clip-vit-base-patch32 revision 3d74acf9a28c67741b2f4f2ea7635f0aaf6f0268; inherited perception remains frozen. Training data: first 512 unique image IDs in the official VSR random training split. Test: 128 new unique image IDs from the official test split, skipping the first 64 eligible IDs and excluding all selected training IDs. VSR revision b2053328fafdd018ff56cf1dfa9643caaa4e69b8. This test set is disjoint from the prior random-initialization experiment's 64 images. No tuning used heldout examples.

Human VSR caption truth labels become explicit supported/unsupported candidate targets. Training: seed 17, 10 Adam epochs, learning rate 0.001, gradient norm limit 1, 224×224 RGB resize. Model preprocessing uses explicit bicubic square resizing and CLIP channel normalization, not CLIP's default aspect-preserving crop. Patch coordinates refer to the supplied image tensor, which the example resizes before calling Scene. CLIP provides learned aligned global image/text features in addition to patch/text workspace inputs. Zero-workspace ablation retains those global foundation features.

Measured result

On 128 heldout images, full-model accuracy decreases from 55.47% to 50.78%; cross entropy increases from 0.69266 to 4.64355. Blank-image accuracy is 53.91%; zero-workspace and bypass-workspace each achieve 52.34%. Training mean loss decreases to 0.46472. This combination indicates overfitting and does not establish grounding or useful cognitive workspace contribution. These are small-subset results, not the published full VSR benchmark.

See report.json and provenance.json. A fresh process loading the complete saved checkpoint reproduces the same metrics. Weights are separate from experience and optimizer state; no source photographs or conversation history are included.

Use

from tensorcode.tools.scene import Scene
model = Scene.from_pretrained('./model')
# Supply normalized CHW float pixels, question, source_id, and explicit candidates.
result = model(inputs)

Source: examples/train_scene.py in TensorCode. Construction does not download; Scene.from_foundation(repo_id, revision=...) explicitly imports foundation perception while initializing a fresh ranking model.

Attribution

CLIP: https://huggingface.co/openai/clip-vit-base-patch32 (upstream MIT license). VSR annotations: https://huggingface.co/datasets/cambridgeltl/vsr_random (CC-BY-4.0), https://github.com/cambridgeltl/visual-spatial-reasoning . Underlying COCO photographs retain their original Flickr/COCO rights and are not included here.