zMANNNNNNNN/cross-model-state-transfer

Cross-model hidden-state map: Qwen2.5-3B-Instruct → gpt-oss-20b

Reconstructed checkpoint. This is a rebuild, not one of the three maps behind the published study. Those three checkpoints were lost in September 2026 and are no longer available. This map was retrained afterwards with the same recipe. All results in the first table below are this checkpoint's own, verified with the code in this repository.

Built with Qwen. Research and evaluation use only (see Licensing).

A small network (4.1M parameters) that moves a short structured record from one language model's internal state into a different model's internal state. The receiving model never sees the record as text. It is then asked questions about the record, including kinds of question the map was never trained on.

This repository accompanies an independent research write-up by Waylon Zabriskie.

What this checkpoint is, and is not

• The study reported results from three independently trained maps. Those three checkpoints were lost in September 2026, after their results had been recorded. They are not available.
• This is a rebuilt map. It was retrained on September 23, 2026 with the same recipe and seed 1, on regenerated training targets. Its weights are not identical to the original seed-1 map.
• Its scores below are its own. They were measured on October 30, 2026 with the code in this repository. They do not reproduce the original three-map results exactly. The original results appear only for comparison.

How it works

1. Sender: Qwen2.5-3B-Instruct (frozen) reads a synthetic record such as F1=B F2=D F3=A F4=C N1=3 N2=5 N3=1. Its final hidden state (after the final norm) is read at each of the seven value tokens: 7 × 2,048 numbers.
2. Map: the map turns these into 7 × 2,880 numbers. It is applied as map(record) − map(placeholder), so a placeholder record adds nothing.
3. Receiver: gpt-oss-20b (frozen) gets a prompt with a placeholder record (all A / all 1) and a question. After its first four blocks, the map's output is added to the residual stream at the seven value positions. The model then continues and its answer is read as the highest-scoring valid answer token.

The map was trained only to reproduce gpt-oss-20b's own layer-4 representation of each record, relative to the placeholder. It never saw a question or an answer.

Question types:

• four used during development: retrieve, numcmp, membership, count;
• equality: never used to train, tune or select;
• mapping: apply a private lookup table given in the question. Sealed and run once in the original study.

Results

Measured October 30, 2026 with this code (results/verification_run.json), inference only, on Apple Silicon with MLX 0.31.2 and mlx-lm 0.31.3. The run took about 5 minutes.

Question type	n	correct	swap follow	conv. under swap	zero	random	ceiling	text
retrieve (dev)	60	1.000	1.000	0.000	0.250	0.233	1.000	1.000
numcmp (dev)	60	1.000	0.967	0.033	0.650	0.467	1.000	0.950
membership (dev)	60	0.967	0.983	0.017	0.583	0.583	0.950	0.917
count (dev)	60	0.967	1.000	0.000	0.200	0.200	0.950	0.983
equality (held out)	60	0.917	0.783	0.217	0.267	0.283	0.900	0.883
mapping (final holdout)	80	0.787	0.812	0.062	0.237	0.250	0.775	0.800

On the two held-out types, this rebuilt map scores 0.917 (equality) and 0.787 (mapping). Both fall inside the range the original three maps reported.

For comparison only: the original three maps. These are the study's reported results. Their checkpoints were lost, so they cannot be checked from this repository, and this checkpoint does not reproduce those experiments exactly.

Question type	correct, original three maps (range)
retrieve (dev)	1.000
numcmp (dev)	0.983–1.000
membership (dev)	0.933
count (dev)	0.967–1.000
equality (held out)	0.900–0.917
mapping (final holdout)	0.775–0.787

Checks against the recorded study. The zero, ceiling and text conditions do not depend on the map, so they test whether this code poses the same questions in the same way:

• Mapping: ceiling 0.775 and text 0.800 match the study exactly. Zero is 0.237 here against 0.225 recorded, one question out of 80.
• Retrieve, numcmp and equality: zero and ceiling match the recorded seed-1 values exactly. The record does not list the text condition for these types.
• Membership and count: the ceiling differs from the record by one question out of 60 (0.950 against 0.933 and 0.967).

These one-question differences show the exact original evaluation environment was not recreated bit for bit.

Clean-environment check. The package was copied into an empty directory, installed into a fresh Python 3.11 environment from requirements.txt alone, and evaluated again with network access disabled. Every count and every mean margin matched results/verification_run.json exactly.

Earlier record of this checkpoint. A September 23 record of this same checkpoint lists mapping at 0.750, with zero at 0.183. Its zero score does not match the study's question set (0.225), so that evaluation evidently used different questions. The numbers above supersede it.

Conditions:

• correct: the map's output for the true record.
• swap follow: the output for a different record whose answer differs, scored against that record's answer. This tests whether the receiver's answer follows the substituted state.
• conv. under swap: the same run, scored against the true record's answer.
• zero: nothing added.
• random: Gaussian noise at the map output's scale.
• ceiling: gpt-oss-20b's own layer-4 rows for the true record.
• text: the record given as plain text.

Known reproducibility limitations and caveats

• Block 4 of the receiver is not run. The recorded evaluation runs blocks 0–3, inserts the map's output, then continues from block 5. This code keeps that behavior so its results are comparable with the study.
  • All conditions use the same 23-block path, including the zero, random, ceiling and plain-text baselines. This supports comparisons within this implementation; the effect of the skipped block has not been independently established.
  • It is not the unmodified 24-block model. This was found while preparing this release.
• Swap records are now seeded. In the original code, the substituted records for the swap test were drawn from Python's global, unseeded random generator. Here they come from a seeded generator (9000 + sum of the type name's character codes).
  • The question sets themselves use the original seeds (7000 + sum of character codes) and match the study's question sets.
  • Swap-follow numbers can therefore differ slightly from the study's for reasons unrelated to the map.
• Scope:
  • one model pair, one record format, known field positions;
  • about 15 bits of information per record;
  • 60 questions per type (80 for mapping).
• Not a general transfer method. The map relies on the receiver's existing ability to read a record at these positions. Whether it works on field values never seen in training has not been tested.
• Hardware: Apple Silicon with MLX only. About 13 GB of model weights are downloaded on first use.

Usage

pip install -r requirements.txt
python -m latent_transfer.evaluate --map map.safetensors --out results/my_run.json

The code loads both models at pinned revisions (see config.json) and runs all six question types with all controls. No training code is included. The run takes about 6 minutes on Apple Silicon once the models are downloaded. After the first download it can run offline with HF_HUB_OFFLINE=1; this relies on the pinned huggingface_hub version, because a newer version tried to reach the network even with a pinned revision.

Files

File	Contents
map.safetensors	The rebuilt map: 19 float32 tensors, 4,106,816 parameters
config.json	Architecture, insertion point, model revisions, recipe summary, seeds
latent_transfer/map_model.py	Map definition
latent_transfer/models.py	Model loading, state extraction and insertion
latent_transfer/synthetic.py	Record and question generator (seeded)
latent_transfer/evaluate.py	Evaluation with controls
examples/synthetic_examples.json	Two generated questions per type
results/verification_run.json	The verification run reported above
LICENSE	Apache-2.0, for the code only
LICENSING.md, NOTICE, licenses/	Upstream terms and attribution for the weights

Licensing

This repository has two licenses.

Part	License
Code in latent_transfer/	Apache-2.0 (LICENSE). It is original code and contains no upstream model code or weights. Its dependencies (MLX, mlx-lm: MIT; NumPy: BSD) are installed separately, not included.
Weights in map.safetensors	Not Apache-2.0. Released under the conservative reading of the Qwen RESEARCH LICENSE AGREEMENT: research and evaluation use only.
Configuration, examples, results, this card	Describe the weights and follow their terms.

Built with Qwen. The map was trained on hidden states of Qwen2.5-3B-Instruct, which is licensed under the Qwen RESEARCH LICENSE AGREEMENT.

• Section 4b of that Agreement requires "Built with Qwen" to be displayed for models trained using its outputs.
• Whether the map is also a "derivative work" under the Agreement is unresolved: the term is not defined, and the map contains no Qwen parameters. This release follows the stricter reading. The full Agreement and its required notice are included (licenses/, NOTICE). For commercial use, request a license from Alibaba Cloud.
• LICENSING.md gives the clauses and the open questions.

gpt-oss. gpt-oss-20b is by OpenAI, under Apache 2.0 and its usage policy.

No upstream weights. No Qwen or gpt-oss weights are included. Running the code downloads them, and using Qwen2.5-3B-Instruct requires accepting its Agreement.