Qwen3.5-9B-PsiLM

Qwen3.5 9B — a decoder that is mostly recurrent — coupled to a physics model through trained latent bridges, runnable on a Mac with one command.

Research generated by Claude (Anthropic) under the direction of Ryoji Furui; see the AI generation disclosure.

What this is

PsiLM (ΨLM) runs a frozen language model and a frozen physics model together while one answer is produced. Nothing is fine-tuned and no text crosses the interface: small trainable bridges read the physical problem out of the language model's hidden states, the physics model computes, and its result flows back into the language model's residual stream as a few soft tokens through a gated cross-attention. This repository packages that system for a third backbone family, and the first one that is not an attention stack:

part	what	trained?	where it comes from
language model	Qwen3.5 9B text tower, NVFP4 (32 layers: 24 Gated DeltaNet linear-attention layers carrying a recurrent state, 8 full-attention; hidden 4096)	frozen	this repo, at its root (config.json, model.safetensors, the tokenizer): Ollama's qwen3.5:9b-mlx release reassembled for mlx-lm, the exact weights the bridges were trained against — see The backbone in this repo
physics model	a 1D Burgers Fourier Neural Operator, 70K parameters (physics/fno_burgers_singlemode.safetensors)	frozen	this repo (also in ryoji-info/PsiLM-physics)
bridges	forward readout + value-token channel + gated injection, 28.4M parameters (bridges/qwen3.5-9b-nvfp4-mlx-1d-value-selective/)	trained	this repo (also in ryoji-info/PsiLM-bridges)

What it can answer today. One family of questions, the one the bridges were trained on:

A velocity field on the periodic domain [0,1) starts as u(x,0) = a · sin(2πx + φ). It evolves by Burgers' equation with viscosity 0.02 until t = 0.5. What is the value of u at x = x₀? Answer with a number rounded to 2 decimal places.

with a in [0.5, 1.5], φ in [0, 6.28], x₀ in [0, 0.99], two decimals each. Qwen3.5 alone answers 3.3% of them (it derives for its whole 768-token budget and, forced to commit, is usually wrong). Coupled through the bridges it answers 100% of the sixty held-out questions within ±0.05 (MAE 0.015) — above the 98.3% it reaches when the true value is written into the prompt as text, the oracle ceiling. On everything else the gate stays shut and the model is Qwen3.5 (guard-rail below).

What it is not. Not a general physics assistant, not a fine-tuned Qwen3.5, and not a model that will know when a different PDE applies: the physics model solves exactly one equation family and the bridges read exactly three quantities (a, φ, x₀) from the text. It is a research artifact: a working, measured instance of latent coupling between a language model and a physics model — here through a residual stream that is carried by recurrent state for most of the depth above the injection — on consumer hardware.

Run it

Apple Silicon Mac (the NVFP4 backbone is 8.0 GB on disk; MLX ≥ 0.32.2 reads NVFP4 natively; training of the bridges peaked at 16 GB on a 24 GB M2, inference needs less), Python 3.11+.

# 1. get this repo (8.0 GB of backbone + ≈114 MB of bridges + 0.5 MB of physics model)
huggingface-cli download ryoji-info/Qwen3.5-9B-PsiLM --local-dir Qwen3.5-9B-PsiLM
cd Qwen3.5-9B-PsiLM

# 2. dependencies (mlx, mlx-lm, transformers, torch, huggingface_hub + the psilm package from GitHub)
pip install -r requirements.txt

# 3. one command
python psilm_infer.py

Nothing else is downloaded: the backbone is the directory you just fetched, and the script uses it by default (run from elsewhere, --backbone ryoji-info/Qwen3.5-9B-PsiLM loads the same files through mlx-lm into the Hugging Face cache). The script answers the default question (a = 1.28, φ = 0.5, x₀ = 0.76) three ways and prints timing:

[1] PsiLM (coupled)     : 'u at x = 0.76 equals -0.26.'
    value               : -0.26   (20.0 s)
[2] backbone alone      : "...The characteristic time for diffusion to smooth out a gradient of size $L$ is $L^2 / \nu$.\n\nAnswer:1.28\n\nWait"
    value               : 1.28   (64.3 s, 786 tokens, answer forced)
[3] physics model (FNO) : u(0.76) = -0.2522   (8 ms; the reference the coupled answer should match to +-0.05)
    spectral solver     : u(0.76) = -0.2517   (ground truth)
PsiLM match (|-0.26--0.25| <= 0.05); backbone alone off

(Recorded on 2026-09-11 against the same weights as a local directory; the coupled arm's 20 s is the release script's uncached staged decode, the guard-rail's KV-cached decoder answers the same questions in 2.35 s.)

Other questions and options:

python psilm_infer.py --a 0.9 --phi 2.1 --x0 0.33     # any (a, phi, x0) in the ranges above
python psilm_infer.py --no-baseline                   # skip the slow backbone-alone arm
python psilm_infer.py --question-only                 # print the exact prompt, load nothing
python psilm_infer.py --help                          # --bridges DIR, --physics FILE, --backbone ID, ...

Any other copy of the same weights works too: python psilm_infer.py --backbone /path/to/qwen3.5-9b-mlx (for instance one you rebuilt yourself with eval/ollama_to_mlx.py qwen3.5:9b-mlx --out /path/to/qwen3.5-9b-mlx from an ollama pull). Without pip install-ing the package, a clone of the GitHub repository works: PSILM_REPO=/path/to/PsiLM python psilm_infer.py.

Loading the pieces yourself, in Python:

import json, mlx.core as mx
from psilm.mlx.gemma_loader import load_backbone_any     # dispatches on the config: Qwen3.5 -> psilm.mlx.qwen35_loader
from psilm.mlx.bridges import PsiBridgesMLX
from psilm.mlx.fno import load_fno_safetensors
from psilm.mlx.model import PsiLMMLX

model, stock, tok = load_backbone_any(".")   # or "ryoji-info/Qwen3.5-9B-PsiLM"
d = "bridges/qwen3.5-9b-nvfp4-mlx-1d-value-selective"
cfg = json.load(open(f"{d}/config.json"))
bridges = PsiBridgesMLX(**cfg["construct"])                # d_model 4096, channel "value", inj_cap 0.2, readout_norm "dim"
bridges.load_weights(f"{d}/bridges.safetensors", strict=False)   # the retired learned-pointer tensors are omitted
fno = load_fno_safetensors("physics/fno_burgers_singlemode.safetensors")
psi = PsiLMMLX(model, tok, fno, bridges, l_fwd=cfg["coupling"]["l_fwd"], l_rev=cfg["coupling"]["l_rev"])   # read @13, inject @26 of 32
# psi.generate(QABuilder(hf_tokenizer), {"a": 1.28, "phi": 0.5, "x0": 0.76}) -- see psilm_infer.py

What it costs and what it buys

component	parameters	on disk	trained?
Qwen3.5 9B text tower, NVFP4 (language model)	8.95B	8.0 GB	frozen
PsiLM bridges, one (backbone, task) pair	28.4M	114 MB	trained
Burgers FNO, the physics hemisphere	0.07M	0.55 MB	frozen, pretrained

The trained part is 0.32% of the backbone's parameters and 1.4% of its checkpoint size. The 8.95B never move.

Measured on one Apple M2 (24 GB): +0.32% parameters and +114 MB turn 3.3% into 100% on the physics task, at 5.7× lower latency (2.35 vs 13.5 seconds per question on the guard-rail's physics set, 16.9 vs 160 generated tokens), with GSM8K, MMLU and BoolQ unchanged — the gate's σ is 0.81 on physics against 0.004–0.014 elsewhere, so the channel is shut when physics is irrelevant. The held-out backbone figure (3.3%) is its own text protocol: it spends the whole 768-token budget deriving, is forced to answer, and is usually wrong.

Results for this backbone

Held-out evaluation, 60 questions, accuracy within ±0.05 (results/stage2_qwen35/final_eval.json in the GitHub repository):

arm	accuracy	MAE	note
Qwen3.5 9B alone, forced to answer	3.3%	0.570	derives for the whole 768-token budget on every item, then commits to a wrong number
PsiLM (this repo)	100%	0.0147	bridges read the prompt, FNO computes, value returns in latent space; largest error 0.047
oracle (true value written into the prompt)	98.3%	0.100	the tool-loop ceiling; its one miss is a forced reply whose parser picked the phase out of the derivation (5.61 for a true −0.202)
always answer 0.00	1.7%	0.308	calibration

Scored again on the numerics the bridges were trained with — the differentiable (pure-MLX) recurrent scan above the injection layer rather than the Metal kernel, final_eval_ops.json — PsiLM is again 100% at MAE 0.015, with 58 of the 60 answers identical to two decimals and the other two within 0.01.

Guard-rail: does the coupled model still do everything else? 100 questions per dataset, three arms — backbone alone / PsiLM / PsiLM with the injection zeroed — with the gate recorded per question (results/bench/guardrail_qwen35_summary.json):

dataset (n=100)	backbone	PsiLM	zeroed	gate σ (PsiLM)	open on
physics QA (this task)	1%	99%	10%	0.81	100%
GSM8K, 384 tokens	83%	83%	83%	0.004	0%
MMLU, 5 subjects, 256 tokens	64%	66%	64%	0.014	0%
BoolQ	90%	89%	90%	0.008	0%

GSM8K is item-identical across the three arms (paired: 83 both, 0 either way). MMLU's +2 is two items gained and none lost (p = 0.5), from a parse rate of 0.86 against 0.82 — the brevity effect the paper's leaky-gate section describes; BoolQ's −1 is one item (p = 1). The backbone's 1% on physics is its 160-token nudge protocol on this bench; the zeroed arm's 10% is what the reply template alone recovers, the same floor as Gemma's.

The gate selectivity comes from a no-harm training arm: 1,194 non-physics prompts (597 distinct — 400 GSM8K train, 197 MMLU validation — each with and without the Answer: nudge line) paired with Qwen3.5's own greedy continuations, on which only the gate is updated with a mean-gate penalty (config.json → training). In the training logs the gate closed on the negatives within the first no-harm chunk (0.001 at their answer positions, against 0.94 on physics prompts) and stayed shut (0.0003 at the end), while the physics rollouts went 93.8%, 100%, 100%.

Training: 11,500 steps on one Apple M2 (24 GB) — 2,000 readout-only warm-up steps at batch 8 (3.4 s/step), 8,000 coupled steps at batch 2 (lr 3e-4; 4.6–8.5 s/step at a 16 GB peak), then 1,500 no-harm steps at lr 1e-4 (batch 2; 7.7 s/step at a 35–38 GB peak, through swap — the negatives are long sequences and the recurrent scan's backward pass keeps its whole recurrence in memory). The recipes are results/qwen35/phaseA_recipe.sh and results/qwen35/coupled_recipe.sh; the coupling-depth probes that put the injection at layer 26 (a memory cliff between 16 and 19 GB, not a design preference) are in results/qwen35/probes.txt. The readout standardizes each hidden dimension with statistics from a 32-prompt calibration pass (readout_norm: "dim"; the two frozen vectors fwd.dim_mu, fwd.dim_sigma are in the checkpoint), the adjustment Gemma needed, simply left on here.

Bridges in this repository

directory	task	physics model	trained params	held-out	status
bridges/qwen3.5-9b-nvfp4-mlx-1d-value-selective/	1D Burgers, single-mode initial conditions, value at x₀	physics/fno_burgers_singlemode.safetensors	28.4M	100% @±0.05, MAE 0.0147 (n=60; oracle 98.3%); GSM8K 83/83	released

How it works, in one paragraph

The prompt runs through the first 13 of Qwen3.5's 32 layers. The forward bridge reads the queried position x₀ by pooling the hidden states over its tokens (a deterministic span pointer computed by the QA builder, plus a 100-bin classifier) and the initial-condition parameters with a learned pool, after a calibrated per-dimension standardization, and emits (a, sin φ, cos φ) and x₀; from these it builds the initial condition on a 128-point grid. The frozen FNO evolves it to t = 0.5. A learned periodic lookup kernel reads the field at x₀, and the value-token channel turns that single number into eight soft tokens through Fourier features. At layer 26 a gated cross-attention injects them into the residual stream, capped at 20% of the stream's RMS; the gate is a small MLP on the residual stream, trained to open on physics prompts and close elsewhere. Layers 26–32 and the answer are Qwen3.5's own — four of those six layers are Gated DeltaNet, so the injected value travels onward mostly through recurrent state rather than attention. Nothing in the bridges knows what kind of layer they read from or write into. Details, the coupling-depth measurement and the failure analyses that produced this design are in the paper (paper/psilm.pdf in the repository, Section 9.8 for this backbone).

Two things the adapter (psilm/mlx/qwen35_loader.py) does that the attention backbones never needed: it hands each Gated DeltaNet layer a boolean key-validity mask derived from the staged forward's additive causal mask (the recurrence would otherwise zero every real token), and it switches only the layers above the injection onto mlx-lm's differentiable recurrent scan during training, because the Metal kernel has no backward rule. The staged forward reproduces the stock model exactly (results/qwen35/setup_summary.txt: maximum logit difference 0.0, also at every real position of a right-padded row).

The backbone in this repo

The root of this repository is a loadable mlx-lm model: the exact Qwen3.5 9B weights the bridges were trained against, so that no separate download or conversion is needed. It is not a new quantization. It is Ollama's qwen3.5:9b-mlx release — NVFP4, 4-bit floats with one 8-bit (E4M3) scale per 16-value block, on every wide linear projection of the decoder — reassembled from Ollama's per-tensor blob store into a Hugging Face-style directory by eval/ollama_to_mlx.py, with the vision tower removed. No Hub NVFP4 build of Qwen3.5-9B is bit-compatible with it (the Hub builds also quantize the embeddings, the output head and the recurrence's gate projections, which this release keeps in bf16), which is why it is republished here rather than referenced.

tensors	dtype	what
200 + 200	uint32 weights + uint8 scales (NVFP4, group 16)	every wide linear projection: q/k/v/o, Gated DeltaNet in_proj_qkv / in_proj_z / out_proj, MLP gate/up/down
203	bf16	embeddings (248,320 × 4,096, untied), output head, all norms, the recurrence's conv1d, dt_bias and its two 32-wide gate projections (in_proj_a, in_proj_b)
24	fp32	the recurrence's decay parameters (A_log)

627 tensors, 7.97 GB on disk, 8.95B parameters. config.json carries the quantization stanza ({"group_size": 16, "bits": 4, "mode": "nvfp4"}) that the Ollama release leaves implicit in tensor metadata; vision_config is removed along with the 333 vision tensors. As a plain language model:

from mlx_lm import load, generate
model, tokenizer = load("ryoji-info/Qwen3.5-9B-PsiLM")      # mlx>=0.32.2, mlx-lm>=0.31.3
prompt = tokenizer.apply_chat_template([{"role": "user", "content": "What is Burgers' equation?"}],
                                       add_generation_prompt=True, enable_thinking=False)
print(generate(model, tokenizer, prompt=prompt, max_tokens=64))

Provenance and checks: ollama pull qwen3.5:9b-mlx (manifest registry.ollama.ai/library/qwen3.5/9b-mlx), upstream Qwen/Qwen3.5-9B, Apache-2.0. The converter renames Ollama's <w>.weight.scale to the <w>.scales that mlx.nn.QuantizedLinear expects, writes the quantization stanza, drops vision_tower.* and copies the licence — byte-for-byte the same weights. PsiLM's staged forward reproduces this model's stock forward exactly (results/qwen35/setup_summary.txt in the GitHub repository: maximum logit difference 0.0 at batch 1, and 0.0 at every real position of a right-padded row).

Is the answer really coming through the channel?

Two controls on the earlier backbones, and the second is decisive. Zeroing the injection while running everything else — readout, FNO, value tokens, gate — removes the physics result (0% for Qwen3-8B, 10% for Gemma, which is what the reply template alone recovers). Corrupting only the number — feeding the value encoder another question's answer at matched magnitude, with prompt, readout, gate, reply length and parsing untouched — makes the frozen model report the corruption: the spoken answer lands within ±0.05 of the injected value on 99 of 100 held-out questions and within ±0.05 of the truth on 9, while the KL to the base model is unchanged. Run on non-physics prompts the same swap changes nothing, which separates what the channel does by its presence from what it does by its content. Records: results/bench/leaky_8b_shuf_guardrail_summary.json and §9.7 of the paper. The zeroed arm of this backbone's guard-rail (table above) repeats the first control here.

Limitations

• One task family. The bridges read exactly the three quantities of the trained question and the FNO solves exactly one equation family; a different PDE, boundary condition, viscosity or final time is out of scope, and the gate closing on non-physics text does not mean it can recognize other physics. Free-text initial conditions are not supported.
• The pointer is task-supplied. Which tokens hold x₀ is computed by the QA builder from the prompt (QABuilder.x0_span), not learned from the words. psilm_infer.py builds the prompt itself for that reason; a paraphrased question is not the trained input.
• This exact backbone. The bridges were trained through these NVFP4 weights and expect them: the ones at the root of this repo, or a directory rebuilt from Ollama's qwen3.5:9b-mlx with eval/ollama_to_mlx.py. No other Qwen3.5-9B quantization on the Hub is bit-compatible (they also quantize the embeddings, the output head and the recurrence's gate projections, which this one keeps in bf16); the script refuses a backbone of another width or depth. MLX's quantized kernels are not guaranteed bit-identical across mlx versions or Apple chips, so a given question can land a hundredth away from the recorded run; the numbers above are from mlx 0.32.2 / mlx-lm 0.31.3 on an M2.
• Two scan paths. Training used mlx-lm's pure-MLX recurrent scan above the injection (the Metal kernel has no backward rule); inference uses the kernel. The two differ by 1.4e-2 in maximum relative logit difference with identical argmax; on the held-out set they give the same 60 answers to two decimals on 58 items and differ by at most 0.01 on the other two.
• Thinking is off. The backbone is driven with enable_thinking=False, and its chat template then opens every assistant turn with an empty <think></think> block; answer positions and token budgets count from after it. With thinking on, nothing here has been measured.
• Inputs with two decimals, inside the training ranges. The readout was trained on numbers formatted like the training set; psilm_infer.py rounds inputs to two decimals and warns outside [0.5, 1.5] × [0, 6.28].
• Evaluation scope. The guard-rail covers GSM8K, a five-subject MMLU slice, BoolQ and this physics set at n=100 each (results/bench/guardrail_qwen35_guardrail_summary.json); nothing else has been measured, and the leaky-gate sweep of the paper has not been run on this backbone.

Beyond physics

The bridges here couple a frozen language model to a frozen physics model, but the recipe (read a fixed set of quantities from text; let a frozen quantitative model compute; return one value through a selective gate) is not specific to PDEs. A calibrated market or event-probability model in the physics model's seat would be the same architecture, and the appeal is the same: a language model's forecast grounded in a model that can be validated separately, with a gate that stays shut when the model does not apply. Nothing in this repository has been trained or tested on financial data; the physics results relied on exact oracles, deterministic targets and no distribution shift, none of which markets provide. This is a research direction, not a capability, and not a basis for investment decisions.

Files

psilm_infer.py          the one-command CLI (PsiLM / backbone alone / physics model)
requirements.txt        pip dependencies, including the psilm package from GitHub
bridges/qwen3.5-9b-nvfp4-mlx-1d-value-selective/
    bridges.safetensors     the trained bridges (28.4M params, fp32, 114 MB)
    config.json             construction, coupling depths, training record, per-chunk held-out scores
physics/fno_burgers_singlemode.safetensors   the frozen FNO (torch key names; complex spectral weights as .real/.imag)
MANIFEST.md             what each file is and its sha256
psilm-banner.png        banner
README.md               this card

Related

• Code, training scripts, evaluation records and the paper: github.com/ryoji-info/PsiLM
• Bridges for every backbone (Qwen2.5-0.5B, Qwen3-1.7B, Qwen3-8B, Gemma 4 12B, Qwen3.5 9B): ryoji-info/PsiLM-bridges
• The frozen physics models: ryoji-info/PsiLM-physics
• The Gemma 4 12B release, same layout: ryoji-info/Gemma-4-12B-PsiLM

Support

PsiLM is independent research, run on a single Apple M2. If it is useful to you, you can support the work at ko-fi.com/ryojifurui.

Citation

@misc{furui2026psilm,
  title  = {PsiLM: Coupling Frozen Language and Physics Models through Trainable Latent Bridges},
  author = {Furui, Ryoji},
  year   = {2026},
  url    = {https://github.com/ryoji-info/PsiLM},
  note   = {Research generated by Claude (Anthropic) under the author's direction}
}

License

The bridges, the FNO and the code in this repository are released under Apache-2.0. The backbone at the root is a repackaging of the Qwen Team's Apache-2.0 Qwen3.5 release (Qwen/Qwen3.5-9B) as shipped by Ollama; its licence text is in LICENSE.

AI generation disclosure

This model, its training recipe, the evaluations and this card were generated by Claude (Anthropic; the Opus 5 and Fable 5.1 models, on the recipe the earlier backbones' campaigns established), operating as an autonomous research agent under the direction and review of Ryoji Furui, who set the research question and the hardware constraint, approved each stage, and bears responsibility for the published claims. All numbers on this card are taken from committed evaluation records in the repository, cited by file name above.