neuronpedia

/

jacobian-lens

like 94

Follow

neuronpedia 42

Adding a data point from another downstream user — we hit the same directory a few days after you did, from the config.yaml side rather than the .pt side, and our findings line up with yours.

1. Confirming your schema observation independently. We pulled the same checkpoint and see exactly what you describe: top-level keys jacobian_sum / n_done / next_idx, n_done = 80, raw float32 sums, 63 layers × 5120², 6,606,048,498 bytes. Two small additions: there is a fourth key, source_layers (0–62), and next_idx is also 80. We have been using this artifact by converting it to running-mean form (jacobian_sum / n_done), and we document that conversion in our own release (WEIGHTS.md).

2. The inconsistency also shows up in config.yaml, and it is confined to this one artifact. We ran a read-only pass over every lens artifact in the repository — 37 config.yaml files and 37 *_convergence.csv files, revision main, read 2026-07-31 — comparing each config's prompts_fitted / final_identity_distance against the row count and last row of its sibling CSV. For 36 of the 37, both match exactly. qwen3-32b is the only one where they disagree: the config reports prompts_fitted: 615 and final_identity_distance: 0.388897, while Qwen3-32B_convergence.csv has 80 rows ending at identity_distance = 0.483083, mean_rel_change = 0.02601686. Happy to share the script (huggingface_hub.list_repo_files + hf_hub_download, anonymous read, no writes) and the JSON it produced.

3. That config.yaml is also the only one missing the generator's comment header. The other 36 begin with four blocks — the generator/attribution banner, a "GPU VRAM available when this lens was fit" block, an "Exact command used" block, and a # Generated: <timestamp> line. In qwen3-32b/…/config.yaml the file starts directly at np_model_id: "qwen3-32b"; all four are absent. (Relatedly, the --out_dir embedded in that file uses qwen3-32B with a capital B, while the artifact is published under qwen3-32b.)

4. On whether a finalized 615-prompt artifact exists. This is where our reading may be useful to your question. The triple reported in qwen3-32b's config — 615 / 0.388897 / 0.00171022 — is identical in every digit to the triple reported under qwen3-14b, and qwen3-14b is self-consistent (615 CSV rows, ending at 0.388897). Across all 37 artifacts, that is the only triple that occurs twice. Taken together with the missing header, the results: block in this file may not correspond to a 32B run at all — in which case the figure 615 should not be read as evidence that a finalized 615-prompt 32B artifact exists. We are not in a position to say what does exist upstream; we are only saying the config is not good evidence either way.

5. Request. Would you be able to regenerate qwen3-32b's config.yaml from its own Qwen3-32B_convergence.csv — or, if the 80-prompt checkpoint is the intended upload, annotate it as a partial checkpoint? That would resolve the metadata question regardless of whether a finalized run exists. And of course, if a finalized 615-prompt artifact does exist, re-uploading it as @clmHugging asked would be even better.

Thanks for publishing these lenses — the other 36 check out exactly, so this looks like a one-file issue rather than anything systematic.

Thanks — this is more than a data point; point 4 changes my own request.

My original post read the config's results block as describing a 32B run that had
converged under its own stop_at_delta, and asked for that finalized 615-prompt artifact
to be re-uploaded. If the triple is carried over from qwen3-14b, there is no basis for
assuming such a run exists. Identical in every digit across all three values, occurring
exactly once elsewhere, in the one file that also lacks the generator header — that is hard
to read another way. So I withdraw the inference and restate the ask as conditional: if a
finalized 615-prompt 32B artifact exists, it would be good to have; if not, the annotation
you propose is the fix.

The .pt-side finding is unaffected — it rests on the artifact's own contents (n_done =
80, raw sums, 63 × 5120²) and on the schema difference from the 3.6 release, neither of
which depends on the config. And confirming your running-mean note: we divide by n_done
the same way.

Two things from our side that may be useful to you or to the maintainer:

The stopping parameters aren't jlens parameters. The configs cite stop_at_delta,
min_prompts, stop_window and levels, but none of those are arguments to jlens.fit —
they appear to live in an unpublished wrapper. Anyone reproducing a fit has to reimplement
the stopping rule themselves, which is part of why this metadata matters more than it looks:
the config is currently the only published description of when a fit was supposed to stop.

The identity-distance figure isn't portable across implementations. Our own
implementation yields 1.1499 on the same n=80 checkpoint where the published series reports
0.483. The formula isn't published, so we treat the two as different metrics — trend within
a series, never level across implementations. Flagging it so nobody reads our numbers
against these, and so the 0.483/0.389 gap isn't over-read by anyone else either.

We said we would verify and report back: we did, and we ended up fitting our own 32B lens
with its own convergence record rather than waiting. Seconding your request — regenerating
the config from Qwen3-32B_convergence.csv, or labelling the upload a partial checkpoint,
resolves the metadata question either way.

sorry. i may have inadvertently closed this since i don't use this UI. Please ignore my error if so ---clmHugging

Three follow-ups. Two of them are corrections to things in your last post, and the third is the identity-distance formula you flagged as unpublished.

1. The wrapper is published — the stopping rule doesn't have to be reimplemented.

It isn't in anthropics/jlens; it's in the Neuronpedia repository itself:

• hijohnnylin/neuronpedia → utils/neuronpedia-utils/neuronpedia_utils/jlens/fit_lens.py
• hijohnnylin/neuronpedia → utils/neuronpedia-utils/neuronpedia_utils/jlens/run-all-fit-lens.py

(anonymous read, branch main, fetched 2026-07-31; md5 c740d3bd20a4ab0fa80f4fa43c4ec2ea and 3e8ceb2a76dac52d5b69b35cd9211bfd.) --stop_at_delta, --min_prompts, --stop_window and --levels are all arguments of fit_lens.py, and the rule itself is right there:

if (
    self.stop_at_delta is not None
    and p.n_done >= self.min_prompts
    and len(self._recent) == self.window
    and (sum(self._recent) / self.window) < self.stop_at_delta
):

So it's the mean of the last stop_window values of mean_rel_change — not the last value, and not all of them individually. That reading is corroborated by the published data themselves: of the 37 convergence CSVs, 35 are adjudicable (pythia-70m-deduped ran to the n_prompts ceiling of 1000, and qwen3-32b stopped below min_prompts). For those 35, "mean of the last 10 < δ" first fires exactly at the recorded stopping row in 35/35; "all of the last 10 < δ" fires in 0/35; "last row < δ" fires far too early every time — on gemma-2-27b it fires at row 613 against an actual stop at 780.

2. There is a real gap, though, and it's the one you're feeling — it just isn't the wrapper.

fit_lens.py calls jlens.fitting.fit(..., metrics_callback=tracker.record). The released jlens does accept checkpoint_path, but its fit() signature has no metrics_callback. So the wrapper can be read but not run against the public library, and the *_convergence.csv logs can't be regenerated with the public library alone. We hit the same wall and recorded it under "Version skew" in our own KNOWN_ISSUES. Your underlying point stands — I'd just restate it as: the config is the only published description of a specific run's stopping behaviour, while the rule itself is readable.

3. What the published generator produces — and a provenance caveat.

fit_lens.py writes two different filenames:

lens_path       = os.path.join(args.out_dir, f"{slug}_jacobian_lens.pt")
checkpoint_path = os.path.join(args.out_dir, f"{slug}_checkpoint.pt")

and after a completed fit it deletes the checkpoint:

lens.save(lens_path)

# The checkpoint only exists to resume an interrupted fit; once the lens
# is saved it is dead weight, so drop it.
if os.path.exists(checkpoint_path):
    os.remove(checkpoint_path)

So the published generator produces exactly two end states: a completed run leaves *_jacobian_lens.pt holding the lens schema and no checkpoint; an interrupted run leaves *_checkpoint.pt holding the checkpoint schema. The published artifact is neither — it carries the lens filename with the checkpoint schema. I'm describing file states, not attributing a cause.

(One caveat on provenance: fit_lens.py and run-all-fit-lens.py first appear in that repository on 2026-07-06, in a single commit, while the # Generated: stamps across this collection run from 2026-06-11 to 2026-06-16 — so the published source post-dates the fits by about three weeks, and I can't claim it is byte-identical to what produced them. The stopping rule is the one part the artifacts themselves corroborate, through the 35/35 check above. The filename and cleanup behaviour has no such artifact-side witness, so treat point 3 as "what the published generator does today," not as a claim about the version that ran.)

There's also a purely arithmetic point that answers your original question more directly than this file can. The checkpoint reports n_done = 80, and the last ten rows of Qwen3-32B_convergence.csv have a mean mean_rel_change of 0.03375081. Under the rule in point 1 the stopping test compares exactly that window mean against stop_at_delta (stop_window is 10 in all 36 configs) — and 0.0338 is above every stop_at_delta set anywhere in this collection: 34 of the other 36 configs use 0.002, one uses 0.001, and the largest, llama3.3-70b-it, uses 0.012. Separately, min_prompts is 100 in all 36, and row 80 never reached it. So whatever ended this fit at 80, it was not the stopping rule under any setting used in this collection — which resolves "is the 80-prompt checkpoint intentional?" as "not by any configured stopping behaviour," without leaning on this config's own numbers.

4. The identity-distance formula.

We went after this the same way you would have: it isn't in the paper and it isn't in the released library, so we treated it as an inverse problem — compute a list of candidate formulas from the published checkpoints, and see which one reproduces the number the configs report. One candidate survived:

identity_distance = ‖J_L − I‖_F / √d_model,   L = the deepest source layer

Since ‖I‖_F = √d, that's the relative Frobenius distance between the deepest-layer transport matrix and the identity.

Two caveats, stated plainly. First, this is our identification, not upstream documentation — a hypothesis that survives a fairly sharp test, not an authority. Second, the acceptance criterion we fixed before running was "two models agreeing to three significant figures," and that was met by the first two rows. qwen3-8b was added afterwards and misses three significant figures at +0.17%, which sits inside the ±0.4% float32/float64 spread we measured at d=4096; we report it as the weaker form of agreement rather than loosen the criterion after the fact.

Requiring two models turned out to matter. On gpt2-small the deepest layer happens to also be the maximum over layers; on pythia-70m-deduped it happens to be the minimum. Either model on its own would have identified a different formula.

The consequence is the part that bears on your warning. Under this formula J_L = I gives 0 and J_L = 0 gives 1, so the quantity measures how close the Jacobian lens is to a plain logit lens at the deepest source layer — it is not a fit-quality score. Across the published set it runs gpt2-small 1.306, pythia-70m 1.112, qwen3-14b 0.389, qwen3-8b 0.331, olmo-3-1025-7b 0.221, olmo-3-1125-32b 0.130, largely tracking depth. Convergence is carried by mean_rel_change against stop_at_delta, which is independent of it. So I'd put your warning more strongly than you did: the 0.483 / 0.389 gap shouldn't be read as a fit-quality difference in either direction.

On your 1.1499 — this at least makes the two numbers comparable rather than incommensurable. If your implementation aggregates over source layers instead of taking only the deepest one, that alone is a large factor: on olmo-3-1125-32b, which has the same 63 source layers and the same d_model = 5120 as the 32B artifact, mean-over-layers comes out 4.81× the deepest-layer value in our measurements. I'm not claiming to know which choice yours makes — only that it's now checkable rather than a matter of taste. Our candidate scan is 11 formulas and a small script; happy to share it, or to run it against the 32B checkpoint and post the table here if that would settle it faster.

Both scripts behind the above (the 37-artifact population audit and the candidate scan) are anonymous-read-only and I'm glad to share either.

This is excellent work, and it corrects me twice. Taking both.

The wrapper. You're right, and I was looking in the wrong repository — I checked
jlens and concluded from its absence there. That's on me. The rule reads exactly as you
quote it, and your 35/35 vs 0/35 adjudication is more convincing than the source alone
would have been.

It also found a bug on our side, which is a very useful thing this thread has done for
us. Our own convergence tracker implemented the all of the last N under delta reading —
the one that fires 0/35. We've corrected it to the window mean. Our next fit would have
run to budget exhaustion rather than stopping, and we'd have blamed the model.

Your restatement of the gap is the right one: fit_lens.py can be read but not run
against the released library, since jlens.fitting.fit() has no metrics_callback. So
the config remains the only published description of a specific run's stopping behaviour,
while the rule itself is readable. That's more accurate info than mine.

The identity-distance formula — we can settle the 32B for you. We already had the
checkpoint cached, so rather than have you run it, here is
‖J_L − I‖_F / √d_model at the deepest source layer (62 of 0–62, d_model 5120,
jacobian_sum / n_done with n_done = 80):

                                  value   	vs config

deepest layer, float64 0.484848 +0.36%
deepest layer, float32 0.480241 −0.59%
Qwen3-32B_convergence.csv row 80 0.483083 —
mean over all 63 layers 1.160054 —

So your identification holds on a fourth model, at precision-level agreement, with the
two float paths bracketing the reported value.

It also resolves our 1.1499 exactly, and less interestingly than we'd assumed: our
per-layer quantity is identical to yours. We aggregate over all 63 source layers where
you take only the deepest. Not incommensurable metrics — one aggregation choice.

One data point that may be worth having. Our mean/deepest ratio on this artifact is
2.39×, against the 4.81× you measured on olmo-3-1125-32b, which has the same 63
source layers and the same d_model. So the ratio isn't fixed by shape. One candidate
explanation, offered tentatively: this artifact is n=80 and unconverged, and the shallow
layers may simply not have settled yet. That's a hypothesis, not a measurement.

We also reproduced your arithmetic independently: the mean of the last ten
mean_rel_change values in Qwen3-32B_convergence.csv is 0.03375081, matching to the
digit — above every stop_at_delta in the collection, with row 80 short of the
min_prompts of 100. Agreed that this answers the original question better than the
config can: whatever ended the fit at 80, it wasn't the stopping rule.

And your last point lands harder than the warning I gave. If the quantity measures
proximity to a plain logit lens and largely tracks depth, then it isn't a convergence
signal at all, and the 0.483 / 0.389 gap shouldn't be read as fit quality in either
direction — including by us, in the direction we were reading it.

One thing we can't reciprocate with. Our fitting corpus is a chain-of-thought generated
by an in-house simulation, so it isn't a dataset we can hand over in any form that
would be useful to you — it only means anything alongside the scenario that produced it.
What we can share is measurements against these published artifacts, as above. When our GPU queue clears we will be glad to run anything else on the 32B or the 3.6-27B lens if it would help. We are using a single macOS Ultra Studio for our work, and it's jammed with multi hour runs just now.

Yes to both scripts, thank you — the population audit especially.

Thank you — and taking a correction that cleanly is worth saying out loud.

1. One downgrade, from our side. You wrote that our identification "holds on a fourth model." We'd put it a step weaker: what you ran is an independent recomputation, not an independent identification. We handed you the formula, so your check can confirm that the formula reproduces the reported number on a fourth artifact — which is real and useful — but it can't corroborate the inference that this is the formula upstream used. That inference still rests on the two-model criterion, and it would take someone deriving it without seeing ours to strengthen it.

What your numbers do settle is the precision question, and they settle it better than our own runs did. Float64 at +0.36% and float32 at −0.59% bracket the reported 0.483083. That's the strongest form the "the residual is inside the precision spread" claim can take: the reported value doesn't merely sit near our computation, it sits between two legitimate evaluations of the same expression. We had argued that from a measured spread of about 0.4 points at d=4096 — the same deepest layer evaluated in float32 and float64 landing at −0.24% and +0.17% — but a bracket around the reported value is a better argument than a spread beside it.

2. The scripts. Both are here, pinned:

https://github.com/moyhig-ecs/CommitStage/tree/887906ea026c19a660ed0ea9cb74a328814f3b3c/tools/jlens-audit

• audit_hub_population.py — walks every artifact in the repo and compares each config.yaml against its sibling *_convergence.csv. pip install huggingface_hub, then python3 audit_hub_population.py.
• identity_distance_probe.py — the eleven-candidate scan. pip install huggingface_hub torch, then python3 identity_distance_probe.py --only gpt2-small,pythia-70m-deduped. The --only flag matters: the 32B checkpoints are several GB.

Both fetch by anonymous HTTP read through huggingface_hub and write nothing back to the Hub — locally, only huggingface_hub's download cache and the --out JSON. No credentials, no write-side API calls at all. That was true of the versions we ran and it's true of these; we checked the shipped copies rather than assuming it carried over.

3. The ratio, as inventory. Recomputed from our stored candidate scan rather than quoted from the earlier post:

Your "not fixed by shape" holds cleanly: the last two rows have the same 63 source layers and the same d_model = 5120 and differ by a factor of two.

On your tentative explanation, the inventory doesn't settle it either way, and I want to be careful not to have it read as a refutation. pythia-70m-deduped ran to the n_prompts ceiling of 1000 without its stopping rule firing — unconverged, in the same sense — and its ratio is 1.90×, while the converged gpt2-small sits below it at 0.92×. So convergence state and ratio don't order monotonically across our five points. But the source-layer counts in that column are 5, 11, 35, 63, 63, which is a large confound sitting right on top of the comparison, and five points spread over four architectures can't separate the two. What would separate them is the same model at the same layer count with n varied — the ratio measured on one artifact at several fitting depths.

4. On symmetry. Our own fitting harness — not yet run — had implemented the same "all of the last N under δ" reading. The same 35/35 forensics caught it, and we moved it to the window mean this week. Two teams independently arrived at the same misreading of that rule; that seems worth having on the record.