Alpha BLAH Jacobian Lens
This is a blah-jacobian-lens v1 artifact consumed by evals.blah.dev. It is fitted to exactly ajaxdavis/alpha-60m-chat at immutable revision ab1c5be13a12c0feb2d5e2c9af89bd5924a0e8b0 and will not be applied to a checkpoint with a different native weight fingerprint.
Architecture and sites
Alpha is a custom TypeScript tensor/autograd model with a native Helios Vulkan backend. The adapter does not replace that implementation. It captures one token-aligned post-residual representation after each complete decoder block. The target is the final post-block representation immediately before the exact final RMSNorm and tied token-embedding projection. Every published site has the target width and basis, so ordinary Logit Lens decoding is valid.
The bundle stores 16 dense matrices as J[output_dimension, input_dimension], plus a source mean for every site and one target mean. Application is affine: target_mean + (h - source_mean) @ transpose(J), followed by Alpha's exact final decoding path. No low-rank approximation is used.
This is a readout instrument, not a sparse autoencoder: it contains no learned sparse dictionary, sparsity objective, dead-feature handling, or feature interpretation. It also does not establish global-workspace, broadcast, ignition, persistence, or causal-necessity claims. Those require separate interventions and controls.
Fit
- Corpus: alpha-semantic-v4-reviewed-synthetic (alpha-semantic-v4-reviewed-synthetic-fit-100, revision sha256:fd0573038f787a922af9d48fa0228cea39c56a8604025019b38d69b5524cae6f)
- Visibility: synthetic
- Prompts / tokens: 100 / 8267
- Maximum sequence length: 128
- Leading positions excluded: 16
- Native fitting backend: helios
- Fitting dtype: float32; artifact dtype: float16
- Hugging Face model.safetensors SHA-256: sha256:a5214ebad501b8bd3b09f7552c0db67417d18c3b66432f66f847de0e723dd688
- Estimator: dimension-batched native VJP with prompt- and dimension-specific Rademacher position probes; matching source-position signs give an unbiased Hutchinson estimate of the mean same-position Jacobian while cancelling cross-position causal terms in expectation
- Estimator kind: same_position
- Centering: affine source/target activation means
See fit-report.json and validation.json for convergence, parity, and finite-difference measurements.
The published same-position matrix is estimated with one deterministic Rademacher position probe per fitting prompt and output dimension. This gives an unbiased estimate of the mean diagonal position Jacobian while keeping the native fit tractable; it is not an exhaustive enumeration of every position's full Jacobian.
Runtime
Execution mode: remote_http. The public blah-lens-http/1 runtime is https://alpha.donto.org/lens-runtime.
The runtime uses the exact native tokenizer and chat template, preserves exact token IDs and unprettified vocabulary items, and includes authoritative bytes_base64 for every byte-BPE token. That byte side channel preserves identity even when an isolated vocabulary item is not valid UTF-8; the visible GPT-2 surrogate spelling remains display-only. The runtime checks the checkpoint fingerprint before loading transports and returns top-k readouts rather than full vocabulary logits. Full completion text is decoded only after the complete token sequence is assembled.
Reproduce
From the Alpha source revision recorded in fit-report.json:
alpha lens fit --checkpoint=<native-checkpoint> --prompts=fit-prompts.jsonl --samples=100 --max-seq-len=128 --skip-first=16 --dim-batch=32 --estimator-kind=same_position --dtype=float16 --checkpoint-every=5 --output=dist/blah-lens
The model and this artifact use the apache-2.0 license. Fitting prompt provenance and visibility are recorded in fit-report.json. The license-safe synthetic fitting prompts are included as fit-prompts.jsonl for exact reproduction.