echoforest/market-psych-classifier-v1

Market-psychology classifier v1 (Qwen3.5-9B-Base, LoRA and 10 heads)

A research checkpoint. It labels one X/Twitter post, with its local context, on 10 market-psychology fields in a single forward pass.

Architecture: a LoRA adapter on Qwen/Qwen3.5-9B-Base (text decoder only), plus one linear head per field.

Pooling: the heads read the final-norm hidden state at the last non-padding token of the whole input. That is always the last token of the \n### LABELS suffix, which classifier.formats.encode appends after the target post, its context posts and the FACTS block. It is not the end of the post text. Batches are right-padded.

Training labels: Astra (gpt-6-astra, medium reasoning) teacher labels on 30,227 posts from Oct 12–15, 2026. The training data is not included here.

Files

The layout is the run directory written by classifier.train:

Path	What
best/adapter/	PEFT LoRA adapter (r 32, alpha 32), base Qwen/Qwen3.5-9B-Base
best/heads.pt	torch.save state dict of the 10 linear heads (4096 → K per field)
config.json	Training settings, library versions, source commit
taxonomy.json	Fields and options (logit order), hard rules, gate order
temperatures.json	Per-field temperatures fitted on dev
metrics_dev.json, train_log.jsonl	Dev metrics at the selected step; training log
scores/	Local evaluation outputs (aggregates and internal record IDs only)
check_env.json	Pod environment: GPU, versions, fast-kernel check
code/classifier/	Inference and training code (HANDOFF.md first); code/SOURCE_COMMIT.txt names the project commit
SHA256SUMS.txt	Checksums of every file except this card and itself

The adapter's base_model_name_or_path is the pod's local path. Point --model at a local copy of Qwen/Qwen3.5-9B-Base.

Use

Read code/classifier/HANDOFF.md before serving. This repo carries the code it needs in code/classifier/. That folder is a copy of the project's evaluation/market_psych_v2/classifier/ at the commit in code/SOURCE_COMMIT.txt, without tests. The code that trained the model is the tree named in config.json; its Python modules are the same.

What the code does, so nothing needs to be guessed:

1. formats.encode: an evidence-v1 row's segments, in stored order (target post, its context posts, FACTS), become token ids. It applies segment-wise truncation to 2,048 tokens and appends the \n### LABELS suffix.
2. model.pad: right-pads, then pools at the last real token of the whole input, the suffix's last token.
3. calibrate.gated: applies the per-field temperatures, then the hard-rule gate in taxonomy.GATE_ORDER.

If the input format is guessed, the labels are wrong without any error.

Rows: evidence-v1 rows are built by the project's exporter (evaluation/market_psych_v2/classifier_export.py), not on the server.

Check before use: predict the project's 1,100-post benchmark.jsonl and compare the result with the project run's reference logits (HANDOFF section 5).

Running it (from code/):

python -m classifier.predict --run .. --model <Qwen3.5-9B-Base dir> --input rows.jsonl --out logits.jsonl

• Output: raw logits per field, in taxonomy.json option order. Keep them.
• Labels: the argmax of each gated, calibrated head:

  import json
  from classifier.taxonomy import Taxonomy, argmax
  from classifier.calibrate import gated
  tax = Taxonomy.load('../taxonomy.json'); temps = json.load(open('../temperatures.json'))
  labels = argmax(gated(row_from_logits_jsonl, tax, temps))
  

Runtime environment: torch 2.8 (CUDA 12.8), transformers 5.20.1, peft 0.21.2, flash-linear-attention 0.5.2 and causal-conv1d 1.7.0 (code/classifier/requirements-train.txt).

Served check (2026-10-17): the node12 server (mpsy-server) passed the benchmark check.

• 3 near-tie label flips in 11,000 decisions.
• The largest probability difference was 0.020.
• Token counts were identical.

Training

• Run size: 2 epochs, 1,890 steps of 32 posts each, 1.04 h on one H100 SXM. The selected checkpoint is step 1,890, with dev mean field log-loss 0.164.
• Optimiser: AdamW. The learning rate is 2e-4 for LoRA and 1e-3 for the heads, on a cosine schedule with 3% warmup.
• Precision and length: bf16, max length 2,048 tokens.
• Loss: per-step masked cross-entropy, with no class weights.

Evaluation (pre-registered)

Data: a 1,100-post benchmark held out from training; d3 is compared on its 1,000 calibration posts.

Scoring: labels are the argmax of the gated, calibrated heads. Intervals are 95% paired author-group bootstrap intervals.

Class	F1, this model	F1, d3	Difference [95% CI]
Bullish	0.758	0.611	+0.148 [0.034, 0.265]
Positive emotion	0.798	0.430	+0.368 [0.275, 0.463]
Negative emotion	0.642	0.531	+0.110 [−0.071, 0.279]
Crowd reference	0.704	0.429	+0.275 [0.140, 0.417]
Primary population	0.902	0.649	+0.253 [0.206, 0.304]
Relevance	0.933	0.873 (P ≥ 0.05 gate)	+0.060 [0.035, 0.085]

Acceptance items:

Item	Result
1. Beats d3 (4 of 5 classes above 0, none below)	Pass. Without the 209 overlap posts, bullish's interval reaches 0: 3 of 5 classes
2. Relevance above d3	Pass
3. Calibration (ECE ≤ 0.05; key-class shares within 2 points)	Pass. ECE 0.013–0.016; share gaps ≤ 0.5 points
4a. Determinism (dev re-predicted from the checkpoint; benchmark repeated)	Pass (identical)
4b. Robustness (≤ 3 of 60 changed decisions per set)	Fail. Injection 30/60, context attack 12/60, date shift 0/60

Agreement with the teacher (100 pilot posts, 1,000 labels): 95.4% with Astra r1. For comparison, Astra r1 and r2 agree on 97.2%.

Learning curve (8k posts against the full set): mean macro-F1 over relevance, direction, valence and crowd rises from 0.648 to 0.758. That is +11.1 points, interval [2.3, 14.8].

Limitations

• Robustness failure: instruction-like text in the post or its context changes the labels, mostly from "not relevant" or "not evaluable" toward relevant, evaluable or bullish. The model has seen no adversarial examples. Treat its output on spam-like posts with care.
• Training window: the labels come from one teacher on Oct 12–15, 2026 posts. Performance on later weeks is not yet measured.