naver-psy-classifier-gemma4-12b-screen-b-mean

LoRA adapter and per-field linear heads on the Gemma 4 12B text decoder for the nsb-v2 taxonomy of Naver 종목토론실 posts (naver-psy, docs/classifier-plan.md).

• Agreement figures are agreement with the teacher labels, not accuracy against ground truth.
• Input: nsb-input-v1 render, segments joined with newlines, suffix ### LABELS, one BOS.
• Output: raw logits per field; calibrated with temperatures.json, then gated (training/taxonomy.py).
• No training data, predictions or post text are in this repository.

Setting	Value
arm	B-mean
checkpoint	base
pool	mean
epochs	1.0
max_len	2048
lora_r	32
lora_alpha	32
lr_lora	0.0002
lr_heads	0.001
seed	20261017
data_version	v20261019-1009
data_manifest_sha256	b3bf46a948289240f59e08ca95c654022a6f748636d483540e8c4d46b35ed8c7

Metrics

metrics_val

{
 "step": 250,
 "selection_loss": 0.2983155878000662,
 "log_loss": {
  "relevance": 0.09018880741980341,
  "target": 0.294391183402869,
  "direction": 0.43876729825393007,
  "sarcasm": 0.2049240072556455,
  "behavior": 0.2270070845821015,
  "pnl": 0.14452966331089212,
  "emotion_fine": 0.6938292753210247,
  "crowd": 0.40611958800298387,
  "commercial": 0.0281342175210785,
  "emotion_valence": 0.45526475293033297
 },
 "accuracy": {
  "relevance": 0.9681486396814863,
  "target": 0.8907945397269863,
  "direction": 0.8330379298121233,
  "sarcasm": 0.9376968848442422,
  "behavior": 0.9156457822891144,
  "pnl": 0.9488974448722436,
  "emotion_fine": 0.7325866293314666,
  "crowd": 0.8456422821141057,
  "commercial": 0.9923689449236894,
  "emotion_valence": 0.8092404620231012
 },
 "rows": 3014,
 "eval_rows": 2237,
 "screen_primary": 0.17661745790850816
}

screen

{
 "winner": "B-last",
 "beats_default": [],
 "scores": {
  "B-last": {
   "primary": 0.1703279423207186,
   "secondary": 0.7503783758084439,
   "eval_rows": 2237,
   "run": "/workspace/runs/screen-B-last"
  },
  "I-last": {
   "primary": 0.1879070587120423,
   "secondary": 0.6152050313786558,
   "eval_rows": 2237,
   "run": "/workspace/runs/screen-I-last"
  },
  "B-mean": {
   "primary": 0.17661745790850816,
   "secondary": 0.6749563706026908,
   "eval_rows": 2237,
   "run": "/workspace/runs/screen-B-mean"
  },
  "I-mean": {
   "primary": 0.20586273438668987,
   "secondary": 0.5690600738310523,
   "eval_rows": 2237,
   "run": "/workspace/runs/screen-I-mean"
  }
 }
}

Screen arm

This is one of four screening arms (classifier-plan §4.4), not the production model: seeded 8,000-post subset of the nsb-v2 training split (bundle v20261019-1009), 1 epoch, scored on the 2,237 random-sample validation posts. temperatures.json was fitted on this arm's validation predictions. The screen chose B-last (screen.json); the production model is echoforest/naver-psy-classifier-gemma4-12b (B-last, all 27,118 training posts, 2 epochs).