echoforest/yt-chat-qwen3.5-9b-screen-gen-instruct

YouTube live-chat market classifier: Qwen3.5-9B, screening arm gen-instruct

Screening checkpoint, not a production model. This is one of four arms trained on the same 15,000-turn sample for one epoch to choose the configuration for the full training run. Scores are on a fixed 4,000-turn validation sample, before calibration. The model classifies one author's turn in Korean bitcoin-futures YouTube live chat into eight labels.

• Base model: Qwen/Qwen3.5-9B (instruct checkpoint), text decoder only.
• Arm: LoRA adapter on the text decoder that writes eight field: option lines. Labels are scored, not sampled: for each field in order, the softmax of the next-token logits restricted to that field's option first tokens gives the distribution, and the argmax option is appended before the next field. Eight forward passes per turn.
• Labels: stored reference labels from an LLM labeller. Every score measures agreement with those labels, not ground truth.

Screening results

Correct direction views at the lowest gate τ with ≥80% val precision, and mean field agreement, on the same 4,000 val turns. Both backbones were screened with the same recipe and data; the production model is the Qwen3.5-9B heads-instruct arm retrained in full (echoforest/yt-chat-classifier-qwen3.5-9b).

Arm	Qwen3.5-9B (this screen): correct views / agreement	Gemma 4 12B: correct views / agreement
heads-base	707 / 86.4%	936 / 88.3%
heads-instruct	717 / 86.2%	0 / 68.9%
gen-base	705 / 86.6%	903 / 88.6%
gen-instruct (this model)	624 / 86.1%	876 / 88.5%

This arm: mean field agreement 86.1%; all eight fields identical on 42.5% of turns; direction views: 624 correct of 780 accepted (precision 80.0%, recall 45.5%) at τ 0.63.

Per-field val agreement: relevance 94.0 · direction 77.5 · direction_basis 80.2 · target 89.4 · own_position_or_trade 92.8 · pnl_state 96.7 · expressed_emotion 72.1 · sarcasm 85.7.

Fields

Field	Options (in order)
relevance	market, off_topic, unclear
direction	bullish, bearish, rangebound, mixed, none, unclear
direction_basis	explicit_forecast, own_position, recommendation_to_others, wish_or_hope, reaction_to_move, not_applicable
target	bitcoin, ethereum, named_altcoin, altcoins_general, crypto_market, traditional, multiple, none
own_position_or_trade	long, short, took_profit, stopped_or_liquidated, hedged, flat_waiting, none
pnl_state	profit, loss, liquidated_or_large_loss, none
expressed_emotion	fear, fomo, greed_euphoria, enthusiasm, frustration, regret, mockery, uncertainty, calm, mixed, none
sarcasm	likely, not_evident, unclear

Input format

Transcripts must be rendered exactly as in training (the project's N1 template): a stream preamble with a slang glossary, recent earlier chat, and the target author's lines fenced as <<<...>>>. Qwen has no BOS token; tokenize without special tokens.

Usage

import json, torch
from huggingface_hub import snapshot_download
from peft import PeftModel
from transformers import AutoConfig, AutoTokenizer, Qwen3_5ForCausalLM

repo = snapshot_download("echoforest/yt-chat-qwen3.5-9b-screen-gen-instruct")
base = "Qwen/Qwen3.5-9B"
lm = Qwen3_5ForCausalLM.from_pretrained(  # text decoder only; vision and MTP weights are ignored
    base, config=AutoConfig.from_pretrained(base).text_config, dtype=torch.bfloat16, device_map="cuda")
text = PeftModel.from_pretrained(lm.model, repo).eval()
W = lm.lm_head.weight
labels = json.load(open(f"{repo}/labels.json"))
tok = AutoTokenizer.from_pretrained(base)
piece = lambda s: tok(s, add_special_tokens=False)["input_ids"]

def prompt(transcript):
    chat = tok.apply_chat_template([{"role": "user", "content": transcript + "\n\nLabel the TARGET TURN."}],
                                   tokenize=False, add_generation_prompt=True, enable_thinking=False)
    return piece(chat)

def classify(transcript):
    ids, out = prompt(transcript), {}
    for f in labels["fields"]:  # one forward pass per field, in this order
        firsts = [piece(" " + o + "\n")[0] for o in labels["options"][f]]
        x = torch.tensor([ids + piece(f + ":")], device="cuda")
        with torch.no_grad():
            h = text(input_ids=x).last_hidden_state[0, -1].float()
        p = torch.softmax(h @ W[firsts].float().T, -1)
        out[f] = dict(zip(labels["options"][f], p.tolist()))
        ids += piece(f + ":") + piece(" " + max(out[f], key=out[f].get) + "\n")
    return out

Needs transformers ≥ 5.20 and peft ≥ 0.21. Install flash-linear-attention and causal-conv1d for the fast Gated DeltaNet kernels; on Hopper with torch 2.8 (Triton 3.4), flash-linear-attention also needs tilelang. Probabilities are uncalibrated (screening runs fit no temperatures).

Training

• Data: export v20261016-1835 (splits by whole stream). A stream-proportional 15,000-turn train sample (seed 13), 1 epoch, max length 1024 tokens, nothing truncated. Turns that needed stream audio trained at half weight.
• Method: LoRA r 32 / α 32 / dropout 0.05 on attention q/k/v/o, the Gated DeltaNet projections (in_proj_qkv, in_proj_z, out_proj) and MLP gate/up/down; cross-entropy on the label tokens only. LoRA LR 0.0002, cosine schedule, 3% warmup, AdamW, weight decay 0.01, 32 sequences per step, bf16, gradient checkpointing.
• Compute: 1× H100 80GB, about 5,800 tokens/s.
• Versions: torch 2.8.0+cu128, transformers 5.20.1, peft 0.21.2.

Limitations

• Screening checkpoint: trained on about a fifth of the data for one epoch, uncalibrated, and evaluated only on a val sample from 2 streams. The test set is reserved for the final run.
• Narrow data: 15 train streams over 6 days (2026-10-10 to 10-15), all Korean crypto-futures chat.
• Chat text that imitates the transcript fences or gives instructions can move labels; the robustness suite is run on the final model only.