Model Card for ethicalabs/Echo-DSRN-v0.1.4-Research-Intent-CLF

6-class research paper intent classifier built on the Echo-DSRN architecture (98.3M params), fine-tuned from ethicalabs/Echo-DSRN-v0.1.3-Research-Intent-CLF (5-class).

Labels: Methodology · Dataset · Review · Applied · Theoretical · Unclassifiable

The Unclassifiable class is new in v0.1.4 — papers flagged by >1 LLM judge as garbled / non-English / non-academic.

Training

Data: ethicalabs/Research-Intent-Collab (10,001 papers, LLM-judge majority consensus + human curation). Official train (7,999) / validation (1,001) / test (1,001) splits.
Recipe: head swap 5→6, differential AdamW (backbone 2e-5, head 2e-4), batch 16, 3 epochs (1,500 steps), bf16, cosine LR + 5% warmup, early stopping on the official validation split.

Results (official `test` split, 1,001 rows)

Label	Precision	Recall	F1	Support
Methodology	0.8140	0.7883	0.8009	222
Dataset	0.7097	1.0000	0.8302	22
Review	0.6400	0.5845	0.6110	219
Applied	0.7162	0.8112	0.7608	392
Theoretical	0.7333	0.7500	0.7416	44
Unclassifiable	0.9091	0.5882	0.7143	102
Macro avg	0.7537	0.7537	0.7431	1001

Accuracy: 0.7353 (736/1001).

Comparison with v0.1.3 (same official `test` split)

v0.1.3 (5-class) evaluates on 899 rows only — its head cannot produce Unclassifiable, so all 102 such test rows would be misclassified.

v0.1.4 covers all 1,001 rows and catches 60 of the 102 with 91% precision.

Metric	v0.1.3 (5-class)	v0.1.4 (6-class)
Accuracy, all evaluated rows	0.7197 (647/899)	0.7353 (736/1001)
Accuracy, 5-class subset	0.7197 (647/899)	0.7519 (676/899)
Macro F1, 5-class subset	0.7328	0.7489
Macro F1, all 6 classes	—	0.7431
Unclassifiable	— (all 102 misclassified)	F1 0.7143 (P 0.909)

Per-class F1 (test split):

Label	v0.1.3	v0.1.4	Δ
Methodology	0.7563	0.8009	+0.045
Dataset	0.8627	0.8302	−0.033
Review	0.5876	0.6110	+0.023
Applied	0.7524	0.7608	+0.008
Theoretical	0.7048	0.7416	+0.037
Unclassifiable	—	0.7143	new

v0.1.4 gains accuracy on the original classes (+3.2 pts on the 5-class subset) and adds the Unclassifiable safety class.

Review remains the weakest class for both models (mostly confused with Applied).

Usage

from echo_dsrn import pipeline

# Requires echo_dsrn >= 0.1.6 (chat-template support for
# text-classification pipelines). Tiny model — CPU load + inference
# is faster than GPU for this size.
pipe = pipeline(
    "text-classification",
    model="ethicalabs/Echo-DSRN-v0.1.4-Research-Intent-CLF",
    trust_remote_code=True,
    device="cpu",
)

# Raw title+abstract string — the pipeline applies the model's chat
# template (system prompt + user phrasing) automatically.
pipe(
    "Title: Attention Is All You Need\n"
    "Abstract: The dominant sequence transduction models are based on complex "
    "recurrent or convolutional neural networks that include an encoder and a "
    "decoder. The best performing models also connect the encoder and decoder "
    "through an attention mechanism. We propose a new simple network "
    "architecture, the Transformer, based solely on attention mechanisms, "
    "dispensing with recurrence and convolutions entirely. Experiments on two "
    "machine translation tasks show these models to be superior in quality "
    "while being more parallelizable and requiring significantly less time to "
    "train. Our model achieves 28.4 BLEU on the WMT 2014 English-to-German "
    "translation task, improving over the existing best results, including "
    "ensembles, by over 2 BLEU."
)
# [{'label': 'Methodology', 'score': 0.9518}]