scaling-mmbert-40k-rams (Stage B, RAMS fine-tune)

Research checkpoint from the mmBERT head-init data-scaling curve (private). The warmed base whr778/scaling-mmbert-40k (mmBERT-base from_encoder, warmed on ~40,000 structure/argument records) fine-tuned on RAMS under the fixed mmbert-base-rams recipe (15 epochs, bce_posweight 4.0, native long-context, argument-strict checkpoint selection).

RAMS blind-test (871 docs), strict micro-F1

metric score
event_argument_strict 0.115
event_trigger_strict 0.706

This is the N=40k point's y-value on the curve: RAMS argument-strict F1 vs Stage-A corpus size. Curve context: N=0 (fresh heads → RAMS) = 0.050; a DeBERTa-v3 fastino warm-start (254K) reaches 0.462 as a cross-encoder reference.

Purpose

Quantifies whether warming mmBERT's heads on ~40,000 records lifts downstream RAMS argument extraction. See SCALING_CURVE_EXPERIMENT.md and PAPER.md §10.

Caveats

Research artifact, private. One point on a scaling curve, not a general release.

Downloads last month
-
Safetensors
Model size
0.3B params
Tensor type
F32
·
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support