BDRC/Bo-Boundary-mmbert-0.2

Bo-Boundary-mmBERT 0.2

A ModernBERT token classifier fine-tuned to detect text / outline boundaries in Tibetan Buddhist volume text. Each token is labelled B (boundary) or O (non-boundary).

This is the 0.2 release of BDRC/Bo-Boundary-mmBert. Weights are checkpoint-46000 from run run_bo_outline_neg06 (epoch 2 of 3). The run later early-stopped at step 52,000; 46k is the checkpoint that scored highest micro-F1 on the frozen 100-document benchmark after a decoder sweep.

Hosted as BDRC/Bo-Boundary-mmbert-0.2.

Model details

Property	Value
Base model	jhu-clsp/mmBERT-base
Architecture	ModernBertForTokenClassification
Parameters	~307 M
Hidden size	768
Layers	22
Attention heads	12
Max sequence length	8,192
Vocab size	256,000
Labels	O (id 0), B (id 1)
Checkpoint	checkpoint-46000 (step 46,000 / epoch 2)

Training data

Prepared from BDRC outline annotations (data/Bo_Outline_neg06). Splits are work-level and stratified by print_format, so volumes of the same BDRC work never appear in more than one split. Mixed-format works keep only majority-format volumes (119 minority-format volumes dropped). There is no test split in the training set — evaluation uses a separately curated 100-document benchmark.

Item	Value
Source volumes scanned	17,381
Unique BDRC works	10,578
Volumes kept	17,262
Annotated breakpoints	204,514
Tokenizer	jhu-clsp/mmBERT-base
Window size	8,192 tokens
Stride	256 tokens
Boundary radius	±3 characters around the boundary char
Prepare-time O-only keep ratio	0.6
Train-time extra negative sampling	none (ratio = 1.0)
Seed	42

Splits

Split	Works	Volumes	Windows
train	10,050	16,356	374,660
validation	528	906	32,376
test	0	0	0 (hand-crafted benchmark instead)

Stratification by print_format (work-level)

print_format	Train works	Val works
modern_book	1,146	60
pering	8,904	468

Train label distribution

The boundary class is extremely rare. All windows that contain a B label are kept; only O-only train windows are downsampled at prepare time.

Stage	Windows	B labels	O labels	B:O
Raw (before downsampling)	555,837	1,233,749	4,486,541,054	1 : 3,636
Final (after 0.6 O-only keep)	374,660	1,233,749	3,021,774,477	1 : 2,449

Training

Run directory: output/run_bo_outline_neg06. Started 2026-09-11. The full run early-stopped at step 52,000 after 105 h 42 m (patience = 10 evals with no doc-level F2 gain). This Hub checkpoint is step 46,000.

Hyperparameters

Parameter	Value
Epochs planned	3
Batch size (per GPU)	4
Gradient accumulation	4
Effective batch size	16
Optimiser	AdamW
Learning rate	1 × 10⁻⁵
Schedule	cosine decay with warmup
Warmup	7,024 steps (10 %)
Steps / epoch	23,416
Planned total steps	70,248
This checkpoint	46,000 (epoch 2)
Weight decay	0.01
Max grad norm	1.0
Loss	Focal (γ = 2.0, α = [O: 0.20, B: 0.80])
Mixed precision	bfloat16
torch.compile	enabled
Gradient checkpointing	enabled
Neg-sample ratio (train loader)	1.0
Eval / save interval	every 2,000 steps
In-loop eval threshold	0.70
Selection metric (run best/)	doc-level F2 on 80 frozen val docs
Early-stop patience	10 evals
Seed	42

In-loop validation used the production decoder at threshold 0.70. Those numbers are not the benchmark operating point below.

Infrastructure

• GPU: 1 × NVIDIA A100-SXM4-40GB (39.5 GB)
• Time to this checkpoint: ~92 h 55 m (step 46,000)
• Full run: 105 h 42 m (early stop at step 52,000)

Validation at this checkpoint (80 holdout docs, threshold 0.70)

Metric	Value
Precision	0.788
Recall	0.817
F1	0.802
F2	0.811
TP / FP / FN	887 / 239 / 199
Train loss	4.04 × 10⁻⁵

Key in-loop milestones (same 80-doc, threshold-0.70 decoder):

Step	Epoch	Train loss	Val P	Val R	Val F1	Val F2
2,000	1	3.74e-3	0.320	0.378	0.347	0.365
8,000	1	1.01e-3	0.840	0.712	0.771	0.734
16,000	1	5.35e-4	0.894	0.768	0.826	0.790
34,000	2	4.24e-5	0.807	0.828	0.817	0.824
46,000	2	4.04e-5	0.788	0.817	0.802	0.811
52,000	3	2.97e-5	0.823	0.669	0.738	0.695

Step 34,000 was the run’s best in-loop F2 (saved as checkpoints/best). Step 46,000 is released here because it generalises better on the frozen benchmark once the decode threshold is swept.

Evaluation on the benchmark

Frozen 100-document outline-boundary benchmark (78 works, 1,643 gold breakpoints, 23.7 M characters). A predicted boundary matches gold if it falls within ±25 characters. Decoder: centre-weighted sliding-window B probabilities → threshold → merge nearby peaks.

Documents	100 (81 with ≥1 gold boundary, 19 with none)
Gold breakpoints	1,643
Sources	bdrc_outline 27, tei 18, qwen_ocr 18, google_books 14, ocrv1 12, google_vision 11
Print format	pering 64, modern_book 36

Recommended operating point

threshold = 0.55, merge window = 75 characters (best micro-F1 on this checkpoint).

Metric	Value
Micro precision	0.836
Micro recall	0.847
Micro F1	0.842
F2	0.845
Work-macro F1	0.599
has_gt F1 (81 docs)	0.849
TP / FP / FN	1,392 / 273 / 251
Predicted / gold	1,665 / 1,643
FP on zero-GT docs	27

Other useful decode settings

postprocess did not change scores on this sweep.

Threshold	Merge	P	R	F1	F2	FP	FN	Notes
0.45	25	0.749	0.890	0.813	0.858	490	181	best F2 / highest recall
0.50	50	0.798	0.870	0.833	0.855	362	213	balanced
0.55	75	0.836	0.847	0.842	0.845	273	251	recommended (best F1)
0.60	25	0.862	0.816	0.838	0.824	215	303	slightly more precise
0.70	50	0.909	0.712	0.799	0.744	117	473	in-training eval setting

Source breakdown (threshold 0.70, merge 50)

Source	P	R	F1
tei	0.953	0.920	0.936
bdrc_outline	0.853	0.799	0.825
google_vision	0.908	0.688	0.783
qwen_ocr	0.966	0.644	0.773
ocrv1	0.930	0.524	0.670
google_books	0.892	0.294	0.443

Print format (threshold 0.70, merge 50)

Format	P	R	F1
pering	0.923	0.820	0.869
modern_book	0.884	0.569	0.692

How to use

from transformers import AutoModelForTokenClassification, AutoTokenizer

repo_id = "BDRC/Bo-Boundary-mmbert-0.2"
tokenizer = AutoTokenizer.from_pretrained(repo_id)
model = AutoModelForTokenClassification.from_pretrained(repo_id)

Production decode (centre-weighted windows, threshold, 50-char merge) is implemented in the training package:

mmbert-boundary predict input.txt --model BDRC/Bo-Boundary-mmbert-0.2 --threshold 0.55

Or via OpenPecha/outline-detection:

from outline_detection import MmBertDetector

det = MmBertDetector(model_name_or_path="BDRC/Bo-Boundary-mmbert-0.2")
boundaries = det.predict(text)  # [(offset, confidence, "mmbert"), ...]

Recommended defaults for this checkpoint: --threshold 0.55, merge nearby peaks within 75 characters, match gold within ±25 characters. Raise the threshold (0.60–0.70) if you need fewer false positives; lower it (0.45–0.50) if you need higher recall.

Limitations

• Trained on BDRC outline-annotated Tibetan volumes (pering and modern_book). Other layouts, scripts, or OCR engines may degrade.
• Sparse and zero-boundary volumes still produce extra detections. Downstream use should keep a confidence threshold.
• modern_book recall is lower than pering at the high-precision 0.70 setting (0.57 vs 0.82).
• In-loop checkpoint selection used F2 at threshold 0.70 on 80 val docs; published scores use a held-out 100-doc benchmark and a swept decoder.

Credits

Trained for the Buddhist Digital Resource Center (BDRC) etext corpus, with funding from the Khyentse Foundation. Successor to BDRC/Bo-Boundary-mmBert.