repleeka/tagin-ner-v0.1

Tagin NER v0.1

tagin-ner-v0.1 is the first public named-entity recognition (NER) model for Tagin (ISO 639-3: tgj). Tagin is an endangered Tani (Tibeto-Burman) language spoken mainly in the Upper Subansiri district of Arunachal Pradesh, India. The model tags five entity types: persons, locations, organisations, dates, and CULT (cultural entities) such as rituals, festivals, myths, deities and traditional roles. Cultural entities are the most frequent type in Tagin and are not covered by standard NER schemes.

The model is XLM-RoBERTa-base (277.5 M parameters), trained in two stages:

1. Domain adaptation: continued MLM pre-training of xlm-roberta-base on about 115K Tagin segments (repleeka/tagin-monolingual-corpus).
2. NER fine-tuning: token classification on the gold-standard repleeka/tagin-ner-corpus (CuNER Gold).

We benchmarked six domain-adapted backbones under one identical protocol. This model had the highest test entity-level F1 (84.62), the highest validation F1 (87.56) and the highest token accuracy (95.49). See Variant comparison.

At a glance
Task	Token classification / NER (BIO tagging, 11 labels)
Test entity-level P / R / F1 (micro)	85.55 / 83.71 / 84.62
Test macro F1	87.68
Best entity type	PER, F1 92.49
Hardest entity type	CULT, F1 76.00
Companion MLM model	repleeka/tagin-mlm-v0.1
Paper	Dugi & Sambyo, Sādhanā (Springer Nature), under review
Code	github.com/repleeka/tagin-ner-mlm

Note: this is a token-classification model only. It cannot do fill-mask or MLM, because the MLM head was replaced by the NER classifier. For MLM, use repleeka/tagin-mlm-v0.1.

---

Table of contents

1. Model details
2. Uses
3. How to get started
4. Training data
5. Training procedure
6. Evaluation
7. Error analysis
8. Variant comparison
9. Tokenization analysis
10. Bias, risks and limitations
11. Environmental impact
12. Technical specifications
13. Files in this repository
14. Version history
15. License
16. Citation
17. Glossary
18. Model card authors and contact

---

Model details

Description

Tagin had no annotated NER data and no NER model. This release goes with the CuNER Gold corpus (10,600 manually annotated sentences) and the accompanying paper. The paper benchmarks BERT-family encoders for Tagin NER after domain-adaptive MLM pre-training. This model is the best of those encoders.

Developed by	Tungon Dugi and Koj Sambyo, Department of Computer Science and Engineering, National Institute of Technology Arunachal Pradesh, Itanagar 791113, Arunachal Pradesh, India
Shared by	repleeka
Model type	Transformer encoder with a linear token-classification head (XLMRobertaForTokenClassification)
Language	Tagin (tgj), lowercase Latin script
Base model	xlm-roberta-base → Tagin MLM-adapted (intermediate, unreleased checkpoint xlm-roberta-ginlish-domain-adapted) → this NER model
Labels	11 BIO tags: B-/I- × {CULT, DATE, LOC, ORG, PER} plus O
Version	v0.1 (first public release)
License	CC BY-NC 4.0
Funding	Not specified

Model sources

Paper	Developing a Tagin NER Corpus and Benchmarking BERT-Based Models in a Low-Resource Setting, Tungon Dugi and Koj Sambyo, Sādhanā (Springer Nature), under review
Code	github.com/repleeka/tagin-ner-mlm: training notebooks, logs, plots and results for all six backbones
NER dataset	repleeka/tagin-ner-corpus
MLM dataset	repleeka/tagin-monolingual-corpus
Companion model	repleeka/tagin-mlm-v0.1 (best Tagin MLM)

Entity types

Tag	Meaning	Corpus share (spans)	Examples from the corpus
CULT	Cultural entities: rituals, festivals, myths, deities, spirits, traditional roles, culturally specific objects and concepts	41%	si-donyi, riumching, riumdo, pinchi, jarkw-pajok
PER	Persons	26%	anand, dondhup tsring
LOC	Locations: villages, rivers, mountains, regions	16%	sipi, lwtur, tumlv, nyarbom, taktv
DATE	Dates and temporal expressions	9%	apo
ORG	Organisations	8%

Label mapping

The ids in config.json follow alphabetical order:

id	0	1	2	3	4	5	6	7	8	9	10
label	B-CULT	B-DATE	B-LOC	B-ORG	B-PER	I-CULT	I-DATE	I-LOC	I-ORG	I-PER	O

⚠️ This is different from the ClassLabel order in repleeka/tagin-ner-corpus (O, B-PER, I-PER, B-LOC, …). When you evaluate this model on the dataset, map labels by name (dataset.features["ner_tags"].feature.int2str ↔ model.config.label2id), never by integer id.

---

Uses

Direct use

• Extracting persons, locations, organisations, dates and cultural entities from Tagin text.
• Language documentation and archives: indexing oral-history transcripts, folk narratives and community write-ups by the people, places and rituals they mention.
• Cultural-heritage research: finding mentions of rituals, festivals and deities across a Tagin text collection.
• Pre-annotation: producing first-pass BIO tags for human annotators to correct when extending Tagin NER data.

Downstream use

• A baseline for Tagin and other Tani-language NER research, and a starting point for transfer to related languages (for example Nyishi, Galo, Adi or Apatani). This is untested.
• A component in Tagin information-extraction, search or knowledge-graph pipelines.

Out-of-scope use

• Fill-mask or text generation (this model has no MLM head).
• Commercial use (non-commercial licence).
• Languages other than Tagin, cased text, or Tagin written in other orthographies.
• High-stakes or surveillance use, such as identifying, profiling or tracking individuals or communities.
• Treating the output as authoritative cultural classification. What counts as a CULT entity reflects the annotation guidelines, not community consensus.

---

How to get started

⚠️ The training data is entirely lowercased. Always lowercase your input.

Requirements: transformers (files saved with 5.3.0), plus torch. The tokenizer is the standard XLM-R SentencePiece tokenizer.

Pipeline

from transformers import pipeline

ner = pipeline(
    "token-classification",
    model="repleeka/tagin-ner-v0.1",
    aggregation_strategy="first",   # labels were trained on the first sub-token of each word
)

text = "riumching nv riumdo bunyi ham charching kanye dvbv bvnglin tho".lower()
for ent in ner(text):
    print(f"{ent['word']:<20s} {ent['entity_group']:<6s} {ent['score']:.4f}  [{ent['start']}:{ent['end']}]")

Use aggregation_strategy="first" (or "max"), not "simple". Only the first sub-token of each word was trained with a label. With "simple", one word can be split into several spans: for example, si-donyi hilo comes out as si-do + nyi hilo.

Word-level prediction (pre-tokenized input)

This reproduces the evaluation setup exactly: the label of each word is the label predicted for its first sub-token.

import torch
from transformers import AutoTokenizer, AutoModelForTokenClassification

tok = AutoTokenizer.from_pretrained("repleeka/tagin-ner-v0.1")
model = AutoModelForTokenClassification.from_pretrained("repleeka/tagin-ner-v0.1").eval()

def tag(words):
    words = [w.lower() for w in words]
    enc = tok(words, is_split_into_words=True, return_tensors="pt", truncation=True, max_length=128)
    with torch.no_grad():
        pred = model(**enc).logits.argmax(-1)[0].tolist()
    labels, prev = [], None
    for i, w in enumerate(enc.word_ids()):
        if w is not None and w != prev:
            labels.append(model.config.id2label[pred[i]])
        prev = w
    return list(zip(words, labels))

print(tag(["si-donyi", "hilo", "kilin", "koongv"]))

Evaluating on the public dataset

from datasets import load_dataset   # the dataset is gated: accept its terms on the Hub first
import evaluate

ds = load_dataset("repleeka/tagin-ner-corpus", split="test")
names = ds.features["ner_tags"].feature.names          # dataset label order
seqeval = evaluate.load("seqeval")

preds = [[lab for _, lab in tag(x["tokens"])] for x in ds]
refs  = [[names[i] for i in x["ner_tags"]][:len(p)] for x, p in zip(ds, preds)]
print(seqeval.compute(predictions=preds, references=refs))

---

Training data

Stage 1: MLM domain adaptation (unlabeled)

115,323 lowercased Tagin segments (1.02 M words):

Source	Segments
GinLish Corpus v0.1 (Tagin side)	103,275
Tagin NER corpus, labels stripped	10,600
Tagin monolingual community text	1,448

The corpus was split 90/5/5 with seed=42 (103,790 / 5,766 / 5,767). The public repleeka/tagin-monolingual-corpus is the cleaned release of this corpus (114,994 rows). This model was trained on the pre-release build listed above.

Stage 2: NER fine-tuning (labeled)

repleeka/tagin-ner-corpus (CuNER Gold) has 10,600 manually annotated sentences, 59,820 tokens, 10,182 word types and 11,413 entity spans (8,133 single-token, 3,280 multi-token). Sentences are short: mean 5.6 tokens, median 5, maximum 237.

Entity	B- tags (spans)	I- tags	Span share	Test-set spans
CULT	4,734	969	41%	472
PER	2,925	1,072	26%	298
LOC	1,844	313	16%	194
DATE	1,038	642	9%	116
ORG	872	1,193	8%	80
Total	11,413	4,189		1,160
O tokens	44,218 (73.9% of tokens)

Split: 80 / 10 / 10 with 🤗 datasets train_test_split (10% test first, then 1/9 of the remainder for validation), seed=42:

Split	Sentences
Train	8,480
Validation	1,060
Test	1,060

These sizes match the published dataset splits.

Preprocessing

• Words: gold tokens are used as given (is_split_into_words=True). Punctuation stays attached to words as in the corpus (for example sipi,).
• Sub-word label alignment: the first sub-token of each word gets the word's label. Following sub-tokens and special tokens (<s>, </s>) get -100 and are ignored by the loss and by evaluation.
• Truncation: 128 sub-word tokens. Padding is dynamic, with DataCollatorForTokenClassification.
• Corpus cleaning: a guard skips malformed non-BIO tags while computing statistics. All 11 labels above are the only ones present in the gold data.

---

Training procedure

Stage 1: MLM domain adaptation of XLM-R

Hyperparameter	Value
Starting checkpoint	xlm-roberta-base (278.3 M parameters with the MLM head)
Objective	MLM, 15% dynamic masking (80/10/10)
Max length	128
Epochs / steps	5 / 64,870
Learning rate / schedule	2e-5, linear decay, 10% warm-up
Batch size	8
Weight decay	0.01
Precision	FP16
Seed	42
Model selection	Lowest validation MLM loss (epoch 5)
Training time	621.9 min (≈ 10.4 h) on 1 × RTX 4000 Ada

Epoch	Mean train loss	Val MLM loss	Val PPL
1	2.7089	NaN*	–
2	1.7400	1.5525	4.72
3	1.4883	1.3888	4.01
4	1.3508	1.2874	3.62
5	1.2757	1.2357	3.44

* The epoch-1 validation loss came out as NaN, most likely an FP16 overflow during evaluation. Training loss stayed finite throughout, and later evaluations were normal.

Adapted XLM-R, final held-out MLM results: validation loss 1.2408 (perplexity 3.46); test loss 1.2113 (perplexity 3.36); masked-token Top-1 / Top-5 accuracy 58.05% / 74.68%.

Stage 2: NER fine-tuning

The same hyperparameters were used for every backbone in the benchmark.

Hyperparameter	Value
Head	Linear layer (768 → 11) on the final hidden states, newly initialised; dropout 0.1
Loss	Token-level cross-entropy (sub-tokens other than the first ignored)
Max epochs	15
Early stopping	Patience 3 evaluations on validation F1 (not triggered before epoch 15)
Optimizer	AdamW (β₁ 0.9, β₂ 0.999, ε 1e-8)
Peak learning rate	3e-5
LR schedule	Linear warm-up over 10% of steps, then linear decay to 0
Weight decay	0.01
Batch size	16 (1 GPU, no accumulation)
Steps	7,950 (530 per epoch)
Precision	FP16 mixed precision
Seed	42
Evaluation / checkpointing	Every epoch; at most 2 checkpoints kept
Model selection	load_best_model_at_end=True on validation entity-level F1. Best: epoch 12, step 6,360
Hardware	1 × NVIDIA RTX 4000 Ada Generation (20 GB)
Training time	80.7 min (≈ 1.64 steps/s, ≈ 26 sentences/s)
Total FLOPs (Trainer estimate)	2.67 × 10¹⁵

Training dynamics (validation set, seqeval entity-level)

Epoch	Mean train loss	Val loss	Val P	Val R	Val F1	Val token acc
1	0.7706	0.2428	74.95	72.74	73.83	92.58
2	0.2272	0.1593	80.62	81.22	80.92	94.64
3	0.1534	0.1761	78.55	85.64	81.94	94.40
4	0.1153	0.1628	86.58	84.35	85.45	95.58
5	0.0816	0.1532	84.84	85.54	85.19	95.76
6	0.0622	0.1808	82.36	87.29	84.76	95.49
7	0.0507	0.1801	87.46	85.45	86.45	96.05
8	0.0398	0.2155	85.14	87.57	86.34	95.93
9	0.0316	0.2190	87.00	86.92	86.96	96.29
10	0.0264	0.2239	85.62	87.20	86.41	96.03
11	0.0207	0.2255	87.13	86.65	86.89	96.27
12 ✔	0.0156	0.2412	87.00	88.12	87.56	96.34
13	0.0152	0.2458	86.95	87.11	87.03	96.34
14	0.0121	0.2523	87.26	87.66	87.46	96.34
15	0.0091	0.2582	87.25	87.57	87.41	96.34

Validation loss is lowest at epoch 5 and then rises, while validation F1 keeps improving until epoch 12. Training loss falls to about 0.01, so the model fits the training set almost perfectly.

Why selection is by F1, not by loss. This divergence is the normal pattern when fine-tuning for NER, and it is not a sign of failure. Cross-entropy loss heavily penalises the few mistakes the model makes with high confidence, so loss rises late in training. Meanwhile, the number of correct argmax predictions, which is what entity-level F1 counts, keeps increasing. All six backbones in the benchmark show the same pattern under the identical protocol:

Backbone	Epochs run	Lowest val loss (epoch → F1)	Best val F1 (epoch → loss)
Tagin-XLMR (this model)	15	epoch 5 → F1 0.852	epoch 12 → F1 0.876 (loss 0.241)
Tagin-BERT	11	epoch 2 → F1 0.824	epoch 8 → F1 0.875 (loss 0.207)
Tagin-IndicBERT	13	epoch 4 → F1 0.837	epoch 10 → F1 0.861 (loss 0.216)
Tagin-NE-BERT	12	epoch 2 → F1 0.814	epoch 9 → F1 0.861 (loss 0.250)
Tagin-ModernBERT	15	epoch 2 → F1 0.793	epoch 12 → F1 0.856 (loss 0.245)
Tagin-RoBERTa	12	epoch 4 → F1 0.846	epoch 9 → F1 0.860 (loss 0.215)

Selecting this model by lowest validation loss would have cost 2.4 F1 points on validation (0.852 vs 0.876). Selecting on the target metric is therefore the right choice. The one side effect is that the model's confidence scores are over-confident (see Limitations). The predicted labels are not affected.

---

Evaluation

Testing data, factors and metrics

• Test set: 1,060 held-out sentences with 1,160 gold entity spans, never used for training or model selection.
• Metric: seqeval via 🤗 evaluate, in default (non-strict, IOB2) mode. Scores are entity-level exact match: a predicted entity is correct only if both its boundaries and its type match the gold span. Precision, recall and F1 are micro-averaged over entities (the primary metric). Macro and support-weighted averages are also given. Token accuracy is the fraction of words whose BIO tag is correct.
• Disaggregation: results are reported per entity type, since entity frequency and difficulty vary widely across types.

Overall results

Split	Precision	Recall	F1	Token accuracy
Validation	87.00	88.12	87.56	96.34
Test	85.55	83.71	84.62	95.49

Test averaging	Precision	Recall	F1
Micro (primary)	85.55	83.71	84.62
Macro	88.56	86.85	87.68
Support-weighted	85.56	83.71	84.61

Macro F1 is higher than micro F1 because the four smaller classes score above 86 F1, while the largest class (CULT, 41% of spans) scores 76.

Per-entity results (test)

Entity	Precision	Recall	F1	Support
PER	94.10	90.94	92.49	298
ORG	93.59	91.25	92.41	80
DATE	92.86	89.66	91.23	116
LOC	85.00	87.63	86.29	194
CULT	77.24	74.79	76.00	472

---

Error analysis

198 of the 1,060 test sentences (18.7%) contain at least one word-level tag error. In other words, 81.3% of test sentences are tagged entirely correctly.

This sentence-level measure is strict: a single wrong tag anywhere marks the whole sentence as wrong, even though 95.5% of individual word tags are correct. This model is among the best of the six backbones on it:

Backbone	Test sentences with ≥ 1 error	Fully correct sentences
Tagin-BERT	195 (18.4%)	81.6%
Tagin-XLMR (this model)	198 (18.7%)	81.3%
Tagin-NE-BERT	211 (19.9%)	80.1%
Tagin-IndicBERT	213 (20.1%)	79.9%
Tagin-ModernBERT	226 (21.3%)	78.7%
Tagin-RoBERTa	230 (21.7%)	78.3%

Typical error patterns, with real test-set examples:

1. Adjacent CULT entities merged (boundary error). Two consecutive cultural entities are predicted as one span:

Token	Gold	Predicted
pinchi	B-CULT	B-CULT
svching-nyedo	B-CULT	I-CULT ✗

2. Rare cultural compounds missed (false negatives). Long, rare words are predicted as O:

Token	Gold	Predicted
lamrwpadubv	B-CULT	O ✗
mingkw	B-CULT	O ✗
raanv-rapv	B-CULT	O ✗

3. Confusion between entity types. This affects culturally ambiguous names and non-Tagin names:

Token(s)	Gold	Predicted
sipi, hvlv, (place names in a list)	B-LOC, B-LOC	B-CULT, I-CULT ✗
anand	B-PER	B-LOC ✗
dondhup tsring	B-PER I-PER	B-CULT I-LOC ✗

4. Spurious entities (false positives). Temporal-looking or entity-adjacent common words are tagged:

Token(s)	Gold	Predicted
alu loongu	O O	B-DATE I-DATE ✗
namw mwa	B-CULT O	B-CULT I-CULT ✗

Takeaways: most errors involve CULT, either at its boundaries or in confusion with LOC and PER. This reflects real ambiguity, since many Tagin place names and personal names are also mythological or ritual names. Names from outside the Tagin naming system (for example anand, dondhup tsring) are also error-prone.

---

Variant comparison

Each backbone was MLM-adapted on the same Tagin corpus and then fine-tuned with the identical NER protocol above (same split, seed and hyperparameters). The table shows test-set, entity-level micro scores in %, ordered by test F1.

Adapted backbone	Base checkpoint	Val F1	Test P	Test R	Test F1	Token acc
Tagin-XLMR (this model)	xlm-roberta-base	87.56	85.55	83.71	84.62	95.49
Tagin-BERT	bert-base-uncased	87.51	83.96	84.83	84.39	94.97
Tagin-IndicBERT	ai4bharat/IndicBERTv2-MLM-only	86.12	84.02	82.93	83.47	95.11
Tagin-NE-BERT	MWirelabs/ne-bert	86.07	83.86	82.41	83.13	95.06
Tagin-ModernBERT	answerdotai/ModernBERT-base	85.58	85.94	80.09	82.91	94.70
Tagin-RoBERTa	roberta-base	86.02	82.98	81.55	82.26	94.55

Per-entity test F1 across backbones:

Backbone	CULT (472)	DATE (116)	LOC (194)	ORG (80)	PER (298)
Tagin-XLMR (this model)	76.00	91.23	86.29	92.41	92.49
Tagin-BERT	77.46	87.76	85.29	91.72	91.55
Tagin-IndicBERT	76.38	88.98	83.07	91.03	90.76
Tagin-NE-BERT	75.16	92.98	83.25	89.17	90.29
Tagin-ModernBERT	73.85	90.75	83.82	92.41	90.48
Tagin-RoBERTa	73.90	89.66	81.84	91.03	90.51

Observations

• XLM-R has the best overall F1, the best scores on LOC, ORG and PER, and the best token accuracy.
• BERT-base-uncased is a close second (−0.23 F1). It has the highest recall and the best CULT F1.
• NE-BERT, the best intrinsic MLM model (tagin-mlm-v0.1), is best on DATE. Lower MLM perplexity did not translate directly into higher NER F1.
• All six models score between 82.3 and 84.6 F1. The ranking comes from one seed, so differences under about 1 F1 point should not be read as significant. In particular, the lead of XLM-R over BERT (+0.23 F1; BERT has 3 fewer test sentences with errors) is within single-seed noise. A multi-seed comparison (mean ± SD over 3–5 seeds) is planned to confirm the ranking.

Why XLM-R was chosen for release: it has the best test F1, the best validation F1 (the model-selection criterion), the best token accuracy, the best F1 on 3 of 5 entity types (LOC, ORG, PER), and the second-fewest sentence-level errors. It is also the model reported as best in the accompanying manuscript.

Machine-readable results are in results/.

---

Tokenization analysis

The XLM-R vocabulary (250,002 SentencePiece pieces) was not changed. On the 59,820 running words of the NER corpus:

Statistic	Value
Fertility (sub-words per word)	1.99
Words split into ≥ 2 pieces	69.3%
<unk> tokens	0

Word	XLM-R pieces
si-donyi	▁si · - · do · nyi
riumchingjering	▁ · rium · ching · je · ring
lamrwpadubv	▁la · mr · w · padu · b · v
koongv	▁ko · ong · v

Only the first piece of each word is used for classification. Heavily fragmented rare words such as lamrwpadubv (6 pieces) are among the most common false negatives.

---

Bias, risks and limitations

Technical limitations

• Single seed: all scores come from one run (seed 42). No confidence intervals, multi-seed variance or significance tests have been computed yet.
• Unlabeled test text seen during stage-1 MLM adaptation: the raw text of the whole NER corpus, including the test sentences, was part of the MLM corpus. Labels were never used, but test-set NER scores may be slightly optimistic compared with completely unseen Tagin text.
• Short, single-domain sentences: sentences have a mean length of 5.6 words and are mostly cultural and folkloric content. Performance on long-form, news, social-media or conversational Tagin has not been measured.
• Over-confident scores (poor calibration): the checkpoint was selected for F1 at epoch 12, after validation loss had started rising (lowest at epoch 5). Its softmax scores (score in the pipeline output) are therefore over-confident and should not be read as calibrated probabilities. This affects only the confidence values, not the predicted labels. It matters only if you filter entities with a score threshold. If you need calibrated scores, apply post-hoc temperature scaling on the validation set; no retraining is needed.
• Lowercase only, and limited to 128 sub-word tokens per input. Split longer documents into sentences first.
• CULT is the weakest class (76 F1). Rare, multi-word or hyphenated cultural entities, and adjacent entities, are the most error-prone.

Sociolinguistic limitations and risks

• No standard orthography: Tagin has no standardised writing system. The model follows the corpus's transcription conventions (lowercase, v/w for central vowels, hyphenated compounds) and may miss spelling variants.
• Annotation subjectivity: the boundary between CULT, PER and LOC (for example a deity, a mythical ancestor, or a sacred place) depends on the annotation guidelines and may not match every community member's view.
• Names from outside the Tagin naming system (other Indian or Tibetan names) are under-represented and more often mislabelled.
• Cultural sensitivity: some entities (deities, rituals, clan or personal names) are sacred or private. Use outputs respectfully and, where appropriate, in consultation with the Tagin community.
• Privacy: do not use the model to extract or aggregate personal names from private communications.

Recommendations

• Lowercase inputs, process text sentence by sentence, and use aggregation_strategy="first".
• For archival or published use, have a Tagin speaker review the predictions, especially CULT spans.
• When comparing with this model, report results over several seeds and evaluate on text that does not overlap the MLM corpus where possible.

---

Environmental impact

Estimated with the method of Lacoste et al. (2019). These are GPU-only upper-bound estimates at board power:

Hardware type	1 × NVIDIA RTX 4000 Ada Generation (130 W board power)
Hours used	≈ 10.4 h (stage-1 MLM adaptation) + ≈ 1.35 h (stage-2 NER) = ≈ 11.7 h
Energy	≈ 1.52 kWh
Compute region	India
Carbon emitted	≈ 1.1 kg CO₂e (at about 0.71 kg CO₂e/kWh, Indian grid average)
Cloud provider	None (local workstation)

---

Technical specifications

Architecture

Architecture	XLM-RoBERTa (XLMRobertaForTokenClassification)
Parameters	277,461,515 (FP32), of which ≈ 192 M are the 250K × 768 embedding matrix
Layers / hidden / heads	12 / 768 / 12
Intermediate size	3,072 (GELU)
Positional encoding	Learned absolute, 514 positions (trained at 128)
Dropout	0.1 (hidden and attention)
Classification head	Dropout 0.1 → Linear(768, 11)
Vocabulary	250,002 (SentencePiece)
Special tokens	<s> (0, CLS/BOS), <pad> (1), </s> (2, SEP/EOS), <unk> (3), <mask>

Compute infrastructure

Hardware	Single workstation, 1 × NVIDIA RTX 4000 Ada Generation (20 GB)
Software	Python 3.13.9 · PyTorch (CUDA) · 🤗 Transformers 5.3.0 · 🤗 Datasets · 🤗 Evaluate · seqeval · scikit-learn · TensorBoard

Note on weight naming

The saved model.safetensors stores LayerNorm parameters under the legacy names LayerNorm.gamma / LayerNorm.beta. 🤗 Transformers renames these to weight / bias automatically when loading, so no action is needed.

---

Files in this repository

File	Description
model.safetensors	Model weights (FP32, ≈ 1.11 GB): encoder and NER classification head
config.json	Model configuration, including id2label and label2id
tokenizer.json, tokenizer_config.json	XLM-R tokenizer
assets/tag_distribution.png	BIO tag distribution of the corpus (log scale)
assets/ner_learning_curves.png	Training and validation loss, and validation F1, per epoch
assets/ner_confusion_matrix.png	Token-level BIO confusion matrix on the test set
results/xlmr_ner_results.json	Final metrics, per-entity scores, data sizes and hyperparameters
results/all_variants_ner_comparison.csv	Overall NER comparison of all six backbones
results/all_variants_ner_per_entity.csv	Per-entity NER comparison of all six backbones
results/trainer_state.json	Full Trainer log history (every 50 steps; per-epoch evaluations)

---

Version history

Version	Date	Notes
v0.1	2026	First public release. XLM-R-base, Tagin MLM-adapted, then fine-tuned on CuNER Gold (best checkpoint at epoch 12).

Ideas for future versions: multi-seed training with confidence intervals, a CRF or span-based head for better boundaries, more CULT and non-Tagin-name training data, MLM adaptation that excludes NER test text, and possibly joint MLM+NER (multi-task) training.

---

License

• Model weights: CC BY-NC 4.0, for non-commercial research and educational use, with attribution.
• Training data: repleeka/tagin-ner-corpus and repleeka/tagin-monolingual-corpus are licensed CC BY-NC-ND 4.0.
• Base model: xlm-roberta-base is MIT-licensed.

---

Citation

The accompanying manuscript is under review at Sādhanā (Springer Nature). Until it is published, please cite it as:

@article{dugi_sambyo_tagin_ner,
  title     = {Developing a Tagin NER Corpus and Benchmarking BERT-Based Models in a Low-Resource Setting},
  author    = {Dugi, Tungon and Sambyo, Koj},
  journal   = {S\={a}dhan\={a}},
  publisher = {Springer Nature},
  note      = {Under review}
}

This entry will be updated with volume, pages and DOI once the paper is published.

Please also cite XLM-R:

@inproceedings{conneau2020unsupervised,
  title     = {Unsupervised Cross-lingual Representation Learning at Scale},
  author    = {Conneau, Alexis and Khandelwal, Kartikay and Goyal, Naman and Chaudhary, Vishrav and Wenzek, Guillaume and Guzm{\'a}n, Francisco and Grave, Edouard and Ott, Myle and Zettlemoyer, Luke and Stoyanov, Veselin},
  booktitle = {Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics},
  pages     = {8440--8451},
  year      = {2020}
}

---

Glossary

Term	Meaning
NER	Named-entity recognition: finding and classifying names and other entities in text
BIO tagging	B-X marks the first word of an entity of type X, I-X marks a following word of the same entity, and O marks a word outside any entity
Entity-level F1	Harmonic mean of precision and recall over whole entities; an entity counts as correct only if its span and type both match exactly
Micro / macro average	Micro pools all entities, so frequent types weigh more. Macro averages the per-type scores equally.
MLM domain adaptation	Continued masked-language-model pre-training on target-language text before task fine-tuning
Fertility	Average number of sub-word tokens per word
CULT	Cultural-entity class introduced for Tagin: rituals, festivals, myths, deities, traditional roles and objects

---

Model card authors and contact

• Model card authors: Tungon Dugi and Koj Sambyo
• Contact: Tungon Dugi (corresponding author), tungondugi@gmail.com · Koj Sambyo, sambyo.koj@gmail.com
• Affiliation: Department of Computer Science and Engineering, National Institute of Technology Arunachal Pradesh, Itanagar, India

Acknowledgements

We thank the Tagin-speaking community, whose knowledge, texts and recordings made this corpus possible, and the annotators who created the gold-standard NER labels.