kk-bert-small-bpe-kazqad

BERT-Small finetuned on KazQAD (Kazakh Reading Comprehension) starting from Abzalbek89/kk-bert-small-bpe — pretrained with the bpe tokenizer.

Part of a comparative study of subword tokenizers for Kazakh small language models.

Test set results

metric	value
Exact Match	2.0273
F1 (token-level)	8.72

Training

Base model	Abzalbek89/kk-bert-small-bpe
Dataset	IS2AI/KazQAD reading-comprehension v1.0
Examples	2,713 test, ~10K train
Epochs	10
Batch size	12
Max length	384 (stride 128)
Learning rate	3e-5 (warmup 10%, weight decay 0.01)
Precision	fp16
Hardware	RTX A4000
Time	1.4 min
Seed	42

Usage

from transformers import AutoTokenizer, AutoModelForQuestionAnswering, pipeline
tok = AutoTokenizer.from_pretrained("Abzalbek89/kk-bert-small-bpe-kazqad")
model = AutoModelForQuestionAnswering.from_pretrained("Abzalbek89/kk-bert-small-bpe-kazqad")
qa = pipeline("question-answering", model=model, tokenizer=tok)
qa(question="Кардиннің туған жылы қашан?",
   context="Belcalis Marlenis Almánzar 1992 жылы 11 қазанда дүниеге келген...")

Companion artifacts

• Pretrained base: Abzalbek89/kk-bert-small-bpe
• KazNERD finetune: Abzalbek89/kk-bert-small-bpe-kaznerd
• Aggregate metrics: Abzalbek89/kk-tokenizer-fertility-baseline