AkanT5 base: vocabulary replacement

The 12 release models are grouped in the AkanT5 collection.

This repository contains the selected continued-pretraining (CPT) checkpoint of IARG-UF/akant5-base-replacement, adapted from google/mt5-base. The associated vocabulary initialization is IARG-UF/akant5-base-replacement-init.

Property	Value
Architecture	mT5 encoder–decoder
Vocabulary strategy	replacement
Parameters	296,729,856
Selected CPT update	49,000
Stored weights	float32 safetensors
CPT dropout rate	0.0

The replacement tokenizer has 64,100 entries and 64,128 embedding/output rows. The extension tokenizer has 295,031 entries and 295,040 embedding/output rows. Each repository includes its own unchanged tokenizer and model configuration.

CPT uses span corruption on M1xEN: normalized Twi text with English replay, targeting 80% Twi and 20% English by retained training characters. The adaptation study compares small, base, and large mT5 backbones under replacement and extension, with a limit of 50,000 updates. The selected checkpoint can precede the final update. These weights precede downstream machine-translation fine-tuning; task-specific fine-tuning is required for translation.

Loading and fine-tuning

The artifacts were prepared with Transformers 5.16.1. Load the tokenizer saved with this repository to preserve vocabulary IDs and byte-fallback behavior.

from transformers import AutoModelForSeq2SeqLM, AutoTokenizer

repo_id = "IARG-UF/akant5-base-replacement"
tokenizer = AutoTokenizer.from_pretrained(repo_id, fix_mistral_regex=False)
model = AutoModelForSeq2SeqLM.from_pretrained(repo_id)
# mT5 shares encoder/decoder input embeddings; its output head is separate.
model.config.tie_word_embeddings = False

The serialized CPT configuration retains dropout_rate=0.0, as trained. The paper's main supervised English–Twi MT protocol sets dropout_rate=0.1 for every model before constructing the fine-tuning model. To reproduce that setting, load the stored configuration, set config.dropout_rate = 0.1, and pass that configuration to from_pretrained; set tie_word_embeddings=False after loading, as above. Merely editing the config of an already constructed model does not change existing dropout modules.

Data and provenance

The table records the corresponding CPT mixture's pinned source declarations. Several sources are synthetic, and most Twi sources carry noncommercial licenses. Source labels do not establish that every document was human-authored or reviewed by a native speaker. The assembled M1xEN corpus and MT outputs are excluded from this release. Source licenses and attribution remain applicable.

Dataset	Configuration	Pinned revision	Declared license
ghananlpcommunity/pristine-twi-english	default	`89c1d085fd677387e4dad29df6a3a304359eee0e`	cc-by-nc-4.0
ghanaopenai/ghok-parallel-en-twi	default	`8525742d5dd782b96b0a5271dddb1bf6ae36e5e3`	cc-by-nc-4.0
ghanaopenai/ghana-farmer-qa-twi	default	`66fbd932df7b86a6b1c6aeab7f9fbd981010a0ed`	cc-by-nc-4.0
ghananlpcommunity/true-asr-ipa-pairs-twi	default	`c93adcc9bd2b18deac68131b2938b35ff66c541d`	cc-by-nc-4.0
ghanaopenai/kumawood-speech-transcriptions	default	`80dc82d4813e26eddb251ac83643c6e519fdb068`	cc-by-nc-4.0
wikimedia/wikipedia	20231101.tw	`b04c8d1ceb2f5cd4588862100d08de323dccfbaa`	cc-by-sa-3.0
ghanaopenai/ghana-speech	Akuapem_Twi_twi	`884f35c11eef4f1f6d4b472528125f9d1b6319a9`	cc-by-nc-4.0
ghanaopenai/ghana-speech	Asante_Twi_twi	`884f35c11eef4f1f6d4b472528125f9d1b6319a9`	cc-by-nc-4.0
allenai/c4	en-multi	`1588ec454efa1a09f29cd18ddd04fe05fc8653a2`	odc-by

sources.json records these source identities. release_provenance.json records the selected artifact, parameter count, and inference-file SHA-256 hashes. Code and experiment configurations are designated for Intelligent-Agents-Research-Group/akant5.

Use and limitations

Research use only. See LICENSE, NOTICE, and the upstream Apache 2.0 license. The release supports research on vocabulary adaptation, reconstruction, English retention, and subsequent task adaptation. Evaluation in the associated study concerns Twi (Asante and Akuapem) and English; it does not establish coverage of every Akan variety. Reconstruction scores do not establish translation quality, factual accuracy, or suitability for deployment.

Associated manuscript: AkanT5: Vocabulary Adaptation, Language Retention, and Model Capacity in Continued Pretraining, Kweku Yamoah, Kevin Scroggins, Chibuzor Okocha, Christan Grant, and Emmanuel Dorley (2026).