AkanT5 base: vocabulary replacement initialization

The 12 release models are grouped in the AkanT5 collection.

IARG-UF/akant5-base-replacement-init contains the vocabulary-adapted initialization derived from google/mt5-base, before Akan continued pretraining (CPT) or downstream MT fine-tuning. Its corresponding selected CPT checkpoint is IARG-UF/akant5-base-replacement.

Property	Value
Architecture	mT5 encoder–decoder
Vocabulary strategy	replacement
Parameters	296,729,856
CPT updates	0
Stored weights	float32 safetensors
Stored dropout rate	0.1

Vocabulary replacement substitutes an Akan–English vocabulary with 64,100 tokenizer entries and 64,128 embedding/output rows. Vocabulary extension keeps the original mT5 vocabulary IDs and adds Akan–English pieces, producing 295,031 tokenizer entries and 295,040 embedding/output rows. New vocabulary rows are transferred from the corresponding mT5 parameters using the recorded initialization procedure. Each repository includes its own unchanged tokenizer and model configuration.

Loading and fine-tuning

The artifacts were prepared with Transformers 5.16.1. Load the tokenizer saved with this initialization to preserve vocabulary IDs and byte-fallback behavior.

from transformers import AutoModelForSeq2SeqLM, AutoTokenizer

repo_id = "IARG-UF/akant5-base-replacement-init"
tokenizer = AutoTokenizer.from_pretrained(repo_id, fix_mistral_regex=False)
model = AutoModelForSeq2SeqLM.from_pretrained(repo_id)
# mT5 shares encoder/decoder input embeddings; its output head is separate.
model.config.tie_word_embeddings = False

This initialization stores dropout_rate=0.1. The associated CPT checkpoint stores dropout_rate=0.0, as trained. The paper's main supervised MT fine-tuning uses dropout_rate=0.1 for every model; this must be set in the configuration before constructing the fine-tuning model so the instantiated dropout modules use that rate.

Source records and provenance

For reference, the table records the corresponding CPT mixture's pinned sources. It is not a tokenizer-fitting manifest. Tokenizer-fitting source selection and English proportions differ from CPT; the replacement tokenizer predates the final CPT partitions. No CPT on this mixture has been applied to the initialization in this repository.

Dataset	Configuration	Pinned revision	Declared license
ghananlpcommunity/pristine-twi-english	default	`89c1d085fd677387e4dad29df6a3a304359eee0e`	cc-by-nc-4.0
ghanaopenai/ghok-parallel-en-twi	default	`8525742d5dd782b96b0a5271dddb1bf6ae36e5e3`	cc-by-nc-4.0
ghanaopenai/ghana-farmer-qa-twi	default	`66fbd932df7b86a6b1c6aeab7f9fbd981010a0ed`	cc-by-nc-4.0
ghananlpcommunity/true-asr-ipa-pairs-twi	default	`c93adcc9bd2b18deac68131b2938b35ff66c541d`	cc-by-nc-4.0
ghanaopenai/kumawood-speech-transcriptions	default	`80dc82d4813e26eddb251ac83643c6e519fdb068`	cc-by-nc-4.0
wikimedia/wikipedia	20231101.tw	`b04c8d1ceb2f5cd4588862100d08de323dccfbaa`	cc-by-sa-3.0
ghanaopenai/ghana-speech	Akuapem_Twi_twi	`884f35c11eef4f1f6d4b472528125f9d1b6319a9`	cc-by-nc-4.0
ghanaopenai/ghana-speech	Asante_Twi_twi	`884f35c11eef4f1f6d4b472528125f9d1b6319a9`	cc-by-nc-4.0
allenai/c4	en-multi	`1588ec454efa1a09f29cd18ddd04fe05fc8653a2`	odc-by

sources.json records the corresponding CPT source identities; release_provenance.json records the initialization identity, parameter count, and inference-file SHA-256 hashes. The assembled M1xEN corpus and MT outputs are excluded from this release. Code and experiment configurations are designated for Intelligent-Agents-Research-Group/akant5.

Use and limitations

Research use only. See LICENSE, NOTICE, and the upstream Apache 2.0 license. This initialization supports controlled comparisons of vocabulary transfer before and after CPT. It has received no task-specific translation training. The associated study evaluates Twi (Asante and Akuapem) and English; it does not establish coverage of every Akan variety. Source licensing and provenance limitations remain relevant to the learned vocabulary and adaptation artifacts.

Associated manuscript: AkanT5: Vocabulary Adaptation, Language Retention, and Model Capacity in Continued Pretraining, Kweku Yamoah, Kevin Scroggins, Chibuzor Okocha, Christan Grant, and Emmanuel Dorley (2026).