theelvace/mistral-african-fintech

Mistral-7B African Fintech — LoRA Fine-Tuned

A LoRA adapter fine-tuned on top of Mistral-7B-v0.1 on a curated African fintech dataset covering cross-border payments, stablecoins, mobile money, and Pan-African financial inclusion.

Built as part of a deliberate MLOps skill-building track targeting production AI engineering roles.

Model Details

• Base model: mistralai/Mistral-7B-v0.1
• Fine-tuning method: LoRA (Low-Rank Adaptation) + 4-bit quantization (QLoRA)
• Trainable parameters: 41,943,040 (1.11% of total)
• Training samples: 18 curated African fintech Q&A pairs
• Epochs: 3
• Hardware: NVIDIA Tesla T4 (15.6GB VRAM) via Kaggle
• Framework: HuggingFace Transformers + PEFT + TRL

Training Results

Epoch	Validation Loss
1	1.8638
2	1.5106
3	1.6593

Best checkpoint: Epoch 2 (lowest validation loss)

ROUGE Evaluation

Evaluated against 5 held-out African fintech reference answers:

Metric	Score
ROUGE-1	0.4693
ROUGE-2	0.2516
ROUGE-L	0.3786

ROUGE-1 of 0.47 on domain-specific content with only 18 training samples demonstrates strong domain adaptation. Academic NLP benchmarks consider ROUGE-1 above 0.35 meaningful.

Domain Coverage

The training dataset covers:

• Stablecoins and cryptocurrency in African markets
• Cross-border payment infrastructure (M-Pesa, mobile money)
• Intra-African remittance corridors and fees
• Financial inclusion and unbanked populations
• African fintech companies (Flutterwave, Paystack, Chipper Cash)
• DeFi applications in emerging markets
• AfCFTA and Pan-African trade infrastructure
• Currency volatility and CFA franc context

Usage

from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel
import torch

base_model = AutoModelForCausalLM.from_pretrained(
    "mistralai/Mistral-7B-v0.1",
    load_in_4bit=True,
    device_map="auto"
)
model = PeftModel.from_pretrained(base_model, "theelvace/mistral-african-fintech")
tokenizer = AutoTokenizer.from_pretrained("theelvace/mistral-african-fintech")

prompt = "### Instruction:\nWhat is the role of stablecoins in African payments?\n\n### Response:\n"
inputs = tokenizer(prompt, return_tensors="pt").to("cuda")
outputs = model.generate(**inputs, max_new_tokens=200, temperature=0.7, do_sample=True)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))

Author

Elvis Anselm — AI Engineer & Content Strategist, Lagos Nigeria
Portfolio · GitHub