YAML Metadata Warning:empty or missing yaml metadata in repo card
Check out the documentation for more information.
license: apache-2.0 base_model: google/gemma-3-270m-it library_name: transformers pipeline_tag: text-generation tags:
- gemma
- gemma-3
- financial
- mpesa
- kenya
- sms
- information-extraction
- structured-output
- lora
- gguf
- llama-cpp
- on-device
- edge-ai
- african-ai
- fintech
Gemma 3 270M Financial Intelligence
A lightweight, on-device financial SMS extraction model specialized for Kenyan M-Pesa transaction messages.
This model fine-tunes Google Gemma 3 270M IT with LoRA to transform incoming M-Pesa SMS notifications into a strict five-field financial transaction representation.
The resulting structured records can be stored in a local transaction ledger and analyzed using deterministic Python logic for cash flow, spending patterns, balances, and other financial insights.
The architecture is designed for offline-first, privacy-conscious financial intelligence on resource-constrained hardware.
Model Summary
| Property | Value |
|---|---|
| Base model | google/gemma-3-270m-it |
| Architecture | Gemma 3 |
| Parameters | ~268M |
| Fine-tuning | LoRA |
| LoRA rank | 8 |
| LoRA alpha | 16 |
| LoRA dropout | 0.05 |
| Training epochs | 5 |
| Learning rate | 1e-4 |
| Training dtype | FP32 |
| Maximum sequence length | 512 |
| Training dataset | 200 M-Pesa SMS records |
| Train / validation / test | 140 / 30 / 30 |
| Deployment format | GGUF |
| Quantization | Q4_K_M |
| Quantized size | ~235 MiB |
| Runtime | llama.cpp |
| Intended environment | Local / edge inference |
The training pipeline reported a final validation loss of approximately 0.0513 after five epochs.
What This Model Does
The model is trained to understand M-Pesa transaction SMS messages and extract five canonical fields:
{
"entity": "Ann Mueni",
"amount": 20000.0,
"balance": 159583.0,
"date": "2026-03-05",
"type": "income"
}
Output Schema
| Field | Description |
|---|---|
entity |
Person, merchant, bank, business, or organization involved |
amount |
Transaction amount in KES |
balance |
Observed M-Pesa balance after the transaction |
date |
Transaction date in YYYY-MM-DD format |
type |
income or expense |
The model is intentionally trained to produce only these canonical fields rather than transaction IDs, phone numbers, account numbers, merchant IDs, fees, or additional fields.
Example
Input
You have received Ksh20,000.00 from Ann Mueni.
Transaction cost Ksh0.00.
New balance Ksh159,583.00.
Output
{
"entity": "Ann Mueni",
"amount": 20000.0,
"balance": 159583.0,
"date": "2026-03-05",
"type": "income"
}
The notebook's post-training test confirmed that the model produced all five required fields, returned no extra fields, produced a valid transaction type, extracted the balance, and generated parseable JSON.
Architecture
The system deliberately separates language understanding from financial computation.
M-Pesa SMS
│
▼
Gemma 3 270M + LoRA
│
▼
Structured JSON
│
▼
Local Transaction Ledger
│
┌────────┴────────┐
▼ ▼
Cash-flow analysis Spending analysis
│ │
└────────┬────────┘
▼
Financial Intelligence
The Model Handles
- M-Pesa SMS language understanding
- Transaction entity extraction
- Amount extraction
- Balance extraction
- Date extraction
- Income/expense classification
- Structured JSON generation
Deterministic Python Handles
- Ledger construction
- Balance tracking
- Cash-flow calculations
- Weekly and monthly spending
- Spending by entity
- Spending by category
- Savings-rate calculations
- Financial summaries
- Validation
- Illustrative loan-affordability calculations
This separation prevents the 270M-parameter language model from being responsible for financial arithmetic and bookkeeping.
Training
The model was fine-tuned using a cleaned dataset containing 200 M-Pesa SMS records.
Training : 140
Validation : 30
Test : 30
Total : 200
The dataset pipeline performs:
- SMS and target detection
- JSON normalization
- Five-field schema canonicalization
- Duplicate removal
- Conflicting-label detection
- Leakage-safe dataset splitting
- JSONL export
The split was specifically designed to prevent normalized versions of the same SMS from appearing across multiple splits.
LoRA Configuration
LoRA rank : 8
LoRA alpha : 16
LoRA dropout : 0.05
Learning rate : 1e-4
Epochs : 5
Maximum sequence : 512
Gradient accumulation : 8
Batch size : 1
Weight decay : 0.01
Max gradient norm : 1.0
Training dtype : FP32
Seed : 42
LoRA adapters were applied to:
q_proj
k_proj
v_proj
o_proj
gate_proj
up_proj
down_proj
Only approximately 0.7% of the model parameters were trainable during fine-tuning.
Training Results
Validation loss decreased consistently during training:
| Epoch | Training Loss | Validation Loss |
|---|---|---|
| 1 | 1.666162 | 0.373569 |
| 2 | 0.265191 | 0.099188 |
| 3 | 0.104493 | 0.069702 |
| 4 | 0.070435 | 0.056365 |
| 5 | 0.060221 | 0.051316 |
The project also performs numerical sanity checks covering parameter finiteness, forward-pass stability, gradient finiteness, and gradient norms.
Deployment
The trained model follows this export pipeline:
LoRA Adapter
│
▼
Merged Hugging Face Model
│
▼
FP16 GGUF
│
▼
Q4_K_M GGUF
│
▼
llama.cpp
The final deployment artifact is:
gemma3-financial-intelligence-Q4_K_M.gguf
The Q4_K_M artifact was approximately 235.16 MiB, while the FP16 GGUF was approximately 517.69 MB.
Performance
An ADTC participant-mode benchmark was performed using the Q4_K_M GGUF.
| Metric | Result |
|---|---|
| Parameters | 268,098,176 |
| Generation throughput | 44.32 tokens/sec |
| First-token latency | 1,985.11 ms |
| Peak RSS | 382.34 MB |
| Steady-state RSS | 354.44 MB |
| CPU p99 | 58.8% |
| CPU throttling | False |
| GPU | None |
The benchmark environment was a CPU-based Kaggle environment using an Intel Xeon CPU and 31.3 GB RAM.
Important: These measurements demonstrate successful local GGUF execution but should not be interpreted as performance measurements from an 8 GB consumer laptop. An actual 8 GB laptop benchmark remains future work.
Financial Intelligence Layer
The model is intended to be one component of a larger local financial intelligence system.
Once transaction SMS messages have been converted into structured records, deterministic analytics can answer questions such as:
Where is most of my money going?
How much did I spend this week?
How much did I spend this month?
What is my account balance?
Show me my cash flow.
What amount of loan can I apply?
Example:
Your largest spending entity is SHELL WESTLANDS,
where you spent KES 5,600.00.
Your last observed M-PESA balance is
KES 139,583.00.
Your recorded income is KES 15,300.00,
your recorded expenses are KES 13,890.00,
and your net cash flow is KES 1,410.00.
The loan component is an illustrative analytical heuristic only and should not be interpreted as a lending decision, credit approval, or guaranteed borrowing amount.
Intended Use
This model is intended for:
- M-Pesa SMS transaction extraction
- Local transaction ledger construction
- Offline-first financial applications
- Personal financial tracking
- SME financial intelligence prototypes
- Edge AI / on-device fintech applications
- Structured information extraction from Kenyan financial SMS
- Research into lightweight African-language and financial AI systems
The broader design minimizes the need for users to upload large M-Pesa statements and allows transaction processing to happen locally on the device.
Privacy
The architecture is designed around local processing.
M-Pesa SMS
│
▼
Local model
│
▼
Local transaction ledger
│
▼
Local analytics
│
▼
Local financial intelligence
The core workflow does not require sending transaction messages to a remote inference API.
However, deploying this system in a real financial application still requires appropriate:
- Device security
- Encryption
- Access controls
- Data retention policies
- User consent
- Secure local storage
- Protection against unauthorized SMS access
Limitations
This model is a research/prototype system, not a production banking model.
Important limitations include:
- The training dataset contains only 200 records.
- The model is specialized for M-Pesa-style financial SMS extraction.
- Extraction quality depends on information available in the source SMS.
- Broader SMS-template diversity has not yet been comprehensively evaluated.
- The deterministic analytics layer depends on correct extraction.
- The loan component is only an illustrative heuristic.
- The reported ADTC ARC-Easy score is not a measure of M-Pesa extraction accuracy.
- The reported profiler performance was measured in Kaggle rather than on an 8 GB consumer laptop.
Evaluation
The project currently includes:
- Training/validation loss evaluation
- JSON validity checks
- Required-field validation
- Transaction-type validation
- Balance extraction checks
- A post-training extraction test
- ADTC profiler benchmarking
- General ARC-Easy benchmarking
The ADTC profiler reported an ARC-Easy score of 0.52 on 50 samples. This is a general-purpose benchmark and should not be interpreted as the model's M-Pesa extraction accuracy.
A larger task-specific evaluation should measure:
Exact-record accuracy
Entity accuracy
Amount accuracy
Balance accuracy
Date accuracy
Transaction-type accuracy
JSON validity
Roadmap
Future development includes:
- Expand the M-Pesa SMS dataset
- Evaluate exact-record extraction accuracy
- Measure field-level extraction accuracy
- Add more M-Pesa transaction templates
- Test spelling and formatting variations
- Add multilingual/local-language SMS handling
- Improve structured-output reliability
- Benchmark on an actual 8 GB laptop
- Implement continuous SMS ingestion
- Add encrypted local storage
- Evaluate financial intelligence against independently verified ledgers
These are consistent with the project's documented future-work plan.
Citation
If you use this model in research or applications, please reference the project as:
@misc{wanga2026gemmafinancialintelligence,
title = {Gemma 3 270M Financial Intelligence},
author = {Wanga, Peter Otieno},
year = {2026},
note = {LoRA-specialized Gemma 3 270M model for Kenyan M-Pesa SMS
transaction extraction and on-device financial intelligence}
}
License
This repository should include the applicable license before publication.
The base model's licensing terms should also be reviewed and respected when redistributing derivatives.
Project Concept
Turn lightweight local language understanding into structured financial data, then let deterministic analytics turn that data into useful financial intelligence.
The goal is to demonstrate that useful financial AI can run locally on resource-constrained hardware without requiring cloud-scale infrastructure, while keeping the core transaction-processing workflow on-device.
- Downloads last month
- 85
4-bit