MakiniAI/makini-bias-v2

MAKINI v2 — African Language Bias Detection Model

Model Description

MAKINI v2 is a 4-class bias detection model covering Swahili, French, isiZulu, and Hausa, fine-tuned on the AfricaBias dataset (60,234 sentences). It succeeds MAKINI v1 (Swahili/French only) — this is a new backbone and two additional languages, not an incremental v1.1.

Validated for production use: Swahili, French, isiZulu — all four classes. Hausa: neutral-vs-not detection only. Hausa's stereotype, counter-stereotype, and derogation classes are not reliable in this release — see Known Limitations before using Hausa output for anything beyond flagging "neutral or not."

Base Model

Davlan/afro-xlmr-base, not vanilla xlm-roberta-base (used in v1). Chosen because standard XLM-R's pretraining did not meaningfully cover isiZulu; AfroXLMR extends coverage to isiZulu, Hausa, and Chichewa (planned next language) in one backbone.

Training Approach

Two-stage fine-tune, same recipe as v1:

• Stage 1 — binary (neutral vs. biased), warm-start. F1 macro: 0.963
• Stage 2 — 4-class, initialized from Stage 1's encoder, weighted cross-entropy for class imbalance

Note: this release uses the same Swahili/French label distribution as v1 (24,220 neutral / 9,605 stereotype / 1,288 counter-stereotype / 172 derogation, combined). The separate counterstereotype/derogation rebalancing effort for Swahili/French has not yet been incorporated into any shipped model.

Labels

ID	Label
0	neutral
1	stereotype
2	counter-stereotype
3	derogation

Performance — Held-Out Test Set (n=6,024)

Overall

Class	Precision	Recall	F1	Support
neutral	0.97	0.98	0.97	3,963
stereotype	0.95	0.94	0.95	1,837
counter-stereotype	1.00	0.89	0.94	141
derogation	0.78	0.64	0.70	83
accuracy			0.96	6,024
macro avg	0.92	0.86	0.89	6,024
weighted avg	0.96	0.96	0.96	6,024

The overall row is not a safe substitute for the per-language breakdown below — see Known Limitations.

Per Language

French (n=1,149) — accuracy 0.95, macro F1 0.96

Class	Precision	Recall	F1	Support
neutral	0.98	0.93	0.95	600
stereotype	0.91	0.97	0.94	443
counter-stereotype	1.00	1.00	1.00	91
derogation	0.88	1.00	0.94	15

Swahili (n=2,418) — accuracy 0.98, macro F1 0.74 (derogation support = 1, statistically meaningless — see Known Limitations)

Class	Precision	Recall	F1	Support
neutral	0.99	0.99	0.99	1,840
stereotype	0.97	0.96	0.97	541
counter-stereotype	1.00	0.97	0.99	36
derogation	0.00	0.00	0.00	1

isiZulu (n=1,486) — accuracy 0.99, macro F1 0.72 (counter-stereotype: no examples exist in source data — see Known Limitations)

Class	Precision	Recall	F1	Support
neutral	0.99	0.99	0.99	667
stereotype	0.99	0.99	0.99	783
counter-stereotype	—	—	—	0
derogation	0.94	0.89	0.91	36

Hausa (n=971) — accuracy 0.88, macro F1 0.32. Neutral detection works; the other three classes do not.

Class	Precision	Recall	F1	Support
neutral	0.90	0.98	0.94	856
stereotype	0.19	0.04	0.07	70
counter-stereotype	0.00	0.00	0.00	14
derogation	0.38	0.19	0.26	31

Known Limitations

Hausa: stereotype, counter-stereotype, and derogation detection are not usable. Recall on stereotype is 0.04 — the model misses 96% of actual Hausa stereotype content. Counter-stereotype recall is 0.00. Only neutral-vs-not-neutral is reliable for Hausa in this release. Root cause: Hausa's own 4-class split is heavily neutral-skewed (89%), and its 718/124/293 stereotype/counter-stereotype/derogation examples are a small fraction of the pooled multilingual training signal for those classes — global class weighting does not protect a single language's minority share within an already-weighted class. Addressing this needs either more Hausa non-neutral annotation or language-aware reweighting, not just longer training.

isiZulu has no counter-stereotype examples in the source data — this class cannot be evaluated for isiZulu and its true performance on isiZulu counter-stereotype content is unknown, not zero.

Swahili derogation test support is 1 example. The 0.00 F1 shown above reflects a single data point, not a real measurement of Swahili derogation performance.

isiZulu's domain field is unreliable for ~43% of source rows (see dataset card) — do not use domain-level breakdowns for isiZulu.

Limitations carried over from v1, not addressed by this release:

• Negation handling: stereotypes stated then negated are not reliably detected
• Sheng (informal Kenyan Swahili slang) is underrepresented in training data
• Implicit stereotyping conveyed through French sentence structure may be missed

Languages

Swahili, French, isiZulu, Hausa (Hausa scoped per Known Limitations above).

License

CC BY-NC 4.0. Commercial use requires explicit permission from Makini AI Limited.

Citation

@model{wachira2026makiniv2,
  title     = {MAKINI v2: African Language Bias Detection Model},
  author    = {Wachira, David Maina},
  year      = {2026},
  publisher = {Algedi Intelligence Labs},
  url       = {https://huggingface.co/MakiniAI/makini-v2},
}

Contact

David Maina Wachira — dave@makini.tech — https://makini.tech