mustaphaidris/hausa-agri-ner

Hausa Agricultural Named Entity Recognition

A Hausa named entity recognition (NER) model fine-tuned from Davlan/afro-xlmr-base to identify livestock-related entities in text. It recognises ten entity types, including animal types, breeds, diseases, symptoms, medicines and feeds.

Version covered by this card: the retrained candidate using 1,636 training sentences. Its reported test entity F1 is 97.54%. These results describe the supplied training logs; they have not been independently reproduced. Before publishing this card, ensure the repository contains the corresponding retrained model and tokenizer rather than the original checkpoint.

Model details

• Repository: mustaphaidris/hausa-agri-ner
• Language: Hausa, with some English names and agricultural terminology.
• Base checkpoint: Davlan/afro-xlmr-base
• Architecture: XLMRobertaForTokenClassification; 12 layers, hidden size 768, 12 attention heads.
• Task: token classification with 21 BIO labels: O and B-/I- tags for ten entity types.
• License: not yet specified for this fine-tuned release. Review the base model and dataset terms before assigning a license.

Entity types

Label	Scope and examples
ANIMAL_TYPE	Sheep and goat expressions, such as rago, tunkiya and awaki
BREED	Breed designations, such as Balami, Yankasa, Red Sokoto, Sahel and Sahelian
DISEASE	Named or explicitly reported conditions, such as FMD, tsutsar ciki and cutar huhu
SYMPTOM	Observed signs, such as tari, zawo and wahalar numfashi
MEDICATION	Named drugs, drug classes and therapeutic preparations, such as ivermectin and maganin tsutsar ciki
FEED_TYPE	Named feeds and ingredients, such as dusa and kulin gyada
QUANTITY	Counts and measurements, such as guda biyu and buhu shida
DATE_TIME	Dates, relative times, durations, ages and frequencies
LOCATION	Geographic places and named farm spaces, such as Kano and pen 3
PERSONNEL	Personal names and explicit farm-work roles, such as likitan dabbobi

B- starts an entity, I- continues it, and O indicates text outside the schema. The schema uses flat, non-overlapping spans. Body parts, equipment and management activities are outside its scope. In the reviewed supplement, generic vaccination activity expressions remain O; cutar huhu is treated as a complete reported disease expression. An extracted condition does not establish a diagnosis.

Training data

The original gold freeze, dated 9 September 2026, contains 2,000 sentences and 8,568 entity mentions. It uses one annotator's submitted labels as the project reference. “Gold” does not imply independent second annotation, veterinary validation or measured inter-annotator agreement.

The original corpus contains 1,700 source-linked examples, including earlier revisions, and 300 newly authored examples. It is an assisted/synthetic corpus, not 2,000 independently collected farm observations.

The retrained candidate adds 36 AI-authored sentences reviewed and approved by the project owner to the original training split. These additions target breed terminology, medication/feed distinctions, reported conditions and negative examples. Eight additions contain no entities.

Split	Sentences	Entity mentions
Training, original	1,600	6,837
Reviewed training supplement	36	74
Training, combined	1,636	6,911
Validation	200	861
Test	200	870

Validation and test files remain unchanged. The expanded collection totals 2,036 sentences.

The original split groups recognised source-template descendants and similar entity-masked examples to reduce leakage. The split assignment uses seed 42 and no model scores, according to its release documentation. An audit found no shared source/group IDs or exact normalised sentence duplicates across the original splits. The supplement contains no case/punctuation-normalised duplicate sentences from those splits. These checks do not prove that all paraphrases or source dependencies are independent.

Training procedure

Words are tokenised into subwords. Only the first subword of each original word receives its BIO label; special tokens, padding and later subwords use -100 and do not contribute to the loss or reported metrics.

Training uses truncation. The notebook does not report a subword truncation audit.

Setting	Value
Learning rate	0.00002
Epochs	3
Training batch size per device	8
Evaluation batch size per device	8
Weight decay	0.01
Evaluation and saving	Every epoch
Checkpoint selection	Highest validation F1
Completed training steps	615

These settings come from the supplied notebook and retraining logs. Exact package versions and checkpoint revisions for the retrained run have not been separately verified.

Evaluation

The notebook computes entity precision, recall and F1 using seqeval through the evaluate library, after removing -100 positions. Accuracy measures token-label accuracy. The code uses default seqeval scoring, not explicitly configured strict IOB2 evaluation.

Metric	Final validation	Test
Precision	98.60%	97.15%
Recall	98.49%	97.93%
Entity F1	98.55%	97.54%
Token accuracy	99.43%	98.81%
Loss	0.031035	0.072005

Comparison with the original run

Run	Training sentences	Validation F1	Test F1
Original	1,600	98.26%	97.71%
Retrained candidate	1,636	98.55%	97.54%

The candidate improved validation F1 but decreased test F1 by approximately 0.17 percentage points. One run per configuration does not establish statistical significance or overall superiority.

User-supplied development predictions showed Sahelian changing from MEDICATION to BREED, cutar huhu changing from fragmented labels to DISEASE, and rigakafi no longer receiving FEED_TYPE. These examples motivated the supplement, so they are development checks, not independent evidence of generalisation.

Per-entity scores, confidence intervals, multiple-seed results and a separate natural-text evaluation have not yet been reported. The original held-out set has been inspected during development; further comparisons on it should be treated as benchmark comparisons rather than a fresh blind final evaluation.

Usage

Use word boundaries consistent with the training data and retain only the first subword prediction per word. The tokenizer pattern below reproduced all saved tokens and character offsets in the original 2,000 examples. This validates compatibility with that corpus; it does not prove that the pattern handles every possible Hausa spelling convention.

The standard pipeline's simple aggregation can expose predictions from subwords ignored during training. Its first aggregation produced a word-boundary fallback warning in the supplied environment. The following helper avoids that aggregation heuristic and returns exact slices of the original text.

import re
import torch
from transformers import AutoTokenizer, AutoModelForTokenClassification

repo_id = "mustaphaidris/hausa-agri-ner"
tokenizer = AutoTokenizer.from_pretrained(repo_id, use_fast=True)
model = AutoModelForTokenClassification.from_pretrained(repo_id)
model.eval()

word_pattern = re.compile(r"['’ʼ]?\w+(?:['’ʼ-]\w+)*|[^\w\s]")

def extract_entities(text):
    words = list(word_pattern.finditer(text))
    if not words:
        return []

    batch = tokenizer(
        [word.group() for word in words],
        is_split_into_words=True,
        return_tensors="pt",
        truncation=False,
    )
    word_ids = batch.word_ids()
    # XLM-R position embeddings reserve positions for padding.
    position_limit = model.config.max_position_embeddings - model.config.pad_token_id - 1
    limit = min(tokenizer.model_max_length, position_limit)
    if batch["input_ids"].shape[1] > limit:
        raise ValueError("Input is too long; split it into shorter sentences.")

    device = next(model.parameters()).device
    with torch.no_grad():
        probs = model(**{k: v.to(device) for k, v in batch.items()}).logits[0].softmax(-1)

    spans, current, seen = [], None, set()
    for token_index, word_id in enumerate(word_ids):
        if word_id is None or word_id in seen:
            continue
        seen.add(word_id)
        score, label_id = probs[token_index].max(-1)
        tag = model.config.id2label[int(label_id)]
        if tag == "O":
            if current is not None:
                spans.append(current)
                current = None
            continue
        prefix, label = tag.split("-", 1)
        start, end = words[word_id].span()
        if prefix == "I" and current is not None and current["label"] == label:
            current["end"] = end
            current["scores"].append(float(score))
        else:
            if current is not None:
                spans.append(current)
            current = {"start": start, "end": end, "label": label, "scores": [float(score)]}
    if current is not None:
        spans.append(current)
    return [
        {"text": text[s["start"]:s["end"]], "label": s["label"],
         "start": s["start"], "end": s["end"],
         "score": sum(s["scores"]) / len(s["scores"])}
        for s in spans
    ]

print(extract_entities("Awaki irin Sahelian suna a Zaria."))

The helper starts a new span when an I- tag has no matching preceding entity. This is a display/decoding policy, not strict BIO validation. Scores average the selected first-subword probabilities and are not calibrated probabilities of correctness. The helper has not been independently run against the retrained Hub checkpoint.

Intended uses and limitations

Intended uses include research, annotation assistance and extracting structured information from Hausa livestock text with human review. This model does not generate answers, diagnose animals or recommend treatment.

• Test vocabulary substantially overlaps training: 824 of 870 original test entity mentions have the same text and label in the original training set. All six distinct test breed forms and 31 medication forms occur in training.
• Disease and quantity have only 15 and 18 test mentions respectively. Aggregate F1 can hide errors on these labels.
• Validation and test contain no entity-free sentences. The expanded training set has nine, so false positives on unrelated text remain insufficiently evaluated.
• Performance on independently collected farm text, dialect variations, unfamiliar names, spelling errors and longer documents remains unverified.
• User-reviewed synthetic additions can retain linguistic or annotation mistakes. No independent agreement measurement is available.
• The model can assign high confidence to an incorrect entity type. Human review remains necessary when using extracted information.

Release status

This card documents a retrained candidate. Confirm that the uploaded weights correspond to this run before using its results to describe the live repository. Dataset download links, a release license and a pinned model revision remain to be supplied.