smutuvi/sikia-interviewer-lora-v4

Sikia interviewer v4 (LoRA adapter, card format 12n)

A LoRA adapter on stock Gemma 4 E2B that reads a respondent's reply during an offline survey interview and returns one short JSON reply. It follows the Sikia offline interview design: the app leads the interview (question order, skip logic, repeat rounds, follow-ups, everything that is spoken) and the model only listens. Each call gets the standing instructions (instructions.txt) and one short card.

Versions. v4 (this repo) replaces ndizi-interviewer-lora-v3, which used a bean-only card on the ndizi base model and also wrote the spoken probes. The two are not interchangeable: v4 needs stock Gemma 4 E2B, the shared 12n card and instructions.txt from this repo.

Status: research pilot. Tested on real Swahili bean-trial replies and on drafted interviews. Not yet tested on real speech from other surveys, or on a phone. Don't use it for real data collection yet.

What it does

Card	Returns
Question (any form question: single or multiple choice, whole number, decimal, date, text)	{"act", "value", "say"} — act is answer, clarify, skip_request, impatient, correction (an earlier answer) or off_topic; say is always empty
CURRENT: yes/no (another round? skip this question?)	{"act": "yes"/"no"/"clarify", "say": ""}
TASK: list items (split a spoken list)	{"items": [...]} in the respondent's words
TASK: probe (follow-up check on a researcher's goal)	{"new_items": [...], "goal_met": bool, "follow_up": ""}

goal_met is true only when the goal states a target ("Target: at least 3 points.") and that many points have been heard; otherwise it is false and the app decides when to stop.

Example

LANG: sw
CURRENT: q=top_constraints type=select_multiple "Changamoto kubwa ni zipi?"
OPTIONS: 1=Ukame 2=Wadudu 3=Magonjwa 4=Udongo duni 5=Soko 6=Mbegu duni 7=Hakuna
CONTEXT: -
RESPONDENT SAID: "Wadudu ni wengi, na pia soko halina bei nzuri."

{"act": "answer", "value": "2 5", "say": ""}

Cards must match the training format byte for byte. slmlib/cards.py in the project code writes every card and applies the app's post-processing (option numbers back to codes, the respondent's own words for text, the year of a date).

Try it

Accept the Gemma licence on Hugging Face first, and use transformers 5.17 or later (earlier 5.x versions compute Gemma 4 E2B differently with the cache on, and the adapter then has no effect).

import torch
from huggingface_hub import hf_hub_download
from peft import PeftModel
from transformers import AutoModelForImageTextToText, AutoProcessor

BASE, ADAPTER = "google/gemma-4-E2B-it", "smutuvi/sikia-interviewer-lora-v4"
proc = AutoProcessor.from_pretrained(BASE)
model = AutoModelForImageTextToText.from_pretrained(BASE, dtype=torch.bfloat16, device_map="auto")
model = PeftModel.from_pretrained(model, ADAPTER).eval()
system = open(hf_hub_download(ADAPTER, "instructions.txt"), encoding="utf-8").read().strip()

card = """LANG: sw
CURRENT: q=household_size type=integer "Watu wangapi wanaishi kwenye kaya yako?"
RANGE: 1 to 30
CONTEXT: -
RESPONDENT SAID: "Tuko saba, mimi na mke wangu na watoto watano." """
msgs = [{"role": "system", "content": system}, {"role": "user", "content": card.strip()}]
prompt = proc.apply_chat_template(msgs, tokenize=False, add_generation_prompt=True, enable_thinking=False)
enc = proc.tokenizer(prompt, return_tensors="pt", add_special_tokens=False).to(model.device)
out = model.generate(**enc, max_new_tokens=80, do_sample=False)
print(proc.tokenizer.decode(out[0, enc["input_ids"].shape[1]:], skip_special_tokens=True))
# {"act": "answer", "value": "7", "say": ""}

With the project code, a whole interview on any form, typed or spoken:

python demo/form_interview.py --adapter smutuvi/sikia-interviewer-lora-v4 --form bean_tricot          # terminal
python demo/form_interview.py --adapter smutuvi/sikia-interviewer-lora-v4 --form clinic --lang en
python demo/form_app.py                                                      # browser; lists this adapter

Speech is transcribed separately (the base model with the adapter switched off, or the ndizi speech model): one skill per adapter.

Training data

About 6,000 examples; the data itself is not published (it contains what farmers said).

• Real speech (38%): hand-labelled Swahili bean tricot replies (like / dislike per variety, and follow-ups), split by farmer. Farmer and recording ids are one-way hashes; no names or household data.
• Drafted interviews (62%): Swahili and English, on household, crops, seed trader, extension officer, school, market-price and mobile-money forms, with every question and reply type. Drafted by an AI assistant; native-speaker review pending.
• Held out entirely for testing: the livestock and clinic-access forms. Validation: the mobile-money form.

Evaluation

Test set: 210 real bean replies and 198 real follow-up checks (hand-reviewed), plus drafted interviews. Baseline = stock Gemma 4 E2B without the adapter, given 4 worked examples per task.

On real bean speech	Baseline	This adapter
Correct reply type	0.79	0.98
Answers misread as refusal, correction or off-topic	14.5%	0%
Multiple-choice F1 (traits)	0.54	0.89
"Nothing" recognised	0.53	0.84
Goal met correct	0.70	0.91
Follow-up points, word overlap	0.71	0.70

On drafted forms never used in training, reply type is 1.00, multiple-choice F1 0.96, numbers 0.96, decimals and dates 1.00, list items 0.90. The drafted forms share sentence patterns, so these mostly show the model reads any form's options, types and ranges; real speech outside beans is not yet tested. Full scores by group and language: scores_v12.json. One training run; differences of about ±0.05 on the 210 real replies are within run-to-run noise.

Known weak spots: "nothing" on real speech (0.84); follow-up points learned from rough automatic labels in training; list splitting (the app keeps only items the respondent said).

Training procedure

LoRA (PEFT) on the language model's linear layers only; the vision and audio towers are untouched. Loss on the JSON reply only. Settings in train_config.json (rank 16, alpha 32, 3 epochs, effective batch 16, bf16, one GPU, transformers 5.17).