Gyde Complaint Classifier: in-rbi (preview)
A LoRA adapter for Qwen3-8B that files an Indian bank or NBFC customer complaint into the complaint categories of the Reserve Bank of India's Integrated Ombudsman Scheme (RB-IOS 2026), says whether it is against a bank or an NBFC, and tags what went wrong (28 issue tags).
Preview: trained and evaluated on synthetic complaints with AI-agreed (silver) labels. It has not yet been evaluated on real complaints. Read "Evaluation" before relying on the numbers.
Part of the Gyde Complaint Classifier family: each regulator's own complaint categories, mapped to one shared
top-level taxonomy (universal-taxonomy.json), with one adapter per regulator.
| Jurisdiction | Taxonomy | Training data | Evaluated on | Status |
|---|---|---|---|---|
| US | CFPB product and issue | 8,000 real public complaints | 1,999 held-out real complaints | Released |
| India | RBI Integrated Ombudsman Scheme categories | 6,404 synthetic complaints (written by Reflection Beam) | Synthetic complaints with silver labels; no real-complaint evaluation yet | This preview |
What it outputs
{"category": "deposit_accounts", "entity_type": "bank", "tags": ["kyc_records", "account_freeze"]}
- category, one of the 10 the RBI complaint portal (cms.rbi.org.in) lists for a bank: Opening/Operation of Deposit accounts, ATM/CDM/Debit card, Credit Card, Mobile / Electronic Banking, Loans and Advances, Notes and Coins, Remittance and Collection of instruments, Pension related, Para-Banking, Other products and services. For an NBFC the portal lists 7: Notes and Coins, Remittance and Collection of instruments and Pension related are bank-only.
- entity_type:
bank,nbfcorunclear. - tags: what went wrong, for example
charges_fees,recovery_conduct,unauthorised_transaction,kyc_records. RBI has no separate category for charges, recovery agents, KYC or mis-selling; they sit under the product, and the tags carry them.
categories.json has the ids, names, definitions and tags. Category names are RBI's; definitions, tags and rules
are Gyde's.
Evaluation
463 synthetic complaints written by Claude Sonnet 5.5 across all categories, banks and NBFCs, many personas and styles (formal, angry, Hinglish, on behalf of a parent). A complaint counts only if three AI labellers agreed on its category, with each other and with the category it was written for (455 unanimous, 8 two-of-three with that category). These are silver labels: no person reviewed them.
| Model | Category accuracy | Category macro-F1 | Bank / NBFC | Tags F1 |
|---|---|---|---|---|
| Claude Sonnet 5.5, prompt only (full instructions) | 99.4% | 0.992 | 98.0% | 0.869 |
| GPT-5.4, prompt only (full instructions, default) | 99.1% | 0.989 | 96.0% | 0.847 |
| This adapter (Qwen3-8B + LoRA, rank 16) | 96.3% | 0.966 | 99.6% | 0.787 |
| Qwen3-8B, 5 retrieved training complaints as examples | 95.0% | 0.944 | 97.8% | 0.764 |
| Qwen3-8B, prompt only (full instructions) | 91.4% | 0.917 | 96.2% | 0.684 |
Read these numbers with three caveats:
- Frontier models from two providers score about 99% on these categories. Sonnet wrote the complaints and Claude Opus helped label them, but GPT-5.4, which had no part in building the set, also scores 99.1%. The RBI's 10 categories are well defined, and general-purpose models already know them: on category, this adapter is genuinely about 3 points behind. Its case here is cost, data residency and the bank/NBFC field (99.6%). Where labels follow an organisation's own conventions, training matters: on the 82 US complaint issues, the US adapter is 16 to 21 points ahead of GPT-5.4 (us-cfpb).
- Agreement removes the hardest cases. 37 complaints on which the labellers disagreed are held out; on those the adapter picks one of the labellers' choices 34 times (agreement, not accuracy).
- Synthetic hard cases were not very hard. A further 143 complaints were written as hard cases and labelled the same way (without one of the three labellers, so it could be compared fairly elsewhere). Written to a stated category, they mostly stay clear: Sonnet scored 143/143, GPT-5.4 and this adapter 142/143.
Hard-case set (143), category accuracy by case:
| Case | Complaints | This adapter | Qwen3-8B, 5 examples | Claude Sonnet 5.5, prompt only | GPT-5.4, prompt only |
|---|---|---|---|---|---|
| Hindi (Devanagari) | 20 | 20 | 20 | 20 | 20 |
| Noisy (typos, SMS style, Hinglish) | 19 | 19 | 17 | 19 | 19 |
| One line | 19 | 19 | 19 | 19 | 19 |
| Rambling, problem buried | 15 | 14 | 13 | 15 | 15 |
| Two issues, one main | 22 | 22 | 21 | 22 | 21 |
| Bank or NBFC not stated | 16 | 16 | 16 | 16 | 16 |
| Near-miss categories (e.g. account problem met in the app) | 23 | 23 | 22 | 23 | 23 |
| Firm outside the Ombudsman scheme | 9 | 9 | 8 | 9 | 9 |
| All | 143 | 142 (99.3%) | 136 (95.1%) | 143 (100%) | 142 (99.3%) |
The adapter's remaining confusions on the 463: Opening/Operation of Deposit accounts read as Mobile / Electronic Banking (4), Loans and Advances read as Deposit accounts (4), Pension related read as Deposit accounts (2).
Cost per 1,000 complaints (USD)
| Model | Real-time | Batch |
|---|---|---|
| This adapter, self-hosted | about the US adapter's ~$0.012 | ~$0.014 to ~$0.034 (as US) |
| Claude Sonnet 5.5, prompt only | ~$6.0 | ~$1.0 |
| GPT-5.4, prompt only | ~$4.5 | ~$2.2 |
Estimates. Sonnet 5.5 at list prices ($2/$10 per million input/output tokens, cached reads $0.20) on the tokens measured for these 463 complaints (about 2,550 input, of which about 2,240 are the shared instructions, and 94 output per complaint): real-time without caching; batch at the 50% discount with the instructions cached. GPT-5.4 at list prices ($2.50/$15 per million input/output tokens) on the tokens measured here (about 1,570 input, 35 output; no cached input was reported), batch at its 50% discount. Self-hosted: not measured for this adapter; its prompt (about 260 tokens) is shorter than the US adapter's, so the US figures (one NVIDIA A30 in India at about $0.41/hour, about 10 complaints/s when busy) are a fair guide. At low volume a dedicated GPU costs about $9.80/day whatever the traffic.
Use
Base model Qwen/Qwen3-8B (revision b968826d9c46dd6066d109eabc6255188de91218). Greedy decoding (temperature 0),
max_tokens 160, thinking disabled. The adapter was trained on exactly this prompt:
- System:
You file Indian bank and NBFC customer complaints under the RBI Integrated Ombudsman Scheme. Reply with JSON only: {"category": ..., "entity_type": ..., "tags": [...]}. - User:
Complaint: {text}\nAnswer:
vllm serve Qwen/Qwen3-8B --revision b968826d9c46dd6066d109eabc6255188de91218 \
--enable-lora --max-lora-rank 16 --lora-modules in-rbi=Gydeai/complaint-classifier-in-rbi
from openai import OpenAI
client = OpenAI(base_url="http://localhost:8000/v1", api_key="local")
SYSTEM = ('You file Indian bank and NBFC customer complaints under the RBI Integrated Ombudsman Scheme. '
'Reply with JSON only: {"category": ..., "entity_type": ..., "tags": [...]}.')
r = client.chat.completions.create(
model="in-rbi", temperature=0, max_tokens=160,
extra_body={"chat_template_kwargs": {"enable_thinking": False}},
messages=[{"role": "system", "content": SYSTEM},
{"role": "user", "content": "Complaint: UPI payment of Rs 4,500 failed but money was debited from my "
"savings account 10 days ago, still not reversed.\nAnswer:"}])
print(r.choices[0].message.content) # {"category": "mobile_electronic_banking", "entity_type": "bank", ...}
Validate the answer against categories.json (a category id, and for an NBFC not one of the three bank-only ones);
constrained JSON decoding makes an invalid answer impossible.
Training
6,404 synthetic complaints written by Reflection Beam (Beam-501B-A23B) from the category definitions, with personas,
styles and channels; Beam then classified each one without being told its intended category, and only matching ones
were kept (7,115 of 7,775; 711 held out for validation). No evaluation text was used to write them, and no output of
Claude or of the other labellers was used for training. Labels are the category and entity type each complaint was
written for, and the tags Beam reported. LoRA rank 16, alpha 32, dropout 0.05 on all attention and MLP projections;
1 epoch, learning rate 2e-4; about 25 minutes on one A100 40 GB (run.json). Training size: 2,964 complaints gave
93.7%, 4,444 gave 95.2%, 6,404 gave 96.3% on the 463.
Limits
- No real-complaint evaluation yet. Real complaints are messier than synthetic ones; expect lower accuracy.
- Indian bank and NBFC complaints only. Entities not covered by the Ombudsman scheme (for example peer-to-peer lending platforms) go to RBI's Consumer Education and Protection Cell; the adapter has no output for that and will still pick a category.
- English, Hinglish and some Hindi in training; mainly English in evaluation.
- For routing and triage, not for deciding a customer's case or any credit decision. A person reviews outcomes.
- Base model origin: Qwen (Alibaba Cloud). A version on another base model is available on request.
Data and licence
All complaints are synthetic; no real customer data was used. Adapter: Apache 2.0. Base model: Qwen3-8B under its own licence (Apache 2.0). Category names follow the RBI complaint portal; this adapter is not affiliated with or endorsed by the Reserve Bank of India.
Gyde serves this model inside India with data-residency, policy and failover controls, and trains versions on your own categories and complaints: gyde.ai.
Built by Gyde (DAI Labs Private Limited, Pune, India).
- Downloads last month
- 7