tinydistill-banking77-static

A small text classifier distilled from Qwen/Qwen2.5-7B-Instruct into static with tinydistill, exported to ONNX for fast CPU inference.

Labels (77): activate_my_card, age_limit, apple_pay_or_google_pay, atm_support, automatic_top_up, balance_not_updated_after_bank_transfer, balance_not_updated_after_cheque_or_cash_deposit, beneficiary_not_allowed, cancel_transfer, card_about_to_expire, card_acceptance, card_arrival, card_delivery_estimate, card_linking, card_not_working, card_payment_fee_charged, card_payment_not_recognised, card_payment_wrong_exchange_rate, card_swallowed, cash_withdrawal_charge, cash_withdrawal_not_recognised, change_pin, compromised_card, contactless_not_working, country_support, declined_card_payment, declined_cash_withdrawal, declined_transfer, direct_debit_payment_not_recognised, disposable_card_limits, edit_personal_details, exchange_charge, exchange_rate, exchange_via_app, extra_charge_on_statement, failed_transfer, fiat_currency_support, get_disposable_virtual_card, get_physical_card, getting_spare_card, getting_virtual_card, lost_or_stolen_card, lost_or_stolen_phone, order_physical_card, passcode_forgotten, pending_card_payment, pending_cash_withdrawal, pending_top_up, pending_transfer, pin_blocked, receiving_money, Refund_not_showing_up, request_refund, reverted_card_payment?, supported_cards_and_currencies, terminate_account, top_up_by_bank_transfer_charge, top_up_by_card_charge, top_up_by_cash_or_cheque, top_up_failed, top_up_limits, top_up_reverted, topping_up_by_card, transaction_charged_twice, transfer_fee_charged, transfer_into_account, transfer_not_received_by_recipient, transfer_timing, unable_to_verify_identity, verify_my_identity, verify_source_of_funds, verify_top_up, virtual_card_not_working, visa_or_mastercard, why_verify_identity, wrong_amount_of_cash_received, wrong_exchange_rate_for_cash_withdrawal

Usage

from tinydistill import Predictor  # pip install tinydistill

clf = Predictor("hf:Shroomy/tinydistill-banking77-static")
clf.predict(["some text"])

Or from the command line:

tinydistill predict hf:Shroomy/tinydistill-banking77-static "some text"
tinydistill serve hf:Shroomy/tinydistill-banking77-static

Results

Model	Device	Accuracy	Macro-F1	Agreement with teacher	Latency, batch 1 (ms)	Latency, batch 32 (ms)	Throughput (texts/s)	Size (MB)	Speed-up, batch 1
teacher, 8-shot, 4-bit	cuda:0	0.783	0.779	n/a	n/a	n/a	n/a	n/a	n/a
student, PyTorch	cpu	0.803 ± 0.001	0.801 ± 0.001	0.824 ± 0.001	0.2	6.0	5309.9	31	n/a
student, ONNX fp32	cpu	0.802	0.801	0.823	0.1	4.9	6529.4	31	n/a
student, ONNX int8	cpu	0.802	0.801	0.823	0.1	5.1	6328.5	8	n/a

Task banking77, 3080 gold-labelled test texts. Teacher: Qwen/Qwen2.5-7B-Instruct. Student: static. The teacher's prompt shows 8 gold training examples per class (616 texts). The student learns from the teacher's soft labels on all 10003 training texts, plus cross-entropy on 616 gold-labelled texts (8 per class; weight 1.0). Values with ± are the mean and standard deviation over 3 training seeds; the ONNX rows are the exported first-seed model. CPU rows use 2 threads. Latency is the median over repeated runs. Size is the teacher's weights as loaded and the students' model files. The teacher packs several texts into one forward pass; it is timed at batch 32 only when that takes under three minutes, otherwise its throughput is 1 / batch-1 latency.

Cascade: student first, unsure texts to the teacher

Share sent to the teacher	Student confidence threshold	Accuracy	Macro-F1	Mean latency per text (ms)
0%	0.049	0.802	0.801	n/a
10%	0.336	0.801	0.799	n/a
20%	0.469	0.803	0.800	n/a
30%	0.576	0.804	0.800	n/a
50%	0.787	0.793	0.789	n/a
100%	1.000	0.783	0.779	n/a

The student answers every text and passes its least confident ones to the teacher. Thresholds are set on the test texts to hit each share; in deployment, set them on a validation split. Latency is the student's batch-1 latency plus the share times the batch-1 latency of the teacher.

License

Check the licences of the student model, the teacher and the training data before using this model. The uploader chose cc-by-4.0.