Yuvaraj-ml/choicefit-roberta

ChoiceFit RoBERTa

RoBERTa-base (125M parameters) fine-tuned to pick the best option for a message. Each (question + message, option description) pair gets one score from a single-output classification head; a softmax across a question's options gives probabilities. Trained on about 486k examples (2-8 options, 256 tokens) from tasksource/tasksource-jev-typed-decisions, a collection of roughly 400 source datasets with mixed licenses (some non-commercial).

Intended use: research and education only. Not for commercial use. This is a personal learning project.

Results (hard accuracy)

Test set	Untrained RoBERTa	ChoiceFit	JEV (TypeSafe)
Held-out mixed benchmark (6,597 rows)	37.0%	66.7%	78.3%
banking77, 4 options (500 rows)	12.2%	67.0%	96.2%
clinc_oos, 4 options (500 rows)	16.2%	82.8%	99.4%

Limitations

• Trained on 2-8 options; more options are untested and likely less accurate.
• Weak on knowledge-heavy and many-similar-label questions.
• Overconfident: stated probabilities run higher than actual accuracy.

Code: https://github.com/yuvarajd2588/ChoiceFit-RoBERTa