Support Ticket Outcome Classifier
Reads a whole support conversation (customer and agent turns) and predicts how it ended: refund issued, workaround given, record updated, question answered, fixed by the team, account unlocked, replacement sent, bug logged, kept with a pause, seats reduced or cancelled.
Every name, number and address it was trained on is made up. Nothing comes from a real person, patient, customer or business.
Why it exists
The value of support data for AI training sits in the outcome: a conversation that ends in a fix, a refund or a lost customer teaches what worked. Many help desk exports keep the messages but not a clean outcome field. This model labels the outcome so tickets can be counted, sampled and checked by result.
Use it
from transformers import pipeline
classify = pipeline("text-classification", model="aitrainingdatabroker/support-ticket-outcome-classifier", top_k=3)
print(classify("customer: I was charged twice this month.\nagent: I see two charges on the 3rd. I have refunded the second one."))
How it was trained
Fine-tuned from distilroberta-base for 6 epochs on one RTX 3070. Training data: synthetic-support-tickets-with-outcomes (400 rows), all invented. The scripts that built the data, trained the model and scored it are in source/; the full scores are in scores.json.
Test scores
On 100 held-out rows: accuracy 1.0, macro F1 1.0.
| Label | F1 | Count |
|---|---|---|
account_unlocked |
1.0 | 8 |
answered |
1.0 | 20 |
bug_logged |
1.0 | 6 |
cancelled |
1.0 | 1 |
fixed_by_team |
1.0 | 7 |
refund_issued |
1.0 | 19 |
replacement_sent |
1.0 | 7 |
retained_with_pause |
1.0 | 3 |
seats_reduced |
1.0 | 6 |
updated |
1.0 | 9 |
workaround_given |
1.0 | 14 |
Limits
- Trained and tested on invented conversations from the same generator, so its perfect score says little about real tickets. Test it on a labelled sample of your own help desk export.
- English only, and conversations are cut at 256 tokens; long threads lose their end, which is where the outcome usually is.
- This model is not legal advice and not a certification. Whether a de-identification method meets HIPAA Safe Harbor or Expert Determination, CCPA and CPRA, GDPR or UK GDPR anonymisation, PIPEDA, Quebec Law 25 or another rule is a question for your counsel and, where a rule requires one, a qualified expert. A person should review its output before any data is shared.
Read more
License and credit
Apache 2.0. Please credit AI Training Data Broker and link to https://aitrainingdatabroker.com/ when you share or build on it.
About
AI Training Data Broker is a broker and technical advisor for companies that sell or license their data to AI labs. It guides the technical side, including export, format, de-identification and compliance, and publishes free checklists on de-identifying data and keeping it within the privacy rules that apply.
- Downloads last month
- 20
Model tree for aitrainingdatabroker/support-ticket-outcome-classifier
Base model
distilbert/distilroberta-base