Model Details
This model is designed to classify customer service inquiries into five categories: Technical Issues, Billing & Payment, Product Inquiries, Account Management, and Policy Questions.
Training Data
The model was trained on a balanced dataset of 28000 entries composed of anonymized customer service inquiries. Each category contained a similar number of examples to prevent class imbalance. https://github.com/amosproj/amos2023ws01-ticket-chat-ai/tree/main/Backend/app/model/test_data/test_data_with_gpt
Training Procedure
The model was fine-tuned over four epochs for a sequence classification task. We utilized a batch size of 4 and an Adam optimizer with a learning rate of 2e-5.
Model Performance
The model's performance was evaluated using a confusion matrix and a learning curve, as detailed below:
Confusion Matrix Analysis
Learning Curve Analysis
- Training Loss: This line starts at approximately 1.2 and steadily decreases to about 0.6, suggesting that the model is learning and improving its ability to classify the training data correctly.
- Validation Loss: The validation loss begins around 0.9 and decreases to just under 0.7 after three epochs. The validation loss mirrors the training loss's downward trend, which is a good sign that the model is generalizing well and not overfitting the training data.
- Downloads last month
- 67