---
license: mit
language:
- en
metrics:
- confusion_matrix
---
## Model Details
This model is designed to classify customer service inquiries into five categories: Technical Issues, Billing & Payment, Product Inquiries, Account Management, and Policy Questions.

## Training Data
The model was trained on a balanced dataset of 53000 entries composed of anonymized customer service inquiries. Each category contained a similar number of examples to prevent class imbalance.
https://github.com/amosproj/amos2023ws01-ticket-chat-ai/tree/main/Backend/app/model/test_data/test_data_with_gpt

## Training Procedure
The model was fine-tuned over four epochs for a sequence classification task. We utilized a batch size of 4 and an Adam optimizer with a learning rate of 2e-5.

## Model Performance
The model's performance was evaluated using a confusion matrix and a learning curve, as detailed below:

- ### Confusion Matrix Analysis
    ![image/png](https://cdn-uploads.huggingface.co/production/uploads/654f564ebebf0c4c51d7290a/-MfnMl8PJO3DTo6tuhflu.png)
  
    - **Technical Issues**: Some misclassification noted, particularly with Policy Questions.
    - **Billing & Payment**: High accuracy with some instances misclassified as Technical Issues.
    - **Product Inquiries**: Very high accuracy with few misclassifications.
    - **Account Management**: Notable confusion with Policy Questions.
    - **Policy Questions**: Highest accuracy among all categories.
      
- ### Learning Curve Analysis
  ![image/png](https://cdn-uploads.huggingface.co/production/uploads/654f564ebebf0c4c51d7290a/_MX90aoAGcXTBszQ3vJzQ.png)
  - The training loss decreased consistently, indicating good model learning.
  - The validation loss remained close to the training loss and did not show signs of increasing, suggesting that the model was generalizing well without overfitting.
 
- ### Interprating Model's Output:
  - LABEL_0 stands for Account Management
  - LABEL_1 stands for Billing & Payment
  - LABEL_2 stands for Policy Questions
  - LABEL_3 stands for Product Inquiries
  - LABEL_4 stands for Technical Issues