---
license: mit
base_model: microsoft/xtremedistil-l6-h384-uncased
tags:
- generated_from_trainer
metrics:
- accuracy
model-index:
- name: xtremedistil-l6-h384-uncased-v1.1
  results: []
---

<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# xtremedistil-l6-h384-uncased-v1.1

This model is a fine-tuned version of [microsoft/xtremedistil-l6-h384-uncased](https://huggingface.co/microsoft/xtremedistil-l6-h384-uncased) on the None dataset.
It achieves the following results on the evaluation set:
- Loss: 0.5278
- F1 Macro: 0.6999
- F1 Micro: 0.7000
- Accuracy Balanced: 0.7017
- Accuracy: 0.7000
- Precision Macro: 0.7009
- Recall Macro: 0.7017
- Precision Micro: 0.7000
- Recall Micro: 0.7000

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 2e-05
- train_batch_size: 16
- eval_batch_size: 128
- seed: 40
- gradient_accumulation_steps: 2
- total_train_batch_size: 32
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- lr_scheduler_warmup_ratio: 0.06
- num_epochs: 3

### Training results

| Training Loss | Epoch | Step | Validation Loss | F1 Macro | F1 Micro | Accuracy Balanced | Accuracy | Precision Macro | Recall Macro | Precision Micro | Recall Micro |
|:-------------:|:-----:|:----:|:---------------:|:--------:|:--------:|:-----------------:|:--------:|:---------------:|:------------:|:---------------:|:------------:|
| 0.6275        | 0.17  | 200  | 0.6177          | 0.3647   | 0.5463   | 0.5039            | 0.5463   | 0.6163          | 0.5039       | 0.5463          | 0.5463       |
| 0.5811        | 0.34  | 400  | 0.5808          | 0.5807   | 0.6194   | 0.5976            | 0.6194   | 0.6331          | 0.5976       | 0.6194          | 0.6194       |
| 0.5769        | 0.51  | 600  | 0.5680          | 0.6564   | 0.6585   | 0.6703            | 0.6585   | 0.6796          | 0.6703       | 0.6585          | 0.6585       |
| 0.5647        | 0.68  | 800  | 0.5634          | 0.6703   | 0.6728   | 0.6855            | 0.6728   | 0.6976          | 0.6855       | 0.6728          | 0.6728       |
| 0.5607        | 0.85  | 1000 | 0.5720          | 0.6176   | 0.6448   | 0.6264            | 0.6448   | 0.6569          | 0.6264       | 0.6448          | 0.6448       |
| 0.5645        | 1.02  | 1200 | 0.5617          | 0.6523   | 0.6601   | 0.6521            | 0.6601   | 0.6581          | 0.6521       | 0.6601          | 0.6601       |
| 0.5665        | 1.19  | 1400 | 0.5479          | 0.6802   | 0.6840   | 0.6986            | 0.6840   | 0.7172          | 0.6986       | 0.6840          | 0.6840       |
| 0.5432        | 1.35  | 1600 | 0.5540          | 0.6642   | 0.6665   | 0.6644            | 0.6665   | 0.6641          | 0.6644       | 0.6665          | 0.6665       |
| 0.5427        | 1.52  | 1800 | 0.5520          | 0.6533   | 0.6617   | 0.6532            | 0.6617   | 0.6601          | 0.6532       | 0.6617          | 0.6617       |
| 0.5453        | 1.69  | 2000 | 0.5487          | 0.6756   | 0.6781   | 0.6755            | 0.6781   | 0.6757          | 0.6755       | 0.6781          | 0.6781       |
| 0.5528        | 1.86  | 2200 | 0.5492          | 0.6720   | 0.6771   | 0.6713            | 0.6771   | 0.6747          | 0.6713       | 0.6771          | 0.6771       |
| 0.531         | 2.03  | 2400 | 0.5476          | 0.6799   | 0.6803   | 0.6882            | 0.6803   | 0.6911          | 0.6882       | 0.6803          | 0.6803       |
| 0.5199        | 2.2   | 2600 | 0.5454          | 0.6823   | 0.6824   | 0.6863            | 0.6824   | 0.6856          | 0.6863       | 0.6824          | 0.6824       |
| 0.535         | 2.37  | 2800 | 0.5441          | 0.6797   | 0.6803   | 0.6817            | 0.6803   | 0.6804          | 0.6817       | 0.6803          | 0.6803       |
| 0.5246        | 2.54  | 3000 | 0.5453          | 0.6746   | 0.6750   | 0.6771            | 0.6750   | 0.6759          | 0.6771       | 0.6750          | 0.6750       |
| 0.5405        | 2.71  | 3200 | 0.5408          | 0.6824   | 0.6861   | 0.6819            | 0.6861   | 0.6836          | 0.6819       | 0.6861          | 0.6861       |
| 0.5414        | 2.88  | 3400 | 0.5404          | 0.6826   | 0.6834   | 0.6841            | 0.6834   | 0.6828          | 0.6841       | 0.6834          | 0.6834       |


### Framework versions

- Transformers 4.33.3
- Pytorch 2.5.1+cu121
- Datasets 2.14.7
- Tokenizers 0.13.3