End of training

README.md

@@ -0,0 +1,86 @@
+---
+license: mit
+base_model: microsoft/xtremedistil-l6-h384-uncased
+tags:
+- generated_from_trainer
+metrics:
+- accuracy
+model-index:
+- name: xtremedistil-l6-h384-uncased-v1.1
+  results: []
+---
+
+<!-- This model card has been generated automatically according to the information the Trainer had access to. You
+should probably proofread and complete it, then remove this comment. -->
+
+# xtremedistil-l6-h384-uncased-v1.1
+
+This model is a fine-tuned version of [microsoft/xtremedistil-l6-h384-uncased](https://huggingface.co/microsoft/xtremedistil-l6-h384-uncased) on the None dataset.
+It achieves the following results on the evaluation set:
+- Loss: 0.5278
+- F1 Macro: 0.6999
+- F1 Micro: 0.7000
+- Accuracy Balanced: 0.7017
+- Accuracy: 0.7000
+- Precision Macro: 0.7009
+- Recall Macro: 0.7017
+- Precision Micro: 0.7000
+- Recall Micro: 0.7000
+
+## Model description
+
+More information needed
+
+## Intended uses & limitations
+
+More information needed
+
+## Training and evaluation data
+
+More information needed
+
+## Training procedure
+
+### Training hyperparameters
+
+The following hyperparameters were used during training:
+- learning_rate: 2e-05
+- train_batch_size: 16
+- eval_batch_size: 128
+- seed: 40
+- gradient_accumulation_steps: 2
+- total_train_batch_size: 32
+- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
+- lr_scheduler_type: linear
+- lr_scheduler_warmup_ratio: 0.06
+- num_epochs: 3
+
+### Training results
+
+| Training Loss | Epoch | Step | Validation Loss | F1 Macro | F1 Micro | Accuracy Balanced | Accuracy | Precision Macro | Recall Macro | Precision Micro | Recall Micro |
+|:-------------:|:-----:|:----:|:---------------:|:--------:|:--------:|:-----------------:|:--------:|:---------------:|:------------:|:---------------:|:------------:|
+| 0.6275        | 0.17  | 200  | 0.6177          | 0.3647   | 0.5463   | 0.5039            | 0.5463   | 0.6163          | 0.5039       | 0.5463          | 0.5463       |
+| 0.5811        | 0.34  | 400  | 0.5808          | 0.5807   | 0.6194   | 0.5976            | 0.6194   | 0.6331          | 0.5976       | 0.6194          | 0.6194       |
+| 0.5769        | 0.51  | 600  | 0.5680          | 0.6564   | 0.6585   | 0.6703            | 0.6585   | 0.6796          | 0.6703       | 0.6585          | 0.6585       |
+| 0.5647        | 0.68  | 800  | 0.5634          | 0.6703   | 0.6728   | 0.6855            | 0.6728   | 0.6976          | 0.6855       | 0.6728          | 0.6728       |
+| 0.5607        | 0.85  | 1000 | 0.5720          | 0.6176   | 0.6448   | 0.6264            | 0.6448   | 0.6569          | 0.6264       | 0.6448          | 0.6448       |
+| 0.5645        | 1.02  | 1200 | 0.5617          | 0.6523   | 0.6601   | 0.6521            | 0.6601   | 0.6581          | 0.6521       | 0.6601          | 0.6601       |
+| 0.5665        | 1.19  | 1400 | 0.5479          | 0.6802   | 0.6840   | 0.6986            | 0.6840   | 0.7172          | 0.6986       | 0.6840          | 0.6840       |
+| 0.5432        | 1.35  | 1600 | 0.5540          | 0.6642   | 0.6665   | 0.6644            | 0.6665   | 0.6641          | 0.6644       | 0.6665          | 0.6665       |
+| 0.5427        | 1.52  | 1800 | 0.5520          | 0.6533   | 0.6617   | 0.6532            | 0.6617   | 0.6601          | 0.6532       | 0.6617          | 0.6617       |
+| 0.5453        | 1.69  | 2000 | 0.5487          | 0.6756   | 0.6781   | 0.6755            | 0.6781   | 0.6757          | 0.6755       | 0.6781          | 0.6781       |
+| 0.5528        | 1.86  | 2200 | 0.5492          | 0.6720   | 0.6771   | 0.6713            | 0.6771   | 0.6747          | 0.6713       | 0.6771          | 0.6771       |
+| 0.531         | 2.03  | 2400 | 0.5476          | 0.6799   | 0.6803   | 0.6882            | 0.6803   | 0.6911          | 0.6882       | 0.6803          | 0.6803       |
+| 0.5199        | 2.2   | 2600 | 0.5454          | 0.6823   | 0.6824   | 0.6863            | 0.6824   | 0.6856          | 0.6863       | 0.6824          | 0.6824       |
+| 0.535         | 2.37  | 2800 | 0.5441          | 0.6797   | 0.6803   | 0.6817            | 0.6803   | 0.6804          | 0.6817       | 0.6803          | 0.6803       |
+| 0.5246        | 2.54  | 3000 | 0.5453          | 0.6746   | 0.6750   | 0.6771            | 0.6750   | 0.6759          | 0.6771       | 0.6750          | 0.6750       |
+| 0.5405        | 2.71  | 3200 | 0.5408          | 0.6824   | 0.6861   | 0.6819            | 0.6861   | 0.6836          | 0.6819       | 0.6861          | 0.6861       |
+| 0.5414        | 2.88  | 3400 | 0.5404          | 0.6826   | 0.6834   | 0.6841            | 0.6834   | 0.6828          | 0.6841       | 0.6834          | 0.6834       |
+
+
+### Framework versions
+
+- Transformers 4.33.3
+- Pytorch 2.5.1+cu121
+- Datasets 2.14.7
+- Tokenizers 0.13.3

config.json

@@ -27,7 +27,7 @@
   "pad_token_id": 0,
   "position_embedding_type": "absolute",
   "problem_type": "single_label_classification",
-  "torch_dtype": "float32",
+  "torch_dtype": "float16",
   "transformers_version": "4.33.3",
   "type_vocab_size": 2,
   "use_cache": true,

pytorch_model.bin

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:cb3aaef3d30e2b57474be8170ba528a2d3c3a022b7f57c2a4847230554f1b531
-size 90891822
+oid sha256:cc9784ccb8ade0cd06803fc824d515cc4d2dd31b1976be8e15a4e134ba7d61f1
+size 45463854