PiGrieco/Llama3-q4_k_m

README.md

@@ -1,12 +1,11 @@
 ---
-license: llama2
-library_name: peft
+license: mit
+base_model: roberta-base
 tags:
-- trl
-- sft
-- unsloth
 - generated_from_trainer
-base_model: unsloth/llama-3-8b-bnb-4bit
+metrics:
+- accuracy
+- f1
 model-index:
 - name: Llama3-q4_k_m
   results: []
@@ -17,7 +16,11 @@ should probably proofread and complete it, then remove this comment. -->
 
 # Llama3-q4_k_m
 
-This model is a fine-tuned version of [unsloth/llama-3-8b-bnb-4bit](https://huggingface.co/unsloth/llama-3-8b-bnb-4bit) on an unknown dataset.
+This model is a fine-tuned version of [roberta-base](https://huggingface.co/roberta-base) on the None dataset.
+It achieves the following results on the evaluation set:
+- Loss: 0.0938
+- Accuracy: 0.9825
+- F1: 0.9827
 
 ## Model description
 
@@ -36,21 +39,31 @@ More information needed
 ### Training hyperparameters
 
 The following hyperparameters were used during training:
-- learning_rate: 0.0002
-- train_batch_size: 2
+- learning_rate: 5e-05
+- train_batch_size: 8
 - eval_batch_size: 8
-- seed: 3407
-- gradient_accumulation_steps: 4
-- total_train_batch_size: 8
+- seed: 42
 - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
 - lr_scheduler_type: linear
-- lr_scheduler_warmup_steps: 5
-- training_steps: 60
+- num_epochs: 8
+
+### Training results
+
+| Training Loss | Epoch | Step | Validation Loss | Accuracy | F1     |
+|:-------------:|:-----:|:----:|:---------------:|:--------:|:------:|
+| 0.3823        | 1.0   | 129  | 0.1932          | 0.9532   | 0.9535 |
+| 0.1585        | 2.0   | 258  | 0.3872          | 0.8977   | 0.9057 |
+| 0.3048        | 3.0   | 387  | 0.1816          | 0.9474   | 0.9477 |
+| 0.2353        | 4.0   | 516  | 0.1817          | 0.9591   | 0.9605 |
+| 0.2928        | 5.0   | 645  | 0.2058          | 0.9503   | 0.9524 |
+| 0.2452        | 6.0   | 774  | 0.1246          | 0.9737   | 0.9742 |
+| 0.348         | 7.0   | 903  | 0.0932          | 0.9825   | 0.9827 |
+| 0.1316        | 8.0   | 1032 | 0.0938          | 0.9825   | 0.9827 |
+
 
 ### Framework versions
 
-- PEFT 0.11.1
-- Transformers 4.41.0
+- Transformers 4.41.2
 - Pytorch 2.3.0+cu121
-- Datasets 2.19.1
-- Tokenizers 0.19.1
+- Datasets 2.19.2
+- Tokenizers 0.19.1

logs/events.out.tfevents.1717779800.e3a39cc1c013.424.3

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:4c1f4034fe09049a8cfe01ce0367531f93eda09b20ff8b3d4cbc285cbd6d6ef4
+size 29803

logs/events.out.tfevents.1717780430.e3a39cc1c013.424.4

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:243abaf63319bb4e0f5ac52853e2d782112c7de8a7b5587f0ed7ee8464e3cc35
+size 29894

logs/events.out.tfevents.1717781196.e3a39cc1c013.424.5

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:777a1308f8a540ec39d89fec884ea0f50c6ebf36ab62959bbd912be8f70a67d6
+size 29894

model.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:a3ed68c11cc3249509e926842da977d0250b14d6c170a2fce9ddb41d24a6512a
+oid sha256:6f9595da539b149233c9546da281b84469159c4fbba67540a934f193344c394c
 size 498612824