End of training

README.md

@@ -4,9 +4,6 @@ library_name: peft
 tags:
 - axolotl
 - generated_from_trainer
-- unsloth
-- trl
-- sft
 base_model: deepseek-ai/deepseek-coder-6.7b-instruct
 model-index:
 - name: diff-deepseek-code-ir
@@ -40,7 +37,7 @@ test_datasets:
     split: test
     type: oasst
 
-output_dir: ./outputs/dscoder-code-ir-3
+output_dir: ./outputs/dscoder-code-ir-4
 
 sequence_len: 4096
 sample_packing: true
@@ -110,7 +107,7 @@ special_tokens:
 
 This model is a fine-tuned version of [deepseek-ai/deepseek-coder-6.7b-instruct](https://huggingface.co/deepseek-ai/deepseek-coder-6.7b-instruct) on the None dataset.
 It achieves the following results on the evaluation set:
-- Loss: 0.2677
+- Loss: 0.2549
 
 ## Model description
 
@@ -147,10 +144,10 @@ The following hyperparameters were used during training:
 
 | Training Loss | Epoch | Step | Validation Loss |
 |:-------------:|:-----:|:----:|:---------------:|
-| 0.6921        | 0.03  | 1    | 0.7832          |
-| 0.5453        | 0.25  | 10   | 0.5221          |
-| 0.3129        | 0.51  | 20   | 0.2985          |
-| 0.2527        | 0.76  | 30   | 0.2677          |
+| 0.5233        | 0.02  | 1    | 0.5554          |
+| 0.3976        | 0.25  | 13   | 0.3534          |
+| 0.3354        | 0.51  | 26   | 0.2805          |
+| 0.2759        | 0.76  | 39   | 0.2549          |
 
 
 ### Framework versions
@@ -158,5 +155,5 @@ The following hyperparameters were used during training:
 - PEFT 0.10.0
 - Transformers 4.40.0.dev0
 - Pytorch 2.3.0+cu121
-- Datasets 2.15.0
+- Datasets 2.20.0
 - Tokenizers 0.15.0

adapter_model.bin

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:2a374dcebcc148069ae22aec3b398b504514f6fb5b9980a3a763e7f69b983b02
+oid sha256:df02e68614751317dbc9dadbdc89b6f78bff3b81bdd9c242a2cd91fc0a00dd03
 size 848460690