Hikiyo commited on
Commit
db00956
1 Parent(s): c4b7230

update model card README.md

Browse files
Files changed (1) hide show
  1. README.md +3 -28
README.md CHANGED
@@ -6,7 +6,6 @@ tags:
6
  model-index:
7
  - name: llama-3-WP
8
  results: []
9
- library_name: peft
10
  ---
11
 
12
  <!-- This model card has been generated automatically according to the information the Trainer had access to. You
@@ -16,7 +15,7 @@ should probably proofread and complete it, then remove this comment. -->
16
 
17
  This model is a fine-tuned version of [meta-llama/Meta-Llama-3-8B](https://huggingface.co/meta-llama/Meta-Llama-3-8B) on an unknown dataset.
18
  It achieves the following results on the evaluation set:
19
- - Loss: 2.2532
20
 
21
  ## Model description
22
 
@@ -32,28 +31,6 @@ More information needed
32
 
33
  ## Training procedure
34
 
35
-
36
- The following `bitsandbytes` quantization config was used during training:
37
- - load_in_8bit: False
38
- - load_in_4bit: True
39
- - llm_int8_threshold: 6.0
40
- - llm_int8_skip_modules: None
41
- - llm_int8_enable_fp32_cpu_offload: False
42
- - llm_int8_has_fp16_weight: False
43
- - bnb_4bit_quant_type: nf4
44
- - bnb_4bit_use_double_quant: True
45
- - bnb_4bit_compute_dtype: float16
46
-
47
- The following `bitsandbytes` quantization config was used during training:
48
- - load_in_8bit: False
49
- - load_in_4bit: True
50
- - llm_int8_threshold: 6.0
51
- - llm_int8_skip_modules: None
52
- - llm_int8_enable_fp32_cpu_offload: False
53
- - llm_int8_has_fp16_weight: False
54
- - bnb_4bit_quant_type: nf4
55
- - bnb_4bit_use_double_quant: True
56
- - bnb_4bit_compute_dtype: float16
57
  ### Training hyperparameters
58
 
59
  The following hyperparameters were used during training:
@@ -70,14 +47,12 @@ The following hyperparameters were used during training:
70
 
71
  | Training Loss | Epoch | Step | Validation Loss |
72
  |:-------------:|:-----:|:----:|:---------------:|
73
- | 2.2498 | 0.4 | 200 | 2.2861 |
74
- | 2.2186 | 0.81 | 400 | 2.2532 |
75
 
76
 
77
  ### Framework versions
78
 
79
- - PEFT 0.4.0
80
- - PEFT 0.4.0
81
  - Transformers 4.31.0
82
  - Pytorch 2.1.2
83
  - Datasets 2.19.1
 
6
  model-index:
7
  - name: llama-3-WP
8
  results: []
 
9
  ---
10
 
11
  <!-- This model card has been generated automatically according to the information the Trainer had access to. You
 
15
 
16
  This model is a fine-tuned version of [meta-llama/Meta-Llama-3-8B](https://huggingface.co/meta-llama/Meta-Llama-3-8B) on an unknown dataset.
17
  It achieves the following results on the evaluation set:
18
+ - Loss: 2.1808
19
 
20
  ## Model description
21
 
 
31
 
32
  ## Training procedure
33
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
34
  ### Training hyperparameters
35
 
36
  The following hyperparameters were used during training:
 
47
 
48
  | Training Loss | Epoch | Step | Validation Loss |
49
  |:-------------:|:-----:|:----:|:---------------:|
50
+ | 2.2077 | 0.4 | 200 | 2.2084 |
51
+ | 2.3065 | 0.81 | 400 | 2.1808 |
52
 
53
 
54
  ### Framework versions
55
 
 
 
56
  - Transformers 4.31.0
57
  - Pytorch 2.1.2
58
  - Datasets 2.19.1