Training in progress, step 10

24_10_23_results_1st_group.csv

@@ -4,3 +4,5 @@ run_number,items_per_minute,changed_settings,total_time_taken,rouge_scores_unnes
 3,1167.961557490705,num_train_epochs set 4,22.6034836769104,0,0.2017668631411228,0.2943660462891026,0.2275745277256124,0.2402018073982986,0.3337487272152939,0.2622230969724901,0.2800257917490945,0.3723264196525258,0.2985025000752551,0.0681875943274993,0.0902994027619567,0.0732690208470744,0.088492522537141,0.1148011192777671,0.0942046380946422,0.1095363410196173,0.143483247069912,0.1164429090551231
 4,1162.7142153040672,gradient_accumulation_steps set 2,22.705493450164795,0,0.2377524270451348,0.2987412399762022,0.2500155939457053,0.279676230316319,0.3364800986139911,0.2881491459471711,0.3237935645348746,0.3735174802097811,0.3265839526773124,0.0855126216551136,0.0949357351919399,0.0863749530085826,0.1105430616720297,0.1193682783654687,0.1089689115281058,0.1387884155416608,0.1450061706561381,0.1338769366004306
 5,0.0,0,0.0,0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0
+6,0.0,0,0.0,0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0
+7,0.0,0,0.0,0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0

README.md

@@ -424,6 +424,18 @@ The following `bitsandbytes` quantization config was used during training:
 - bnb_4bit_use_double_quant: True
 - bnb_4bit_compute_dtype: bfloat16
 
+The following `bitsandbytes` quantization config was used during training:
+- quant_method: bitsandbytes
+- load_in_8bit: False
+- load_in_4bit: True
+- llm_int8_threshold: 6.0
+- llm_int8_skip_modules: None
+- llm_int8_enable_fp32_cpu_offload: False
+- llm_int8_has_fp16_weight: False
+- bnb_4bit_quant_type: nf4
+- bnb_4bit_use_double_quant: True
+- bnb_4bit_compute_dtype: bfloat16
+
 The following `bitsandbytes` quantization config was used during training:
 - quant_method: bitsandbytes
 - load_in_8bit: False
@@ -472,5 +484,6 @@ The following `bitsandbytes` quantization config was used during training:
 - PEFT 0.5.0
 - PEFT 0.5.0
 - PEFT 0.5.0
+- PEFT 0.5.0
 
 - PEFT 0.5.0

adapter_model.bin

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:90b976e7c8748ad1b00936ae23d84dcbdd6918a4d10d983ebc66489de6f3f6ff
+oid sha256:feb8ccca740ee04cac83d7a7f25646c3abefa92177d846bedb4ba0cc05984db5
 size 100733709

adapter_model.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:6285e1a80a9f7e48077509b90c5687599369635ce95c6a9f6f78f7691632f8ac
+oid sha256:87b9ed10847317ae86f52d8d365efbb83dbd73a0b37e8742fa26e66cbe3c2a37
 size 100690288

training_args.bin

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:20545a7b3c481007b95896ba869ef3f39097a7d6a9b2091de4c4f169fb292497
-size 4091
+oid sha256:de8bf979ccca0a6b419cb0fd720c17969bd5ece7562962498d87a71d7f3f4140
+size 4283