End of training

README.md

@@ -1,11 +1,13 @@
 ---
 base_model: sanchit-gandhi/Mistral-7B-v0.1-6-layer
 tags:
+- alignment-handbook
+- generated_from_trainer
 - trl
 - sft
 - generated_from_trainer
 datasets:
-- generator
+- HuggingFaceH4/ultrachat_200k
 model-index:
 - name: sanchit-gandhi/Mistral-7B-v0.1-6-layer
   results: []
@@ -16,7 +18,7 @@ should probably proofread and complete it, then remove this comment. -->
 
 # sanchit-gandhi/Mistral-7B-v0.1-6-layer
 
-This model is a fine-tuned version of [sanchit-gandhi/Mistral-7B-v0.1-6-layer](https://huggingface.co/sanchit-gandhi/Mistral-7B-v0.1-6-layer) on the generator dataset.
+This model is a fine-tuned version of [sanchit-gandhi/Mistral-7B-v0.1-6-layer](https://huggingface.co/sanchit-gandhi/Mistral-7B-v0.1-6-layer) on the HuggingFaceH4/ultrachat_200k dataset.
 It achieves the following results on the evaluation set:
 - Loss: 2.1183
 

config.json

@@ -21,6 +21,6 @@
   "tie_word_embeddings": false,
   "torch_dtype": "bfloat16",
   "transformers_version": "4.36.2",
-  "use_cache": false,
+  "use_cache": true,
   "vocab_size": 32000
 }

wandb/debug-internal.log

@@ -4501,3 +4501,9 @@
 2024-02-01 19:26:12,253 DEBUG   HandlerThread:239784 [handler.py:handle_request():146] handle_request: stop_status
 2024-02-01 19:26:12,253 DEBUG   SenderThread:239784 [sender.py:send_request():409] send_request: stop_status
 2024-02-01 19:26:13,626 INFO    Thread-12 :239784 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-ssft/wandb/run-20240201_175850-i93q0p12/files/output.log
+2024-02-01 19:26:15,629 INFO    Thread-12 :239784 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-ssft/wandb/run-20240201_175850-i93q0p12/files/output.log
+2024-02-01 19:26:16,138 DEBUG   HandlerThread:239784 [handler.py:handle_request():146] handle_request: status_report
+2024-02-01 19:26:17,631 INFO    Thread-12 :239784 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-ssft/wandb/run-20240201_175850-i93q0p12/files/output.log
+2024-02-01 19:26:21,019 DEBUG   SenderThread:239784 [sender.py:send():382] send: stats
+2024-02-01 19:26:21,637 INFO    Thread-12 :239784 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-ssft/wandb/run-20240201_175850-i93q0p12/files/output.log
+2024-02-01 19:26:22,020 DEBUG   HandlerThread:239784 [handler.py:handle_request():146] handle_request: status_report

wandb/run-20240201_175850-i93q0p12/files/output.log

@@ -1731,3 +1731,19 @@ Training completed. Do not forget to share your model on huggingface.co/models =
 [INFO|tokenization_utils_base.py:2432] 2024-02-01 19:26:11,031 >> tokenizer config file saved in ./tokenizer_config.json
 [INFO|tokenization_utils_base.py:2441] 2024-02-01 19:26:11,033 >> Special tokens file saved in ./special_tokens_map.json
 [INFO|modelcard.py:452] 2024-02-01 19:26:11,224 >> Dropping the following result as it does not have all the necessary fields:
+{'task': {'name': 'Causal Language Modeling', 'type': 'text-generation'}, 'dataset': {'name': 'generator', 'type': 'generator', 'config': 'default', 'split': 'train', 'args': 'default'}}
+events.out.tfevents.1706815561.ip-26-0-165-24.239318.1: 100%|██████████████████████████| 359/359 [00:00<00:00, 4.01kB/s]
+run-i93q0p12.wandb: 100%|██████████████████████████████████████████████████████████| 1.57M/1.57M [00:00<00:00, 11.3MB/s]
+Upload 2 LFS files: 100%|█████████████████████████████████████████████████████████████████| 2/2 [00:00<00:00,  6.81it/s]
+[INFO|modelcard.py:452] 2024-02-01 19:26:15,132 >> Dropping the following result as it does not have all the necessary fields:
+{'task': {'name': 'Causal Language Modeling', 'type': 'text-generation'}, 'dataset': {'name': 'HuggingFaceH4/ultrachat_200k', 'type': 'HuggingFaceH4/ultrachat_200k', 'config': 'default', 'split': 'train', 'args': 'default'}}
+[INFO|configuration_utils.py:483] 2024-02-01 19:26:15,137 >> Configuration saved in ./config.json
+[INFO|trainer.py:2889] 2024-02-01 19:26:16,239 >> Saving model checkpoint to ./
+[INFO|configuration_utils.py:483] 2024-02-01 19:26:16,242 >> Configuration saved in ./config.json
+[INFO|configuration_utils.py:594] 2024-02-01 19:26:16,244 >> Configuration saved in ./generation_config.json
+2024-02-01 19:26:14 - INFO - __main__ - Model saved to ./
+2024-02-01 19:26:15 - INFO - __main__ - Pushing to hub...
+[INFO|modeling_utils.py:2382] 2024-02-01 19:26:19,933 >> Model weights saved in ./pytorch_model.bin
+[INFO|tokenization_utils_base.py:2432] 2024-02-01 19:26:19,936 >> tokenizer config file saved in ./tokenizer_config.json
+[INFO|tokenization_utils_base.py:2441] 2024-02-01 19:26:19,938 >> Special tokens file saved in ./special_tokens_map.json
+[INFO|modelcard.py:452] 2024-02-01 19:26:20,002 >> Dropping the following result as it does not have all the necessary fields:

wandb/run-20240201_175850-i93q0p12/logs/debug-internal.log

@@ -4501,3 +4501,9 @@
 2024-02-01 19:26:12,253 DEBUG   HandlerThread:239784 [handler.py:handle_request():146] handle_request: stop_status
 2024-02-01 19:26:12,253 DEBUG   SenderThread:239784 [sender.py:send_request():409] send_request: stop_status
 2024-02-01 19:26:13,626 INFO    Thread-12 :239784 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-ssft/wandb/run-20240201_175850-i93q0p12/files/output.log
+2024-02-01 19:26:15,629 INFO    Thread-12 :239784 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-ssft/wandb/run-20240201_175850-i93q0p12/files/output.log
+2024-02-01 19:26:16,138 DEBUG   HandlerThread:239784 [handler.py:handle_request():146] handle_request: status_report
+2024-02-01 19:26:17,631 INFO    Thread-12 :239784 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-ssft/wandb/run-20240201_175850-i93q0p12/files/output.log
+2024-02-01 19:26:21,019 DEBUG   SenderThread:239784 [sender.py:send():382] send: stats
+2024-02-01 19:26:21,637 INFO    Thread-12 :239784 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-ssft/wandb/run-20240201_175850-i93q0p12/files/output.log
+2024-02-01 19:26:22,020 DEBUG   HandlerThread:239784 [handler.py:handle_request():146] handle_request: status_report

wandb/run-20240201_175850-i93q0p12/run-i93q0p12.wandb

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:18cc7a0b99001252605aad90d7a9da2c46834f3f63607f44557cc2ada6b58562
-size 1573084
+oid sha256:861cad702742507ee3bc6bdf7695a0c667fb97b99178e244dc422c91a63feac2
+size 1608059