Femboyuwu2000/bloomz-1b1-vn-chat

README.md

@@ -39,12 +39,14 @@ The following hyperparameters were used during training:
 - train_batch_size: 2
 - eval_batch_size: 8
 - seed: 42
-- gradient_accumulation_steps: 4
-- total_train_batch_size: 8
 - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
 - lr_scheduler_type: cosine
-- training_steps: 1000
-- mixed_precision_training: Native AMP
+- lr_scheduler_warmup_ratio: 0.03
+- training_steps: 60
+
+### Training results
+
+
 
 ### Framework versions
 

wandb/debug-internal.log

@@ -108,3 +108,4 @@ subprocess.TimeoutExpired: Command '['conda', 'env', 'export']' timed out after
 2024-04-12 07:28:13,812 DEBUG   SenderThread:216 [sender.py:send_request():406] send_request: summary_record
 2024-04-12 07:28:13,812 INFO    SenderThread:216 [sender.py:_save_file():1390] saving file wandb-summary.json with policy end
 2024-04-12 07:28:14,043 INFO    Thread-12 :216 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240412_072736-gyl3aena/files/wandb-summary.json
+2024-04-12 07:28:18,814 DEBUG   HandlerThread:216 [handler.py:handle_request():146] handle_request: status_report

wandb/run-20240412_072736-gyl3aena/logs/debug-internal.log

@@ -108,3 +108,4 @@ subprocess.TimeoutExpired: Command '['conda', 'env', 'export']' timed out after
 2024-04-12 07:28:13,812 DEBUG   SenderThread:216 [sender.py:send_request():406] send_request: summary_record
 2024-04-12 07:28:13,812 INFO    SenderThread:216 [sender.py:_save_file():1390] saving file wandb-summary.json with policy end
 2024-04-12 07:28:14,043 INFO    Thread-12 :216 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240412_072736-gyl3aena/files/wandb-summary.json
+2024-04-12 07:28:18,814 DEBUG   HandlerThread:216 [handler.py:handle_request():146] handle_request: status_report