| 05/20/2024 15:00:49 - INFO - transformers.tokenization_utils_base - loading file vocab.json from cache at /home/featurize/.cache/huggingface/hub/models--Qwen--Qwen1.5-7B-Chat/snapshots/5f4f5e69ac7f1d508f8369e977de208b4803444b/vocab.json | |
| 05/20/2024 15:00:49 - INFO - transformers.tokenization_utils_base - loading file merges.txt from cache at /home/featurize/.cache/huggingface/hub/models--Qwen--Qwen1.5-7B-Chat/snapshots/5f4f5e69ac7f1d508f8369e977de208b4803444b/merges.txt | |
| 05/20/2024 15:00:49 - INFO - transformers.tokenization_utils_base - loading file tokenizer.json from cache at /home/featurize/.cache/huggingface/hub/models--Qwen--Qwen1.5-7B-Chat/snapshots/5f4f5e69ac7f1d508f8369e977de208b4803444b/tokenizer.json | |
| 05/20/2024 15:00:49 - INFO - transformers.tokenization_utils_base - loading file added_tokens.json from cache at None | |
| 05/20/2024 15:00:49 - INFO - transformers.tokenization_utils_base - loading file special_tokens_map.json from cache at None | |
| 05/20/2024 15:00:49 - INFO - transformers.tokenization_utils_base - loading file tokenizer_config.json from cache at /home/featurize/.cache/huggingface/hub/models--Qwen--Qwen1.5-7B-Chat/snapshots/5f4f5e69ac7f1d508f8369e977de208b4803444b/tokenizer_config.json | |
| 05/20/2024 15:00:49 - WARNING - transformers.tokenization_utils_base - Special tokens have been added in the vocabulary, make sure the associated word embeddings are fine-tuned or trained. | |
| 05/20/2024 15:00:49 - INFO - llamafactory.data.template - Replace eos token: <|im_end|> | |
| 05/20/2024 15:00:49 - INFO - llamafactory.data.loader - Loading dataset svjack/ShareGPT-Genshin-Impact-Human-Gpt-half... | |
| 05/20/2024 15:00:59 - INFO - transformers.configuration_utils - loading configuration file config.json from cache at /home/featurize/.cache/huggingface/hub/models--Qwen--Qwen1.5-7B-Chat/snapshots/5f4f5e69ac7f1d508f8369e977de208b4803444b/config.json | |
| 05/20/2024 15:00:59 - INFO - transformers.configuration_utils - Model config Qwen2Config { | |
| "_name_or_path": "Qwen/Qwen1.5-7B-Chat", | |
| "architectures": [ | |
| "Qwen2ForCausalLM" | |
| ], | |
| "attention_dropout": 0.0, | |
| "bos_token_id": 151643, | |
| "eos_token_id": 151645, | |
| "hidden_act": "silu", | |
| "hidden_size": 4096, | |
| "initializer_range": 0.02, | |
| "intermediate_size": 11008, | |
| "max_position_embeddings": 32768, | |
| "max_window_layers": 28, | |
| "model_type": "qwen2", | |
| "num_attention_heads": 32, | |
| "num_hidden_layers": 32, | |
| "num_key_value_heads": 32, | |
| "rms_norm_eps": 1e-06, | |
| "rope_theta": 1000000.0, | |
| "sliding_window": 32768, | |
| "tie_word_embeddings": false, | |
| "torch_dtype": "bfloat16", | |
| "transformers_version": "4.41.0", | |
| "use_cache": true, | |
| "use_sliding_window": false, | |
| "vocab_size": 151936 | |
| } | |
| 05/20/2024 15:00:59 - INFO - llamafactory.model.utils.quantization - Quantizing model to 4 bit. | |
| 05/20/2024 15:00:59 - INFO - transformers.modeling_utils - loading weights file model.safetensors from cache at /home/featurize/.cache/huggingface/hub/models--Qwen--Qwen1.5-7B-Chat/snapshots/5f4f5e69ac7f1d508f8369e977de208b4803444b/model.safetensors.index.json | |
| 05/20/2024 15:00:59 - INFO - transformers.modeling_utils - Instantiating Qwen2ForCausalLM model under default dtype torch.float16. | |
| 05/20/2024 15:00:59 - INFO - transformers.generation.configuration_utils - Generate config GenerationConfig { | |
| "bos_token_id": 151643, | |
| "eos_token_id": 151645 | |
| } | |
| 05/20/2024 15:01:11 - INFO - transformers.modeling_utils - All model checkpoint weights were used when initializing Qwen2ForCausalLM. | |
| 05/20/2024 15:01:11 - INFO - transformers.modeling_utils - All the weights of Qwen2ForCausalLM were initialized from the model checkpoint at Qwen/Qwen1.5-7B-Chat. | |
| If your task is similar to the task the model of the checkpoint was trained on, you can already use Qwen2ForCausalLM for predictions without further training. | |
| 05/20/2024 15:01:11 - INFO - transformers.generation.configuration_utils - loading configuration file generation_config.json from cache at /home/featurize/.cache/huggingface/hub/models--Qwen--Qwen1.5-7B-Chat/snapshots/5f4f5e69ac7f1d508f8369e977de208b4803444b/generation_config.json | |
| 05/20/2024 15:01:11 - INFO - transformers.generation.configuration_utils - Generate config GenerationConfig { | |
| "bos_token_id": 151643, | |
| "do_sample": true, | |
| "eos_token_id": [ | |
| 151645, | |
| 151643 | |
| ], | |
| "pad_token_id": 151643, | |
| "repetition_penalty": 1.05, | |
| "temperature": 0.7, | |
| "top_k": 20, | |
| "top_p": 0.8 | |
| } | |
| 05/20/2024 15:01:11 - INFO - llamafactory.model.utils.checkpointing - Gradient checkpointing enabled. | |
| 05/20/2024 15:01:11 - INFO - llamafactory.model.utils.attention - Using torch SDPA for faster training and inference. | |
| 05/20/2024 15:01:11 - INFO - llamafactory.model.adapter - Upcasting trainable params to float32. | |
| 05/20/2024 15:01:11 - INFO - llamafactory.model.adapter - Fine-tuning method: LoRA | |
| 05/20/2024 15:01:12 - INFO - llamafactory.model.loader - trainable params: 4194304 || all params: 7725518848 || trainable%: 0.0543 | |
| 05/20/2024 15:01:12 - WARNING - accelerate.utils.other - Detected kernel version 5.4.0, which is below the recommended minimum of 5.5.0; this can cause the process to hang. It is recommended to upgrade the kernel to the minimum version or higher. | |
| 05/20/2024 15:01:12 - INFO - transformers.trainer - Using auto half precision backend | |
| 05/20/2024 15:01:12 - INFO - transformers.trainer - ***** Running training ***** | |
| 05/20/2024 15:01:12 - INFO - transformers.trainer - Num examples = 1,258 | |
| 05/20/2024 15:01:12 - INFO - transformers.trainer - Num Epochs = 3 | |
| 05/20/2024 15:01:12 - INFO - transformers.trainer - Instantaneous batch size per device = 1 | |
| 05/20/2024 15:01:12 - INFO - transformers.trainer - Total train batch size (w. parallel, distributed & accumulation) = 8 | |
| 05/20/2024 15:01:12 - INFO - transformers.trainer - Gradient Accumulation steps = 8 | |
| 05/20/2024 15:01:12 - INFO - transformers.trainer - Total optimization steps = 471 | |
| 05/20/2024 15:01:12 - INFO - transformers.trainer - Number of trainable parameters = 4,194,304 | |
| 05/20/2024 15:02:34 - INFO - llamafactory.extras.callbacks - {'loss': 2.9252, 'learning_rate': 4.9986e-05, 'epoch': 0.03} | |
| 05/20/2024 15:03:56 - INFO - llamafactory.extras.callbacks - {'loss': 2.4825, 'learning_rate': 4.9944e-05, 'epoch': 0.06} | |
| 05/20/2024 15:05:17 - INFO - llamafactory.extras.callbacks - {'loss': 2.2139, 'learning_rate': 4.9875e-05, 'epoch': 0.10} | |
| 05/20/2024 15:06:39 - INFO - llamafactory.extras.callbacks - {'loss': 2.0351, 'learning_rate': 4.9778e-05, 'epoch': 0.13} | |
| 05/20/2024 15:07:59 - INFO - llamafactory.extras.callbacks - {'loss': 2.1238, 'learning_rate': 4.9653e-05, 'epoch': 0.16} | |
| 05/20/2024 15:09:20 - INFO - llamafactory.extras.callbacks - {'loss': 1.9683, 'learning_rate': 4.9501e-05, 'epoch': 0.19} | |
| 05/20/2024 15:10:41 - INFO - llamafactory.extras.callbacks - {'loss': 1.9120, 'learning_rate': 4.9322e-05, 'epoch': 0.22} | |
| 05/20/2024 15:12:03 - INFO - llamafactory.extras.callbacks - {'loss': 1.9160, 'learning_rate': 4.9115e-05, 'epoch': 0.25} | |
| 05/20/2024 15:13:24 - INFO - llamafactory.extras.callbacks - {'loss': 1.8891, 'learning_rate': 4.8882e-05, 'epoch': 0.29} | |
| 05/20/2024 15:14:46 - INFO - llamafactory.extras.callbacks - {'loss': 1.9039, 'learning_rate': 4.8623e-05, 'epoch': 0.32} | |
| 05/20/2024 15:16:05 - INFO - llamafactory.extras.callbacks - {'loss': 1.8915, 'learning_rate': 4.8337e-05, 'epoch': 0.35} | |
| 05/20/2024 15:17:27 - INFO - llamafactory.extras.callbacks - {'loss': 1.8870, 'learning_rate': 4.8025e-05, 'epoch': 0.38} | |
| 05/20/2024 15:18:48 - INFO - llamafactory.extras.callbacks - {'loss': 1.9291, 'learning_rate': 4.7687e-05, 'epoch': 0.41} | |
| 05/20/2024 15:20:07 - INFO - llamafactory.extras.callbacks - {'loss': 1.8363, 'learning_rate': 4.7324e-05, 'epoch': 0.45} | |
| 05/20/2024 15:21:26 - INFO - llamafactory.extras.callbacks - {'loss': 1.8624, 'learning_rate': 4.6937e-05, 'epoch': 0.48} | |
| 05/20/2024 15:22:48 - INFO - llamafactory.extras.callbacks - {'loss': 1.8554, 'learning_rate': 4.6524e-05, 'epoch': 0.51} | |
| 05/20/2024 15:24:10 - INFO - llamafactory.extras.callbacks - {'loss': 1.8476, 'learning_rate': 4.6089e-05, 'epoch': 0.54} | |
| 05/20/2024 15:25:31 - INFO - llamafactory.extras.callbacks - {'loss': 1.7998, 'learning_rate': 4.5629e-05, 'epoch': 0.57} | |
| 05/20/2024 15:26:51 - INFO - llamafactory.extras.callbacks - {'loss': 1.9258, 'learning_rate': 4.5147e-05, 'epoch': 0.60} | |
| 05/20/2024 15:28:11 - INFO - llamafactory.extras.callbacks - {'loss': 1.7941, 'learning_rate': 4.4642e-05, 'epoch': 0.64} | |
| 05/20/2024 15:28:11 - INFO - transformers.trainer - Saving model checkpoint to saves/Qwen1.5-7B-Chat/lora/train_2024-05-20-14-55-51/checkpoint-100 | |
| 05/20/2024 15:28:12 - INFO - transformers.configuration_utils - loading configuration file config.json from cache at /home/featurize/.cache/huggingface/hub/models--Qwen--Qwen1.5-7B-Chat/snapshots/5f4f5e69ac7f1d508f8369e977de208b4803444b/config.json | |
| 05/20/2024 15:28:12 - INFO - transformers.configuration_utils - Model config Qwen2Config { | |
| "architectures": [ | |
| "Qwen2ForCausalLM" | |
| ], | |
| "attention_dropout": 0.0, | |
| "bos_token_id": 151643, | |
| "eos_token_id": 151645, | |
| "hidden_act": "silu", | |
| "hidden_size": 4096, | |
| "initializer_range": 0.02, | |
| "intermediate_size": 11008, | |
| "max_position_embeddings": 32768, | |
| "max_window_layers": 28, | |
| "model_type": "qwen2", | |
| "num_attention_heads": 32, | |
| "num_hidden_layers": 32, | |
| "num_key_value_heads": 32, | |
| "rms_norm_eps": 1e-06, | |
| "rope_theta": 1000000.0, | |
| "sliding_window": 32768, | |
| "tie_word_embeddings": false, | |
| "torch_dtype": "bfloat16", | |
| "transformers_version": "4.41.0", | |
| "use_cache": true, | |
| "use_sliding_window": false, | |
| "vocab_size": 151936 | |
| } | |
| 05/20/2024 15:28:12 - INFO - transformers.tokenization_utils_base - tokenizer config file saved in saves/Qwen1.5-7B-Chat/lora/train_2024-05-20-14-55-51/checkpoint-100/tokenizer_config.json | |
| 05/20/2024 15:28:12 - INFO - transformers.tokenization_utils_base - Special tokens file saved in saves/Qwen1.5-7B-Chat/lora/train_2024-05-20-14-55-51/checkpoint-100/special_tokens_map.json | |
| 05/20/2024 15:29:33 - INFO - llamafactory.extras.callbacks - {'loss': 1.8889, 'learning_rate': 4.4115e-05, 'epoch': 0.67} | |
| 05/20/2024 15:30:53 - INFO - llamafactory.extras.callbacks - {'loss': 1.8287, 'learning_rate': 4.3567e-05, 'epoch': 0.70} | |
| 05/20/2024 15:32:13 - INFO - llamafactory.extras.callbacks - {'loss': 1.7811, 'learning_rate': 4.2999e-05, 'epoch': 0.73} | |
| 05/20/2024 15:33:31 - INFO - llamafactory.extras.callbacks - {'loss': 1.7925, 'learning_rate': 4.2410e-05, 'epoch': 0.76} | |
| 05/20/2024 15:34:52 - INFO - llamafactory.extras.callbacks - {'loss': 1.8646, 'learning_rate': 4.1802e-05, 'epoch': 0.79} | |
| 05/20/2024 15:36:12 - INFO - llamafactory.extras.callbacks - {'loss': 1.8213, 'learning_rate': 4.1176e-05, 'epoch': 0.83} | |
| 05/20/2024 15:37:35 - INFO - llamafactory.extras.callbacks - {'loss': 1.8301, 'learning_rate': 4.0531e-05, 'epoch': 0.86} | |
| 05/20/2024 15:38:54 - INFO - llamafactory.extras.callbacks - {'loss': 1.8081, 'learning_rate': 3.9869e-05, 'epoch': 0.89} | |
| 05/20/2024 15:40:13 - INFO - llamafactory.extras.callbacks - {'loss': 1.8472, 'learning_rate': 3.9191e-05, 'epoch': 0.92} | |
| 05/20/2024 15:41:34 - INFO - llamafactory.extras.callbacks - {'loss': 1.8186, 'learning_rate': 3.8497e-05, 'epoch': 0.95} | |
| 05/20/2024 15:42:53 - INFO - llamafactory.extras.callbacks - {'loss': 1.8542, 'learning_rate': 3.7788e-05, 'epoch': 0.99} | |
| 05/20/2024 15:44:13 - INFO - llamafactory.extras.callbacks - {'loss': 1.6997, 'learning_rate': 3.7064e-05, 'epoch': 1.02} | |
| 05/20/2024 15:45:34 - INFO - llamafactory.extras.callbacks - {'loss': 1.8274, 'learning_rate': 3.6327e-05, 'epoch': 1.05} | |
| 05/20/2024 15:46:55 - INFO - llamafactory.extras.callbacks - {'loss': 1.7660, 'learning_rate': 3.5578e-05, 'epoch': 1.08} | |
| 05/20/2024 15:48:15 - INFO - llamafactory.extras.callbacks - {'loss': 1.7501, 'learning_rate': 3.4817e-05, 'epoch': 1.11} | |
| 05/20/2024 15:49:37 - INFO - llamafactory.extras.callbacks - {'loss': 1.7754, 'learning_rate': 3.4045e-05, 'epoch': 1.14} | |
| 05/20/2024 15:50:54 - INFO - llamafactory.extras.callbacks - {'loss': 1.8031, 'learning_rate': 3.3263e-05, 'epoch': 1.18} | |
| 05/20/2024 15:52:14 - INFO - llamafactory.extras.callbacks - {'loss': 1.7226, 'learning_rate': 3.2471e-05, 'epoch': 1.21} | |
| 05/20/2024 15:53:34 - INFO - llamafactory.extras.callbacks - {'loss': 1.7752, 'learning_rate': 3.1672e-05, 'epoch': 1.24} | |
| 05/20/2024 15:54:55 - INFO - llamafactory.extras.callbacks - {'loss': 1.7353, 'learning_rate': 3.0865e-05, 'epoch': 1.27} | |
| 05/20/2024 15:54:55 - INFO - transformers.trainer - Saving model checkpoint to saves/Qwen1.5-7B-Chat/lora/train_2024-05-20-14-55-51/checkpoint-200 | |
| 05/20/2024 15:54:56 - INFO - transformers.configuration_utils - loading configuration file config.json from cache at /home/featurize/.cache/huggingface/hub/models--Qwen--Qwen1.5-7B-Chat/snapshots/5f4f5e69ac7f1d508f8369e977de208b4803444b/config.json | |
| 05/20/2024 15:54:56 - INFO - transformers.configuration_utils - Model config Qwen2Config { | |
| "architectures": [ | |
| "Qwen2ForCausalLM" | |
| ], | |
| "attention_dropout": 0.0, | |
| "bos_token_id": 151643, | |
| "eos_token_id": 151645, | |
| "hidden_act": "silu", | |
| "hidden_size": 4096, | |
| "initializer_range": 0.02, | |
| "intermediate_size": 11008, | |
| "max_position_embeddings": 32768, | |
| "max_window_layers": 28, | |
| "model_type": "qwen2", | |
| "num_attention_heads": 32, | |
| "num_hidden_layers": 32, | |
| "num_key_value_heads": 32, | |
| "rms_norm_eps": 1e-06, | |
| "rope_theta": 1000000.0, | |
| "sliding_window": 32768, | |
| "tie_word_embeddings": false, | |
| "torch_dtype": "bfloat16", | |
| "transformers_version": "4.41.0", | |
| "use_cache": true, | |
| "use_sliding_window": false, | |
| "vocab_size": 151936 | |
| } | |
| 05/20/2024 15:54:56 - INFO - transformers.tokenization_utils_base - tokenizer config file saved in saves/Qwen1.5-7B-Chat/lora/train_2024-05-20-14-55-51/checkpoint-200/tokenizer_config.json | |
| 05/20/2024 15:54:56 - INFO - transformers.tokenization_utils_base - Special tokens file saved in saves/Qwen1.5-7B-Chat/lora/train_2024-05-20-14-55-51/checkpoint-200/special_tokens_map.json | |
| 05/20/2024 15:56:19 - INFO - llamafactory.extras.callbacks - {'loss': 1.7834, 'learning_rate': 3.0051e-05, 'epoch': 1.30} | |
| 05/20/2024 15:57:40 - INFO - llamafactory.extras.callbacks - {'loss': 1.7668, 'learning_rate': 2.9232e-05, 'epoch': 1.34} | |
| 05/20/2024 15:59:02 - INFO - llamafactory.extras.callbacks - {'loss': 1.7912, 'learning_rate': 2.8408e-05, 'epoch': 1.37} | |
| 05/20/2024 16:00:24 - INFO - llamafactory.extras.callbacks - {'loss': 1.7674, 'learning_rate': 2.7580e-05, 'epoch': 1.40} | |
| 05/20/2024 16:01:48 - INFO - llamafactory.extras.callbacks - {'loss': 1.7827, 'learning_rate': 2.6749e-05, 'epoch': 1.43} | |
| 05/20/2024 16:03:10 - INFO - llamafactory.extras.callbacks - {'loss': 1.8176, 'learning_rate': 2.5917e-05, 'epoch': 1.46} | |
| 05/20/2024 16:04:32 - INFO - llamafactory.extras.callbacks - {'loss': 1.7613, 'learning_rate': 2.5083e-05, 'epoch': 1.49} | |
| 05/20/2024 16:05:52 - INFO - llamafactory.extras.callbacks - {'loss': 1.7506, 'learning_rate': 2.4250e-05, 'epoch': 1.53} | |
| 05/20/2024 16:07:13 - INFO - llamafactory.extras.callbacks - {'loss': 1.7380, 'learning_rate': 2.3417e-05, 'epoch': 1.56} | |
| 05/20/2024 16:08:35 - INFO - llamafactory.extras.callbacks - {'loss': 1.8030, 'learning_rate': 2.2586e-05, 'epoch': 1.59} | |
| 05/20/2024 16:09:56 - INFO - llamafactory.extras.callbacks - {'loss': 1.7351, 'learning_rate': 2.1758e-05, 'epoch': 1.62} | |
| 05/20/2024 16:11:19 - INFO - llamafactory.extras.callbacks - {'loss': 1.7596, 'learning_rate': 2.0933e-05, 'epoch': 1.65} | |
| 05/20/2024 16:12:40 - INFO - llamafactory.extras.callbacks - {'loss': 1.7875, 'learning_rate': 2.0113e-05, 'epoch': 1.69} | |
| 05/20/2024 16:14:02 - INFO - llamafactory.extras.callbacks - {'loss': 1.7871, 'learning_rate': 1.9298e-05, 'epoch': 1.72} | |
| 05/20/2024 16:15:24 - INFO - llamafactory.extras.callbacks - {'loss': 1.7658, 'learning_rate': 1.8489e-05, 'epoch': 1.75} | |
| 05/20/2024 16:16:46 - INFO - llamafactory.extras.callbacks - {'loss': 1.7510, 'learning_rate': 1.7688e-05, 'epoch': 1.78} | |
| 05/20/2024 16:18:05 - INFO - llamafactory.extras.callbacks - {'loss': 1.7664, 'learning_rate': 1.6895e-05, 'epoch': 1.81} | |
| 05/20/2024 16:19:26 - INFO - llamafactory.extras.callbacks - {'loss': 1.8125, 'learning_rate': 1.6111e-05, 'epoch': 1.84} | |
| 05/20/2024 16:20:47 - INFO - llamafactory.extras.callbacks - {'loss': 1.7941, 'learning_rate': 1.5337e-05, 'epoch': 1.88} | |
| 05/20/2024 16:22:09 - INFO - llamafactory.extras.callbacks - {'loss': 1.7435, 'learning_rate': 1.4573e-05, 'epoch': 1.91} | |
| 05/20/2024 16:22:09 - INFO - transformers.trainer - Saving model checkpoint to saves/Qwen1.5-7B-Chat/lora/train_2024-05-20-14-55-51/checkpoint-300 | |
| 05/20/2024 16:22:10 - INFO - transformers.configuration_utils - loading configuration file config.json from cache at /home/featurize/.cache/huggingface/hub/models--Qwen--Qwen1.5-7B-Chat/snapshots/5f4f5e69ac7f1d508f8369e977de208b4803444b/config.json | |
| 05/20/2024 16:22:10 - INFO - transformers.configuration_utils - Model config Qwen2Config { | |
| "architectures": [ | |
| "Qwen2ForCausalLM" | |
| ], | |
| "attention_dropout": 0.0, | |
| "bos_token_id": 151643, | |
| "eos_token_id": 151645, | |
| "hidden_act": "silu", | |
| "hidden_size": 4096, | |
| "initializer_range": 0.02, | |
| "intermediate_size": 11008, | |
| "max_position_embeddings": 32768, | |
| "max_window_layers": 28, | |
| "model_type": "qwen2", | |
| "num_attention_heads": 32, | |
| "num_hidden_layers": 32, | |
| "num_key_value_heads": 32, | |
| "rms_norm_eps": 1e-06, | |
| "rope_theta": 1000000.0, | |
| "sliding_window": 32768, | |
| "tie_word_embeddings": false, | |
| "torch_dtype": "bfloat16", | |
| "transformers_version": "4.41.0", | |
| "use_cache": true, | |
| "use_sliding_window": false, | |
| "vocab_size": 151936 | |
| } | |
| 05/20/2024 16:22:10 - INFO - transformers.tokenization_utils_base - tokenizer config file saved in saves/Qwen1.5-7B-Chat/lora/train_2024-05-20-14-55-51/checkpoint-300/tokenizer_config.json | |
| 05/20/2024 16:22:10 - INFO - transformers.tokenization_utils_base - Special tokens file saved in saves/Qwen1.5-7B-Chat/lora/train_2024-05-20-14-55-51/checkpoint-300/special_tokens_map.json | |
| 05/20/2024 16:23:32 - INFO - llamafactory.extras.callbacks - {'loss': 1.7548, 'learning_rate': 1.3821e-05, 'epoch': 1.94} | |
| 05/20/2024 16:24:54 - INFO - llamafactory.extras.callbacks - {'loss': 1.8122, 'learning_rate': 1.3082e-05, 'epoch': 1.97} | |
| 05/20/2024 16:26:15 - INFO - llamafactory.extras.callbacks - {'loss': 1.7795, 'learning_rate': 1.2356e-05, 'epoch': 2.00} | |
| 05/20/2024 16:27:36 - INFO - llamafactory.extras.callbacks - {'loss': 1.7171, 'learning_rate': 1.1644e-05, 'epoch': 2.03} | |
| 05/20/2024 16:28:57 - INFO - llamafactory.extras.callbacks - {'loss': 1.6727, 'learning_rate': 1.0947e-05, 'epoch': 2.07} | |
| 05/20/2024 16:30:18 - INFO - llamafactory.extras.callbacks - {'loss': 1.8067, 'learning_rate': 1.0265e-05, 'epoch': 2.10} | |
| 05/20/2024 16:31:40 - INFO - llamafactory.extras.callbacks - {'loss': 1.7300, 'learning_rate': 9.5997e-06, 'epoch': 2.13} | |
| 05/20/2024 16:32:59 - INFO - llamafactory.extras.callbacks - {'loss': 1.7939, 'learning_rate': 8.9516e-06, 'epoch': 2.16} | |
| 05/20/2024 16:34:21 - INFO - llamafactory.extras.callbacks - {'loss': 1.7272, 'learning_rate': 8.3214e-06, 'epoch': 2.19} | |
| 05/20/2024 16:35:42 - INFO - llamafactory.extras.callbacks - {'loss': 1.7387, 'learning_rate': 7.7097e-06, 'epoch': 2.23} | |
| 05/20/2024 16:37:03 - INFO - llamafactory.extras.callbacks - {'loss': 1.7606, 'learning_rate': 7.1172e-06, 'epoch': 2.26} | |
| 05/20/2024 16:38:24 - INFO - llamafactory.extras.callbacks - {'loss': 1.6940, 'learning_rate': 6.5446e-06, 'epoch': 2.29} | |
| 05/20/2024 16:39:47 - INFO - llamafactory.extras.callbacks - {'loss': 1.7982, 'learning_rate': 5.9926e-06, 'epoch': 2.32} | |
| 05/20/2024 16:41:07 - INFO - llamafactory.extras.callbacks - {'loss': 1.7905, 'learning_rate': 5.4616e-06, 'epoch': 2.35} | |
| 05/20/2024 16:42:29 - INFO - llamafactory.extras.callbacks - {'loss': 1.7480, 'learning_rate': 4.9525e-06, 'epoch': 2.38} | |
| 05/20/2024 16:43:51 - INFO - llamafactory.extras.callbacks - {'loss': 1.7059, 'learning_rate': 4.4656e-06, 'epoch': 2.42} | |
| 05/20/2024 16:45:13 - INFO - llamafactory.extras.callbacks - {'loss': 1.7080, 'learning_rate': 4.0015e-06, 'epoch': 2.45} | |
| 05/20/2024 16:46:34 - INFO - llamafactory.extras.callbacks - {'loss': 1.7037, 'learning_rate': 3.5608e-06, 'epoch': 2.48} | |
| 05/20/2024 16:47:56 - INFO - llamafactory.extras.callbacks - {'loss': 1.7715, 'learning_rate': 3.1439e-06, 'epoch': 2.51} | |
| 05/20/2024 16:49:17 - INFO - llamafactory.extras.callbacks - {'loss': 1.7584, 'learning_rate': 2.7514e-06, 'epoch': 2.54} | |
| 05/20/2024 16:49:17 - INFO - transformers.trainer - Saving model checkpoint to saves/Qwen1.5-7B-Chat/lora/train_2024-05-20-14-55-51/checkpoint-400 | |
| 05/20/2024 16:49:18 - INFO - transformers.configuration_utils - loading configuration file config.json from cache at /home/featurize/.cache/huggingface/hub/models--Qwen--Qwen1.5-7B-Chat/snapshots/5f4f5e69ac7f1d508f8369e977de208b4803444b/config.json | |
| 05/20/2024 16:49:18 - INFO - transformers.configuration_utils - Model config Qwen2Config { | |
| "architectures": [ | |
| "Qwen2ForCausalLM" | |
| ], | |
| "attention_dropout": 0.0, | |
| "bos_token_id": 151643, | |
| "eos_token_id": 151645, | |
| "hidden_act": "silu", | |
| "hidden_size": 4096, | |
| "initializer_range": 0.02, | |
| "intermediate_size": 11008, | |
| "max_position_embeddings": 32768, | |
| "max_window_layers": 28, | |
| "model_type": "qwen2", | |
| "num_attention_heads": 32, | |
| "num_hidden_layers": 32, | |
| "num_key_value_heads": 32, | |
| "rms_norm_eps": 1e-06, | |
| "rope_theta": 1000000.0, | |
| "sliding_window": 32768, | |
| "tie_word_embeddings": false, | |
| "torch_dtype": "bfloat16", | |
| "transformers_version": "4.41.0", | |
| "use_cache": true, | |
| "use_sliding_window": false, | |
| "vocab_size": 151936 | |
| } | |
| 05/20/2024 16:49:18 - INFO - transformers.tokenization_utils_base - tokenizer config file saved in saves/Qwen1.5-7B-Chat/lora/train_2024-05-20-14-55-51/checkpoint-400/tokenizer_config.json | |
| 05/20/2024 16:49:18 - INFO - transformers.tokenization_utils_base - Special tokens file saved in saves/Qwen1.5-7B-Chat/lora/train_2024-05-20-14-55-51/checkpoint-400/special_tokens_map.json | |
| 05/20/2024 16:50:40 - INFO - llamafactory.extras.callbacks - {'loss': 1.7246, 'learning_rate': 2.3836e-06, 'epoch': 2.58} | |
| 05/20/2024 16:52:03 - INFO - llamafactory.extras.callbacks - {'loss': 1.6902, 'learning_rate': 2.0409e-06, 'epoch': 2.61} | |
| 05/20/2024 16:53:25 - INFO - llamafactory.extras.callbacks - {'loss': 1.7111, 'learning_rate': 1.7238e-06, 'epoch': 2.64} | |
| 05/20/2024 16:54:47 - INFO - llamafactory.extras.callbacks - {'loss': 1.6956, 'learning_rate': 1.4326e-06, 'epoch': 2.67} | |
| 05/20/2024 16:56:09 - INFO - llamafactory.extras.callbacks - {'loss': 1.7892, 'learning_rate': 1.1675e-06, 'epoch': 2.70} | |
| 05/20/2024 16:57:31 - INFO - llamafactory.extras.callbacks - {'loss': 1.6886, 'learning_rate': 9.2902e-07, 'epoch': 2.73} | |
| 05/20/2024 16:58:52 - INFO - llamafactory.extras.callbacks - {'loss': 1.7672, 'learning_rate': 7.1727e-07, 'epoch': 2.77} | |
| 05/20/2024 17:00:14 - INFO - llamafactory.extras.callbacks - {'loss': 1.7667, 'learning_rate': 5.3253e-07, 'epoch': 2.80} | |
| 05/20/2024 17:01:36 - INFO - llamafactory.extras.callbacks - {'loss': 1.7337, 'learning_rate': 3.7500e-07, 'epoch': 2.83} | |
| 05/20/2024 17:02:58 - INFO - llamafactory.extras.callbacks - {'loss': 1.6704, 'learning_rate': 2.4485e-07, 'epoch': 2.86} | |
| 05/20/2024 17:04:20 - INFO - llamafactory.extras.callbacks - {'loss': 1.6902, 'learning_rate': 1.4223e-07, 'epoch': 2.89} | |
| 05/20/2024 17:05:41 - INFO - llamafactory.extras.callbacks - {'loss': 1.7985, 'learning_rate': 6.7260e-08, 'epoch': 2.93} | |
| 05/20/2024 17:07:02 - INFO - llamafactory.extras.callbacks - {'loss': 1.7464, 'learning_rate': 2.0018e-08, 'epoch': 2.96} | |
| 05/20/2024 17:08:24 - INFO - llamafactory.extras.callbacks - {'loss': 1.7430, 'learning_rate': 5.5612e-10, 'epoch': 2.99} | |
| 05/20/2024 17:08:40 - INFO - transformers.trainer - | |
| Training completed. Do not forget to share your model on huggingface.co/models =) | |
| 05/20/2024 17:08:40 - INFO - transformers.trainer - Saving model checkpoint to saves/Qwen1.5-7B-Chat/lora/train_2024-05-20-14-55-51 | |
| 05/20/2024 17:08:42 - INFO - transformers.configuration_utils - loading configuration file config.json from cache at /home/featurize/.cache/huggingface/hub/models--Qwen--Qwen1.5-7B-Chat/snapshots/5f4f5e69ac7f1d508f8369e977de208b4803444b/config.json | |
| 05/20/2024 17:08:42 - INFO - transformers.configuration_utils - Model config Qwen2Config { | |
| "architectures": [ | |
| "Qwen2ForCausalLM" | |
| ], | |
| "attention_dropout": 0.0, | |
| "bos_token_id": 151643, | |
| "eos_token_id": 151645, | |
| "hidden_act": "silu", | |
| "hidden_size": 4096, | |
| "initializer_range": 0.02, | |
| "intermediate_size": 11008, | |
| "max_position_embeddings": 32768, | |
| "max_window_layers": 28, | |
| "model_type": "qwen2", | |
| "num_attention_heads": 32, | |
| "num_hidden_layers": 32, | |
| "num_key_value_heads": 32, | |
| "rms_norm_eps": 1e-06, | |
| "rope_theta": 1000000.0, | |
| "sliding_window": 32768, | |
| "tie_word_embeddings": false, | |
| "torch_dtype": "bfloat16", | |
| "transformers_version": "4.41.0", | |
| "use_cache": true, | |
| "use_sliding_window": false, | |
| "vocab_size": 151936 | |
| } | |
| 05/20/2024 17:08:42 - INFO - transformers.tokenization_utils_base - tokenizer config file saved in saves/Qwen1.5-7B-Chat/lora/train_2024-05-20-14-55-51/tokenizer_config.json | |
| 05/20/2024 17:08:42 - INFO - transformers.tokenization_utils_base - Special tokens file saved in saves/Qwen1.5-7B-Chat/lora/train_2024-05-20-14-55-51/special_tokens_map.json | |
| 05/20/2024 17:08:42 - WARNING - llamafactory.extras.ploting - No metric eval_loss to plot. | |
| 05/20/2024 17:08:42 - INFO - transformers.modelcard - Dropping the following result as it does not have all the necessary fields: | |
| {'task': {'name': 'Causal Language Modeling', 'type': 'text-generation'}} | |