[2024-01-10 03:41:18,555] [INFO] [real_accelerator.py:158:get_accelerator] Setting ds_accelerator to cuda (auto detect)
[2024-01-10 03:41:20,266] [WARNING] [runner.py:203:fetch_hostfile] Unable to find hostfile, will proceed with training with local resources only.
[2024-01-10 03:41:20,266] [INFO] [runner.py:570:main] cmd = /root/anaconda3/envs/dj_comp/bin/python -u -m deepspeed.launcher.launch --world_info=eyJsb2NhbGhvc3QiOiBbMCwgMSwgMiwgMywgNCwgNSwgNiwgN119 --master_addr=127.0.0.1 --master_port=50000 --enable_each_rank_log=None train.py --model_name_or_path /home/baaidial/models/llama-2-13b-hf/ --tokenizer /home/baaidial/models/llama-2-13b-hf/ --data_path /home/baaidial/fengduanyu/lima_code/code/data/alpaca/alpaca_predata.jsonl --output_dir /home/baaidial/fengduanyu/lima_code/code/model_save/after_sft/alpaca_Llama-2-13b_all/ --per_device_train_batch_size 8 --gradient_accumulation_steps 4 --lang en --bf16 True --gradient_checkpointing_enable True --num_train_epochs 3 --model_max_length 1024 --learning_rate 2.5e-5 --weight_decay 0 --warmup_ratio 0.03 --evaluation_strategy no --save_strategy no --save_steps -1 --save_total_limit 999 --lr_scheduler_type cosine --logging_steps 1 --tf32 True --deepspeed /home/baaidial/fengduanyu/FT-Data-Ranker/competition_kit/lm-training/train_scripts/deepspeed_configs/ds_config_stage3.json
[2024-01-10 03:41:21,810] [INFO] [real_accelerator.py:158:get_accelerator] Setting ds_accelerator to cuda (auto detect)
[2024-01-10 03:41:23,124] [INFO] [launch.py:138:main] 0 NCCL_VERSION=2.18.1
[2024-01-10 03:41:23,124] [INFO] [launch.py:145:main] WORLD INFO DICT: {'localhost': [0, 1, 2, 3, 4, 5, 6, 7]}
[2024-01-10 03:41:23,124] [INFO] [launch.py:151:main] nnodes=1, num_local_procs=8, node_rank=0
[2024-01-10 03:41:23,124] [INFO] [launch.py:162:main] global_rank_mapping=defaultdict(<class 'list'>, {'localhost': [0, 1, 2, 3, 4, 5, 6, 7]})
[2024-01-10 03:41:23,124] [INFO] [launch.py:163:main] dist_world_size=8
[2024-01-10 03:41:23,124] [INFO] [launch.py:165:main] Setting CUDA_VISIBLE_DEVICES=0,1,2,3,4,5,6,7
[2024-01-10 03:41:25,938] [INFO] [real_accelerator.py:158:get_accelerator] Setting ds_accelerator to cuda (auto detect)
[2024-01-10 03:41:26,237] [INFO] [real_accelerator.py:158:get_accelerator] Setting ds_accelerator to cuda (auto detect)
[2024-01-10 03:41:26,239] [INFO] [real_accelerator.py:158:get_accelerator] Setting ds_accelerator to cuda (auto detect)
[2024-01-10 03:41:26,338] [INFO] [real_accelerator.py:158:get_accelerator] Setting ds_accelerator to cuda (auto detect)
[2024-01-10 03:41:26,339] [INFO] [real_accelerator.py:158:get_accelerator] Setting ds_accelerator to cuda (auto detect)
[2024-01-10 03:41:26,351] [INFO] [real_accelerator.py:158:get_accelerator] Setting ds_accelerator to cuda (auto detect)
[2024-01-10 03:41:26,353] [INFO] [real_accelerator.py:158:get_accelerator] Setting ds_accelerator to cuda (auto detect)
[2024-01-10 03:41:26,357] [INFO] [real_accelerator.py:158:get_accelerator] Setting ds_accelerator to cuda (auto detect)
[2024-01-10 03:41:27,429] [INFO] [comm.py:637:init_distributed] cdb=None
[2024-01-10 03:41:27,429] [INFO] [comm.py:668:init_distributed] Initializing TorchBackend in DeepSpeed with backend nccl
[2024-01-10 03:41:27,709] [INFO] [comm.py:637:init_distributed] cdb=None
[2024-01-10 03:41:27,773] [INFO] [comm.py:637:init_distributed] cdb=None
[2024-01-10 03:41:27,835] [INFO] [comm.py:637:init_distributed] cdb=None
[2024-01-10 03:41:27,866] [INFO] [comm.py:637:init_distributed] cdb=None
[2024-01-10 03:41:27,867] [INFO] [comm.py:637:init_distributed] cdb=None
[2024-01-10 03:41:27,894] [INFO] [comm.py:637:init_distributed] cdb=None
[2024-01-10 03:41:27,933] [INFO] [comm.py:637:init_distributed] cdb=None
Loading model from /home/baaidial/models/llama-2-13b-hf/
[2024-01-10 03:41:30,329] [INFO] [partition_parameters.py:347:__exit__] finished initializing model - num_params = 363, num_elems = 13.02B
gradient_checkpointing_enable
Model class: <class 'transformers.models.llama.modeling_llama.LlamaForCausalLM'>
Tokenizer class: <class 'transformers.models.llama.tokenization_llama.LlamaTokenizer'>
+---------+---------+------------+---------+------------+---------+------------+---------+------------+----------------+
|         |pad_token|pad_token_id|bos_token|bos_token_id|eos_token|eos_token_id|unk_token|unk_token_id|model_max_length|
+---------+---------+------------+---------+------------+---------+------------+---------+------------+----------------+
|tokenizer|  <unk>  |     0      |   <s>   |     1      |   </s>  |     2      |  <unk>  |     0      |      1024      |
+---------+---------+------------+---------+------------+---------+------------+---------+------------+----------------+
|  model  |    -    |     0      |    -    |     1      |    -    |     2      |    -    |     -      |       -        |
+---------+---------+------------+---------+------------+---------+------------+---------+------------+----------------+
There are 52002 training samples in data path
Formatting inputs...
Tokenizing inputs... This may take some time...
PROMPT:
'Below is an instruction that describes a task. Write a response that appropriately completes the request.\n\n### Instruction:\nWhen did the Second World War end?\n\n### Response:'
RESPONSE:
'The Second World War ended on September 2, 1945 with the official surrender of Japan. The Allies officially declared victory over Germany on May 8, 1945, which is known as V-E Day, or Victory in Europe Day.</s>'
====================================================================================================
PROMPT:
'Below is an instruction that describes a task. Write a response that appropriately completes the request.\n\n### Instruction:\nExplain why the following statement is wrong: "Cloud computing stores your data on the Internet."\n\n### Response:'
RESPONSE:
'The statement is incorrect because cloud computing does not store data on the Internet. Instead, cloud computing stores data in a remote server which is usually connected to the Internet. Data can be accessed and retrieved from the remote server through the Internet using an application.</s>'
====================================================================================================
PROMPT:
'Below is an instruction that describes a task, paired with an input that provides further context. Write a response that appropriately completes the request.\n\n### Instruction:\nIdentify 5 major differences between two similar items.\n\n### Input:\nBlackberry and Raspberry\n\n### Response:'
RESPONSE:
'1. Blackberries have a rounded shape, while raspberries have a distinct hollow core when picked off the plant.\n2. Raspberries have a rich and intense flavor profile, while blackberries are sweeter and less tart.\n3. Blackberries typically have very tiny seeds that are barely noticeable when eaten, while raspberries have larger, more noticeable seeds.\n4. Blackberries tend to be darker in color than raspberries, which tend to be a pinkish-red.\n5. Raspberries are generally more fragile, and can spoil or bruise more easily than blackberries.</s>'
====================================================================================================
PROMPT:
'Below is an instruction that describes a task, paired with an input that provides further context. Write a response that appropriately completes the request.\n\n### Instruction:\nClassify the following text as opinion, fact, or non-credible\n\n### Input:\nThis new book is amazing!\n\n### Response:'
RESPONSE:
'Opinion.</s>'
====================================================================================================
PROMPT:
'Below is an instruction that describes a task, paired with an input that provides further context. Write a response that appropriately completes the request.\n\n### Instruction:\nClean the HTML code by removing all the unnecessary tags.\n\n### Input:\n<html><head><title>Website</title></head><body><p>This is a paragraph.</p><p>This is the second paragraph.</p></body></html>\n\n### Response:'
RESPONSE:
'<html><head><title>Website</title></head><body>This is a paragraph.<p>This is the second paragraph.</p></body></html></s>'
====================================================================================================
ORI NUMBER: 52002, AFTER FILETER: 52000, DROP NUMBER: 2
Total 52000 samples [ 6.56M tokens] in training!
Parameter Offload: Total persistent parameters: 414720 in 81 params
{'loss': 1.3013, 'learning_rate': 1.3157894736842106e-06, 'epoch': 0.0}
{'loss': 1.2825, 'learning_rate': 2.631578947368421e-06, 'epoch': 0.01}
{'loss': 1.3755, 'learning_rate': 3.9473684210526315e-06, 'epoch': 0.01}
{'loss': 1.2644, 'learning_rate': 5.263157894736842e-06, 'epoch': 0.02}
{'loss': 1.2507, 'learning_rate': 6.578947368421053e-06, 'epoch': 0.02}
{'loss': 1.1368, 'learning_rate': 7.894736842105263e-06, 'epoch': 0.03}
{'loss': 1.1033, 'learning_rate': 9.210526315789474e-06, 'epoch': 0.03}
{'loss': 1.0729, 'learning_rate': 1.0526315789473684e-05, 'epoch': 0.04}
{'loss': 1.0756, 'learning_rate': 1.1842105263157895e-05, 'epoch': 0.04}
{'loss': 1.0396, 'learning_rate': 1.3157894736842106e-05, 'epoch': 0.05}
{'loss': 1.0539, 'learning_rate': 1.4473684210526317e-05, 'epoch': 0.05}
{'loss': 1.085, 'learning_rate': 1.5789473684210526e-05, 'epoch': 0.06}
{'loss': 1.0104, 'learning_rate': 1.7105263157894737e-05, 'epoch': 0.06}
{'loss': 1.0773, 'learning_rate': 1.8421052631578947e-05, 'epoch': 0.07}
{'loss': 1.0052, 'learning_rate': 1.9736842105263158e-05, 'epoch': 0.07}
{'loss': 1.0603, 'learning_rate': 2.105263157894737e-05, 'epoch': 0.08}
{'loss': 1.0628, 'learning_rate': 2.236842105263158e-05, 'epoch': 0.08}
{'loss': 1.0233, 'learning_rate': 2.368421052631579e-05, 'epoch': 0.09}
{'loss': 1.0953, 'learning_rate': 2.5e-05, 'epoch': 0.09}
{'loss': 1.0786, 'learning_rate': 2.499982279551347e-05, 'epoch': 0.1}
{'loss': 1.0389, 'learning_rate': 2.4999291187078104e-05, 'epoch': 0.1}
{'loss': 1.0168, 'learning_rate': 2.4998405189766444e-05, 'epoch': 0.11}
{'loss': 1.036, 'learning_rate': 2.4997164828698926e-05, 'epoch': 0.11}
{'loss': 1.0171, 'learning_rate': 2.4995570139043158e-05, 'epoch': 0.12}
{'loss': 1.0811, 'learning_rate': 2.4993621166012923e-05, 'epoch': 0.12}
{'loss': 1.0955, 'learning_rate': 2.4991317964866905e-05, 'epoch': 0.13}
{'loss': 1.0038, 'learning_rate': 2.4988660600907115e-05, 'epoch': 0.13}
{'loss': 1.0558, 'learning_rate': 2.4985649149477043e-05, 'epoch': 0.14}
{'loss': 1.0707, 'learning_rate': 2.4982283695959525e-05, 'epoch': 0.14}
{'loss': 1.1162, 'learning_rate': 2.4978564335774313e-05, 'epoch': 0.15}
{'loss': 1.006, 'learning_rate': 2.4974491174375374e-05, 'epoch': 0.15}
{'loss': 1.0481, 'learning_rate': 2.497006432724791e-05, 'epoch': 0.16}
{'loss': 1.0609, 'learning_rate': 2.4965283919905065e-05, 'epoch': 0.16}
{'loss': 1.0946, 'learning_rate': 2.4960150087884376e-05, 'epoch': 0.17}
{'loss': 1.0222, 'learning_rate': 2.4954662976743944e-05, 'epoch': 0.17}
{'loss': 1.0427, 'learning_rate': 2.494882274205827e-05, 'epoch': 0.18}
{'loss': 1.0987, 'learning_rate': 2.4942629549413894e-05, 'epoch': 0.18}
{'loss': 1.1212, 'learning_rate': 2.4936083574404646e-05, 'epoch': 0.19}
{'loss': 1.0986, 'learning_rate': 2.4929185002626714e-05, 'epoch': 0.19}
{'loss': 1.05, 'learning_rate': 2.492193402967336e-05, 'epoch': 0.2}
{'loss': 1.0784, 'learning_rate': 2.4914330861129376e-05, 'epoch': 0.2}
{'loss': 1.0997, 'learning_rate': 2.4906375712565245e-05, 'epoch': 0.21}
{'loss': 1.0774, 'learning_rate': 2.489806880953106e-05, 'epoch': 0.21}
{'loss': 1.0832, 'learning_rate': 2.4889410387550093e-05, 'epoch': 0.22}
{'loss': 1.0649, 'learning_rate': 2.488040069211214e-05, 'epoch': 0.22}
{'loss': 1.0684, 'learning_rate': 2.487103997866655e-05, 'epoch': 0.23}
{'loss': 1.0857, 'learning_rate': 2.4861328512615e-05, 'epoch': 0.23}
{'loss': 1.0896, 'learning_rate': 2.485126656930394e-05, 'epoch': 0.24}
{'loss': 1.0719, 'learning_rate': 2.4840854434016808e-05, 'epoch': 0.24}
{'loss': 1.0917, 'learning_rate': 2.4830092401965942e-05, 'epoch': 0.25}
{'loss': 1.0893, 'learning_rate': 2.4818980778284202e-05, 'epoch': 0.25}
{'loss': 1.0136, 'learning_rate': 2.4807519878016313e-05, 'epoch': 0.26}
{'loss': 1.0472, 'learning_rate': 2.4795710026109954e-05, 'epoch': 0.26}
{'loss': 1.0625, 'learning_rate': 2.4783551557406515e-05, 'epoch': 0.27}
{'loss': 1.1197, 'learning_rate': 2.4771044816631635e-05, 'epoch': 0.27}
{'loss': 1.0531, 'learning_rate': 2.4758190158385402e-05, 'epoch': 0.28}
{'loss': 1.0767, 'learning_rate': 2.4744987947132318e-05, 'epoch': 0.28}
{'loss': 1.0852, 'learning_rate': 2.473143855719095e-05, 'epoch': 0.29}
{'loss': 1.093, 'learning_rate': 2.4717542372723333e-05, 'epoch': 0.29}
{'loss': 1.0991, 'learning_rate': 2.4703299787724065e-05, 'epoch': 0.3}
{'loss': 1.0982, 'learning_rate': 2.468871120600913e-05, 'epoch': 0.3}
{'loss': 1.1232, 'learning_rate': 2.4673777041204478e-05, 'epoch': 0.31}
{'loss': 1.0692, 'learning_rate': 2.465849771673427e-05, 'epoch': 0.31}
{'loss': 1.1136, 'learning_rate': 2.4642873665808873e-05, 'epoch': 0.31}
{'loss': 1.0771, 'learning_rate': 2.4626905331412608e-05, 'epoch': 0.32}
{'loss': 1.0464, 'learning_rate': 2.4610593166291143e-05, 'epoch': 0.32}
{'loss': 1.0578, 'learning_rate': 2.4593937632938695e-05, 'epoch': 0.33}
{'loss': 1.0322, 'learning_rate': 2.457693920358491e-05, 'epoch': 0.33}
{'loss': 1.0573, 'learning_rate': 2.455959836018145e-05, 'epoch': 0.34}
{'loss': 1.083, 'learning_rate': 2.454191559438836e-05, 'epoch': 0.34}
{'loss': 1.0426, 'learning_rate': 2.4523891407560107e-05, 'epoch': 0.35}
{'loss': 1.0706, 'learning_rate': 2.4505526310731376e-05, 'epoch': 0.35}
{'loss': 1.089, 'learning_rate': 2.448682082460257e-05, 'epoch': 0.36}
{'loss': 1.0918, 'learning_rate': 2.446777547952507e-05, 'epoch': 0.36}
{'loss': 1.0869, 'learning_rate': 2.4448390815486165e-05, 'epoch': 0.37}
{'loss': 1.0751, 'learning_rate': 2.4428667382093762e-05, 'epoch': 0.37}
{'loss': 1.0684, 'learning_rate': 2.440860573856081e-05, 'epoch': 0.38}
{'loss': 1.0704, 'learning_rate': 2.438820645368942e-05, 'epoch': 0.38}
{'loss': 1.0671, 'learning_rate': 2.4367470105854766e-05, 'epoch': 0.39}
{'loss': 1.0434, 'learning_rate': 2.4346397282988666e-05, 'epoch': 0.39}
{'loss': 1.0605, 'learning_rate': 2.432498858256292e-05, 'epoch': 0.4}
{'loss': 1.0536, 'learning_rate': 2.4303244611572372e-05, 'epoch': 0.4}
{'loss': 1.0807, 'learning_rate': 2.4281165986517696e-05, 'epoch': 0.41}
{'loss': 1.0605, 'learning_rate': 2.425875333338792e-05, 'epoch': 0.41}
{'loss': 1.056, 'learning_rate': 2.4236007287642664e-05, 'epoch': 0.42}
{'loss': 1.0294, 'learning_rate': 2.421292849419416e-05, 'epoch': 0.42}
{'loss': 1.0571, 'learning_rate': 2.4189517607388918e-05, 'epoch': 0.43}
{'loss': 1.1025, 'learning_rate': 2.416577529098921e-05, 'epoch': 0.43}
{'loss': 1.1459, 'learning_rate': 2.4141702218154232e-05, 'epoch': 0.44}
{'loss': 1.0755, 'learning_rate': 2.4117299071421023e-05, 'epoch': 0.44}
{'loss': 1.0686, 'learning_rate': 2.4092566542685123e-05, 'epoch': 0.45}
{'loss': 1.0586, 'learning_rate': 2.4067505333180938e-05, 'epoch': 0.45}
{'loss': 1.1033, 'learning_rate': 2.404211615346187e-05, 'epoch': 0.46}
{'loss': 1.0253, 'learning_rate': 2.401639972338016e-05, 'epoch': 0.46}
{'loss': 1.0888, 'learning_rate': 2.399035677206651e-05, 'epoch': 0.47}
{'loss': 1.0677, 'learning_rate': 2.396398803790936e-05, 'epoch': 0.47}
{'loss': 1.0582, 'learning_rate': 2.393729426853399e-05, 'epoch': 0.48}
{'loss': 1.0223, 'learning_rate': 2.3910276220781314e-05, 'epoch': 0.48}
{'loss': 1.0885, 'learning_rate': 2.3882934660686418e-05, 'epoch': 0.49}
{'loss': 1.0553, 'learning_rate': 2.3855270363456833e-05, 'epoch': 0.49}
{'loss': 1.1293, 'learning_rate': 2.382728411345058e-05, 'epoch': 0.5}
{'loss': 1.1106, 'learning_rate': 2.3798976704153906e-05, 'epoch': 0.5}
{'loss': 1.0647, 'learning_rate': 2.3770348938158805e-05, 'epoch': 0.51}
{'loss': 1.078, 'learning_rate': 2.3741401627140242e-05, 'epoch': 0.51}
{'loss': 1.0344, 'learning_rate': 2.371213559183316e-05, 'epoch': 0.52}
{'loss': 1.095, 'learning_rate': 2.3682551662009203e-05, 'epoch': 0.52}
{'loss': 1.0794, 'learning_rate': 2.3652650676453188e-05, 'epoch': 0.53}
{'loss': 0.9938, 'learning_rate': 2.362243348293931e-05, 'epoch': 0.53}
{'loss': 1.1374, 'learning_rate': 2.3591900938207147e-05, 'epoch': 0.54}
{'loss': 1.0443, 'learning_rate': 2.3561053907937315e-05, 'epoch': 0.54}
{'loss': 1.0452, 'learning_rate': 2.352989326672696e-05, 'epoch': 0.55}
{'loss': 1.0359, 'learning_rate': 2.349841989806495e-05, 'epoch': 0.55}
{'loss': 1.0973, 'learning_rate': 2.346663469430683e-05, 'epoch': 0.56}
{'loss': 1.138, 'learning_rate': 2.3434538556649515e-05, 'epoch': 0.56}
{'loss': 1.0806, 'learning_rate': 2.3402132395105737e-05, 'epoch': 0.57}
{'loss': 1.1112, 'learning_rate': 2.336941712847825e-05, 'epoch': 0.57}
{'loss': 1.0837, 'learning_rate': 2.3336393684333778e-05, 'epoch': 0.58}
{'loss': 1.0794, 'learning_rate': 2.3303062998976712e-05, 'epoch': 0.58}
{'loss': 1.0757, 'learning_rate': 2.3269426017422576e-05, 'epoch': 0.59}
{'loss': 0.9923, 'learning_rate': 2.3235483693371214e-05, 'epoch': 0.59}
{'loss': 1.0442, 'learning_rate': 2.3201236989179766e-05, 'epoch': 0.6}
{'loss': 1.0608, 'learning_rate': 2.3166686875835365e-05, 'epoch': 0.6}
{'loss': 1.0806, 'learning_rate': 2.313183433292764e-05, 'epoch': 0.61}
{'loss': 1.0728, 'learning_rate': 2.3096680348620886e-05, 'epoch': 0.61}
{'loss': 1.0979, 'learning_rate': 2.3061225919626124e-05, 'epoch': 0.62}
{'loss': 1.0827, 'learning_rate': 2.3025472051172755e-05, 'epoch': 0.62}
{'loss': 1.0715, 'learning_rate': 2.2989419756980134e-05, 'epoch': 0.62}
{'loss': 1.0668, 'learning_rate': 2.295307005922879e-05, 'epoch': 0.63}
{'loss': 1.0726, 'learning_rate': 2.2916423988531437e-05, 'epoch': 0.63}
{'loss': 1.1038, 'learning_rate': 2.2879482583903783e-05, 'epoch': 0.64}
{'loss': 1.0698, 'learning_rate': 2.2842246892735053e-05, 'epoch': 0.64}
{'loss': 1.0782, 'learning_rate': 2.2804717970758286e-05, 'epoch': 0.65}
{'loss': 1.0834, 'learning_rate': 2.2766896882020422e-05, 'epoch': 0.65}
{'loss': 1.0536, 'learning_rate': 2.2728784698852114e-05, 'epoch': 0.66}
{'loss': 1.0867, 'learning_rate': 2.2690382501837344e-05, 'epoch': 0.66}
{'loss': 1.0369, 'learning_rate': 2.2651691379782764e-05, 'epoch': 0.67}
{'loss': 1.0474, 'learning_rate': 2.2612712429686845e-05, 'epoch': 0.67}
{'loss': 1.129, 'learning_rate': 2.2573446756708754e-05, 'epoch': 0.68}
{'loss': 1.0561, 'learning_rate': 2.2533895474137047e-05, 'epoch': 0.68}
{'loss': 1.1228, 'learning_rate': 2.2494059703358073e-05, 'epoch': 0.69}
{'loss': 1.1673, 'learning_rate': 2.24539405738242e-05, 'epoch': 0.69}
{'loss': 1.0184, 'learning_rate': 2.2413539223021794e-05, 'epoch': 0.7}
{'loss': 1.0699, 'learning_rate': 2.237285679643895e-05, 'epoch': 0.7}
{'loss': 1.0924, 'learning_rate': 2.2331894447533035e-05, 'epoch': 0.71}
{'loss': 1.0106, 'learning_rate': 2.229065333769796e-05, 'epoch': 0.71}
{'loss': 1.0525, 'learning_rate': 2.2249134636231285e-05, 'epoch': 0.72}
{'loss': 1.1001, 'learning_rate': 2.2207339520301033e-05, 'epoch': 0.72}
{'loss': 1.0832, 'learning_rate': 2.216526917491234e-05, 'epoch': 0.73}
{'loss': 0.9838, 'learning_rate': 2.2122924792873827e-05, 'epoch': 0.73}
{'loss': 1.0705, 'learning_rate': 2.2080307574763824e-05, 'epoch': 0.74}
{'loss': 1.1082, 'learning_rate': 2.2037418728896282e-05, 'epoch': 0.74}
{'loss': 1.0794, 'learning_rate': 2.1994259471286545e-05, 'epoch': 0.75}
{'loss': 1.0738, 'learning_rate': 2.1950831025616873e-05, 'epoch': 0.75}
{'loss': 1.0687, 'learning_rate': 2.190713462320173e-05, 'epoch': 0.76}
{'loss': 0.9987, 'learning_rate': 2.186317150295289e-05, 'epoch': 0.76}
{'loss': 1.0749, 'learning_rate': 2.1818942911344283e-05, 'epoch': 0.77}
{'loss': 1.053, 'learning_rate': 2.17744501023767e-05, 'epoch': 0.77}
{'loss': 1.0967, 'learning_rate': 2.1729694337542196e-05, 'epoch': 0.78}
{'loss': 1.1154, 'learning_rate': 2.168467688578834e-05, 'epoch': 0.78}
{'loss': 1.0552, 'learning_rate': 2.163939902348225e-05, 'epoch': 0.79}
{'loss': 1.0798, 'learning_rate': 2.1593862034374365e-05, 'epoch': 0.79}
{'loss': 1.073, 'learning_rate': 2.15480672095621e-05, 'epoch': 0.8}
{'loss': 1.0787, 'learning_rate': 2.15020158474532e-05, 'epoch': 0.8}
{'loss': 1.034, 'learning_rate': 2.145570925372894e-05, 'epoch': 0.81}
{'loss': 1.0699, 'learning_rate': 2.1409148741307107e-05, 'epoch': 0.81}
{'loss': 0.9939, 'learning_rate': 2.1362335630304768e-05, 'epoch': 0.82}
{'loss': 1.0616, 'learning_rate': 2.131527124800086e-05, 'epoch': 0.82}
{'loss': 0.992, 'learning_rate': 2.1267956928798525e-05, 'epoch': 0.83}
{'loss': 1.0463, 'learning_rate': 2.1220394014187312e-05, 'epoch': 0.83}
{'loss': 1.071, 'learning_rate': 2.1172583852705115e-05, 'epoch': 0.84}
{'loss': 1.0274, 'learning_rate': 2.1124527799899953e-05, 'epoch': 0.84}
{'loss': 1.0593, 'learning_rate': 2.1076227218291535e-05, 'epoch': 0.85}
{'loss': 1.0286, 'learning_rate': 2.1027683477332623e-05, 'epoch': 0.85}
{'loss': 1.0657, 'learning_rate': 2.0978897953370204e-05, 'epoch': 0.86}
{'loss': 1.0616, 'learning_rate': 2.0929872029606472e-05, 'epoch': 0.86}
{'loss': 1.0425, 'learning_rate': 2.088060709605962e-05, 'epoch': 0.87}
{'loss': 1.0024, 'learning_rate': 2.08311045495244e-05, 'epoch': 0.87}
{'loss': 1.0252, 'learning_rate': 2.078136579353255e-05, 'epoch': 0.88}
{'loss': 1.0623, 'learning_rate': 2.0731392238312985e-05, 'epoch': 0.88}
{'loss': 1.0712, 'learning_rate': 2.0681185300751814e-05, 'epoch': 0.89}
{'loss': 1.0976, 'learning_rate': 2.0630746404352168e-05, 'epoch': 0.89}
{'loss': 1.0481, 'learning_rate': 2.058007697919386e-05, 'epoch': 0.9}
{'loss': 1.0502, 'learning_rate': 2.0529178461892785e-05, 'epoch': 0.9}
{'loss': 1.0707, 'learning_rate': 2.0478052295560253e-05, 'epoch': 0.91}
{'loss': 1.0669, 'learning_rate': 2.0426699929762035e-05, 'epoch': 0.91}
{'loss': 1.0067, 'learning_rate': 2.037512282047726e-05, 'epoch': 0.92}
{'loss': 1.1202, 'learning_rate': 2.0323322430057166e-05, 'epoch': 0.92}
{'loss': 1.074, 'learning_rate': 2.0271300227183598e-05, 'epoch': 0.92}
{'loss': 1.0394, 'learning_rate': 2.02190576868274e-05, 'epoch': 0.93}
{'loss': 1.0368, 'learning_rate': 2.016659629020658e-05, 'epoch': 0.93}
{'loss': 1.0733, 'learning_rate': 2.0113917524744308e-05, 'epoch': 0.94}
{'loss': 1.0594, 'learning_rate': 2.0061022884026762e-05, 'epoch': 0.94}
{'loss': 1.0401, 'learning_rate': 2.0007913867760763e-05, 'epoch': 0.95}
{'loss': 1.0903, 'learning_rate': 1.995459198173127e-05, 'epoch': 0.95}
{'loss': 1.1046, 'learning_rate': 1.9901058737758667e-05, 'epoch': 0.96}
{'loss': 1.0478, 'learning_rate': 1.9847315653655915e-05, 'epoch': 0.96}
{'loss': 1.0747, 'learning_rate': 1.9793364253185518e-05, 'epoch': 0.97}
{'loss': 1.0484, 'learning_rate': 1.9739206066016307e-05, 'epoch': 0.97}
{'loss': 1.0897, 'learning_rate': 1.9684842627680088e-05, 'epoch': 0.98}
{'loss': 1.0715, 'learning_rate': 1.963027547952808e-05, 'epoch': 0.98}
{'loss': 1.0671, 'learning_rate': 1.9575506168687247e-05, 'epoch': 0.99}
{'loss': 0.9719, 'learning_rate': 1.9520536248016403e-05, 'epoch': 0.99}
{'loss': 1.0384, 'learning_rate': 1.946536727606219e-05, 'epoch': 1.0}
{'loss': 0.7968, 'learning_rate': 1.941000081701492e-05, 'epoch': 1.0}
{'loss': 0.6873, 'learning_rate': 1.9354438440664167e-05, 'epoch': 1.01}
{'loss': 0.6435, 'learning_rate': 1.929868172235433e-05, 'epoch': 1.01}
{'loss': 0.6083, 'learning_rate': 1.9242732242939893e-05, 'epoch': 1.02}
{'loss': 0.6499, 'learning_rate': 1.9186591588740667e-05, 'epoch': 1.02}
{'loss': 0.6342, 'learning_rate': 1.913026135149678e-05, 'epoch': 1.03}
{'loss': 0.6186, 'learning_rate': 1.9073743128323558e-05, 'epoch': 1.03}
{'loss': 0.6751, 'learning_rate': 1.9017038521666232e-05, 'epoch': 1.04}
{'loss': 0.6175, 'learning_rate': 1.8960149139254513e-05, 'epoch': 1.04}
{'loss': 0.5863, 'learning_rate': 1.8903076594057013e-05, 'epoch': 1.05}
{'loss': 0.6271, 'learning_rate': 1.88458225042355e-05, 'epoch': 1.05}
{'loss': 0.6416, 'learning_rate': 1.8788388493099033e-05, 'epoch': 1.06}
{'loss': 0.653, 'learning_rate': 1.8730776189057918e-05, 'epoch': 1.06}
{'loss': 0.6631, 'learning_rate': 1.8672987225577552e-05, 'epoch': 1.07}
{'loss': 0.6022, 'learning_rate': 1.8615023241132127e-05, 'epoch': 1.07}
{'loss': 0.645, 'learning_rate': 1.855688587915813e-05, 'epoch': 1.08}
{'loss': 0.6546, 'learning_rate': 1.8498576788007775e-05, 'epoch': 1.08}
{'loss': 0.6453, 'learning_rate': 1.8440097620902284e-05, 'epoch': 1.09}
{'loss': 0.6509, 'learning_rate': 1.838145003588497e-05, 'epoch': 1.09}
{'loss': 0.5689, 'learning_rate': 1.832263569577427e-05, 'epoch': 1.1}
{'loss': 0.5919, 'learning_rate': 1.8263656268116576e-05, 'epoch': 1.1}
{'loss': 0.6308, 'learning_rate': 1.8204513425138952e-05, 'epoch': 1.11}
{'loss': 0.6666, 'learning_rate': 1.8145208843701742e-05, 'epoch': 1.11}
{'loss': 0.584, 'learning_rate': 1.808574420525101e-05, 'epoch': 1.12}
{'loss': 0.5634, 'learning_rate': 1.8026121195770874e-05, 'epoch': 1.12}
{'loss': 0.5975, 'learning_rate': 1.7966341505735695e-05, 'epoch': 1.13}
{'loss': 0.6015, 'learning_rate': 1.7906406830062156e-05, 'epoch': 1.13}
{'loss': 0.5879, 'learning_rate': 1.7846318868061212e-05, 'epoch': 1.14}
{'loss': 0.6187, 'learning_rate': 1.7786079323389893e-05, 'epoch': 1.14}
{'loss': 0.617, 'learning_rate': 1.7725689904003006e-05, 'epoch': 1.15}
{'loss': 0.6135, 'learning_rate': 1.766515232210473e-05, 'epoch': 1.15}
{'loss': 0.6676, 'learning_rate': 1.7604468294100034e-05, 'epoch': 1.16}
{'loss': 0.6294, 'learning_rate': 1.754363954054605e-05, 'epoch': 1.16}
{'loss': 0.6308, 'learning_rate': 1.7482667786103256e-05, 'epoch': 1.17}
{'loss': 0.5851, 'learning_rate': 1.7421554759486614e-05, 'epoch': 1.17}
{'loss': 0.6303, 'learning_rate': 1.736030219341651e-05, 'epoch': 1.18}
{'loss': 0.6124, 'learning_rate': 1.7298911824569676e-05, 'epoch': 1.18}
{'loss': 0.6291, 'learning_rate': 1.7237385393529916e-05, 'epoch': 1.19}
{'loss': 0.6457, 'learning_rate': 1.7175724644738773e-05, 'epoch': 1.19}
{'loss': 0.6475, 'learning_rate': 1.7113931326446054e-05, 'epoch': 1.2}
{'loss': 0.6723, 'learning_rate': 1.705200719066028e-05, 'epoch': 1.2}
{'loss': 0.6456, 'learning_rate': 1.6989953993098998e-05, 'epoch': 1.21}
{'loss': 0.613, 'learning_rate': 1.6927773493139014e-05, 'epoch': 1.21}
{'loss': 0.6202, 'learning_rate': 1.68654674537665e-05, 'epoch': 1.22}
{'loss': 0.5821, 'learning_rate': 1.6803037641527002e-05, 'epoch': 1.22}
{'loss': 0.6698, 'learning_rate': 1.674048582647538e-05, 'epoch': 1.23}
{'loss': 0.6762, 'learning_rate': 1.6677813782125592e-05, 'epoch': 1.23}
{'loss': 0.6087, 'learning_rate': 1.6615023285400432e-05, 'epoch': 1.23}
{'loss': 0.6154, 'learning_rate': 1.6552116116581133e-05, 'epoch': 1.24}
{'loss': 0.6162, 'learning_rate': 1.648909405925691e-05, 'epoch': 1.24}
{'loss': 0.624, 'learning_rate': 1.6425958900274362e-05, 'epoch': 1.25}
{'loss': 0.5986, 'learning_rate': 1.6362712429686846e-05, 'epoch': 1.25}
{'loss': 0.6152, 'learning_rate': 1.629935644070369e-05, 'epoch': 1.26}
{'loss': 0.6114, 'learning_rate': 1.6235892729639378e-05, 'epoch': 1.26}
{'loss': 0.603, 'learning_rate': 1.617232309586261e-05, 'epoch': 1.27}
{'loss': 0.6791, 'learning_rate': 1.6108649341745262e-05, 'epoch': 1.27}
{'loss': 0.6662, 'learning_rate': 1.604487327261133e-05, 'epoch': 1.28}
{'loss': 0.6407, 'learning_rate': 1.59809966966857e-05, 'epoch': 1.28}
{'loss': 0.629, 'learning_rate': 1.591702142504291e-05, 'epoch': 1.29}
{'loss': 0.6191, 'learning_rate': 1.5852949271555787e-05, 'epoch': 1.29}
{'loss': 0.6332, 'learning_rate': 1.5788782052844015e-05, 'epoch': 1.3}
{'loss': 0.642, 'learning_rate': 1.572452158822265e-05, 'epoch': 1.3}
{'loss': 0.6257, 'learning_rate': 1.5660169699650503e-05, 'epoch': 1.31}
{'loss': 0.6622, 'learning_rate': 1.5595728211678525e-05, 'epoch': 1.31}
{'loss': 0.6221, 'learning_rate': 1.553119895139803e-05, 'epoch': 1.32}
{'loss': 0.641, 'learning_rate': 1.546658374838894e-05, 'epoch': 1.32}
{'loss': 0.6199, 'learning_rate': 1.5401884434667864e-05, 'epoch': 1.33}
{'loss': 0.6204, 'learning_rate': 1.5337102844636188e-05, 'epoch': 1.33}
{'loss': 0.6315, 'learning_rate': 1.5272240815028067e-05, 'epoch': 1.34}
{'loss': 0.6727, 'learning_rate': 1.520730018485832e-05, 'epoch': 1.34}
{'loss': 0.6628, 'learning_rate': 1.5142282795370305e-05, 'epoch': 1.35}
{'loss': 0.6742, 'learning_rate': 1.5077190489983728e-05, 'epoch': 1.35}
{'loss': 0.6381, 'learning_rate': 1.5012025114242355e-05, 'epoch': 1.36}
{'loss': 0.6467, 'learning_rate': 1.49467885157617e-05, 'epoch': 1.36}
{'loss': 0.6497, 'learning_rate': 1.4881482544176627e-05, 'epoch': 1.37}
{'loss': 0.6165, 'learning_rate': 1.4816109051088931e-05, 'epoch': 1.37}
{'loss': 0.6183, 'learning_rate': 1.4750669890014807e-05, 'epoch': 1.38}
{'loss': 0.6557, 'learning_rate': 1.468516691633233e-05, 'epoch': 1.38}
{'loss': 0.6581, 'learning_rate': 1.4619601987228832e-05, 'epoch': 1.39}
{'loss': 0.6562, 'learning_rate': 1.4553976961648244e-05, 'epoch': 1.39}
{'loss': 0.6587, 'learning_rate': 1.4488293700238401e-05, 'epoch': 1.4}
{'loss': 0.6124, 'learning_rate': 1.4422554065298286e-05, 'epoch': 1.4}
{'loss': 0.6168, 'learning_rate': 1.435675992072521e-05, 'epoch': 1.41}
{'loss': 0.6346, 'learning_rate': 1.4290913131961997e-05, 'epoch': 1.41}
{'loss': 0.6851, 'learning_rate': 1.4225015565944072e-05, 'epoch': 1.42}
{'loss': 0.652, 'learning_rate': 1.4159069091046526e-05, 'epoch': 1.42}
{'loss': 0.6331, 'learning_rate': 1.4093075577031157e-05, 'epoch': 1.43}
{'loss': 0.6402, 'learning_rate': 1.4027036894993442e-05, 'epoch': 1.43}
{'loss': 0.6072, 'learning_rate': 1.396095491730951e-05, 'epoch': 1.44}
{'loss': 0.6314, 'learning_rate': 1.3894831517583017e-05, 'epoch': 1.44}
{'loss': 0.5981, 'learning_rate': 1.3828668570592069e-05, 'epoch': 1.45}
{'loss': 0.6045, 'learning_rate': 1.3762467952236027e-05, 'epoch': 1.45}
{'loss': 0.6773, 'learning_rate': 1.369623153948235e-05, 'epoch': 1.46}
{'loss': 0.6185, 'learning_rate': 1.3629961210313353e-05, 'epoch': 1.46}
{'loss': 0.6684, 'learning_rate': 1.3563658843672985e-05, 'epoch': 1.47}
{'loss': 0.5952, 'learning_rate': 1.3497326319413539e-05, 'epoch': 1.47}
{'loss': 0.6146, 'learning_rate': 1.3430965518242362e-05, 'epoch': 1.48}
{'loss': 0.6532, 'learning_rate': 1.336457832166852e-05, 'epoch': 1.48}
{'loss': 0.6521, 'learning_rate': 1.3298166611949467e-05, 'epoch': 1.49}
{'loss': 0.6157, 'learning_rate': 1.3231732272037672e-05, 'epoch': 1.49}
{'loss': 0.6474, 'learning_rate': 1.3165277185527234e-05, 'epoch': 1.5}
{'loss': 0.5977, 'learning_rate': 1.3098803236600462e-05, 'epoch': 1.5}
{'loss': 0.624, 'learning_rate': 1.3032312309974482e-05, 'epoch': 1.51}
{'loss': 0.6255, 'learning_rate': 1.2965806290847768e-05, 'epoch': 1.51}
{'loss': 0.6181, 'learning_rate': 1.2899287064846725e-05, 'epoch': 1.52}
{'loss': 0.6159, 'learning_rate': 1.2832756517972185e-05, 'epoch': 1.52}
{'loss': 0.6414, 'learning_rate': 1.2766216536545984e-05, 'epoch': 1.53}
{'loss': 0.5773, 'learning_rate': 1.2699669007157435e-05, 'epoch': 1.53}
{'loss': 0.627, 'learning_rate': 1.2633115816609867e-05, 'epoch': 1.54}
{'loss': 0.5812, 'learning_rate': 1.2566558851867106e-05, 'epoch': 1.54}
{'loss': 0.6219, 'learning_rate': 1.25e-05, 'epoch': 1.54}
{'loss': 0.6287, 'learning_rate': 1.2433441148132897e-05, 'epoch': 1.55}
{'loss': 0.5863, 'learning_rate': 1.236688418339014e-05, 'epoch': 1.55}
{'loss': 0.5904, 'learning_rate': 1.2300330992842566e-05, 'epoch': 1.56}
{'loss': 0.6398, 'learning_rate': 1.223378346345402e-05, 'epoch': 1.56}
{'loss': 0.6025, 'learning_rate': 1.2167243482027816e-05, 'epoch': 1.57}
{'loss': 0.6047, 'learning_rate': 1.2100712935153283e-05, 'epoch': 1.57}
{'loss': 0.6134, 'learning_rate': 1.2034193709152231e-05, 'epoch': 1.58}
{'loss': 0.6624, 'learning_rate': 1.196768769002552e-05, 'epoch': 1.58}
{'loss': 0.6065, 'learning_rate': 1.190119676339954e-05, 'epoch': 1.59}
{'loss': 0.6245, 'learning_rate': 1.1834722814472771e-05, 'epoch': 1.59}
{'loss': 0.6223, 'learning_rate': 1.1768267727962328e-05, 'epoch': 1.6}
{'loss': 0.638, 'learning_rate': 1.1701833388050536e-05, 'epoch': 1.6}
{'loss': 0.6685, 'learning_rate': 1.1635421678331485e-05, 'epoch': 1.61}
{'loss': 0.6236, 'learning_rate': 1.1569034481757644e-05, 'epoch': 1.61}
{'loss': 0.6174, 'learning_rate': 1.150267368058646e-05, 'epoch': 1.62}
{'loss': 0.6431, 'learning_rate': 1.1436341156327016e-05, 'epoch': 1.62}
{'loss': 0.6326, 'learning_rate': 1.137003878968665e-05, 'epoch': 1.63}
{'loss': 0.6585, 'learning_rate': 1.1303768460517657e-05, 'epoch': 1.63}
{'loss': 0.653, 'learning_rate': 1.1237532047763974e-05, 'epoch': 1.64}
{'loss': 0.6169, 'learning_rate': 1.1171331429407934e-05, 'epoch': 1.64}
{'loss': 0.5823, 'learning_rate': 1.1105168482416984e-05, 'epoch': 1.65}
{'loss': 0.6307, 'learning_rate': 1.1039045082690496e-05, 'epoch': 1.65}
{'loss': 0.6119, 'learning_rate': 1.0972963105006556e-05, 'epoch': 1.66}
{'loss': 0.6236, 'learning_rate': 1.0906924422968846e-05, 'epoch': 1.66}
{'loss': 0.5794, 'learning_rate': 1.0840930908953477e-05, 'epoch': 1.67}
{'loss': 0.6227, 'learning_rate': 1.0774984434055932e-05, 'epoch': 1.67}
{'loss': 0.6367, 'learning_rate': 1.0709086868038005e-05, 'epoch': 1.68}
{'loss': 0.6325, 'learning_rate': 1.064324007927479e-05, 'epoch': 1.68}
{'loss': 0.6073, 'learning_rate': 1.0577445934701717e-05, 'epoch': 1.69}
{'loss': 0.6616, 'learning_rate': 1.0511706299761602e-05, 'epoch': 1.69}
{'loss': 0.6514, 'learning_rate': 1.044602303835176e-05, 'epoch': 1.7}
{'loss': 0.6421, 'learning_rate': 1.0380398012771169e-05, 'epoch': 1.7}
{'loss': 0.6286, 'learning_rate': 1.0314833083667672e-05, 'epoch': 1.71}
{'loss': 0.6372, 'learning_rate': 1.0249330109985198e-05, 'epoch': 1.71}
{'loss': 0.6379, 'learning_rate': 1.0183890948911074e-05, 'epoch': 1.72}
{'loss': 0.6726, 'learning_rate': 1.0118517455823374e-05, 'epoch': 1.72}
{'loss': 0.6715, 'learning_rate': 1.0053211484238303e-05, 'epoch': 1.73}
{'loss': 0.6376, 'learning_rate': 9.987974885757647e-06, 'epoch': 1.73}
{'loss': 0.6052, 'learning_rate': 9.922809510016277e-06, 'epoch': 1.74}
{'loss': 0.6083, 'learning_rate': 9.857717204629696e-06, 'epoch': 1.74}
{'loss': 0.6343, 'learning_rate': 9.792699815141685e-06, 'epoch': 1.75}
{'loss': 0.631, 'learning_rate': 9.727759184971936e-06, 'epoch': 1.75}
{'loss': 0.6598, 'learning_rate': 9.662897155363814e-06, 'epoch': 1.76}
{'loss': 0.6396, 'learning_rate': 9.598115565332142e-06, 'epoch': 1.76}
{'loss': 0.6778, 'learning_rate': 9.533416251611064e-06, 'epoch': 1.77}
{'loss': 0.6252, 'learning_rate': 9.468801048601972e-06, 'epoch': 1.77}
{'loss': 0.6119, 'learning_rate': 9.404271788321482e-06, 'epoch': 1.78}
{'loss': 0.6523, 'learning_rate': 9.339830300349498e-06, 'epoch': 1.78}
{'loss': 0.6336, 'learning_rate': 9.275478411777354e-06, 'epoch': 1.79}
{'loss': 0.592, 'learning_rate': 9.211217947155986e-06, 'epoch': 1.79}
{'loss': 0.6204, 'learning_rate': 9.147050728444215e-06, 'epoch': 1.8}
{'loss': 0.6273, 'learning_rate': 9.08297857495709e-06, 'epoch': 1.8}
{'loss': 0.5883, 'learning_rate': 9.0190033033143e-06, 'epoch': 1.81}
{'loss': 0.6282, 'learning_rate': 8.955126727388672e-06, 'epoch': 1.81}
{'loss': 0.6745, 'learning_rate': 8.89135065825474e-06, 'epoch': 1.82}
{'loss': 0.6146, 'learning_rate': 8.827676904137397e-06, 'epoch': 1.82}
{'loss': 0.5944, 'learning_rate': 8.76410727036062e-06, 'epoch': 1.83}
{'loss': 0.6359, 'learning_rate': 8.700643559296313e-06, 'epoch': 1.83}
{'loss': 0.5846, 'learning_rate': 8.637287570313159e-06, 'epoch': 1.84}
{'loss': 0.6315, 'learning_rate': 8.574041099725644e-06, 'epoch': 1.84}
{'loss': 0.5993, 'learning_rate': 8.510905940743093e-06, 'epoch': 1.85}
{'loss': 0.652, 'learning_rate': 8.447883883418866e-06, 'epoch': 1.85}
{'loss': 0.6542, 'learning_rate': 8.38497671459957e-06, 'epoch': 1.85}
{'loss': 0.6168, 'learning_rate': 8.32218621787441e-06, 'epoch': 1.86}
{'loss': 0.632, 'learning_rate': 8.25951417352462e-06, 'epoch': 1.86}
{'loss': 0.6016, 'learning_rate': 8.196962358473e-06, 'epoch': 1.87}
{'loss': 0.5967, 'learning_rate': 8.134532546233505e-06, 'epoch': 1.87}
{'loss': 0.6398, 'learning_rate': 8.072226506860988e-06, 'epoch': 1.88}
{'loss': 0.6485, 'learning_rate': 8.010046006901001e-06, 'epoch': 1.88}
{'loss': 0.588, 'learning_rate': 7.947992809339723e-06, 'epoch': 1.89}
{'loss': 0.6784, 'learning_rate': 7.88606867355395e-06, 'epoch': 1.89}
{'loss': 0.6223, 'learning_rate': 7.824275355261233e-06, 'epoch': 1.9}
{'loss': 0.5966, 'learning_rate': 7.762614606470084e-06, 'epoch': 1.9}
{'loss': 0.6124, 'learning_rate': 7.701088175430327e-06, 'epoch': 1.91}
{'loss': 0.637, 'learning_rate': 7.639697806583493e-06, 'epoch': 1.91}
{'loss': 0.6198, 'learning_rate': 7.578445240513393e-06, 'epoch': 1.92}
{'loss': 0.6397, 'learning_rate': 7.517332213896743e-06, 'epoch': 1.92}
{'loss': 0.6308, 'learning_rate': 7.456360459453952e-06, 'epoch': 1.93}
{'loss': 0.5814, 'learning_rate': 7.395531705899969e-06, 'epoch': 1.93}
{'loss': 0.5953, 'learning_rate': 7.334847677895276e-06, 'epoch': 1.94}
{'loss': 0.6071, 'learning_rate': 7.274310095997e-06, 'epoch': 1.94}
{'loss': 0.6304, 'learning_rate': 7.213920676610112e-06, 'epoch': 1.95}
{'loss': 0.6411, 'learning_rate': 7.153681131938791e-06, 'epoch': 1.95}
{'loss': 0.6012, 'learning_rate': 7.093593169937849e-06, 'epoch': 1.96}
{'loss': 0.6503, 'learning_rate': 7.033658494264309e-06, 'epoch': 1.96}
{'loss': 0.6363, 'learning_rate': 6.973878804229131e-06, 'epoch': 1.97}
{'loss': 0.6212, 'learning_rate': 6.914255794748989e-06, 'epoch': 1.97}
{'loss': 0.6694, 'learning_rate': 6.8547911562982595e-06, 'epoch': 1.98}
{'loss': 0.6147, 'learning_rate': 6.7954865748610515e-06, 'epoch': 1.98}
{'loss': 0.6615, 'learning_rate': 6.736343731883425e-06, 'epoch': 1.99}
{'loss': 0.602, 'learning_rate': 6.677364304225731e-06, 'epoch': 1.99}
{'loss': 0.604, 'learning_rate': 6.618549964115033e-06, 'epoch': 2.0}
{'loss': 0.4458, 'learning_rate': 6.559902379097721e-06, 'epoch': 2.0}
{'loss': 0.3383, 'learning_rate': 6.501423211992224e-06, 'epoch': 2.01}
{'loss': 0.3288, 'learning_rate': 6.443114120841874e-06, 'epoch': 2.01}
{'loss': 0.3252, 'learning_rate': 6.384976758867875e-06, 'epoch': 2.02}
{'loss': 0.3226, 'learning_rate': 6.32701277442245e-06, 'epoch': 2.02}
{'loss': 0.3122, 'learning_rate': 6.269223810942086e-06, 'epoch': 2.03}
{'loss': 0.2519, 'learning_rate': 6.211611506900973e-06, 'epoch': 2.03}
{'loss': 0.3158, 'learning_rate': 6.1541774957645045e-06, 'epoch': 2.04}
{'loss': 0.2846, 'learning_rate': 6.096923405942989e-06, 'epoch': 2.04}
{'loss': 0.2747, 'learning_rate': 6.03985086074549e-06, 'epoch': 2.05}
{'loss': 0.2876, 'learning_rate': 5.982961478333768e-06, 'epoch': 2.05}
{'loss': 0.2596, 'learning_rate': 5.926256871676443e-06, 'epoch': 2.06}
{'loss': 0.28, 'learning_rate': 5.86973864850322e-06, 'epoch': 2.06}
{'loss': 0.2694, 'learning_rate': 5.813408411259331e-06, 'epoch': 2.07}
{'loss': 0.2707, 'learning_rate': 5.757267757060109e-06, 'epoch': 2.07}
{'loss': 0.2814, 'learning_rate': 5.701318277645675e-06, 'epoch': 2.08}
{'loss': 0.2916, 'learning_rate': 5.645561559335832e-06, 'epoch': 2.08}
{'loss': 0.3062, 'learning_rate': 5.589999182985087e-06, 'epoch': 2.09}
{'loss': 0.2818, 'learning_rate': 5.534632723937809e-06, 'epoch': 2.09}
{'loss': 0.263, 'learning_rate': 5.479463751983603e-06, 'epoch': 2.1}
{'loss': 0.2992, 'learning_rate': 5.424493831312757e-06, 'epoch': 2.1}
{'loss': 0.2966, 'learning_rate': 5.369724520471925e-06, 'epoch': 2.11}
{'loss': 0.2803, 'learning_rate': 5.315157372319915e-06, 'epoch': 2.11}
{'loss': 0.2749, 'learning_rate': 5.2607939339836945e-06, 'epoch': 2.12}
{'loss': 0.2728, 'learning_rate': 5.2066357468144824e-06, 'epoch': 2.12}
{'loss': 0.2545, 'learning_rate': 5.152684346344087e-06, 'epoch': 2.13}
{'loss': 0.2722, 'learning_rate': 5.0989412622413345e-06, 'epoch': 2.13}
{'loss': 0.3021, 'learning_rate': 5.0454080182687314e-06, 'epoch': 2.14}
{'loss': 0.2483, 'learning_rate': 4.992086132239238e-06, 'epoch': 2.14}
{'loss': 0.2703, 'learning_rate': 4.938977115973242e-06, 'epoch': 2.15}
{'loss': 0.2829, 'learning_rate': 4.886082475255694e-06, 'epoch': 2.15}
{'loss': 0.2634, 'learning_rate': 4.833403709793424e-06, 'epoch': 2.15}
{'loss': 0.2684, 'learning_rate': 4.780942313172602e-06, 'epoch': 2.16}
{'loss': 0.3123, 'learning_rate': 4.728699772816407e-06, 'epoch': 2.16}
{'loss': 0.2837, 'learning_rate': 4.6766775699428365e-06, 'epoch': 2.17}
{'loss': 0.2915, 'learning_rate': 4.624877179522742e-06, 'epoch': 2.17}
{'loss': 0.305, 'learning_rate': 4.573300070237973e-06, 'epoch': 2.18}
{'loss': 0.2537, 'learning_rate': 4.5219477044397525e-06, 'epoch': 2.18}
{'loss': 0.2782, 'learning_rate': 4.470821538107219e-06, 'epoch': 2.19}
{'loss': 0.3202, 'learning_rate': 4.419923020806148e-06, 'epoch': 2.19}
{'loss': 0.2547, 'learning_rate': 4.36925359564783e-06, 'epoch': 2.2}
{'loss': 0.2707, 'learning_rate': 4.318814699248188e-06, 'epoch': 2.2}
{'loss': 0.246, 'learning_rate': 4.268607761687019e-06, 'epoch': 2.21}
{'loss': 0.2737, 'learning_rate': 4.218634206467449e-06, 'epoch': 2.21}
{'loss': 0.2921, 'learning_rate': 4.1688954504756e-06, 'epoch': 2.22}
{'loss': 0.2677, 'learning_rate': 4.119392903940384e-06, 'epoch': 2.22}
{'loss': 0.2979, 'learning_rate': 4.07012797039353e-06, 'epoch': 2.23}
{'loss': 0.2459, 'learning_rate': 4.021102046629799e-06, 'epoch': 2.23}
{'loss': 0.2745, 'learning_rate': 3.972316522667381e-06, 'epoch': 2.24}
{'loss': 0.2943, 'learning_rate': 3.923772781708468e-06, 'epoch': 2.24}
{'loss': 0.2403, 'learning_rate': 3.875472200100051e-06, 'epoch': 2.25}
{'loss': 0.2781, 'learning_rate': 3.827416147294888e-06, 'epoch': 2.25}
{'loss': 0.2766, 'learning_rate': 3.7796059858126927e-06, 'epoch': 2.26}
{'loss': 0.2738, 'learning_rate': 3.7320430712014796e-06, 'epoch': 2.26}
{'loss': 0.3136, 'learning_rate': 3.6847287519991434e-06, 'epoch': 2.27}
{'loss': 0.2902, 'learning_rate': 3.637664369695233e-06, 'epoch': 2.27}
{'loss': 0.2516, 'learning_rate': 3.590851258692894e-06, 'epoch': 2.28}
{'loss': 0.2668, 'learning_rate': 3.5442907462710605e-06, 'epoch': 2.28}
{'loss': 0.2789, 'learning_rate': 3.4979841525468016e-06, 'epoch': 2.29}
{'loss': 0.2794, 'learning_rate': 3.451932790437898e-06, 'epoch': 2.29}
{'loss': 0.273, 'learning_rate': 3.4061379656256345e-06, 'epoch': 2.3}
{'loss': 0.2828, 'learning_rate': 3.3606009765177555e-06, 'epoch': 2.3}
{'loss': 0.2836, 'learning_rate': 3.3153231142116617e-06, 'epoch': 2.31}
{'loss': 0.2781, 'learning_rate': 3.270305662457805e-06, 'epoch': 2.31}
{'loss': 0.2911, 'learning_rate': 3.225549897623302e-06, 'epoch': 2.32}
{'loss': 0.3071, 'learning_rate': 3.181057088655719e-06, 'epoch': 2.32}
{'loss': 0.2583, 'learning_rate': 3.136828497047116e-06, 'epoch': 2.33}
{'loss': 0.2976, 'learning_rate': 3.0928653767982695e-06, 'epoch': 2.33}
{'loss': 0.2727, 'learning_rate': 3.0491689743831277e-06, 'epoch': 2.34}
{'loss': 0.2843, 'learning_rate': 3.005740528713458e-06, 'epoch': 2.34}
{'loss': 0.2655, 'learning_rate': 2.9625812711037206e-06, 'epoch': 2.35}
{'loss': 0.2759, 'learning_rate': 2.91969242523618e-06, 'epoch': 2.35}
{'loss': 0.272, 'learning_rate': 2.87707520712617e-06, 'epoch': 2.36}
{'loss': 0.2734, 'learning_rate': 2.834730825087663e-06, 'epoch': 2.36}
{'loss': 0.312, 'learning_rate': 2.7926604796989676e-06, 'epoch': 2.37}
{'loss': 0.2608, 'learning_rate': 2.7508653637687176e-06, 'epoch': 2.37}
{'loss': 0.3084, 'learning_rate': 2.709346662302041e-06, 'epoch': 2.38}
{'loss': 0.3209, 'learning_rate': 2.6681055524669694e-06, 'epoch': 2.38}
{'loss': 0.2781, 'learning_rate': 2.6271432035610513e-06, 'epoch': 2.39}
{'loss': 0.2379, 'learning_rate': 2.5864607769782084e-06, 'epoch': 2.39}
{'loss': 0.2945, 'learning_rate': 2.5460594261757996e-06, 'epoch': 2.4}
{'loss': 0.2545, 'learning_rate': 2.5059402966419303e-06, 'epoch': 2.4}
{'loss': 0.285, 'learning_rate': 2.466104525862957e-06, 'epoch': 2.41}
{'loss': 0.2818, 'learning_rate': 2.426553243291249e-06, 'epoch': 2.41}
{'loss': 0.2743, 'learning_rate': 2.3872875703131583e-06, 'epoch': 2.42}
{'loss': 0.2991, 'learning_rate': 2.3483086202172377e-06, 'epoch': 2.42}
{'loss': 0.2778, 'learning_rate': 2.309617498162657e-06, 'epoch': 2.43}
{'loss': 0.2723, 'learning_rate': 2.271215301147887e-06, 'epoch': 2.43}
{'loss': 0.2684, 'learning_rate': 2.2331031179795792e-06, 'epoch': 2.44}
{'loss': 0.2818, 'learning_rate': 2.1952820292417143e-06, 'epoch': 2.44}
{'loss': 0.2658, 'learning_rate': 2.1577531072649495e-06, 'epoch': 2.45}
{'loss': 0.2865, 'learning_rate': 2.120517416096217e-06, 'epoch': 2.45}
{'loss': 0.2744, 'learning_rate': 2.083576011468562e-06, 'epoch': 2.46}
{'loss': 0.2939, 'learning_rate': 2.0469299407712112e-06, 'epoch': 2.46}
{'loss': 0.3174, 'learning_rate': 2.010580243019865e-06, 'epoch': 2.46}
{'loss': 0.2805, 'learning_rate': 1.9745279488272484e-06, 'epoch': 2.47}
{'loss': 0.2587, 'learning_rate': 1.938774080373884e-06, 'epoch': 2.47}
{'loss': 0.2591, 'learning_rate': 1.9033196513791138e-06, 'epoch': 2.48}
{'loss': 0.28, 'learning_rate': 1.8681656670723663e-06, 'epoch': 2.48}
{'loss': 0.2757, 'learning_rate': 1.833313124164636e-06, 'epoch': 2.49}
{'loss': 0.2684, 'learning_rate': 1.7987630108202355e-06, 'epoch': 2.49}
{'loss': 0.3003, 'learning_rate': 1.764516306628787e-06, 'epoch': 2.5}
{'loss': 0.2826, 'learning_rate': 1.7305739825774228e-06, 'epoch': 2.5}
{'loss': 0.2531, 'learning_rate': 1.6969370010232882e-06, 'epoch': 2.51}
{'loss': 0.277, 'learning_rate': 1.663606315666226e-06, 'epoch': 2.51}
{'loss': 0.2838, 'learning_rate': 1.6305828715217506e-06, 'epoch': 2.52}
{'loss': 0.3017, 'learning_rate': 1.597867604894264e-06, 'epoch': 2.52}
{'loss': 0.2695, 'learning_rate': 1.5654614433504841e-06, 'epoch': 2.53}
{'loss': 0.3005, 'learning_rate': 1.5333653056931684e-06, 'epoch': 2.53}
{'loss': 0.259, 'learning_rate': 1.5015801019350476e-06, 'epoch': 2.54}
{'loss': 0.276, 'learning_rate': 1.4701067332730426e-06, 'epoch': 2.54}
{'loss': 0.2706, 'learning_rate': 1.4389460920626887e-06, 'epoch': 2.55}
{'loss': 0.2808, 'learning_rate': 1.4080990617928571e-06, 'epoch': 2.55}
{'loss': 0.2925, 'learning_rate': 1.3775665170606896e-06, 'epoch': 2.56}
{'loss': 0.2644, 'learning_rate': 1.3473493235468177e-06, 'epoch': 2.56}
{'loss': 0.2484, 'learning_rate': 1.3174483379907996e-06, 'epoch': 2.57}
{'loss': 0.3012, 'learning_rate': 1.2878644081668403e-06, 'epoch': 2.57}
{'loss': 0.3033, 'learning_rate': 1.2585983728597608e-06, 'epoch': 2.58}
{'loss': 0.33, 'learning_rate': 1.2296510618411957e-06, 'epoch': 2.58}
{'loss': 0.2513, 'learning_rate': 1.201023295846093e-06, 'epoch': 2.59}
{'loss': 0.2871, 'learning_rate': 1.1727158865494206e-06, 'epoch': 2.59}
{'loss': 0.2592, 'learning_rate': 1.1447296365431668e-06, 'epoch': 2.6}
{'loss': 0.2672, 'learning_rate': 1.1170653393135847e-06, 'epoch': 2.6}
{'loss': 0.2589, 'learning_rate': 1.0897237792186862e-06, 'epoch': 2.61}
{'loss': 0.2851, 'learning_rate': 1.0627057314660114e-06, 'epoch': 2.61}
{'loss': 0.2319, 'learning_rate': 1.0360119620906426e-06, 'epoch': 2.62}
{'loss': 0.2757, 'learning_rate': 1.0096432279334905e-06, 'epoch': 2.62}
{'loss': 0.2742, 'learning_rate': 9.836002766198379e-07, 'epoch': 2.63}
{'loss': 0.324, 'learning_rate': 9.57883846538135e-07, 'epoch': 2.63}
{'loss': 0.2435, 'learning_rate': 9.324946668190657e-07, 'epoch': 2.64}
{'loss': 0.2795, 'learning_rate': 9.074334573148783e-07, 'epoch': 2.64}
{'loss': 0.3012, 'learning_rate': 8.827009285789778e-07, 'epoch': 2.65}
{'loss': 0.2663, 'learning_rate': 8.582977818457696e-07, 'epoch': 2.65}
{'loss': 0.2469, 'learning_rate': 8.342247090107919e-07, 'epoch': 2.66}
{'loss': 0.2835, 'learning_rate': 8.104823926110811e-07, 'epoch': 2.66}
{'loss': 0.2554, 'learning_rate': 7.870715058058406e-07, 'epoch': 2.67}
{'loss': 0.2571, 'learning_rate': 7.639927123573351e-07, 'epoch': 2.67}
{'loss': 0.2679, 'learning_rate': 7.412466666120846e-07, 'epoch': 2.68}
{'loss': 0.2569, 'learning_rate': 7.188340134823043e-07, 'epoch': 2.68}
{'loss': 0.2704, 'learning_rate': 6.967553884276279e-07, 'epoch': 2.69}
{'loss': 0.3005, 'learning_rate': 6.750114174370797e-07, 'epoch': 2.69}
{'loss': 0.2542, 'learning_rate': 6.536027170113354e-07, 'epoch': 2.7}
{'loss': 0.2783, 'learning_rate': 6.32529894145234e-07, 'epoch': 2.7}
{'loss': 0.2264, 'learning_rate': 6.117935463105809e-07, 'epoch': 2.71}
{'loss': 0.2635, 'learning_rate': 5.913942614391931e-07, 'epoch': 2.71}
{'loss': 0.2658, 'learning_rate': 5.713326179062409e-07, 'epoch': 2.72}
{'loss': 0.2681, 'learning_rate': 5.516091845138377e-07, 'epoch': 2.72}
{'loss': 0.3189, 'learning_rate': 5.322245204749319e-07, 'epoch': 2.73}
{'loss': 0.2983, 'learning_rate': 5.131791753974296e-07, 'epoch': 2.73}
{'loss': 0.2887, 'learning_rate': 4.944736892686278e-07, 'epoch': 2.74}
{'loss': 0.2594, 'learning_rate': 4.761085924398959e-07, 'epoch': 2.74}
{'loss': 0.2366, 'learning_rate': 4.580844056116418e-07, 'epoch': 2.75}
{'loss': 0.3332, 'learning_rate': 4.4040163981855095e-07, 'epoch': 2.75}
{'loss': 0.2593, 'learning_rate': 4.2306079641509094e-07, 'epoch': 2.76}
{'loss': 0.2825, 'learning_rate': 4.0606236706130367e-07, 'epoch': 2.76}
{'loss': 0.2865, 'learning_rate': 3.89406833708858e-07, 'epoch': 2.77}
{'loss': 0.3002, 'learning_rate': 3.73094668587394e-07, 'epoch': 2.77}
{'loss': 0.2941, 'learning_rate': 3.5712633419112686e-07, 'epoch': 2.77}
{'loss': 0.3138, 'learning_rate': 3.4150228326573356e-07, 'epoch': 2.78}
{'loss': 0.3054, 'learning_rate': 3.262229587955232e-07, 'epoch': 2.78}
{'loss': 0.2502, 'learning_rate': 3.112887939908715e-07, 'epoch': 2.79}
{'loss': 0.2679, 'learning_rate': 2.9670021227593965e-07, 'epoch': 2.79}
{'loss': 0.2683, 'learning_rate': 2.824576272766666e-07, 'epoch': 2.8}
{'loss': 0.3101, 'learning_rate': 2.6856144280904967e-07, 'epoch': 2.8}
{'loss': 0.2749, 'learning_rate': 2.550120528676841e-07, 'epoch': 2.81}
{'loss': 0.2455, 'learning_rate': 2.4180984161460004e-07, 'epoch': 2.81}
{'loss': 0.2607, 'learning_rate': 2.2895518336836923e-07, 'epoch': 2.82}
{'loss': 0.2729, 'learning_rate': 2.164484425934879e-07, 'epoch': 2.82}
{'loss': 0.2669, 'learning_rate': 2.0428997389004972e-07, 'epoch': 2.83}
{'loss': 0.3026, 'learning_rate': 1.9248012198368748e-07, 'epoch': 2.83}
{'loss': 0.2394, 'learning_rate': 1.8101922171580022e-07, 'epoch': 2.84}
{'loss': 0.2578, 'learning_rate': 1.699075980340581e-07, 'epoch': 2.84}
{'loss': 0.252, 'learning_rate': 1.5914556598319307e-07, 'epoch': 2.85}
{'loss': 0.2478, 'learning_rate': 1.4873343069606305e-07, 'epoch': 2.85}
{'loss': 0.3517, 'learning_rate': 1.3867148738500313e-07, 'epoch': 2.86}
{'loss': 0.285, 'learning_rate': 1.289600213334505e-07, 'epoch': 2.86}
{'loss': 0.2673, 'learning_rate': 1.195993078878646e-07, 'epoch': 2.87}
{'loss': 0.2644, 'learning_rate': 1.1058961244991144e-07, 'epoch': 2.87}
{'loss': 0.2689, 'learning_rate': 1.0193119046894295e-07, 'epoch': 2.88}
{'loss': 0.2699, 'learning_rate': 9.362428743475565e-08, 'epoch': 2.88}
{'loss': 0.2677, 'learning_rate': 8.566913887062683e-08, 'epoch': 2.89}
{'loss': 0.2846, 'learning_rate': 7.806597032663925e-08, 'epoch': 2.89}
{'loss': 0.2686, 'learning_rate': 7.081499737328634e-08, 'epoch': 2.9}
{'loss': 0.2542, 'learning_rate': 6.391642559535616e-08, 'epoch': 2.9}
{'loss': 0.2678, 'learning_rate': 5.7370450586109737e-08, 'epoch': 2.91}
{'loss': 0.237, 'learning_rate': 5.117725794172995e-08, 'epoch': 2.91}
{'loss': 0.2847, 'learning_rate': 4.533702325605904e-08, 'epoch': 2.92}
{'loss': 0.2397, 'learning_rate': 3.9849912115623446e-08, 'epoch': 2.92}
{'loss': 0.2672, 'learning_rate': 3.471608009493754e-08, 'epoch': 2.93}
{'loss': 0.2767, 'learning_rate': 2.9935672752090525e-08, 'epoch': 2.93}
{'loss': 0.2758, 'learning_rate': 2.5508825624626088e-08, 'epoch': 2.94}
{'loss': 0.2576, 'learning_rate': 2.1435664225688547e-08, 'epoch': 2.94}
{'loss': 0.3037, 'learning_rate': 1.7716304040475697e-08, 'epoch': 2.95}
{'loss': 0.271, 'learning_rate': 1.4350850522956705e-08, 'epoch': 2.95}
{'loss': 0.2347, 'learning_rate': 1.1339399092885616e-08, 'epoch': 2.96}
{'loss': 0.2924, 'learning_rate': 8.682035133096566e-09, 'epoch': 2.96}
{'loss': 0.2847, 'learning_rate': 6.378833987077948e-09, 'epoch': 2.97}
{'loss': 0.2594, 'learning_rate': 4.429860956842169e-09, 'epoch': 2.97}
{'loss': 0.2613, 'learning_rate': 2.8351713010743573e-09, 'epoch': 2.98}
{'loss': 0.2603, 'learning_rate': 1.5948102335572313e-09, 'epoch': 2.98}
{'loss': 0.2915, 'learning_rate': 7.088129218985052e-10, 'epoch': 2.99}
{'loss': 0.2903, 'learning_rate': 1.772044865316902e-10, 'epoch': 2.99}
{'loss': 0.2426, 'learning_rate': 0.0, 'epoch': 3.0}
{'train_runtime': 5691.3236, 'train_samples_per_second': 27.41, 'train_steps_per_second': 0.107, 'train_loss': 0.6601732866517429, 'epoch': 3.0}
[2024-01-10 05:17:55,050] [INFO] [launch.py:347:main] Process 63171 exits successfully.
[2024-01-10 05:17:55,050] [INFO] [launch.py:347:main] Process 63167 exits successfully.
[2024-01-10 05:17:55,051] [INFO] [launch.py:347:main] Process 63166 exits successfully.
[2024-01-10 05:17:56,052] [INFO] [launch.py:347:main] Process 63169 exits successfully.
[2024-01-10 05:17:56,052] [INFO] [launch.py:347:main] Process 63168 exits successfully.
[2024-01-10 05:17:56,052] [INFO] [launch.py:347:main] Process 63165 exits successfully.
[2024-01-10 05:17:56,053] [INFO] [launch.py:347:main] Process 63170 exits successfully.
Finish training...
[2024-01-10 05:18:11,068] [INFO] [launch.py:347:main] Process 63164 exits successfully.