[2024-01-10 02:41:02,998] [INFO] [real_accelerator.py:158:get_accelerator] Setting ds_accelerator to cuda (auto detect) [2024-01-10 02:41:04,718] [WARNING] [runner.py:203:fetch_hostfile] Unable to find hostfile, will proceed with training with local resources only. [2024-01-10 02:41:04,718] [INFO] [runner.py:570:main] cmd = /root/anaconda3/envs/dj_comp/bin/python -u -m deepspeed.launcher.launch --world_info=eyJsb2NhbGhvc3QiOiBbMCwgMSwgMiwgMywgNCwgNSwgNiwgN119 --master_addr=127.0.0.1 --master_port=50000 --enable_each_rank_log=None train.py --model_name_or_path /home/baaidial/juyiming/research/models/llama-7b-v2/ --tokenizer /home/baaidial/juyiming/research/models/llama-7b-v2/ --data_path /home/baaidial/fengduanyu/lima_code/code/data/alpaca/alpaca_predata.jsonl --output_dir /home/baaidial/fengduanyu/lima_code/code/model_save/after_sft/alpaca_Llama-2-7b_all/ --per_device_train_batch_size 8 --gradient_accumulation_steps 4 --lang en --bf16 True --gradient_checkpointing_enable True --num_train_epochs 3 --model_max_length 1024 --learning_rate 2.5e-5 --weight_decay 0 --warmup_ratio 0.03 --evaluation_strategy no --save_strategy no --save_steps -1 --save_total_limit 999 --lr_scheduler_type cosine --logging_steps 1 --tf32 True --deepspeed /home/baaidial/fengduanyu/FT-Data-Ranker/competition_kit/lm-training/train_scripts/deepspeed_configs/ds_config_stage3.json [2024-01-10 02:41:06,392] [INFO] [real_accelerator.py:158:get_accelerator] Setting ds_accelerator to cuda (auto detect) [2024-01-10 02:41:07,849] [INFO] [launch.py:138:main] 0 NCCL_VERSION=2.18.1 [2024-01-10 02:41:07,849] [INFO] [launch.py:145:main] WORLD INFO DICT: {'localhost': [0, 1, 2, 3, 4, 5, 6, 7]} [2024-01-10 02:41:07,849] [INFO] [launch.py:151:main] nnodes=1, num_local_procs=8, node_rank=0 [2024-01-10 02:41:07,849] [INFO] [launch.py:162:main] global_rank_mapping=defaultdict(, {'localhost': [0, 1, 2, 3, 4, 5, 6, 7]}) [2024-01-10 02:41:07,849] [INFO] [launch.py:163:main] dist_world_size=8 [2024-01-10 02:41:07,849] [INFO] [launch.py:165:main] Setting CUDA_VISIBLE_DEVICES=0,1,2,3,4,5,6,7 [2024-01-10 02:41:11,242] [INFO] [real_accelerator.py:158:get_accelerator] Setting ds_accelerator to cuda (auto detect) [2024-01-10 02:41:11,306] [INFO] [real_accelerator.py:158:get_accelerator] Setting ds_accelerator to cuda (auto detect) [2024-01-10 02:41:11,449] [INFO] [real_accelerator.py:158:get_accelerator] Setting ds_accelerator to cuda (auto detect) [2024-01-10 02:41:11,449] [INFO] [real_accelerator.py:158:get_accelerator] Setting ds_accelerator to cuda (auto detect) [2024-01-10 02:41:11,469] [INFO] [real_accelerator.py:158:get_accelerator] Setting ds_accelerator to cuda (auto detect) [2024-01-10 02:41:11,481] [INFO] [real_accelerator.py:158:get_accelerator] Setting ds_accelerator to cuda (auto detect) [2024-01-10 02:41:11,509] [INFO] [real_accelerator.py:158:get_accelerator] Setting ds_accelerator to cuda (auto detect) [2024-01-10 02:41:11,510] [INFO] [real_accelerator.py:158:get_accelerator] Setting ds_accelerator to cuda (auto detect) [2024-01-10 02:41:12,962] [INFO] [comm.py:637:init_distributed] cdb=None [2024-01-10 02:41:13,085] [INFO] [comm.py:637:init_distributed] cdb=None [2024-01-10 02:41:13,085] [INFO] [comm.py:668:init_distributed] Initializing TorchBackend in DeepSpeed with backend nccl [2024-01-10 02:41:13,199] [INFO] [comm.py:637:init_distributed] cdb=None [2024-01-10 02:41:13,236] [INFO] [comm.py:637:init_distributed] cdb=None [2024-01-10 02:41:13,241] [INFO] [comm.py:637:init_distributed] cdb=None [2024-01-10 02:41:13,242] [INFO] [comm.py:637:init_distributed] cdb=None [2024-01-10 02:41:13,285] [INFO] [comm.py:637:init_distributed] cdb=None [2024-01-10 02:41:13,313] [INFO] [comm.py:637:init_distributed] cdb=None Loading model from /home/baaidial/juyiming/research/models/llama-7b-v2/ [2024-01-10 02:41:16,704] [INFO] [partition_parameters.py:347:__exit__] finished initializing model - num_params = 291, num_elems = 6.74B gradient_checkpointing_enable Model class: Tokenizer class: +---------+---------+------------+---------+------------+---------+------------+---------+------------+----------------+ | |pad_token|pad_token_id|bos_token|bos_token_id|eos_token|eos_token_id|unk_token|unk_token_id|model_max_length| +---------+---------+------------+---------+------------+---------+------------+---------+------------+----------------+ |tokenizer| [PAD] | 32000 | | 1 | | 2 | | 0 | 1024 | +---------+---------+------------+---------+------------+---------+------------+---------+------------+----------------+ | model | - | 32000 | - | 1 | - | 2 | - | 0 | - | +---------+---------+------------+---------+------------+---------+------------+---------+------------+----------------+ There are 52002 training samples in data path Formatting inputs... Tokenizing inputs... This may take some time... PROMPT: 'Below is an instruction that describes a task. Write a response that appropriately completes the request.\n\n### Instruction:\nGenerate a poem about nature that has four stanzas.\n\n### Response:' RESPONSE: "Verse 1: \nIn nature everything is alive, \nFrom the blooming flower's to the humming hive. \nOn its own accord the wind freely blows, \nBringing life and joy where ever it goes. \n\nVerse 2:\nFrom the highest peak to the lowest valley, \nNature can be found in every alley. \nFrom the deepest depth of the oceans floor,\nTo the tallest clouds that soar. \n\nVerse 3: \nNature provides us with hope and dreams,\nFills our lives with joy, laughter and streams. \nGives us shelter from the hot summer sun,\nAnd comfort in knowing it will all be done.\n\nVerse 4: \nNature is generous and never asks for a cost, \nIt's simply here for us to enjoy and not to be lost. \nSo let us appreciate and respect it's beauty and grace,\nAnd take care of it with love to keep it alive in this place." ==================================================================================================== PROMPT: 'Below is an instruction that describes a task. Write a response that appropriately completes the request.\n\n### Instruction:\nDesign a database to store information about customers\n\n### Response:' RESPONSE: "A database to store information about customers can be designed using the following schema. The customers table would store the customer's name, contact details, and a unique ID. The orders table would store the ID of the customer, the product purchased, and the order date. The transactions table would store the details of each transaction made by the customer. This could include the credit card details, amount paid, and date of the transaction. Lastly, the feedback table would store the customer's feedback about their experience with the product or service. This could include the customer's ratings, comments, and suggestions. \n\nThis database design would allow customers' information to be stored together in a centralized repository, with tables that are related and easily searchable. This would enable customer management tasks such as tracking orders, processing payments, and responding to customer feedback to be carried out efficiently." ==================================================================================================== PROMPT: 'Below is an instruction that describes a task. Write a response that appropriately completes the request.\n\n### Instruction:\nExplain how artificial intelligence is impacting the aviation industry.\n\n### Response:' RESPONSE: 'Artificial intelligence is transforming the aviation industry by improving aircraft maintenance, reducing fuel costs and emissions, and enhancing the passenger experience. AI can be used to monitor aircraft behavior and maintenance, which helps airlines identify repairs and possible malfunctions before they become an issue. AI can also be used to optimize flight paths to reduce emissions and fuel costs. Finally, AI can be used to enhance the passenger experience, such as by suggesting restaurants and making personalized recommendations.' ==================================================================================================== PROMPT: 'Below is an instruction that describes a task. Write a response that appropriately completes the request.\n\n### Instruction:\nSuggest five books that are suitable for teenagers.\n\n### Response:' RESPONSE: '1. The Fault in Our Stars by John Green\n2. The Hunger Games by Suzanne Collins\n3. 1984 by George Orwell\n4. The Giver by Lois Lowry\n5. Great Expectations by Charles Dickens.' ==================================================================================================== PROMPT: 'Below is an instruction that describes a task, paired with an input that provides further context. Write a response that appropriately completes the request.\n\n### Instruction:\nSimplify the sentence so it conveys the same message while being shorter and easier to understand.\n\n### Input:\nIt was necessary to check the water tank at least once a week due to the consistent growth of bacteria.\n\n### Response:' RESPONSE: 'Check water tank weekly to prevent bacteria growth.' ==================================================================================================== ORI NUMBER: 52002, AFTER FILETER: 52000, DROP NUMBER: 2 Total 52000 samples [ 6.56M tokens] in training! Parameter Offload: Total persistent parameters: 266240 in 65 params {'loss': 1.3628, 'learning_rate': 1.3157894736842106e-06, 'epoch': 0.0} {'loss': 1.3486, 'learning_rate': 2.631578947368421e-06, 'epoch': 0.01} {'loss': 1.4299, 'learning_rate': 3.9473684210526315e-06, 'epoch': 0.01} {'loss': 1.3228, 'learning_rate': 5.263157894736842e-06, 'epoch': 0.02} {'loss': 1.3087, 'learning_rate': 6.578947368421053e-06, 'epoch': 0.02} {'loss': 1.1821, 'learning_rate': 7.894736842105263e-06, 'epoch': 0.03} {'loss': 1.1411, 'learning_rate': 9.210526315789474e-06, 'epoch': 0.03} {'loss': 1.1109, 'learning_rate': 1.0526315789473684e-05, 'epoch': 0.04} {'loss': 1.113, 'learning_rate': 1.1842105263157895e-05, 'epoch': 0.04} {'loss': 1.0936, 'learning_rate': 1.3157894736842106e-05, 'epoch': 0.05} {'loss': 1.1008, 'learning_rate': 1.4473684210526317e-05, 'epoch': 0.05} {'loss': 1.1469, 'learning_rate': 1.5789473684210526e-05, 'epoch': 0.06} {'loss': 1.0595, 'learning_rate': 1.7105263157894737e-05, 'epoch': 0.06} {'loss': 1.1116, 'learning_rate': 1.8421052631578947e-05, 'epoch': 0.07} {'loss': 1.0477, 'learning_rate': 1.9736842105263158e-05, 'epoch': 0.07} {'loss': 1.1064, 'learning_rate': 2.105263157894737e-05, 'epoch': 0.08} {'loss': 1.1002, 'learning_rate': 2.236842105263158e-05, 'epoch': 0.08} {'loss': 1.0617, 'learning_rate': 2.368421052631579e-05, 'epoch': 0.09} {'loss': 1.1376, 'learning_rate': 2.5e-05, 'epoch': 0.09} {'loss': 1.1213, 'learning_rate': 2.499982279551347e-05, 'epoch': 0.1} {'loss': 1.0721, 'learning_rate': 2.4999291187078104e-05, 'epoch': 0.1} {'loss': 1.0531, 'learning_rate': 2.4998405189766444e-05, 'epoch': 0.11} {'loss': 1.0651, 'learning_rate': 2.4997164828698926e-05, 'epoch': 0.11} {'loss': 1.0514, 'learning_rate': 2.4995570139043158e-05, 'epoch': 0.12} {'loss': 1.1169, 'learning_rate': 2.4993621166012923e-05, 'epoch': 0.12} {'loss': 1.1252, 'learning_rate': 2.4991317964866905e-05, 'epoch': 0.13} {'loss': 1.0334, 'learning_rate': 2.4988660600907115e-05, 'epoch': 0.13} {'loss': 1.0856, 'learning_rate': 2.4985649149477043e-05, 'epoch': 0.14} {'loss': 1.1015, 'learning_rate': 2.4982283695959525e-05, 'epoch': 0.14} {'loss': 1.1473, 'learning_rate': 2.4978564335774313e-05, 'epoch': 0.15} {'loss': 1.0341, 'learning_rate': 2.4974491174375374e-05, 'epoch': 0.15} {'loss': 1.0717, 'learning_rate': 2.497006432724791e-05, 'epoch': 0.16} {'loss': 1.0933, 'learning_rate': 2.4965283919905065e-05, 'epoch': 0.16} {'loss': 1.1277, 'learning_rate': 2.4960150087884376e-05, 'epoch': 0.17} {'loss': 1.056, 'learning_rate': 2.4954662976743944e-05, 'epoch': 0.17} {'loss': 1.0766, 'learning_rate': 2.494882274205827e-05, 'epoch': 0.18} {'loss': 1.1247, 'learning_rate': 2.4942629549413894e-05, 'epoch': 0.18} {'loss': 1.1544, 'learning_rate': 2.4936083574404646e-05, 'epoch': 0.19} {'loss': 1.139, 'learning_rate': 2.4929185002626714e-05, 'epoch': 0.19} {'loss': 1.0876, 'learning_rate': 2.492193402967336e-05, 'epoch': 0.2} {'loss': 1.1107, 'learning_rate': 2.4914330861129376e-05, 'epoch': 0.2} {'loss': 1.1297, 'learning_rate': 2.4906375712565245e-05, 'epoch': 0.21} {'loss': 1.1153, 'learning_rate': 2.489806880953106e-05, 'epoch': 0.21} {'loss': 1.1108, 'learning_rate': 2.4889410387550093e-05, 'epoch': 0.22} {'loss': 1.1006, 'learning_rate': 2.488040069211214e-05, 'epoch': 0.22} {'loss': 1.1015, 'learning_rate': 2.487103997866655e-05, 'epoch': 0.23} {'loss': 1.1215, 'learning_rate': 2.4861328512615e-05, 'epoch': 0.23} {'loss': 1.1191, 'learning_rate': 2.485126656930394e-05, 'epoch': 0.24} {'loss': 1.1129, 'learning_rate': 2.4840854434016808e-05, 'epoch': 0.24} {'loss': 1.1249, 'learning_rate': 2.4830092401965942e-05, 'epoch': 0.25} {'loss': 1.119, 'learning_rate': 2.4818980778284202e-05, 'epoch': 0.25} {'loss': 1.0443, 'learning_rate': 2.4807519878016313e-05, 'epoch': 0.26} {'loss': 1.078, 'learning_rate': 2.4795710026109954e-05, 'epoch': 0.26} {'loss': 1.0966, 'learning_rate': 2.4783551557406515e-05, 'epoch': 0.27} {'loss': 1.1596, 'learning_rate': 2.4771044816631635e-05, 'epoch': 0.27} {'loss': 1.087, 'learning_rate': 2.4758190158385402e-05, 'epoch': 0.28} {'loss': 1.121, 'learning_rate': 2.4744987947132318e-05, 'epoch': 0.28} {'loss': 1.1222, 'learning_rate': 2.473143855719095e-05, 'epoch': 0.29} {'loss': 1.1292, 'learning_rate': 2.4717542372723333e-05, 'epoch': 0.29} {'loss': 1.1314, 'learning_rate': 2.4703299787724065e-05, 'epoch': 0.3} {'loss': 1.1266, 'learning_rate': 2.468871120600913e-05, 'epoch': 0.3} {'loss': 1.1486, 'learning_rate': 2.4673777041204478e-05, 'epoch': 0.31} {'loss': 1.0973, 'learning_rate': 2.465849771673427e-05, 'epoch': 0.31} {'loss': 1.145, 'learning_rate': 2.4642873665808873e-05, 'epoch': 0.31} {'loss': 1.1104, 'learning_rate': 2.4626905331412608e-05, 'epoch': 0.32} {'loss': 1.0877, 'learning_rate': 2.4610593166291143e-05, 'epoch': 0.32} {'loss': 1.0961, 'learning_rate': 2.4593937632938695e-05, 'epoch': 0.33} {'loss': 1.0783, 'learning_rate': 2.457693920358491e-05, 'epoch': 0.33} {'loss': 1.0869, 'learning_rate': 2.455959836018145e-05, 'epoch': 0.34} {'loss': 1.1227, 'learning_rate': 2.454191559438836e-05, 'epoch': 0.34} {'loss': 1.0735, 'learning_rate': 2.4523891407560107e-05, 'epoch': 0.35} {'loss': 1.1002, 'learning_rate': 2.4505526310731376e-05, 'epoch': 0.35} {'loss': 1.1313, 'learning_rate': 2.448682082460257e-05, 'epoch': 0.36} {'loss': 1.1208, 'learning_rate': 2.446777547952507e-05, 'epoch': 0.36} {'loss': 1.1212, 'learning_rate': 2.4448390815486165e-05, 'epoch': 0.37} {'loss': 1.1048, 'learning_rate': 2.4428667382093762e-05, 'epoch': 0.37} {'loss': 1.1089, 'learning_rate': 2.440860573856081e-05, 'epoch': 0.38} {'loss': 1.1081, 'learning_rate': 2.438820645368942e-05, 'epoch': 0.38} {'loss': 1.0935, 'learning_rate': 2.4367470105854766e-05, 'epoch': 0.39} {'loss': 1.0858, 'learning_rate': 2.4346397282988666e-05, 'epoch': 0.39} {'loss': 1.0944, 'learning_rate': 2.432498858256292e-05, 'epoch': 0.4} {'loss': 1.0921, 'learning_rate': 2.4303244611572372e-05, 'epoch': 0.4} {'loss': 1.1146, 'learning_rate': 2.4281165986517696e-05, 'epoch': 0.41} {'loss': 1.0932, 'learning_rate': 2.425875333338792e-05, 'epoch': 0.41} {'loss': 1.0857, 'learning_rate': 2.4236007287642664e-05, 'epoch': 0.42} {'loss': 1.0615, 'learning_rate': 2.421292849419416e-05, 'epoch': 0.42} {'loss': 1.0923, 'learning_rate': 2.4189517607388918e-05, 'epoch': 0.43} {'loss': 1.1382, 'learning_rate': 2.416577529098921e-05, 'epoch': 0.43} {'loss': 1.1798, 'learning_rate': 2.4141702218154232e-05, 'epoch': 0.44} {'loss': 1.1104, 'learning_rate': 2.4117299071421023e-05, 'epoch': 0.44} {'loss': 1.1011, 'learning_rate': 2.4092566542685123e-05, 'epoch': 0.45} {'loss': 1.0925, 'learning_rate': 2.4067505333180938e-05, 'epoch': 0.45} {'loss': 1.1313, 'learning_rate': 2.404211615346187e-05, 'epoch': 0.46} {'loss': 1.0656, 'learning_rate': 2.401639972338016e-05, 'epoch': 0.46} {'loss': 1.1277, 'learning_rate': 2.399035677206651e-05, 'epoch': 0.47} {'loss': 1.1044, 'learning_rate': 2.396398803790936e-05, 'epoch': 0.47} {'loss': 1.0957, 'learning_rate': 2.393729426853399e-05, 'epoch': 0.48} {'loss': 1.0594, 'learning_rate': 2.3910276220781314e-05, 'epoch': 0.48} {'loss': 1.1278, 'learning_rate': 2.3882934660686418e-05, 'epoch': 0.49} {'loss': 1.0847, 'learning_rate': 2.3855270363456833e-05, 'epoch': 0.49} {'loss': 1.1694, 'learning_rate': 2.382728411345058e-05, 'epoch': 0.5} {'loss': 1.1414, 'learning_rate': 2.3798976704153906e-05, 'epoch': 0.5} {'loss': 1.0949, 'learning_rate': 2.3770348938158805e-05, 'epoch': 0.51} {'loss': 1.1151, 'learning_rate': 2.3741401627140242e-05, 'epoch': 0.51} {'loss': 1.0679, 'learning_rate': 2.371213559183316e-05, 'epoch': 0.52} {'loss': 1.1304, 'learning_rate': 2.3682551662009203e-05, 'epoch': 0.52} {'loss': 1.1176, 'learning_rate': 2.3652650676453188e-05, 'epoch': 0.53} {'loss': 1.0294, 'learning_rate': 2.362243348293931e-05, 'epoch': 0.53} {'loss': 1.1691, 'learning_rate': 2.3591900938207147e-05, 'epoch': 0.54} {'loss': 1.0793, 'learning_rate': 2.3561053907937315e-05, 'epoch': 0.54} {'loss': 1.0782, 'learning_rate': 2.352989326672696e-05, 'epoch': 0.55} {'loss': 1.0712, 'learning_rate': 2.349841989806495e-05, 'epoch': 0.55} {'loss': 1.1301, 'learning_rate': 2.346663469430683e-05, 'epoch': 0.56} {'loss': 1.1725, 'learning_rate': 2.3434538556649515e-05, 'epoch': 0.56} {'loss': 1.1134, 'learning_rate': 2.3402132395105737e-05, 'epoch': 0.57} {'loss': 1.1455, 'learning_rate': 2.336941712847825e-05, 'epoch': 0.57} {'loss': 1.1175, 'learning_rate': 2.3336393684333778e-05, 'epoch': 0.58} {'loss': 1.1114, 'learning_rate': 2.3303062998976712e-05, 'epoch': 0.58} {'loss': 1.1064, 'learning_rate': 2.3269426017422576e-05, 'epoch': 0.59} {'loss': 1.0249, 'learning_rate': 2.3235483693371214e-05, 'epoch': 0.59} {'loss': 1.0759, 'learning_rate': 2.3201236989179766e-05, 'epoch': 0.6} {'loss': 1.0937, 'learning_rate': 2.3166686875835365e-05, 'epoch': 0.6} {'loss': 1.1164, 'learning_rate': 2.313183433292764e-05, 'epoch': 0.61} {'loss': 1.1097, 'learning_rate': 2.3096680348620886e-05, 'epoch': 0.61} {'loss': 1.1287, 'learning_rate': 2.3061225919626124e-05, 'epoch': 0.62} {'loss': 1.1137, 'learning_rate': 2.3025472051172755e-05, 'epoch': 0.62} {'loss': 1.1086, 'learning_rate': 2.2989419756980134e-05, 'epoch': 0.62} {'loss': 1.0967, 'learning_rate': 2.295307005922879e-05, 'epoch': 0.63} {'loss': 1.1077, 'learning_rate': 2.2916423988531437e-05, 'epoch': 0.63} {'loss': 1.1341, 'learning_rate': 2.2879482583903783e-05, 'epoch': 0.64} {'loss': 1.106, 'learning_rate': 2.2842246892735053e-05, 'epoch': 0.64} {'loss': 1.1166, 'learning_rate': 2.2804717970758286e-05, 'epoch': 0.65} {'loss': 1.1153, 'learning_rate': 2.2766896882020422e-05, 'epoch': 0.65} {'loss': 1.0829, 'learning_rate': 2.2728784698852114e-05, 'epoch': 0.66} {'loss': 1.1223, 'learning_rate': 2.2690382501837344e-05, 'epoch': 0.66} {'loss': 1.0744, 'learning_rate': 2.2651691379782764e-05, 'epoch': 0.67} {'loss': 1.0786, 'learning_rate': 2.2612712429686845e-05, 'epoch': 0.67} {'loss': 1.1594, 'learning_rate': 2.2573446756708754e-05, 'epoch': 0.68} {'loss': 1.0879, 'learning_rate': 2.2533895474137047e-05, 'epoch': 0.68} {'loss': 1.1591, 'learning_rate': 2.2494059703358073e-05, 'epoch': 0.69} {'loss': 1.1975, 'learning_rate': 2.24539405738242e-05, 'epoch': 0.69} {'loss': 1.0487, 'learning_rate': 2.2413539223021794e-05, 'epoch': 0.7} {'loss': 1.1051, 'learning_rate': 2.237285679643895e-05, 'epoch': 0.7} {'loss': 1.1266, 'learning_rate': 2.2331894447533035e-05, 'epoch': 0.71} {'loss': 1.0445, 'learning_rate': 2.229065333769796e-05, 'epoch': 0.71} {'loss': 1.0915, 'learning_rate': 2.2249134636231285e-05, 'epoch': 0.72} {'loss': 1.1328, 'learning_rate': 2.2207339520301033e-05, 'epoch': 0.72} {'loss': 1.1179, 'learning_rate': 2.216526917491234e-05, 'epoch': 0.73} {'loss': 1.0199, 'learning_rate': 2.2122924792873827e-05, 'epoch': 0.73} {'loss': 1.1108, 'learning_rate': 2.2080307574763824e-05, 'epoch': 0.74} {'loss': 1.1442, 'learning_rate': 2.2037418728896282e-05, 'epoch': 0.74} {'loss': 1.1125, 'learning_rate': 2.1994259471286545e-05, 'epoch': 0.75} {'loss': 1.1047, 'learning_rate': 2.1950831025616873e-05, 'epoch': 0.75} {'loss': 1.1058, 'learning_rate': 2.190713462320173e-05, 'epoch': 0.76} {'loss': 1.027, 'learning_rate': 2.186317150295289e-05, 'epoch': 0.76} {'loss': 1.1007, 'learning_rate': 2.1818942911344283e-05, 'epoch': 0.77} {'loss': 1.0835, 'learning_rate': 2.17744501023767e-05, 'epoch': 0.77} {'loss': 1.1269, 'learning_rate': 2.1729694337542196e-05, 'epoch': 0.78} {'loss': 1.1548, 'learning_rate': 2.168467688578834e-05, 'epoch': 0.78} {'loss': 1.0896, 'learning_rate': 2.163939902348225e-05, 'epoch': 0.79} {'loss': 1.1175, 'learning_rate': 2.1593862034374365e-05, 'epoch': 0.79} {'loss': 1.0988, 'learning_rate': 2.15480672095621e-05, 'epoch': 0.8} {'loss': 1.1124, 'learning_rate': 2.15020158474532e-05, 'epoch': 0.8} {'loss': 1.0722, 'learning_rate': 2.145570925372894e-05, 'epoch': 0.81} {'loss': 1.0978, 'learning_rate': 2.1409148741307107e-05, 'epoch': 0.81} {'loss': 1.0294, 'learning_rate': 2.1362335630304768e-05, 'epoch': 0.82} {'loss': 1.0917, 'learning_rate': 2.131527124800086e-05, 'epoch': 0.82} {'loss': 1.0315, 'learning_rate': 2.1267956928798525e-05, 'epoch': 0.83} {'loss': 1.0825, 'learning_rate': 2.1220394014187312e-05, 'epoch': 0.83} {'loss': 1.1014, 'learning_rate': 2.1172583852705115e-05, 'epoch': 0.84} {'loss': 1.0582, 'learning_rate': 2.1124527799899953e-05, 'epoch': 0.84} {'loss': 1.0917, 'learning_rate': 2.1076227218291535e-05, 'epoch': 0.85} {'loss': 1.0564, 'learning_rate': 2.1027683477332623e-05, 'epoch': 0.85} {'loss': 1.0999, 'learning_rate': 2.0978897953370204e-05, 'epoch': 0.86} {'loss': 1.1015, 'learning_rate': 2.0929872029606472e-05, 'epoch': 0.86} {'loss': 1.0807, 'learning_rate': 2.088060709605962e-05, 'epoch': 0.87} {'loss': 1.0348, 'learning_rate': 2.08311045495244e-05, 'epoch': 0.87} {'loss': 1.0644, 'learning_rate': 2.078136579353255e-05, 'epoch': 0.88} {'loss': 1.0882, 'learning_rate': 2.0731392238312985e-05, 'epoch': 0.88} {'loss': 1.109, 'learning_rate': 2.0681185300751814e-05, 'epoch': 0.89} {'loss': 1.1298, 'learning_rate': 2.0630746404352168e-05, 'epoch': 0.89} {'loss': 1.0834, 'learning_rate': 2.058007697919386e-05, 'epoch': 0.9} {'loss': 1.0839, 'learning_rate': 2.0529178461892785e-05, 'epoch': 0.9} {'loss': 1.1053, 'learning_rate': 2.0478052295560253e-05, 'epoch': 0.91} {'loss': 1.0968, 'learning_rate': 2.0426699929762035e-05, 'epoch': 0.91} {'loss': 1.0399, 'learning_rate': 2.037512282047726e-05, 'epoch': 0.92} {'loss': 1.1565, 'learning_rate': 2.0323322430057166e-05, 'epoch': 0.92} {'loss': 1.103, 'learning_rate': 2.0271300227183598e-05, 'epoch': 0.92} {'loss': 1.0688, 'learning_rate': 2.02190576868274e-05, 'epoch': 0.93} {'loss': 1.0667, 'learning_rate': 2.016659629020658e-05, 'epoch': 0.93} {'loss': 1.1085, 'learning_rate': 2.0113917524744308e-05, 'epoch': 0.94} {'loss': 1.085, 'learning_rate': 2.0061022884026762e-05, 'epoch': 0.94} {'loss': 1.0716, 'learning_rate': 2.0007913867760763e-05, 'epoch': 0.95} {'loss': 1.1282, 'learning_rate': 1.995459198173127e-05, 'epoch': 0.95} {'loss': 1.1367, 'learning_rate': 1.9901058737758667e-05, 'epoch': 0.96} {'loss': 1.08, 'learning_rate': 1.9847315653655915e-05, 'epoch': 0.96} {'loss': 1.1025, 'learning_rate': 1.9793364253185518e-05, 'epoch': 0.97} {'loss': 1.0882, 'learning_rate': 1.9739206066016307e-05, 'epoch': 0.97} {'loss': 1.1219, 'learning_rate': 1.9684842627680088e-05, 'epoch': 0.98} {'loss': 1.0971, 'learning_rate': 1.963027547952808e-05, 'epoch': 0.98} {'loss': 1.0991, 'learning_rate': 1.9575506168687247e-05, 'epoch': 0.99} {'loss': 1.0028, 'learning_rate': 1.9520536248016403e-05, 'epoch': 0.99} {'loss': 1.0726, 'learning_rate': 1.946536727606219e-05, 'epoch': 1.0} {'loss': 0.8409, 'learning_rate': 1.941000081701492e-05, 'epoch': 1.0} {'loss': 0.7279, 'learning_rate': 1.9354438440664167e-05, 'epoch': 1.01} {'loss': 0.6879, 'learning_rate': 1.929868172235433e-05, 'epoch': 1.01} {'loss': 0.6526, 'learning_rate': 1.9242732242939893e-05, 'epoch': 1.02} {'loss': 0.6945, 'learning_rate': 1.9186591588740667e-05, 'epoch': 1.02} {'loss': 0.6871, 'learning_rate': 1.913026135149678e-05, 'epoch': 1.03} {'loss': 0.6716, 'learning_rate': 1.9073743128323558e-05, 'epoch': 1.03} {'loss': 0.7348, 'learning_rate': 1.9017038521666232e-05, 'epoch': 1.04} {'loss': 0.6692, 'learning_rate': 1.8960149139254513e-05, 'epoch': 1.04} {'loss': 0.6319, 'learning_rate': 1.8903076594057013e-05, 'epoch': 1.05} {'loss': 0.6803, 'learning_rate': 1.88458225042355e-05, 'epoch': 1.05} {'loss': 0.6941, 'learning_rate': 1.8788388493099033e-05, 'epoch': 1.06} {'loss': 0.702, 'learning_rate': 1.8730776189057918e-05, 'epoch': 1.06} {'loss': 0.7123, 'learning_rate': 1.8672987225577552e-05, 'epoch': 1.07} {'loss': 0.6531, 'learning_rate': 1.8615023241132127e-05, 'epoch': 1.07} {'loss': 0.6997, 'learning_rate': 1.855688587915813e-05, 'epoch': 1.08} {'loss': 0.706, 'learning_rate': 1.8498576788007775e-05, 'epoch': 1.08} {'loss': 0.6839, 'learning_rate': 1.8440097620902284e-05, 'epoch': 1.09} {'loss': 0.6981, 'learning_rate': 1.838145003588497e-05, 'epoch': 1.09} {'loss': 0.6168, 'learning_rate': 1.832263569577427e-05, 'epoch': 1.1} {'loss': 0.6331, 'learning_rate': 1.8263656268116576e-05, 'epoch': 1.1} {'loss': 0.6864, 'learning_rate': 1.8204513425138952e-05, 'epoch': 1.11} {'loss': 0.7184, 'learning_rate': 1.8145208843701742e-05, 'epoch': 1.11} {'loss': 0.6272, 'learning_rate': 1.808574420525101e-05, 'epoch': 1.12} {'loss': 0.61, 'learning_rate': 1.8026121195770874e-05, 'epoch': 1.12} {'loss': 0.6583, 'learning_rate': 1.7966341505735695e-05, 'epoch': 1.13} {'loss': 0.6503, 'learning_rate': 1.7906406830062156e-05, 'epoch': 1.13} {'loss': 0.6473, 'learning_rate': 1.7846318868061212e-05, 'epoch': 1.14} {'loss': 0.658, 'learning_rate': 1.7786079323389893e-05, 'epoch': 1.14} {'loss': 0.6683, 'learning_rate': 1.7725689904003006e-05, 'epoch': 1.15} {'loss': 0.6576, 'learning_rate': 1.766515232210473e-05, 'epoch': 1.15} {'loss': 0.7206, 'learning_rate': 1.7604468294100034e-05, 'epoch': 1.16} {'loss': 0.6805, 'learning_rate': 1.754363954054605e-05, 'epoch': 1.16} {'loss': 0.6856, 'learning_rate': 1.7482667786103256e-05, 'epoch': 1.17} {'loss': 0.6346, 'learning_rate': 1.7421554759486614e-05, 'epoch': 1.17} {'loss': 0.6842, 'learning_rate': 1.736030219341651e-05, 'epoch': 1.18} {'loss': 0.6696, 'learning_rate': 1.7298911824569676e-05, 'epoch': 1.18} {'loss': 0.6832, 'learning_rate': 1.7237385393529916e-05, 'epoch': 1.19} {'loss': 0.6993, 'learning_rate': 1.7175724644738773e-05, 'epoch': 1.19} {'loss': 0.7059, 'learning_rate': 1.7113931326446054e-05, 'epoch': 1.2} {'loss': 0.7253, 'learning_rate': 1.705200719066028e-05, 'epoch': 1.2} {'loss': 0.6968, 'learning_rate': 1.6989953993098998e-05, 'epoch': 1.21} {'loss': 0.6491, 'learning_rate': 1.6927773493139014e-05, 'epoch': 1.21} {'loss': 0.6645, 'learning_rate': 1.68654674537665e-05, 'epoch': 1.22} {'loss': 0.6346, 'learning_rate': 1.6803037641527002e-05, 'epoch': 1.22} {'loss': 0.7157, 'learning_rate': 1.674048582647538e-05, 'epoch': 1.23} {'loss': 0.7311, 'learning_rate': 1.6677813782125592e-05, 'epoch': 1.23} {'loss': 0.6516, 'learning_rate': 1.6615023285400432e-05, 'epoch': 1.23} {'loss': 0.6617, 'learning_rate': 1.6552116116581133e-05, 'epoch': 1.24} {'loss': 0.6757, 'learning_rate': 1.648909405925691e-05, 'epoch': 1.24} {'loss': 0.6776, 'learning_rate': 1.6425958900274362e-05, 'epoch': 1.25} {'loss': 0.6525, 'learning_rate': 1.6362712429686846e-05, 'epoch': 1.25} {'loss': 0.6695, 'learning_rate': 1.629935644070369e-05, 'epoch': 1.26} {'loss': 0.6625, 'learning_rate': 1.6235892729639378e-05, 'epoch': 1.26} {'loss': 0.6569, 'learning_rate': 1.617232309586261e-05, 'epoch': 1.27} {'loss': 0.7276, 'learning_rate': 1.6108649341745262e-05, 'epoch': 1.27} {'loss': 0.7187, 'learning_rate': 1.604487327261133e-05, 'epoch': 1.28} {'loss': 0.693, 'learning_rate': 1.59809966966857e-05, 'epoch': 1.28} {'loss': 0.6845, 'learning_rate': 1.591702142504291e-05, 'epoch': 1.29} {'loss': 0.6641, 'learning_rate': 1.5852949271555787e-05, 'epoch': 1.29} {'loss': 0.6816, 'learning_rate': 1.5788782052844015e-05, 'epoch': 1.3} {'loss': 0.6962, 'learning_rate': 1.572452158822265e-05, 'epoch': 1.3} {'loss': 0.6748, 'learning_rate': 1.5660169699650503e-05, 'epoch': 1.31} {'loss': 0.7169, 'learning_rate': 1.5595728211678525e-05, 'epoch': 1.31} {'loss': 0.6708, 'learning_rate': 1.553119895139803e-05, 'epoch': 1.32} {'loss': 0.6961, 'learning_rate': 1.546658374838894e-05, 'epoch': 1.32} {'loss': 0.6814, 'learning_rate': 1.5401884434667864e-05, 'epoch': 1.33} {'loss': 0.6701, 'learning_rate': 1.5337102844636188e-05, 'epoch': 1.33} {'loss': 0.678, 'learning_rate': 1.5272240815028067e-05, 'epoch': 1.34} {'loss': 0.7226, 'learning_rate': 1.520730018485832e-05, 'epoch': 1.34} {'loss': 0.712, 'learning_rate': 1.5142282795370305e-05, 'epoch': 1.35} {'loss': 0.7174, 'learning_rate': 1.5077190489983728e-05, 'epoch': 1.35} {'loss': 0.697, 'learning_rate': 1.5012025114242355e-05, 'epoch': 1.36} {'loss': 0.6919, 'learning_rate': 1.49467885157617e-05, 'epoch': 1.36} {'loss': 0.7013, 'learning_rate': 1.4881482544176627e-05, 'epoch': 1.37} {'loss': 0.672, 'learning_rate': 1.4816109051088931e-05, 'epoch': 1.37} {'loss': 0.6643, 'learning_rate': 1.4750669890014807e-05, 'epoch': 1.38} {'loss': 0.7121, 'learning_rate': 1.468516691633233e-05, 'epoch': 1.38} {'loss': 0.7121, 'learning_rate': 1.4619601987228832e-05, 'epoch': 1.39} {'loss': 0.7016, 'learning_rate': 1.4553976961648244e-05, 'epoch': 1.39} {'loss': 0.6935, 'learning_rate': 1.4488293700238401e-05, 'epoch': 1.4} {'loss': 0.6644, 'learning_rate': 1.4422554065298286e-05, 'epoch': 1.4} {'loss': 0.6634, 'learning_rate': 1.435675992072521e-05, 'epoch': 1.41} {'loss': 0.6848, 'learning_rate': 1.4290913131961997e-05, 'epoch': 1.41} {'loss': 0.7342, 'learning_rate': 1.4225015565944072e-05, 'epoch': 1.42} {'loss': 0.7043, 'learning_rate': 1.4159069091046526e-05, 'epoch': 1.42} {'loss': 0.6844, 'learning_rate': 1.4093075577031157e-05, 'epoch': 1.43} {'loss': 0.7029, 'learning_rate': 1.4027036894993442e-05, 'epoch': 1.43} {'loss': 0.6458, 'learning_rate': 1.396095491730951e-05, 'epoch': 1.44} {'loss': 0.6892, 'learning_rate': 1.3894831517583017e-05, 'epoch': 1.44} {'loss': 0.6533, 'learning_rate': 1.3828668570592069e-05, 'epoch': 1.45} {'loss': 0.6618, 'learning_rate': 1.3762467952236027e-05, 'epoch': 1.45} {'loss': 0.7309, 'learning_rate': 1.369623153948235e-05, 'epoch': 1.46} {'loss': 0.6675, 'learning_rate': 1.3629961210313353e-05, 'epoch': 1.46} {'loss': 0.7065, 'learning_rate': 1.3563658843672985e-05, 'epoch': 1.47} {'loss': 0.6495, 'learning_rate': 1.3497326319413539e-05, 'epoch': 1.47} {'loss': 0.6695, 'learning_rate': 1.3430965518242362e-05, 'epoch': 1.48} {'loss': 0.7005, 'learning_rate': 1.336457832166852e-05, 'epoch': 1.48} {'loss': 0.7049, 'learning_rate': 1.3298166611949467e-05, 'epoch': 1.49} {'loss': 0.6695, 'learning_rate': 1.3231732272037672e-05, 'epoch': 1.49} {'loss': 0.6984, 'learning_rate': 1.3165277185527234e-05, 'epoch': 1.5} {'loss': 0.6445, 'learning_rate': 1.3098803236600462e-05, 'epoch': 1.5} {'loss': 0.674, 'learning_rate': 1.3032312309974482e-05, 'epoch': 1.51} {'loss': 0.6703, 'learning_rate': 1.2965806290847768e-05, 'epoch': 1.51} {'loss': 0.6647, 'learning_rate': 1.2899287064846725e-05, 'epoch': 1.52} {'loss': 0.6637, 'learning_rate': 1.2832756517972185e-05, 'epoch': 1.52} {'loss': 0.6944, 'learning_rate': 1.2766216536545984e-05, 'epoch': 1.53} {'loss': 0.6246, 'learning_rate': 1.2699669007157435e-05, 'epoch': 1.53} {'loss': 0.6816, 'learning_rate': 1.2633115816609867e-05, 'epoch': 1.54} {'loss': 0.628, 'learning_rate': 1.2566558851867106e-05, 'epoch': 1.54} {'loss': 0.6668, 'learning_rate': 1.25e-05, 'epoch': 1.54} {'loss': 0.6779, 'learning_rate': 1.2433441148132897e-05, 'epoch': 1.55} {'loss': 0.6325, 'learning_rate': 1.236688418339014e-05, 'epoch': 1.55} {'loss': 0.6418, 'learning_rate': 1.2300330992842566e-05, 'epoch': 1.56} {'loss': 0.6853, 'learning_rate': 1.223378346345402e-05, 'epoch': 1.56} {'loss': 0.6517, 'learning_rate': 1.2167243482027816e-05, 'epoch': 1.57} {'loss': 0.6686, 'learning_rate': 1.2100712935153283e-05, 'epoch': 1.57} {'loss': 0.6647, 'learning_rate': 1.2034193709152231e-05, 'epoch': 1.58} {'loss': 0.7095, 'learning_rate': 1.196768769002552e-05, 'epoch': 1.58} {'loss': 0.65, 'learning_rate': 1.190119676339954e-05, 'epoch': 1.59} {'loss': 0.6796, 'learning_rate': 1.1834722814472771e-05, 'epoch': 1.59} {'loss': 0.6685, 'learning_rate': 1.1768267727962328e-05, 'epoch': 1.6} {'loss': 0.6913, 'learning_rate': 1.1701833388050536e-05, 'epoch': 1.6} {'loss': 0.7266, 'learning_rate': 1.1635421678331485e-05, 'epoch': 1.61} {'loss': 0.6682, 'learning_rate': 1.1569034481757644e-05, 'epoch': 1.61} {'loss': 0.6643, 'learning_rate': 1.150267368058646e-05, 'epoch': 1.62} {'loss': 0.6984, 'learning_rate': 1.1436341156327016e-05, 'epoch': 1.62} {'loss': 0.6792, 'learning_rate': 1.137003878968665e-05, 'epoch': 1.63} {'loss': 0.7129, 'learning_rate': 1.1303768460517657e-05, 'epoch': 1.63} {'loss': 0.6972, 'learning_rate': 1.1237532047763974e-05, 'epoch': 1.64} {'loss': 0.6692, 'learning_rate': 1.1171331429407934e-05, 'epoch': 1.64} {'loss': 0.6335, 'learning_rate': 1.1105168482416984e-05, 'epoch': 1.65} {'loss': 0.6787, 'learning_rate': 1.1039045082690496e-05, 'epoch': 1.65} {'loss': 0.6626, 'learning_rate': 1.0972963105006556e-05, 'epoch': 1.66} {'loss': 0.67, 'learning_rate': 1.0906924422968846e-05, 'epoch': 1.66} {'loss': 0.6307, 'learning_rate': 1.0840930908953477e-05, 'epoch': 1.67} {'loss': 0.6691, 'learning_rate': 1.0774984434055932e-05, 'epoch': 1.67} {'loss': 0.6772, 'learning_rate': 1.0709086868038005e-05, 'epoch': 1.68} {'loss': 0.6846, 'learning_rate': 1.064324007927479e-05, 'epoch': 1.68} {'loss': 0.6568, 'learning_rate': 1.0577445934701717e-05, 'epoch': 1.69} {'loss': 0.7119, 'learning_rate': 1.0511706299761602e-05, 'epoch': 1.69} {'loss': 0.6945, 'learning_rate': 1.044602303835176e-05, 'epoch': 1.7} {'loss': 0.6954, 'learning_rate': 1.0380398012771169e-05, 'epoch': 1.7} {'loss': 0.6703, 'learning_rate': 1.0314833083667672e-05, 'epoch': 1.71} {'loss': 0.6854, 'learning_rate': 1.0249330109985198e-05, 'epoch': 1.71} {'loss': 0.6929, 'learning_rate': 1.0183890948911074e-05, 'epoch': 1.72} {'loss': 0.7106, 'learning_rate': 1.0118517455823374e-05, 'epoch': 1.72} {'loss': 0.7181, 'learning_rate': 1.0053211484238303e-05, 'epoch': 1.73} {'loss': 0.691, 'learning_rate': 9.987974885757647e-06, 'epoch': 1.73} {'loss': 0.6584, 'learning_rate': 9.922809510016277e-06, 'epoch': 1.74} {'loss': 0.6575, 'learning_rate': 9.857717204629696e-06, 'epoch': 1.74} {'loss': 0.6865, 'learning_rate': 9.792699815141685e-06, 'epoch': 1.75} {'loss': 0.6692, 'learning_rate': 9.727759184971936e-06, 'epoch': 1.75} {'loss': 0.7082, 'learning_rate': 9.662897155363814e-06, 'epoch': 1.76} {'loss': 0.6894, 'learning_rate': 9.598115565332142e-06, 'epoch': 1.76} {'loss': 0.7292, 'learning_rate': 9.533416251611064e-06, 'epoch': 1.77} {'loss': 0.6795, 'learning_rate': 9.468801048601972e-06, 'epoch': 1.77} {'loss': 0.662, 'learning_rate': 9.404271788321482e-06, 'epoch': 1.78} {'loss': 0.6917, 'learning_rate': 9.339830300349498e-06, 'epoch': 1.78} {'loss': 0.6759, 'learning_rate': 9.275478411777354e-06, 'epoch': 1.79} {'loss': 0.6535, 'learning_rate': 9.211217947155986e-06, 'epoch': 1.79} {'loss': 0.6741, 'learning_rate': 9.147050728444215e-06, 'epoch': 1.8} {'loss': 0.6837, 'learning_rate': 9.08297857495709e-06, 'epoch': 1.8} {'loss': 0.6454, 'learning_rate': 9.0190033033143e-06, 'epoch': 1.81} {'loss': 0.6695, 'learning_rate': 8.955126727388672e-06, 'epoch': 1.81} {'loss': 0.727, 'learning_rate': 8.89135065825474e-06, 'epoch': 1.82} {'loss': 0.6626, 'learning_rate': 8.827676904137397e-06, 'epoch': 1.82} {'loss': 0.6436, 'learning_rate': 8.76410727036062e-06, 'epoch': 1.83} {'loss': 0.6852, 'learning_rate': 8.700643559296313e-06, 'epoch': 1.83} {'loss': 0.6295, 'learning_rate': 8.637287570313159e-06, 'epoch': 1.84} {'loss': 0.679, 'learning_rate': 8.574041099725644e-06, 'epoch': 1.84} {'loss': 0.6501, 'learning_rate': 8.510905940743093e-06, 'epoch': 1.85} {'loss': 0.6976, 'learning_rate': 8.447883883418866e-06, 'epoch': 1.85} {'loss': 0.7009, 'learning_rate': 8.38497671459957e-06, 'epoch': 1.85} {'loss': 0.6754, 'learning_rate': 8.32218621787441e-06, 'epoch': 1.86} {'loss': 0.6816, 'learning_rate': 8.25951417352462e-06, 'epoch': 1.86} {'loss': 0.6554, 'learning_rate': 8.196962358473e-06, 'epoch': 1.87} {'loss': 0.6532, 'learning_rate': 8.134532546233505e-06, 'epoch': 1.87} {'loss': 0.7011, 'learning_rate': 8.072226506860988e-06, 'epoch': 1.88} {'loss': 0.7002, 'learning_rate': 8.010046006901001e-06, 'epoch': 1.88} {'loss': 0.643, 'learning_rate': 7.947992809339723e-06, 'epoch': 1.89} {'loss': 0.7304, 'learning_rate': 7.88606867355395e-06, 'epoch': 1.89} {'loss': 0.673, 'learning_rate': 7.824275355261233e-06, 'epoch': 1.9} {'loss': 0.6368, 'learning_rate': 7.762614606470084e-06, 'epoch': 1.9} {'loss': 0.6589, 'learning_rate': 7.701088175430327e-06, 'epoch': 1.91} {'loss': 0.685, 'learning_rate': 7.639697806583493e-06, 'epoch': 1.91} {'loss': 0.6659, 'learning_rate': 7.578445240513393e-06, 'epoch': 1.92} {'loss': 0.6914, 'learning_rate': 7.517332213896743e-06, 'epoch': 1.92} {'loss': 0.6852, 'learning_rate': 7.456360459453952e-06, 'epoch': 1.93} {'loss': 0.6308, 'learning_rate': 7.395531705899969e-06, 'epoch': 1.93} {'loss': 0.6535, 'learning_rate': 7.334847677895276e-06, 'epoch': 1.94} {'loss': 0.6662, 'learning_rate': 7.274310095997e-06, 'epoch': 1.94} {'loss': 0.6827, 'learning_rate': 7.213920676610112e-06, 'epoch': 1.95} {'loss': 0.6781, 'learning_rate': 7.153681131938791e-06, 'epoch': 1.95} {'loss': 0.6506, 'learning_rate': 7.093593169937849e-06, 'epoch': 1.96} {'loss': 0.7036, 'learning_rate': 7.033658494264309e-06, 'epoch': 1.96} {'loss': 0.6834, 'learning_rate': 6.973878804229131e-06, 'epoch': 1.97} {'loss': 0.6629, 'learning_rate': 6.914255794748989e-06, 'epoch': 1.97} {'loss': 0.7212, 'learning_rate': 6.8547911562982595e-06, 'epoch': 1.98} {'loss': 0.6611, 'learning_rate': 6.7954865748610515e-06, 'epoch': 1.98} {'loss': 0.7121, 'learning_rate': 6.736343731883425e-06, 'epoch': 1.99} {'loss': 0.6365, 'learning_rate': 6.677364304225731e-06, 'epoch': 1.99} {'loss': 0.6527, 'learning_rate': 6.618549964115033e-06, 'epoch': 2.0} {'loss': 0.5056, 'learning_rate': 6.559902379097721e-06, 'epoch': 2.0} {'loss': 0.4116, 'learning_rate': 6.501423211992224e-06, 'epoch': 2.01} {'loss': 0.3913, 'learning_rate': 6.443114120841874e-06, 'epoch': 2.01} {'loss': 0.4092, 'learning_rate': 6.384976758867875e-06, 'epoch': 2.02} {'loss': 0.402, 'learning_rate': 6.32701277442245e-06, 'epoch': 2.02} {'loss': 0.3886, 'learning_rate': 6.269223810942086e-06, 'epoch': 2.03} {'loss': 0.3336, 'learning_rate': 6.211611506900973e-06, 'epoch': 2.03} {'loss': 0.3918, 'learning_rate': 6.1541774957645045e-06, 'epoch': 2.04} {'loss': 0.3541, 'learning_rate': 6.096923405942989e-06, 'epoch': 2.04} {'loss': 0.3317, 'learning_rate': 6.03985086074549e-06, 'epoch': 2.05} {'loss': 0.3406, 'learning_rate': 5.982961478333768e-06, 'epoch': 2.05} {'loss': 0.324, 'learning_rate': 5.926256871676443e-06, 'epoch': 2.06} {'loss': 0.3686, 'learning_rate': 5.86973864850322e-06, 'epoch': 2.06} {'loss': 0.3323, 'learning_rate': 5.813408411259331e-06, 'epoch': 2.07} {'loss': 0.3494, 'learning_rate': 5.757267757060109e-06, 'epoch': 2.07} {'loss': 0.3532, 'learning_rate': 5.701318277645675e-06, 'epoch': 2.08} {'loss': 0.3596, 'learning_rate': 5.645561559335832e-06, 'epoch': 2.08} {'loss': 0.3783, 'learning_rate': 5.589999182985087e-06, 'epoch': 2.09} {'loss': 0.3515, 'learning_rate': 5.534632723937809e-06, 'epoch': 2.09} {'loss': 0.3261, 'learning_rate': 5.479463751983603e-06, 'epoch': 2.1} {'loss': 0.368, 'learning_rate': 5.424493831312757e-06, 'epoch': 2.1} {'loss': 0.3684, 'learning_rate': 5.369724520471925e-06, 'epoch': 2.11} {'loss': 0.3526, 'learning_rate': 5.315157372319915e-06, 'epoch': 2.11} {'loss': 0.3388, 'learning_rate': 5.2607939339836945e-06, 'epoch': 2.12} {'loss': 0.3406, 'learning_rate': 5.2066357468144824e-06, 'epoch': 2.12} {'loss': 0.33, 'learning_rate': 5.152684346344087e-06, 'epoch': 2.13} {'loss': 0.3424, 'learning_rate': 5.0989412622413345e-06, 'epoch': 2.13} {'loss': 0.373, 'learning_rate': 5.0454080182687314e-06, 'epoch': 2.14} {'loss': 0.3332, 'learning_rate': 4.992086132239238e-06, 'epoch': 2.14} {'loss': 0.3413, 'learning_rate': 4.938977115973242e-06, 'epoch': 2.15} {'loss': 0.355, 'learning_rate': 4.886082475255694e-06, 'epoch': 2.15} {'loss': 0.3331, 'learning_rate': 4.833403709793424e-06, 'epoch': 2.15} {'loss': 0.343, 'learning_rate': 4.780942313172602e-06, 'epoch': 2.16} {'loss': 0.3988, 'learning_rate': 4.728699772816407e-06, 'epoch': 2.16} {'loss': 0.3527, 'learning_rate': 4.6766775699428365e-06, 'epoch': 2.17} {'loss': 0.3563, 'learning_rate': 4.624877179522742e-06, 'epoch': 2.17} {'loss': 0.376, 'learning_rate': 4.573300070237973e-06, 'epoch': 2.18} {'loss': 0.3256, 'learning_rate': 4.5219477044397525e-06, 'epoch': 2.18} {'loss': 0.3517, 'learning_rate': 4.470821538107219e-06, 'epoch': 2.19} {'loss': 0.4053, 'learning_rate': 4.419923020806148e-06, 'epoch': 2.19} {'loss': 0.3272, 'learning_rate': 4.36925359564783e-06, 'epoch': 2.2} {'loss': 0.3371, 'learning_rate': 4.318814699248188e-06, 'epoch': 2.2} {'loss': 0.314, 'learning_rate': 4.268607761687019e-06, 'epoch': 2.21} {'loss': 0.347, 'learning_rate': 4.218634206467449e-06, 'epoch': 2.21} {'loss': 0.3671, 'learning_rate': 4.1688954504756e-06, 'epoch': 2.22} {'loss': 0.3479, 'learning_rate': 4.119392903940384e-06, 'epoch': 2.22} {'loss': 0.3732, 'learning_rate': 4.07012797039353e-06, 'epoch': 2.23} {'loss': 0.3116, 'learning_rate': 4.021102046629799e-06, 'epoch': 2.23} {'loss': 0.3443, 'learning_rate': 3.972316522667381e-06, 'epoch': 2.24} {'loss': 0.3662, 'learning_rate': 3.923772781708468e-06, 'epoch': 2.24} {'loss': 0.2995, 'learning_rate': 3.875472200100051e-06, 'epoch': 2.25} {'loss': 0.361, 'learning_rate': 3.827416147294888e-06, 'epoch': 2.25} {'loss': 0.3441, 'learning_rate': 3.7796059858126927e-06, 'epoch': 2.26} {'loss': 0.3462, 'learning_rate': 3.7320430712014796e-06, 'epoch': 2.26} {'loss': 0.3885, 'learning_rate': 3.6847287519991434e-06, 'epoch': 2.27} {'loss': 0.3637, 'learning_rate': 3.637664369695233e-06, 'epoch': 2.27} {'loss': 0.3185, 'learning_rate': 3.590851258692894e-06, 'epoch': 2.28} {'loss': 0.3414, 'learning_rate': 3.5442907462710605e-06, 'epoch': 2.28} {'loss': 0.3493, 'learning_rate': 3.4979841525468016e-06, 'epoch': 2.29} {'loss': 0.3546, 'learning_rate': 3.451932790437898e-06, 'epoch': 2.29} {'loss': 0.3458, 'learning_rate': 3.4061379656256345e-06, 'epoch': 2.3} {'loss': 0.3618, 'learning_rate': 3.3606009765177555e-06, 'epoch': 2.3} {'loss': 0.3519, 'learning_rate': 3.3153231142116617e-06, 'epoch': 2.31} {'loss': 0.3614, 'learning_rate': 3.270305662457805e-06, 'epoch': 2.31} {'loss': 0.3665, 'learning_rate': 3.225549897623302e-06, 'epoch': 2.32} {'loss': 0.3785, 'learning_rate': 3.181057088655719e-06, 'epoch': 2.32} {'loss': 0.3377, 'learning_rate': 3.136828497047116e-06, 'epoch': 2.33} {'loss': 0.3767, 'learning_rate': 3.0928653767982695e-06, 'epoch': 2.33} {'loss': 0.3523, 'learning_rate': 3.0491689743831277e-06, 'epoch': 2.34} {'loss': 0.3635, 'learning_rate': 3.005740528713458e-06, 'epoch': 2.34} {'loss': 0.341, 'learning_rate': 2.9625812711037206e-06, 'epoch': 2.35} {'loss': 0.3467, 'learning_rate': 2.91969242523618e-06, 'epoch': 2.35} {'loss': 0.3493, 'learning_rate': 2.87707520712617e-06, 'epoch': 2.36} {'loss': 0.3379, 'learning_rate': 2.834730825087663e-06, 'epoch': 2.36} {'loss': 0.3951, 'learning_rate': 2.7926604796989676e-06, 'epoch': 2.37} {'loss': 0.3427, 'learning_rate': 2.7508653637687176e-06, 'epoch': 2.37} {'loss': 0.3813, 'learning_rate': 2.709346662302041e-06, 'epoch': 2.38} {'loss': 0.4021, 'learning_rate': 2.6681055524669694e-06, 'epoch': 2.38} {'loss': 0.3559, 'learning_rate': 2.6271432035610513e-06, 'epoch': 2.39} {'loss': 0.3115, 'learning_rate': 2.5864607769782084e-06, 'epoch': 2.39} {'loss': 0.3605, 'learning_rate': 2.5460594261757996e-06, 'epoch': 2.4} {'loss': 0.326, 'learning_rate': 2.5059402966419303e-06, 'epoch': 2.4} {'loss': 0.364, 'learning_rate': 2.466104525862957e-06, 'epoch': 2.41} {'loss': 0.3477, 'learning_rate': 2.426553243291249e-06, 'epoch': 2.41} {'loss': 0.352, 'learning_rate': 2.3872875703131583e-06, 'epoch': 2.42} {'loss': 0.3749, 'learning_rate': 2.3483086202172377e-06, 'epoch': 2.42} {'loss': 0.3521, 'learning_rate': 2.309617498162657e-06, 'epoch': 2.43} {'loss': 0.3528, 'learning_rate': 2.271215301147887e-06, 'epoch': 2.43} {'loss': 0.3462, 'learning_rate': 2.2331031179795792e-06, 'epoch': 2.44} {'loss': 0.3517, 'learning_rate': 2.1952820292417143e-06, 'epoch': 2.44} {'loss': 0.3434, 'learning_rate': 2.1577531072649495e-06, 'epoch': 2.45} {'loss': 0.3679, 'learning_rate': 2.120517416096217e-06, 'epoch': 2.45} {'loss': 0.3568, 'learning_rate': 2.083576011468562e-06, 'epoch': 2.46} {'loss': 0.3727, 'learning_rate': 2.0469299407712112e-06, 'epoch': 2.46} {'loss': 0.3949, 'learning_rate': 2.010580243019865e-06, 'epoch': 2.46} {'loss': 0.3595, 'learning_rate': 1.9745279488272484e-06, 'epoch': 2.47} {'loss': 0.3371, 'learning_rate': 1.938774080373884e-06, 'epoch': 2.47} {'loss': 0.342, 'learning_rate': 1.9033196513791138e-06, 'epoch': 2.48} {'loss': 0.3505, 'learning_rate': 1.8681656670723663e-06, 'epoch': 2.48} {'loss': 0.3533, 'learning_rate': 1.833313124164636e-06, 'epoch': 2.49} {'loss': 0.3303, 'learning_rate': 1.7987630108202355e-06, 'epoch': 2.49} {'loss': 0.3815, 'learning_rate': 1.764516306628787e-06, 'epoch': 2.5} {'loss': 0.3618, 'learning_rate': 1.7305739825774228e-06, 'epoch': 2.5} {'loss': 0.3296, 'learning_rate': 1.6969370010232882e-06, 'epoch': 2.51} {'loss': 0.3413, 'learning_rate': 1.663606315666226e-06, 'epoch': 2.51} {'loss': 0.36, 'learning_rate': 1.6305828715217506e-06, 'epoch': 2.52} {'loss': 0.3716, 'learning_rate': 1.597867604894264e-06, 'epoch': 2.52} {'loss': 0.346, 'learning_rate': 1.5654614433504841e-06, 'epoch': 2.53} {'loss': 0.3628, 'learning_rate': 1.5333653056931684e-06, 'epoch': 2.53} {'loss': 0.3417, 'learning_rate': 1.5015801019350476e-06, 'epoch': 2.54} {'loss': 0.3508, 'learning_rate': 1.4701067332730426e-06, 'epoch': 2.54} {'loss': 0.3418, 'learning_rate': 1.4389460920626887e-06, 'epoch': 2.55} {'loss': 0.3471, 'learning_rate': 1.4080990617928571e-06, 'epoch': 2.55} {'loss': 0.371, 'learning_rate': 1.3775665170606896e-06, 'epoch': 2.56} {'loss': 0.348, 'learning_rate': 1.3473493235468177e-06, 'epoch': 2.56} {'loss': 0.3172, 'learning_rate': 1.3174483379907996e-06, 'epoch': 2.57} {'loss': 0.3701, 'learning_rate': 1.2878644081668403e-06, 'epoch': 2.57} {'loss': 0.3788, 'learning_rate': 1.2585983728597608e-06, 'epoch': 2.58} {'loss': 0.4176, 'learning_rate': 1.2296510618411957e-06, 'epoch': 2.58} {'loss': 0.3323, 'learning_rate': 1.201023295846093e-06, 'epoch': 2.59} {'loss': 0.3603, 'learning_rate': 1.1727158865494206e-06, 'epoch': 2.59} {'loss': 0.3394, 'learning_rate': 1.1447296365431668e-06, 'epoch': 2.6} {'loss': 0.3344, 'learning_rate': 1.1170653393135847e-06, 'epoch': 2.6} {'loss': 0.3331, 'learning_rate': 1.0897237792186862e-06, 'epoch': 2.61} {'loss': 0.3656, 'learning_rate': 1.0627057314660114e-06, 'epoch': 2.61} {'loss': 0.3086, 'learning_rate': 1.0360119620906426e-06, 'epoch': 2.62} {'loss': 0.3505, 'learning_rate': 1.0096432279334905e-06, 'epoch': 2.62} {'loss': 0.3473, 'learning_rate': 9.836002766198379e-07, 'epoch': 2.63} {'loss': 0.3996, 'learning_rate': 9.57883846538135e-07, 'epoch': 2.63} {'loss': 0.3151, 'learning_rate': 9.324946668190657e-07, 'epoch': 2.64} {'loss': 0.3581, 'learning_rate': 9.074334573148783e-07, 'epoch': 2.64} {'loss': 0.3843, 'learning_rate': 8.827009285789778e-07, 'epoch': 2.65} {'loss': 0.3462, 'learning_rate': 8.582977818457696e-07, 'epoch': 2.65} {'loss': 0.3123, 'learning_rate': 8.342247090107919e-07, 'epoch': 2.66} {'loss': 0.3567, 'learning_rate': 8.104823926110811e-07, 'epoch': 2.66} {'loss': 0.3301, 'learning_rate': 7.870715058058406e-07, 'epoch': 2.67} {'loss': 0.3228, 'learning_rate': 7.639927123573351e-07, 'epoch': 2.67} {'loss': 0.3474, 'learning_rate': 7.412466666120846e-07, 'epoch': 2.68} {'loss': 0.3202, 'learning_rate': 7.188340134823043e-07, 'epoch': 2.68} {'loss': 0.3468, 'learning_rate': 6.967553884276279e-07, 'epoch': 2.69} {'loss': 0.3849, 'learning_rate': 6.750114174370797e-07, 'epoch': 2.69} {'loss': 0.3187, 'learning_rate': 6.536027170113354e-07, 'epoch': 2.7} {'loss': 0.3558, 'learning_rate': 6.32529894145234e-07, 'epoch': 2.7} {'loss': 0.2956, 'learning_rate': 6.117935463105809e-07, 'epoch': 2.71} {'loss': 0.3309, 'learning_rate': 5.913942614391931e-07, 'epoch': 2.71} {'loss': 0.339, 'learning_rate': 5.713326179062409e-07, 'epoch': 2.72} {'loss': 0.3394, 'learning_rate': 5.516091845138377e-07, 'epoch': 2.72} {'loss': 0.3898, 'learning_rate': 5.322245204749319e-07, 'epoch': 2.73} {'loss': 0.3797, 'learning_rate': 5.131791753974296e-07, 'epoch': 2.73} {'loss': 0.3644, 'learning_rate': 4.944736892686278e-07, 'epoch': 2.74} {'loss': 0.3346, 'learning_rate': 4.761085924398959e-07, 'epoch': 2.74} {'loss': 0.3028, 'learning_rate': 4.580844056116418e-07, 'epoch': 2.75} {'loss': 0.4031, 'learning_rate': 4.4040163981855095e-07, 'epoch': 2.75} {'loss': 0.3211, 'learning_rate': 4.2306079641509094e-07, 'epoch': 2.76} {'loss': 0.3552, 'learning_rate': 4.0606236706130367e-07, 'epoch': 2.76} {'loss': 0.3539, 'learning_rate': 3.89406833708858e-07, 'epoch': 2.77} {'loss': 0.3764, 'learning_rate': 3.73094668587394e-07, 'epoch': 2.77} {'loss': 0.3776, 'learning_rate': 3.5712633419112686e-07, 'epoch': 2.77} {'loss': 0.3943, 'learning_rate': 3.4150228326573356e-07, 'epoch': 2.78} {'loss': 0.3839, 'learning_rate': 3.262229587955232e-07, 'epoch': 2.78} {'loss': 0.3238, 'learning_rate': 3.112887939908715e-07, 'epoch': 2.79} {'loss': 0.3462, 'learning_rate': 2.9670021227593965e-07, 'epoch': 2.79} {'loss': 0.3338, 'learning_rate': 2.824576272766666e-07, 'epoch': 2.8} {'loss': 0.4009, 'learning_rate': 2.6856144280904967e-07, 'epoch': 2.8} {'loss': 0.3511, 'learning_rate': 2.550120528676841e-07, 'epoch': 2.81} {'loss': 0.3157, 'learning_rate': 2.4180984161460004e-07, 'epoch': 2.81} {'loss': 0.3281, 'learning_rate': 2.2895518336836923e-07, 'epoch': 2.82} {'loss': 0.3573, 'learning_rate': 2.164484425934879e-07, 'epoch': 2.82} {'loss': 0.3414, 'learning_rate': 2.0428997389004972e-07, 'epoch': 2.83} {'loss': 0.386, 'learning_rate': 1.9248012198368748e-07, 'epoch': 2.83} {'loss': 0.3197, 'learning_rate': 1.8101922171580022e-07, 'epoch': 2.84} {'loss': 0.3153, 'learning_rate': 1.699075980340581e-07, 'epoch': 2.84} {'loss': 0.3227, 'learning_rate': 1.5914556598319307e-07, 'epoch': 2.85} {'loss': 0.3259, 'learning_rate': 1.4873343069606305e-07, 'epoch': 2.85} {'loss': 0.3968, 'learning_rate': 1.3867148738500313e-07, 'epoch': 2.86} {'loss': 0.3531, 'learning_rate': 1.289600213334505e-07, 'epoch': 2.86} {'loss': 0.3476, 'learning_rate': 1.195993078878646e-07, 'epoch': 2.87} {'loss': 0.3373, 'learning_rate': 1.1058961244991144e-07, 'epoch': 2.87} {'loss': 0.3372, 'learning_rate': 1.0193119046894295e-07, 'epoch': 2.88} {'loss': 0.3463, 'learning_rate': 9.362428743475565e-08, 'epoch': 2.88} {'loss': 0.3544, 'learning_rate': 8.566913887062683e-08, 'epoch': 2.89} {'loss': 0.3629, 'learning_rate': 7.806597032663925e-08, 'epoch': 2.89} {'loss': 0.3486, 'learning_rate': 7.081499737328634e-08, 'epoch': 2.9} {'loss': 0.3315, 'learning_rate': 6.391642559535616e-08, 'epoch': 2.9} {'loss': 0.3358, 'learning_rate': 5.7370450586109737e-08, 'epoch': 2.91} {'loss': 0.3144, 'learning_rate': 5.117725794172995e-08, 'epoch': 2.91} {'loss': 0.3609, 'learning_rate': 4.533702325605904e-08, 'epoch': 2.92} {'loss': 0.3044, 'learning_rate': 3.9849912115623446e-08, 'epoch': 2.92} {'loss': 0.3391, 'learning_rate': 3.471608009493754e-08, 'epoch': 2.93} {'loss': 0.3522, 'learning_rate': 2.9935672752090525e-08, 'epoch': 2.93} {'loss': 0.3487, 'learning_rate': 2.5508825624626088e-08, 'epoch': 2.94} {'loss': 0.3291, 'learning_rate': 2.1435664225688547e-08, 'epoch': 2.94} {'loss': 0.3867, 'learning_rate': 1.7716304040475697e-08, 'epoch': 2.95} {'loss': 0.3466, 'learning_rate': 1.4350850522956705e-08, 'epoch': 2.95} {'loss': 0.2979, 'learning_rate': 1.1339399092885616e-08, 'epoch': 2.96} {'loss': 0.3577, 'learning_rate': 8.682035133096566e-09, 'epoch': 2.96} {'loss': 0.3616, 'learning_rate': 6.378833987077948e-09, 'epoch': 2.97} {'loss': 0.3405, 'learning_rate': 4.429860956842169e-09, 'epoch': 2.97} {'loss': 0.3453, 'learning_rate': 2.8351713010743573e-09, 'epoch': 2.98} {'loss': 0.3446, 'learning_rate': 1.5948102335572313e-09, 'epoch': 2.98} {'loss': 0.3575, 'learning_rate': 7.088129218985052e-10, 'epoch': 2.99} {'loss': 0.3648, 'learning_rate': 1.772044865316902e-10, 'epoch': 2.99} {'loss': 0.3142, 'learning_rate': 0.0, 'epoch': 3.0} {'train_runtime': 3036.251, 'train_samples_per_second': 51.379, 'train_steps_per_second': 0.201, 'train_loss': 0.7130698823478617, 'epoch': 3.0} [2024-01-10 03:33:18,764] [INFO] [launch.py:347:main] Process 41036 exits successfully. [2024-01-10 03:33:18,764] [INFO] [launch.py:347:main] Process 41035 exits successfully. [2024-01-10 03:33:18,764] [INFO] [launch.py:347:main] Process 41034 exits successfully. [2024-01-10 03:33:19,765] [INFO] [launch.py:347:main] Process 41037 exits successfully. [2024-01-10 03:33:19,766] [INFO] [launch.py:347:main] Process 41039 exits successfully. [2024-01-10 03:33:19,766] [INFO] [launch.py:347:main] Process 41033 exits successfully. [2024-01-10 03:33:19,766] [INFO] [launch.py:347:main] Process 41038 exits successfully. Finish training... [2024-01-10 03:33:27,775] [INFO] [launch.py:347:main] Process 41032 exits successfully.