WARNING:torch.distributed.run: ***************************************** Setting OMP_NUM_THREADS environment variable for each process to be 1 in default, to avoid your system being overloaded, please further tune the variable for optimal performance in your application as needed. ***************************************** WARNING:torch.distributed.run: ***************************************** Setting OMP_NUM_THREADS environment variable for each process to be 1 in default, to avoid your system being overloaded, please further tune the variable for optimal performance in your application as needed. ***************************************** WARNING:torch.distributed.run: ***************************************** Setting OMP_NUM_THREADS environment variable for each process to be 1 in default, to avoid your system being overloaded, please further tune the variable for optimal performance in your application as needed. ***************************************** WARNING:torch.distributed.run: ***************************************** Setting OMP_NUM_THREADS environment variable for each process to be 1 in default, to avoid your system being overloaded, please further tune the variable for optimal performance in your application as needed. ***************************************** [2024-01-28 04:18:55,115] [INFO] [real_accelerator.py:158:get_accelerator] Setting ds_accelerator to cuda (auto detect) [2024-01-28 04:18:55,115] [INFO] [real_accelerator.py:158:get_accelerator] Setting ds_accelerator to cuda (auto detect) [2024-01-28 04:18:55,115] [INFO] [real_accelerator.py:158:get_accelerator] Setting ds_accelerator to cuda (auto detect) [2024-01-28 04:18:55,115] [INFO] [real_accelerator.py:158:get_accelerator] Setting ds_accelerator to cuda (auto detect) [2024-01-28 04:18:55,126] [INFO] [real_accelerator.py:158:get_accelerator] Setting ds_accelerator to cuda (auto detect) [2024-01-28 04:18:55,128] [INFO] [real_accelerator.py:158:get_accelerator] Setting ds_accelerator to cuda (auto detect) [2024-01-28 04:18:55,131] [INFO] [real_accelerator.py:158:get_accelerator] Setting ds_accelerator to cuda (auto detect) [2024-01-28 04:18:55,133] [INFO] [real_accelerator.py:158:get_accelerator] Setting ds_accelerator to cuda (auto detect) [2024-01-28 04:18:54,714] [INFO] [real_accelerator.py:158:get_accelerator] Setting ds_accelerator to cuda (auto detect) [2024-01-28 04:18:54,714] [INFO] [real_accelerator.py:158:get_accelerator] Setting ds_accelerator to cuda (auto detect) [2024-01-28 04:18:54,714] [INFO] [real_accelerator.py:158:get_accelerator] Setting ds_accelerator to cuda (auto detect) [2024-01-28 04:18:54,714] [INFO] [real_accelerator.py:158:get_accelerator] Setting ds_accelerator to cuda (auto detect) [2024-01-28 04:18:54,714] [INFO] [real_accelerator.py:158:get_accelerator] Setting ds_accelerator to cuda (auto detect) [2024-01-28 04:18:54,714] [INFO] [real_accelerator.py:158:get_accelerator] Setting ds_accelerator to cuda (auto detect) [2024-01-28 04:18:54,714] [INFO] [real_accelerator.py:158:get_accelerator] Setting ds_accelerator to cuda (auto detect) [2024-01-28 04:18:54,714] [INFO] [real_accelerator.py:158:get_accelerator] Setting ds_accelerator to cuda (auto detect) [2024-01-28 04:19:17,334] [INFO] [comm.py:637:init_distributed] cdb=None [2024-01-28 04:19:17,334] [INFO] [comm.py:637:init_distributed] cdb=None [2024-01-28 04:19:17,470] [INFO] [comm.py:637:init_distributed] cdb=None [2024-01-28 04:19:17,521] [INFO] [comm.py:637:init_distributed] cdb=None [2024-01-28 04:19:17,527] [INFO] [comm.py:637:init_distributed] cdb=None [2024-01-28 04:19:17,531] [INFO] [comm.py:637:init_distributed] cdb=None [2024-01-28 04:19:17,535] [INFO] [comm.py:637:init_distributed] cdb=None [2024-01-28 04:19:17,537] [INFO] [comm.py:637:init_distributed] cdb=None [INFO] [comm.py:637:init_distributed] cdb=None [2024-01-28 04:19:15,280] [INFO] [comm.py:637:init_distributed] cdb=None [2024-01-28 04:19:18,381] [INFO] [comm.py:637:init_distributed] cdb=None [2024-01-28 04:19:18,381] [INFO] [comm.py:637:init_distributed] cdb=None [2024-01-28 04:19:18,381] [INFO] [comm.py:637:init_distributed] cdb=None [2024-01-28 04:19:18,381] [INFO] [comm.py:637:init_distributed] cdb=None 01/28/2024 04:19:18 - WARNING - __main__ - Process rank: 0, device: cuda:0, n_gpu: 1distributed training: True, 16-bits training: False 01/28/2024 04:19:18 - INFO - __main__ - Training/evaluation parameters TrainingArguments( _n_gpu=1, adafactor=False, adam_beta1=0.9, adam_beta2=0.999, adam_01/28/2024 04:19:18 - WARNING - __main__ - Process rank: 6, device: cuda:6, n_gpu: 1distributed training: True, 16-bits training: False /apdcephfs/share_733425/vinnylywang/jianhuipang/llama2_sft/envs/lib/python3.8/site-packages/datasets/load.py:2089: FutureWarning: 'use_auth_token' was deprecated in favor of 'token' in version 2.14.0 and will be removed in 3.0.0. You can remove this warning by passing 'token=None' instead. warnings.warn( 01/28/2024 04:19:18 - WARNING - __main__ - Process rank: 5, device: cuda:5, n_gpu: 1distributed training: True, 16-bits training: False /apdcephfs/share_733425/vinnylywang/jianhuipang/llama2_sft/envs/lib/python3.8/site-packages/datasets/load.py:2089: FutureWarning: 'use_auth_token' was deprecated in favor of 'token' in version 2.14.0 and will be removed in 3.0.0. You can remove this warning by passing 'token=None' instead. warnings.warn( 01/28/2024 04:19:18 - WARNING - __main__ - Process rank: 4, device: cuda:4, n_gpu: 1distributed training: True, 16-bits training: False 01/28/2024 04:19:18 - WARNING - __main__ - Process rank: 2, device: cuda:2, n_gpu: 1distributed training: True, 16-bits training: False /apdcephfs/share_733425/vinnylywang/jianhuipang/llama2_sft/envs/lib/python3.8/site-packages/datasets/load.py:2089: FutureWarning: 'use_auth_token' was deprecated in favor of 'token' in version 2.14.0 and will be removed in 3.0.0. You can remove this warning by passing 'token=None' instead. warnings.warn( /apdcephfs/share_733425/vinnylywang/jianhuipang/llama2_sft/envs/lib/python3.8/site-packages/datasets/load.py:2089: FutureWarning: 'use_auth_token' was deprecated in favor of 'token' in version 2.14.0 and will be removed in 3.0.0. You can remove this warning by passing 'token=None' instead. warnings.warn( 01/28/2024 04:19:18 - WARNING - __main__ - Process rank: 7, device: cuda:7, n_gpu: 1distributed training: True, 16-bits training: False 01/28/2024 04:19:18 - WARNING - __main__ - Process rank: 3, device: cuda:3, n_gpu: 1distributed training: True, 16-bits training: False 01/28/2024 04:19:18 - WARNING - __main__ - Process rank: 0, device: cuda:0, n_gpu: 1distributed training: True, 16-bits training: False 01/28/2024 04:19:18 - WARNING - __main__ - Process rank: 1, device: cuda:1, n_gpu: 1distributed training: True, 16-bits training: False 01/28/2024 04:19:18 - INFO - __main__ - Training/evaluation parameters TrainingArguments( _n_gpu=1, adafactor=False, adam_beta1=0.9, adam_beta2=0.999, adam_epsilon=1e-08, auto_find_batch_size=False, bf16=True, bf16_full_eval=True, data_seed=None, dataloader_drop_last=False, dataloader_num_workers=8, dataloader_pin_memory=True, ddp_bucket_cap_mb=None, ddp_find_unused_parameters=None, ddp_timeout=72000, debug=[], deepspeed=/apdcephfs/share_733425/vinnylywang/jianhuipang/llama2_sft/train/deepspeed_config_bf16.json, disable_tqdm=False, do_eval=False, do_predict=False, do_train=True, eval_accumulation_steps=None, eval_delay=0, eval_steps=None, evaluation_strategy=no, fp16=False, fp16_backend=auto, fp16_full_eval=False, fp16_opt_level=O1, fsdp=[], fsdp_config={'fsdp_min_num_params': 0, 'xla': False, 'xla_fsdp_grad_ckpt': False}, fsdp_min_num_params=0, fsdp_transformer_layer_cls_to_wrap=None, full_determinism=False, gradient_accumulation_steps=8, gradient_checkpointing=True, greater_is_better=None, group_by_length=False, half_precision_backend=auto, hub_model_id=None, hub_private_repo=False, hub_strategy=every_save, hub_token=, ignore_data_skip=False, include_inputs_for_metrics=False, jit_mode_eval=False, label_names=None, label_smoothing_factor=0.0, learning_rate=2e-05, length_column_name=length, load_best_model_at_end=False, local_rank=0, log_level=passive, log_level_replica=warning, log_on_each_node=True, logging_dir=./checkpoints_ct/ac/allm-ac-7b/runs/Jan28_04-18-54_ts-cbba87c5e7504a249f5127103d9ce40f-worker-1, logging_first_step=False, logging_nan_inf_filter=True, logging_steps=1, logging_strategy=steps, lr_scheduler_type=constant_with_warmup, max_grad_norm=1.0, max_steps=-1, metric_for_best_model=None, mp_parameters=, no_cuda=False, num_train_epochs=1.0, optim=adamw_hf, optim_args=None, output_dir=./checkpoints_ct/ac/allm-ac-7b, overwrite_output_dir=True, past_index=-1, per_device_eval_batch_size=1, per_device_train_batch_size=2, prediction_loss_only=False, push_to_hub=False, push_to_hub_model_id=None, push_to_hub_organization=None, push_to_hub_token=, ray_scope=last, remove_unused_columns=True, report_to=['tensorboard'], resume_from_checkpoint=None, run_name=./checkpoints_ct/ac/allm-ac-7b, save_on_each_node=False, save_steps=500, save_strategy=steps, save_total_limit=1, seed=34, sharded_ddp=[], skip_memory_metrics=True, tf32=None, torch_compile=False, torch_compile_backend=None, torch_compile_mode=None, torchdynamo=None, tpu_metrics_debug=False, tpu_num_cores=None, use_ipex=False, use_legac01/28/2024 04:19:18 - WARNING - __main__ - Process rank: 1, device: cuda:1, n_gpu: 1distributed training: True, 16-bits /apdcephfs/share/apdcephfs/share_733425/vinnylywang/jianhuipang/llama2_sft/envs/lib/python3.8/site-packages/datasets/load.py:2089: FutureWarning: 'use_auth_token' was deprecated in favor of 'token' in version 2.14.0 and will be removed in 3.0.0. You can remove this warning by passing 'token=None' instead. /apdcephfs/share/apdcephfs/share_733425/vinnylywang/jianhuipang/llama2_sft/envs/lib/python3.8/site-packages/datasets/load.py:2089: FutureWarning: 'use_auth_token' was deprecated in favor of 'token' in version 2.14.0 and will be removed in 3.0.0. You can remove this warning by passing 'token=None' instead. /apdcephfs/share/apdcephfs/share_733425/vinnylywang/jianhuipang/llama2_sft/envs/lib/python3.8/site-packages/datasets/load.py:2089: FutureWarning: 'use_auth_token' was deprecated in favor of 'token' in version 2.14.0 and will be removed in 3.0.0. You can remove this warning by passing 'token=None' instead. warnings.warn( No config specified, defaulting to the single config: red_pajama-data-1_t-sample/plain_text 01/28/2024 04:19:18 - INFO - datasets.builder - No config specified, defaulting to the single config: red_pajama-data-1_t-sample/plain_text Loading Dataset Infos from /apdcephfs/share_733425/vinnylywang/jianhuipang/hf_cache2/modules/datasets_modules/datasets/RedPajama-Data-1T-Sample/6ea3bc8ec2e84ec6d2df1930942e9028ace8c5b9d9143823cf911c50bbd92039 01/28/2024 04:19:18 - INFO - datasets.info - Loading Dataset Infos from /apdcephfs/share_733425/vinnylywang/jianhuipang/hf_cache2/modules/datasets_modules/datasets/RedPajama-Data-1T-Sample/6ea3bc8ec2e84ec6d2df1930942e9028ace8c5b9d9143823cf911c50bbd92039 No config specified, defaulting to the single config: red_pajama-data-1_t-sample/plain_text 01/28/2024 04:19:19 - INFO - datasets.builder - No config specified, defaulting to the single config: red_pajama-data-1_t-sample/plain_text Loading Dataset Infos from /apdcephfs/share_733425/vinnylywang/jianhuipang/hf_cache2/modules/datasets_modules/datasets/RedPajama-Data-1T-Sample/6ea3bc8ec2e84ec6d2df1930942e9028ace8c5b9d9143823cf911c50bbd92039 01/28/2024 04:19:19 - INFO - datasets.info - Loading Dataset Infos from /apdcephfs/share_733425/vinnylywang/jianhuipang/hf_cache2/modules/datasets_modules/datasets/RedPajama-Data-1T-Sample/6ea3bc8ec2e84ec6d2df1930942e9028ace8c5b9d9143823cf911c50bbd92039 Overwrite dataset info from restored data version if exists. 01/28/2024 04:19:19 - INFO - datasets.builder - Overwrite dataset info from restored data version if exists. Loading Dataset info from /apdcephfs/share_733425/vinnylywang/jianhuipang/hf_cache2/datasets/red_pajama-data-1_t-sample/plain_text/1.0.0/6ea3bc8ec2e84ec6d2df1930942e9028ace8c5b9d9143823cf911c50bbd92039 01/28/2024 04:19:19 - INFO - datasets.info - Loading Dataset info from /apdcephfs/share_733425/vinnylywang/jianhuipang/hf_cache2/datasets/red_pajama-data-1_t-sample/plain_text/1.0.0/6ea3bc8ec2e84ec6d2df1930942e9028ace8c5b9d9143823cf911c50bbd92039 Found cached dataset red_pajama-data-1_t-sample (/apdcephfs/share_733425/vinnylywang/jianhuipang/hf_cache2/datasets/red_pajama-data-1_t-sample/plain_text/1.0.0/6ea3bc8ec2e84ec6d2df1930942e9028ace8c5b9d9143823cf911c50bbd92039) 01/28/2024 04:19:19 - INFO -No config specified, defaulting to the single config: red_pajama-data-1_t-sample/plain_text 01/28/2024 04:19:20 - INFO - datasets.builder - No config specified, defaulting to the single config: red_pajama-data-1_t-sample/plain_text Loading Dataset Infos from /apdcephfs/share_733425/vinnylywang/jianhuipang/hf_cache2/modules/datasets_modules/datasets/RedPajama-Data-1T-Sample/6ea3bc8ec2e84ec6d2df1930942e9028ace8c5b9d9143823cf911c50bbd92039 01/28/2024 04:19:20 - INFO - datasets.info - Loading Dataset Infos from /apdcephfs/share_733425/vinnylywang/jianhuipang/hf_cache2/modules/datasets_modules/datasets/RedPajama-Data-1T-Sample/6ea3bc8ec2e84ec6d2df1930942e9028ace8c5b9d9143823cf911c50bbd92039 Overwrite dataset info from restored data version if exists. 01/28/2024 04:19:20 - INFO - datasets.builder - Overwrite dataset info from restored data version if exists. Loading Dataset info from /apdcephfs/share_733425/vinnylywang/jianhuipang/hf_cache2/datasets/red_pajama-data-1_t-sample/plain_text/1.0.0/6ea3bc8ec2e84ec6d2df1930942e9028ace8c5b9d9143823cf911c50bbd92039 01/28/2024 04:19:20 - INFO - datasets.info - Loading Dataset info from /apdcephfs/share_733425/vinnylywang/jianhuipang/hf_cache2/datasets/red_pajama-data-1_t-sample/plain_text/1.0.0/6ea3bc8ec2e84ec6d2df1930942e9028ace8c5b9d9143823cf911c50bbd92039 Found cached dataset red_pajama-data-1_t-sample (/apdcephfs/share_733425/vinnylywang/jianhuipang/hf_cache2/datasets/red_pajama-data-1_t-sample/plain_text/1.0.0/6ea3bc8ec2e84ec6d2df1930942e9028ace8c5b9d9143823cf911c50bbd92039) 01/28/2024 04:19:20 - INFO - datasets.builder - Found cached dataset red_pajama-data-1_t-sample (/apdcephfs/share_733425/vinnylywang/jianhuipang/hf_cache2/datasets/red_pajama-data-1_t-sample/plain_text/1.0.0/6ea3bc8ec2e84ec6d2df1930942e9028ace8c5b9d9143823cf911c50bbd92039) Loading Dataset info from /apdcephfs/share_733425/vinnylywang/jianhuipang/hf_cache2/datasets/red_pajama-data-1_t-sample/plain_text/1.0.0/6ea3bc8ec2e84ec6d2df1930942e9028ace8c5b9d9143823cf911c50bbd92039 01/28/2024 04:19:20 - INFO - datasets.info - Loading Dataset info from /apdcephfs/share_733425/vinnylywang/jianhuipang/hf_cache2/datasets/red_pajama-data-1_t-sample/plain_text/1.0.0/6ea3bc8ec2e84ec6d2df1930942e9028ace8c5b9d9143823cf911c50bbd92039 No config specified, defaulting to the single config: red_pajama-data-1_t-sample/plain_text 01/28/2024 04:19:20 - INFO - datasets.builder - No config specified, defaulting to the single config: red_pajama-data-1_t-sample/plain_text Loading Dataset Infos from /apdcephfs/share_733425/vinnylywang/jianhuipang/hf_cache2/modules/datasets_modules/datasets/RedPajama-Data-1T-Sample/6ea3bc8ec2e84ec6d2df1930942e9028ace8c5b9d9143823cf911c50bbd92039 01/28/2024 04:19:20 - INFO - datasets.info - Loading Dataset Infos from /apdcephfs/share_733425/vinnylywang/jianhuipang/hf_cache2/modules/datasets_modules/datasets/RedPajama-Data-1T-Sample/6ea3bc8ec2e84ec6d2df1930942e9028ace8c5b9d9143823cf911c50bbd92039 Overwrite dataset info from restored data version if exists. 01/28/2024 04:19:20 - INFO - datasets.builder - Overwrite dataset info from restored data version if exists. Loading Dataset info from /apdcephfs/share_733425/vinnylywang/jianhuipang/hf_cache2/datasets/red_pajama-data-1_t-sample/plain_text/1.0.0/6ea3bc8ec2e84ec6d2df1930942e9028ace8c5b9d9143823cf911c50bbd92039 01/28/2024 04:19:20 - INFO - datasets.info - Loading Dataset info from /apdcephfs/share_733425/vinnylywang/jianhuipang/hf_cache2/datasets/red_pajama-data-1_t-sample/plain_text/1.0.0/6ea3bc8ec2e84ec6d2df1930942e9028ace8c5b9d9143823cf911c50bbd92039 Found cached dataset red_pajama-data-1_t-sample (/apdcephfs/share_733425/vinnylywang/jianhuipang/hf_cache2/datasets/red_pajama-data-1_t-sample/plain_text/1.0.0/6ea3bc8ec2e84ec6d2df1930942e9028ace8c5b9d9143823cf911c50bbd92039) 01/28/2024 04:19:20 - INFO - datasets.builder - Found cached dataset red_pajama-data-1_t-sample (/apdcephfs/share_733425/vinnylywang/jianhuipang/hf_cache2/datasets/red_pajama-data-1_t-sample/plain_text/1.0.0/6ea3bc8ec2e84ec6d2df1930942e9028ace8c5b9d9143823cf911c50bbd92039) Loading Dataset info from /apdcephfs/share_733425/vinnylywang/jianhuipang/hf_cache2/datasets/red_pajama-data-1_t-sample/plain_text/1.0.0/6ea3bc8ec2e84ec6d2df1930942e9028ace8c5b9d9143823cf911c50bbd92039 01/28/2024 04:19:20 - INFO - datasets.info - Loading Dataset info from /apdcephfs/share_733425/vinnylywang/jianhuipang/hf_cache2/datasets/red_pajama-data-1_t-sample/plain_text/1.0.0/6ea3bc8ec2e84ec6d2df1930942e9028ace8c5b9d9143823cf911c50bbd92039 [INFO|configuration_utils.py:666] 2024-01-28 04:19:20,686 >> loading configuration file /apdcephfs/share_733425/vinnylywang/jianhuipang/opensourcellms/llama2/Llama-2-7b-hf/config.json [INFO|configuration_utils.py:720] 2024-01-28 04:19:20,687 >> Model config LlamaConfig { "_name_or_path": "/apdcephfs/share_733425/vinnylywang/jianhuipang/opensourcellms/llama2/Llama-2-7b-hf", "architectures": [ "LlamaForCausalLM" ], "bos_token_id": 1, "eos_token_id": 2, "hidden_act": "silu", "hidden_size": 4096, "initializer_range": 0.02, "intermediate_size": 11008, "max_position_embeddings": 4096, "model_type": "llama", "num_attention_heads": 32, "num_hidden_layers": 32, "num_key_value_heads": 32, "pad_token_id": 0, "pretraining_tp": 1, "rms_norm_eps": 1e-05, "rope_scaling": null, "tie_word_embeddings": false, "torch_dtype": "float16", "transformers_version": "4.28.0.dev0", "use_cache": true, "vocab_size": 32000 } 01/28/2024 04:19:20 - INFO - __main__ - Tokenizer_kwargs: {'cache_dir': None, 'use_fast': True, 'revision': 'main', 'use_auth_token': None} [INFO|tokenization_utils_base.py:1801] 2024-01-28 04:19:20,692 >> loading file tokenizer.model [INFO|tokenization_utils_base.py:1801] 2024-01-28 04:19:20,692 >> loading file added_tokens.json [INFO|tokenization_utils_base.py:1801] 2024-01-28 04:19:20,692 >> loading file special_tokens_map.json [INFO|tokenization_utils_base.py:1801] 2024-01-28 04:19:20,692 >> loading file tokenizer_config.json 01/28/2024 04:19:20 - INFO - __main__ - Loading checkpoints in dtype: None [INFO|modeling_utils.py:2395] 2024-01-28 04:19:20,710 >> loading weights file /apdcephfs/share_733425/vinnylywang/jianhuipang/opensourcellms/llama2/Llama-2-7b-hf/model.safetensors.index.json [INFO|modeling_utils.py:2487] 2024-01-28 04:19:20,712 >> Detected DeepSpeed ZeRO-3: activating zero.init() for this model [INFO|configuration_utils.py:575] 2024-01-28 04:19:20,718 >> Generate config GenerationConfig { "_from_model_config": true, "bos_token_id": 1, "eos_token_id": 2, "pad_token_id": 0, "transformers_version": "4.28.0.dev0" } ts-cbba87c5e7504a249f5127103d9ce40f-launcher:55261:55261 [0] NCCL INFO Bootstrap : Using eth1:11.219.11.45<0> ts-cbba87c5e7504a249f5127103d9ce40f-launcher:55261:55261 [0] NCCL INFO NET/Plugin : No plugin found (libnccl-net.so), using internal implementation ts-cbba87c5e7504a249f5127103d9ce40f-launcher:55261:55261 [0] NCCL INFO cudaDriverVersion 11070 NCCL version 2.14.3+cuda11.7 ts-cbba87c5e7504a249f5127103d9ce40f-launcher:55266:55266 [5] NCCL INFO cudaDriverVersion 11070 ts-cbba87c5e7504a249f5127103d9ce40f-launcher:55265:55265 [4] NCCL INFO cudaDriverVersion 11070 ts-cbba87c5e7504a249f5127103d9ce40f-launcher:55262:55262 [1] NCCL INFO cudaDriverVersion 11070 ts-cbba87c5e7504a249f5127103d9ce40f-launcher:55266:55266 [5] NCCL INFO Bootstrap : Using eth1:11.219.11.45<0> ts-cbba87c5e7ts-cbba87c5e7504a249f5127103d9ce40f-worker-1:97056:97056 [5] NCCL INFO cudaDriverVts-cbba87c5e7ts-cbba87c5e7504a249f5127103d9ce40f-worker-1:97052:97052 [1] NCCL INFO cudaDriverVersion 11070 ts-cbba87c5e7504a249f5127103d9ce40f-worker-1:97054:97054 [3] NCCL INFO Bootstrap : Using eth1:11.215.64.140<0> ts-cbba87c5e7504a249f5127103d9ce40f-worker-1:97056:97056 [5] NCCL INFO Bootstrap : Using eth1:11.215.64.140<0> ts-cbba87c5e7504a249f5127103d9ce40f-launcher:55266:55266 [5] NCCL INFO NET/Plugin : No plugin found (libnccl-net.so), using internal implementation ts-cbba87c5e7504a249f5127103d9ce40f-launcher:55262:55262 [1] NCCL INFO NET/Plugin : No plugin found (libnccl-net.so), using internal implementation ts-cbba87c5e7504a249f5127103d9ce40f-launcher:55265:55265 [4] NCCL INFO NET/Plugin : No plugin found (libnccl-net.so), using internal implementation ts-cbba87c5e7504a249f5127103d9ce40f-launcher:55263:55263 [2] NCCL INFO NET/Plugin : No plugin found (libnccl-net.so), using internal implementation ts-cbba87c5e7504a249f5127103d9ce40f-launcher:55267:55267 [6] NCCL INFO cudaDriverVersion 11070 ts-cbba87c5e7504a249f5127103d9ce40f-launcher:55267:55267 [6] NCCL INFO Bootstrap : Using eth1:11.219.11.45<0> ts-cbba87c5e7504a249f5127103d9ce40f-launcher:55261:56182 [0] NCCL INFO NET/IB : Using [0]mlx5_2:1/RoCE [RO]; OOB eth1:11.219.11.45<0> ts-cbba87c5e7504a249f5127103d9ce40f-launcher:55261:56182 [0] NCCL INFO Using network IB ts-cbba87c5e7504a249f5127103d9ce40f-launcher:55267:55267 [6] NCCL INFO NET/Plugin : No plugin found (libnccl-net.so), using internal implementation ts-cbba87c5e7504a249f5127103d9ce40f-launcher:55266:56183 [5] NCCL INFO NET/IB : Using [0]mlx5_2:1/RoCE [RO]; OOB eth1:11.219.11.45<0> ts-cbba87c5e7504a249f5127103d9ce40f-launcher:55266:56183 [5] NCCL INFO Using network IB ts-cbba87c5e7504a249f5127103d9ce40f-launcher:55262:56184 [1] NCCL INFO NET/IB : Using [0]mlx5_2:1/RoCE [RO]; OOB eth1:11.219.11.45<0> ts-cbba87c5e7504a249f5127103d9ce40f-launcher:55262:56184 [1] NCCL INFO Using network IB ts-cbba87c5e7504a249f5127103d9ce40f-launcher:55265:56185 [4] NCCL INFO NET/IB : Using [0]mlx5_2:1/RoCE [RO]; OOB eth1:11.219.11.45<0> ts-cbba87c5e7504a249f5127103d9ce40f-launcher:55265:56185 [4] NCCL INFO Using network IB ts-cbba87c5e7504a249f5127103d9ce40f-launcher:55263:56186 [2] NCCL INFO NET/IB : Using [0]mlx5_2:1/RoCE [RO]; OOB eth1:11.219.11.45<0> ts-cbba87c5e7504a249f5127103d9ce40f-launcher:55263:56186 [2] NCCL INFO Using network IB ts-cbba87c5e7504a249f5127103d9ce40f-launcher:55267:56189 [6] NCCL INFO NET/IB : Using [0]mlx5_2:1/RoCE [RO]; OOB eth1:11.219.11.45<0> ts-cbba87c5e7504a249f5127103d9ce40f-launcher:55267:56189 [6] NCCL INFO Using network IB ts-cbba87c5e7504a249f5127103d9ce40f-launcher:55264:55264 [3] NCCL INFO cudaDriverVersion 11070 ts-cbba87c5e7504a249f5127103d9ce40f-launcher:55264:55264 [3] NCCL INFO Bootstrap : Using eth1:11.219.11.45<0> ts-cbba87c5e7504a249f5127103d9ce40f-launcher:55264:55264 [3] NCCL INFO NET/Plugin : No plugin found (libnccl-net.so), using internal implementation ts-cbba87c5e7504a249f5127103d9ce40f-launcher:55268:55268 [7] NCCL INFO cudaDriverVersion 11070 ts-cbba87c5e7504a249f5127103d9ce40f-launcher:55268:55268 [7] NCCL INFO Bootstrap : Using eth1:11.219.11.45<0> ts-cbba87c5e7504a249f5127103d9ce40f-launcher:55264:56194 [3] NCCL INFO NET/IB : Using [0]mlx5_2:1/RoCE [RO]; OOB eth1:11.219.11.45<0> ts-cbba87c5e7504a249f5127103d9ce40f-launcher:55264:56194 [3] NCCL INFO Using network IB ts-cbba87c5e7504a249f5127103d9ce40f-launcher:55268:55268 [7] NCCL INFO NET/Plugin : No plugin found (libnccl-net.so), using internal implementation ts-cbba87c5e7504a249f5127103d9ce40f-launcher:55268:56196 [7] NCCL INFO NET/IB : Using [0]mlx5_2:1/RoCE [RO]; OOB eth1:11.219.11.45<0> ts-cbba87c5e7504a249f5127103d9ce40f-launcher:55268:56196 [7] NCCL INFO Using network IB g network IB ts-cbba87c5e7504a249f5127103d9ce40f-worker-1:97055:97943 [4] NCCL INFO NET/IB : Using [0]mlx5_2:1/RoCE [RO]; OOB eth1:11.215.64.140<0> ts-cbba87c5e7504a249f5127103d9ce40f-worker-1:97055:97943 [4] NCCL INFO Using network IB ts-cbba87c5e7504a249f5127103d9ce40f-worker-1:97057:97940 [6] NCCL INFO NET/IB : Using [0]mlx5_2:1/RoCE [RO]; OOB eth1:11.215.64.140<0> ts-cbba87c5e7504a249f5127103d9ce40f-worker-1:97057:97940 [6] NCCL INFO Using network IB ts-cbba87c5e7504a249f5127103d9ce40f-worker-1:97053:97944 [2] NCCL INFO NET/IB : Using [0]mlx5_2:1/RoCE [RO]; OOB eth1:11.215.64.140<0> ts-cbba87c5e7504a249f5127103d9ce40f-worker-1:97053:97944 [2] NCCL INFO Using network IB 01/28/2024 04:20:18 - WARNING - __main__ - Process rank: 7, device: cuda:7, n_gpu: 1distributed training: True, 16-bits training: False 01/28/2024 04:20:18 - WARNING - __main__ - Process rank: 5, device: cuda:5, n_gpu: 1distributed training: True, 16-bits training: False /apdcephfs/share_733425/vinnylywang/jianhuipang/llama2_sft/envs/lib/python3.8/site-packages/datasets/load.py:2089: FutureWarning: 'use_auth_token' was deprecated in favor of 'token' in version 2.14.0 and will be removed in 3.0.0. You can remove this warning by passing 'token=None' instead. warnings.warn( /apdcephfs/share_733425/vinnylywang/jianhuipang/llama2_sft/envs/lib/python3.8/site-packages/datasets/load.py:2089: FutureWarning: 'use_auth_token' was deprecated in favor of 'token' in version 2.14.0 and will be removed in 3.0.0. You can remove this warning by passing 'token=None' instead. warnings.warn( 01/28/2024 04:20:18 - WARNING - __main__ - Process rank: 3, device: cuda:3, n_gpu: 1distributed training: True, 16-bits training: False 01/28/2024 04:20:18 - WARNING - __main__ - Process rank: 0, device: cuda:0, n_gpu: 1distributed training: True, 16-bits training: False 01/28/2024 04:20:18 - INFO - __main__ - Training/evaluation parameters TrainingArguments( _n_gpu=1, adafactor=False, adam_beta1=0.9, adam_beta2=0.999, adam_epsilon=1e-08, auto_find_batch_size=False, bf16=True, bf16_full_eval=True, data_seed=None, dataloader_drop_last=False, dataloader_num_workers=8, dataloader_pin_memory=True, ddp_bucket_cap_mb=None, ddp_find_unused_parameters=None, ddp_timeout=72000, debug=[], deepspeed=/apdcephfs/share_733425/vinnylywang/jianhuipang/llama2_sft/train/deepspeed_config_bf16.json, disable_tqdm=False, do_eval=False, do_predict=False, do_train=True, eval_accumulation_steps=None, eval_delay=0, eval_steps=None, evaluation_strategy=no, fp16=False, fp16_backend=auto, fp16_full_eval=False, fp16_opt_level=O1, fsdp=[], fsdp_config={'fsdp_min_num_params': 0, 'xla': False, 'xla_fsdp_grad_ckpt': False}, fsdp_min_num_params=0, fsdp_transformer_layer_cls_to_wrap=None, full_determinism=False, gradient_accumulation_steps=8, gradient_checkpointing=True, greater_is_better=None, group_by_length=False, half_precision_backend=auto, hub_model_id=None, hub_private_repo=False, hub_strategy=every_save, hub_token=, ignore_data_skip=False, include_inputs_for_metrics=False, jit_mode_eval=False, label_names=None, label_smoothing_factor=0.0, learning_rate=2e-05, length_column_name=length, load_best_model_at_end=False, local_rank=0, log_level=passive, log_level_replica=warning, log_on_each_node=True, logging_dir=./checkpoints_ct/ac/allm-ac-7b/runs/Jan28_04-18-53_ts-cbba87c5e7504a249f5127103d9ce40f-worker-0, logging_first_step=False, logging_nan_inf_filter=True, logging_steps=1, logging_strategy=steps, lr_scheduler_type=constant_with_warmup, max_grad_norm=1.0, max_steps=-1, metric_for_best_model=None, mp_parameters=, no_cuda=False, num_train_epochs=1.0, optim=adamw_hf, optim_args=None, output_dir=./checkpoints_ct/ac/allm-ac-7b, overwrite_output_dir=True, past_index=-1, per_device_eval_batch_size=1, per_device_train_batch_size=2, prediction_loss_only=False, push_to_hub=False, push_to_hub_model_id=None, push_to_hub_organization=None, push_to_hub_token=, ray_scope=last, remove_unused_columns=True, report_to=['tensorboard'], resume_from_checkpoint=None, run_name=./checkpoints_ct/ac/allm-ac-7b, save_on_each_node=False, save_steps=500, save_strategy=steps, save_total_limit=1, seed=34, sharded_ddp=[], skip_memory_metrics=True, tf32=None, torch_compile=False, torch_compile_backend=None, torch_compile_mode=None, torchdynamo=None, tpu_metrics_debug=False, tpu_num_cores=None, use_ipex=False, use_legacy_prediction_loop=False, use_mps_device=False, warmup_ratio=0.0, warmup_steps=20, weight_decay=0.0, xpu_backend=None, ) /apdcephfs/share_733425/vinnylywang/jianhuipang/llama2_sft/envs/lib/python3.8/site-packages/datasets/load.py:2089: FutureWarning: 'use_auth_token' was deprecated in favor of 'token' in version 2.14.0 and will be removed in 3.0.0. You can remove this warning by passing 'token=None' instead. warnings.warn( /apdcephfs/share_733425/vinnylywang/jianhuipang/llama2_sft/envs/lib/python3.8/site-packages/datasets/load.py:2089: FutureWarning: 'use_auth_token' was deprecated in favor of 'token' in version 2.14.0 and will be removed in 3.0.0. You can remove this warning by passing 'token=None' instead. warnings.warn( 01/28/2024 04:20:18 - WARNING - __main__ - Process rank: 4, device: cuda:4, n_gpu: 1distributed training: True, 16-bits training: False 01/28/2024 04:20:18 - WARNING - __main__ - Process rank: 1, device: cuda:1, n_gpu: 1distributed training: True, 16-bits training: False /apdcephfs/share_733425/vinnylywang/jianhuipang/llama2_sft/envs/lib/python3.8/site-packages/datasets/load.py:2089: FutureWarning: 'use_auth_token' was deprecated in favor of 'token' in version 2.14.0 and will be removed in 3.0.0. You can remove this warning by passing 'token=None' instead. warnings.warn( /apdcephfs/share_733425/vinnylywang/jianhuipang/llama2_sft/envs/lib/python3.8/site-packages/datasets/load.py:2089: FutureWarning: 'use_auth_token' was deprecated in favor of 'token' in version 2.14.0 and will be removed in 3.0.0. You can remove this warning by passing 'token=None' instead. warnings.warn( 01/28/2024 04:20:18 - WARNING - __main__ - Process rank: 6, device: cuda:6, n_gpu: 1distributed training: True, 16-bits training: False /apdcephfs/share_733425/vinnylywang/jianhuipang/llama2_sft/envs/lib/python3.8/site-packages/datasets/load.py:2089: FutureWarning: 'use_auth_token' was deprecated in favor of 'token' in version 2.14.0 and will be removed in 3.0.0. You can remove this warning by passing 'token=None' instead. warnings.warn( 01/28/2024 04:20:18 - WARNING - __main__ - Process rank: 2, device: cuda:2, n_gpu: 1distributed training: True, 16-bits training: False /apdcephfs/share_733425/vinnylywang/jianhuipang/llama2_sft/envs/lib/python3.8/site-packages/datasets/load.py:2089: FutureWarning: 'use_auth_token' was deprecated in favor of 'token' in version 2.14.0 and will be removed in 3.0.0. You can remove this warning by passing 'token=None' instead. warnings.warn( No config specified, defaulting to the single config: red_pajama-data-1_t-sample/plain_text 01/28/2024 04:20:18 - INFO - datasets.builder - No config specified, defaulting to the single config: red_pajama-data-1_t-sample/plain_text Loading Dataset Infos from /apdcephfs/share_733425/vinnylywang/jianhuipang/hf_cache2/modules/datasets_modules/datasets/RedPajama-Data-1T-Sample/6ea3bc8ec2e84ec6d2df1930942e9028ace8c5b9d9143823cf911c50bbd92039 01/28/2024 04:20:18 - INFO - datasets.info - Loading Dataset Infos from /apdcephfs/share_733425/vinnylywang/jianhuipang/hf_cache2/modules/datasets_modules/datasets/RedPajama-Data-1T-Sample/6ea3bc8ec2e84ec6d2df1930942e9028ace8c5b9d9143823cf911c50bbd92039 Overwrite dataset info from restored data version if exists. 01/28/2024 04:20:18 - INFO - datasets.builder - Overwrite dataset info from restored data version if exists. Loading Dataset info from /apdcephfs/share_733425/vinnylywang/jianhuipang/hf_cache2/datasets/red_pajama-data-1_t-sample/plain_text/1.0.0/6ea3bc8ec2e84ec6d2df1930942e9028ace8c5b9d9143823cf911c50bbd92039 01/28/2024 04:20:18 - INFO - datasets.info - Loading Dataset info from /apdcephfs/share_733425/vinnylywang/jianhuipang/hf_cache2/datasets/red_pajama-data-1_t-sample/plain_text/1.0.0/6ea3bc8ec2e84ec6d2df1930942e9028ace8c5b9d9143823cf911c50bbd92039 Found cached dataset red_pajama-data-1_t-sample (/apdcephfs/share_733425/vinnylywang/jianhuipang/hf_cache2/datasets/red_pajama-data-1_t-sample/plain_text/1.0.0/6ea3bc8ec2e84ec6d2df1930942e9028ace8c5b9d9143823cf911c50bbd92039) 01/28/2024 04:20:18 - INFO - datasets.builder - Found cached dataset red_pajama-data-1_t-sample (/apdcephfs/share_733425/vinnylywang/jianhuipang/hf_cache2/datasets/red_pajama-data-1_t-sample/plain_text/1.0.0/6ea3bc8ec2e84ec6d2df1930942e9028ace8c5b9d9143823cf911c50bbd92039) Loading Dataset info from /apdcephfs/share_733425/vinnylywang/jianhuipang/hf_cache2/datasets/red_pajama-data-1_t-sample/plain_text/1.0.0/6ea3bc8ec2e84ec6d2df1930942e9028ace8c5b9d9143823cf911c50bbd92039 01/28/2024 04:20:18 - INFO - datasets.info - Loading Dataset info from /apdcephfs/share_733425/vinnylywang/jianhuipang/hf_cache2/datasets/red_pajama-data-1_t-sample/plain_text/1.0.0/6ea3bc8ec2e84ec6d2df1930942e9028ace8c5b9d9143823cf911c50bbd92039 No config specified, defaulting to the single config: red_pajama-data-1_t-sample/plain_text 01/28/2024 04:20:18 - INFO - datasets.builder - No config specified, defaulting to the single config: red_pajama-data-1_t-sample/plain_text Loading Dataset Infos from /apdcephfs/share_733425/vinnylywang/jianhuipang/hf_cache2/modules/datasets_modules/datasets/RedPajama-Data-1T-Sample/6ea3bc8ec2e84ec6d2df1930942e9028ace8c5b9d9143823cf911c50bbd92039 01/28/2024 04:20:18 - INFO - datasets.info - Loading Dataset Infos from /apdcephfs/share_733425/vinnylywang/jianhuipang/hf_cache2/modules/datasets_modules/datasets/RedPajama-Data-1T-Sample/6ea3bc8ec2e84ec6d2df1930942e9028ace8c5b9d9143823cf911c50bbd92039 Overwrite dataset info from restored data version if exists. 01/28/2024 04:20:18 - INFO - datasets.builder - Overwrite dataset info from restored data version if exists. Loading Dataset info from /apdcephfs/share_733425/vinnylywang/jianhuipang/hf_cache2/datasets/red_pajama-data-1_t-sample/plain_text/1.0.0/6ea3bc8ec2e84ec6d2df1930942e9028ace8c5b9d9143823cf911c50bbd92039 01/28/2024 04:20:18 - INFO - datasets.info - Loading Dataset info from /apdcephfs/share_733425/vinnylywang/jianhuipang/hf_cache2/datasets/red_pajama-data-1_t-sample/plain_text/1.0.0/6ea3bc8ec2e84ec6d2df1930942e9028ace8c5b9d9143823cf911c50bbd92039 Found cached dataset red_pajama-data-1_t-sample (/apdcephfs/share_733425/vinnylywang/jianhuipang/hf_cache2/datasets/red_pajama-data-1_t-sample/plain_text/1.0.0/6ea3bc8ec2e84ec6d2df1930942e9028ace8c5b9d9143823cf911c50bbd92039) 01/28/2024 04:20:18 - INFO - datasets.builder - Found cached dataset red_pajama-data-1_t-sample (/apdcephfs/share_733425/vinnylywang/jianhuipang/hf_cache2/datasets/red_pajama-data-1_t-sample/plain_text/1.0.0/6ea3bc8ec2e84ec6d2df1930942e9028ace8c5b9d9143823cf911c50bbd92039) Loading Dataset info from /apdcephfs/share_733425/vinnylywang/jianhuipang/hf_cache2/datasets/red_pajama-data-1_t-sample/plain_text/1.0.0/6ea3bc8ec2e84ec6d2df1930942e9028ace8c5b9d9143823cf911c50bbd92039 01/28/2024 04:20:18 - INFO - datasets.info - Loading Dataset info from /apdcephfs/share_733425/vinnylywang/jianhuipang/hf_cache2/datasets/red_pajama-data-1_t-sample/plain_text/1.0.0/6ea3bc8ec2e84ec6d2df1930942e9028ace8c5b9d9143823cf911c50bbd92039 No config specified, defaulting to the single config: red_pajama-data-1_t-sample/plain_text 01/28/2024 04:20:19 - INFO - datasets.builder - No config specified, defaulting to the single config: red_pajama-data-1_t-sample/plain_text Loading Dataset Infos from /apdcephfs/share_733425/vinnylywang/jianhuipang/hf_cache2/modules/datasets_modules/datasets/RedPajama-Data-1T-Sample/6ea3bc8ec2e84ec6d2df1930942e9028ace8c5b9d9143823cf911c50bbd92039 01/28/2024 04:20:19 - INFO - datasets.info - Loading Dataset Infos from /apdcephfs/share_733425/vinnylywang/jianhuipang/hf_cache2/modules/datasets_modules/datasets/RedPajama-Data-1T-Sample/6ea3bc8ec2e84ec6d2df1930942e9028ace8c5b9d9143823cf911c50bbd92039 Overwrite dataset info from restored data version if exists. 01/28/2024 04:20:19 - INFO - datasets.builder - Overwrite dataset info from restored data version if exists. Loading Dataset info from /apdcephfs/share_733425/vinnylywang/jianhuipang/hf_cache2/datasets/red_pajama-data-1_t-sample/plain_text/1.0.0/6ea3bc8ec2e84ec6d2df1930942e9028ace8c5b9d9143823cf911c50bbd92039 01/28/2024 04:20:19 - INFO - datasets.info - Loading Dataset info from /apdcephfs/share_733425/vinnylywang/jianhuipang/hf_cache2/datasets/red_pajama-data-1_t-sample/plain_text/1.0.0/6ea3bc8ec2e84ec6d2df1930942e9028ace8c5b9d9143823cf911c50bbd92039 Found cached dataset red_pajama-data-1_t-sample (/apdcephfs/share_733425/vinnylywang/jianhuipang/hf_cache2/datasets/red_pajama-data-1_t-sample/plain_text/1.0.0/6ea3bc8ec2e84ec6d2df1930942e9028ace8c5b9d9143823cf911c50bbd92039) 01/28/2024 04:20:19 - INFO - datasets.builder - Found cached dataset red_pajama-data-1_t-sample (/apdcephfs/share_733425/vinnylywang/jianhuipang/hf_cache2/datasets/red_pajama-data-1_t-sample/plain_text/1.0.0/6ea3bc8ec2e84ec6d2df1930942e9028ace8c5b9d9143823cf911c50bbd92039) Loading Dataset info from /apdcephfs/share_733425/vinnylywang/jianhuipang/hf_cache2/datasets/red_pajama-data-1_t-sample/plain_text/1.0.0/6ea3bc8ec2e84ec6d2df1930942e9028ace8c5b9d9143823cf911c50bbd92039 01/28/2024 04:20:19 - INFO - datasets.info - Loading Dataset info from /apdcephfs/share_733425/vinnylywang/jianhuipang/hf_cache2/datasets/red_pajama-data-1_t-sample/plain_text/1.0.0/6ea3bc8ec2e84ec6d2df1930942e9028ace8c5b9d9143823cf911c50bbd92039 [INFO|configuration_utils.py:666] 2024-01-28 04:20:19,123 >> loading configuration file /apdcephfs/share_733425/vinnylywang/jianhuipang/opensourcellms/llama2/Llama-2-7b-hf/config.json [INFO|configuration_utils.py:720] 2024-01-28 04:20:19,125 >> Model config LlamaConfig { "_name_or_path": "/apdcephfs/share_733425/vinnylywang/jianhuipang/opensourcellms/llama2/Llama-2-7b-hf", "architectures": [ "LlamaForCausalLM" ], "bos_token_id": 1, "eos_token_id": 2, "hidden_act": "silu", "hidden_size": 4096, "initializer_range": 0.02, "intermediate_size": 11008, "max_position_embeddings": 4096, "model_type": "llama", "num_attention_heads": 32, "num_hidden_layers": 32, "num_key_value_heads": 32, "pad_token_id": 0, "pretraining_tp": 1, "rms_norm_eps": 1e-05, "rope_scaling": null, "tie_word_embeddings": false, "torch_dtype": "float16", "transformers_version": "4.28.0.dev0", "use_cache": true, "vocab_size": 32000 } 01/28/2024 04:20:19 - INFO - __main__ - Tokenizer_kwargs: {'cache_dir': None, 'use_fast': True, 'revision': 'main', 'use_auth_token': None} [INFO|tokenization_utils_base.py:1801] 2024-01-28 04:20:19,129 >> loading file tokenizer.model [INFO|tokenization_utils_base.py:1801] 2024-01-28 04:20:19,129 >> loading file added_tokens.json [INFO|tokenization_utils_base.py:1801] 2024-01-28 04:20:19,129 >> loading file special_tokens_map.json [INFO|tokenization_utils_base.py:1801] 2024-01-28 04:20:19,129 >> loading file tokenizer_config.json 01/28/2024 04:20:19 - INFO - __main__ - Loading checkpoints in dtype: None [INFO|modeling_utils.py:2395] 2024-01-28 04:20:19,146 >> loading weights file /apdcephfs/share_733425/vinnylywang/jianhuipang/opensourcellms/llama2/Llama-2-7b-hf/model.safetensors.index.json [INFO|modeling_utils.py:2487] 2024-01-28 04:20:19,146 >> Detected DeepSpeed ZeRO-3: activating zero.init() for this model [INFO|configuration_utils.py:575] 2024-01-28 04:20:19,152 >> Generate config GenerationConfig { "_from_model_config": true, "bos_token_id": 1, "eos_token_id": 2, "pad_token_id": 0, "transformers_version": "4.28.0.dev0" } ts-cbba87c5e7504a249f5127103d9ce40f-worker-0:18322:18322 [5] NCCL INFO cudaDriverVersion 11070 ts-cbba87c5e7504a249f5127103d9ce40f-worker-0:18322:18322 [5] NCCL INFO Bootstrap : Using eth1:11.218.9.169<0> ts-cbba87c5e7504a249f5127103d9ce40f-worker-0:18322:18322 [5] NCCL INFO NET/Plugin : No plugin found (libnccl-net.so), using internal implementation ts-cbba87c5e7504a249f5127103d9ce40f-worker-0:18322:19248 [5] NCCL INFO NET/IB : Using [0]mlx5_2:1/RoCE [RO]; OOB eth1:11.218.9.169<0> ts-cbba87c5e7504a249f5127103d9ce40f-worker-0:18322:19248 [5] NCCL INFO Using network IB ts-cbba87c5ets-cbba87c5e7504a249f5127103d9ce40f-launcher:55266:56183 [5] NCCL INFO Setting affinity for GPU 5 to ffffffff,ffff0000,00000000,ffffffff,ffff0000,00000000 ts-cbba87c5e7504a249f5127103d9ce40f-launcher:55265:56185 [4] NCCL INFO Setting affinity for GPU 4 to ffffffff,ffff0000,00000000,ffffffff,ffff0000,00000000 ts-cbba87c5e7504a249f5127103d9ce40f-launcher:55262:56184 [1] NCCL INFO Setting affinity for GPU 1 to ffff,ffffffff,00000000,0000ffff,ffffffff ts-cbba87c5e7504a249f5127103d9ce40f-launcher:55261:56182 [0] NCCL INFO Setting affinity for GPU 0 to ffff,ffffffff,00000000,0000ffff,ffffffff ts-cbba87c5e7504a249f5127103d9ce40f-launcher:55267:56189 [6] NCCL INFO Setting affinity for GPU 6 to ffffffff,ffff0000,00000000,ffffffff,ffff0000,00000000 ts-cbba87c5e7504a249f5127103d9ce40f-launcher:55268:56196 [7] NCCL INFO Setting affinity for GPU 7 to ffffffff,ffff0000,00000000,ffffffff,ffff0000,00000000 ts-cbba87c5e7504a249f5127103d9ce40f-launcher:55264:56194 [3] NCCL INFO Setting affinity for GPU 3 to ffff,ffffffff,00000000,0000ffff,ffffffff ts-cbba87c5e7504a249f5127103d9ce40f-launcher:55263:56186 [2] NCCL INFO Setting affinity for GPU 2 to ffff,ffffffff,00000000,0000ffff,ffffffff ts-cbba87c5e7504a249f5127103d9ce40f-launcher:55268:56196 [7] NCCL INFO Trees [0] 0/-1/-1->7->6 [1] 0/-1/-1->7->6 ts-cbba87c5e7504a249f5127103d9ce40f-launcher:55267:56189 [6] NCCL INFO Trees [0] 7/-1/-1->6->5 [1] 7/-1/-1->6->5 ts-cbba87c5e7504a249f5127103d9ce40f-launcher:55266:56183 [5] NCCL INFO Trees [0] 6/-1/-1->5->4 [1] 6/-1/-1->5->4 ts-cbba87c5e7504a249f5127103d9ce40f-launcher:55265:56185 [4] NCCL INFO Trees [0] 5/-1/-1->4->3 [1] 5/-1/-1->4->3 ts-cbba87c5e7504a249f5127103d9ce40f-launcher:55263:56186 [2] NCCL INFO Trees [0] 3/18/-1->2->-1 [1] 3/-1/-1->2->10 ts-cbba87c5e7504a249f5127103d9ce40f-launcher:55261:56182 [0] NCCL INFO Channel 00/02 : 0 3 10 15 14 13 12 9 8 11 18 23 22 21 20 17 16 19 26 31 ts-cbba87c5e7504a249f5127103d9ce40f-launcher:55264:56194 [3] NCCL INFO Trees [0] 4/-1/-1->3->2 [1] 4/-1/-1->3->2 ts-cbba87c5e7504a249f5127103d9ce40f-launcher:55261:56182 [0] NCCL INFO Channel 01/02 : 0 3 10 15 14 13 12 9 8 11 18 23 22 21 20 17 16 19 26 31 ts-cbba87c5e7504a249f5127103d9ce40f-launcher:55261:56182 [0] NCCL INFO Trees [0] 1/-1/-1->0->7 [1] 1/-1/-1->0->7 ts-cbba87c5e7504a249f5127103d9ce40f-launcher:55262:56184 [1] NCCL INFO Trees [0] -1/-1/-1->1->0 [1] -1/-1/-1->1->0 ts-cbba87c5e7504a249f5127103d9ce40f-launcher:55265:56185 [4] NCCL INFO Channel 00/0 : 4[93000] -> 1[13000] via P2P/IPC/read ts-cbba87c5e7504a249f5127103d9ce40f-launcher:55263:56186 [2] NCCL INFO Channel 00/0 : 2[4b000] -> 7[d0000] via P2P/IPC/read ts-cbba87c5e7504a249f5127103d9ce40f-launcher:55261:56182 [0] NCCL INFO Channel 00/0 : 0[e000] -> 3[51000] via P2P/IPC/read ts-cbba87c5e7504a249f5127103d9ce40f-launcher:55266:56183 [5] NCCL INFO Channel 00/0 : 5[99000] -> 4[93000] via P2P/IPC/read ts-cbba87c5e7504a249f5127103d9ce40f-launcher:55265:56185 [4] NCCL INFO Channel 01/0 : 4[93000] -> 1[13000] via P2P/IPC/read ts-cbba87c5e7504a249f5127103d9ce40f-launcher:55263:56186 [2] NCCL INFO Channel 01/0 : 2[4b000] -> 7[d0000] via P2P/IPC/read ts-cbba87c5e7504a249f5127103d9ce40f-launcher:55261:56182 [0] NCCL INFO Channel 01/0 : 0[e000] -> 3[51000] via P2P/IPC/read ts-cbba87c5e7504a249f5127103d9ce40f-launcher:55266:56183 [5] NCCL INFO Channel 01/0 : 5[99000] -> 4[93000] via P2P/IPC/read ts-cbba87c5e7504a249f5127103d9ce40f-launcher:55267:56189 [6] NCCL INFO Channel 00/0 : 6[cb000] -> 5[99000] via P2P/IPC/read ts-cbba87c5e7504a249f5127103d9ce40f-launcher:55267:56189 [6] NCCL INFO Channel 01/0 : 6[cb000] -> 5[99000] via P2P/IPC/read ts-cbba87c5e7504a249f5127103d9ce40f-launcher:55268:56196 [7] NCCL INFO Channel 00/0 : 7[d0000] -> 6[cb000] via P2P/IPC/read ts-cbba87c5e7504a249f5127103d9ce40f-launcher:55264:56194 [3] NCCL INFO NCCL_NET_GDR_READ set by environment to 1. ts-cbba87c5e7504a249f5127103d9ce40f-launcher:55262:56184 [1] NCCL INFO Channel 00/0 : 1[13000] -> 0[e000] via P2P/IPC/read ts-cbba87c5e7504a249f5127103d9ce40f-launcher:55268:56196 [7] NCCL INFO Channel 01/0 : 7[d0000] -> 6[cb000] via P2P/IPC/read ts-cbba87c5e7504a249f5127103d9ce40f-launcher:55262:56184 [1] NCCL INFO Channel 01/0 : 1[13000] -> 0[e000] via P2P/IPC/read ts-cbba87c5e7504a249f5127103d9ce40f-launcher:55268:56196 [7] NCCL INFO Connected all rings ts-cbba87c5e7504a249f5127103d9ce40f-launcher:55265:56185 [4] NCCL INFO Connected all rings ts-cbba87c5e7504a249f5127103d9ce40f-launcher:55262:56184 [1] NCCL INFO Connected all rings ts-cbba87c5e7504a249f5127103d9ce40f-launcher:55261:56182 [0] NCCL INFO Connected all rings ts-cbba87c5e7504a249f5127103d9ce40f-launcher:55261:56182 [0] NCCL INFO Channel 00/0 : 0[e000] -> 1[13000] via P2P/IPC/read ts-cbba87c5e7504a249f5127103d9ce40f-launcher:55267:56189 [6] NCCL INFO Connected all rings ts-cbba87c5e7504a249f5127103d9ce40f-launcher:55261:56182 [0] NCCL INFO Channel 01/0 : 0[e000] -> 1[13000] via P2P/IPC/read ts-cbba87c5e7504a249f5127103d9ce40f-launcher:55266:56183 [5] NCCL INFO Connected all rings ts-cbba87c5e7504a249f5127103d9ce40f-launcher:55265:56185 [4] NCCL INFO Channel 00/0 : 4[93000] -> 5[99000] via P2P/IPC/read ts-cbba87c5e7504a249f5127103d9ce40f-launcher:55265:56185 [4] NCCL INFO Channel 01/0 : 4[93000] -> 5[99000] via P2P/IPC/read ts-cbba87c5e7504a249f5127103d9ce40f-launcher:55267:56189 [6] NCCL INFO Channel 00/0 : 6[cb000] -> 7[d0000] via P2P/IPC/read ts-cbba87c5e7504a249f5127103d9ce40f-launcher:55262:56184 [1] NCCL INFO Connected all trees ts-cbba87c5e7504a249f5127103d9ce40f-launcher:55262:56184 [1] NCCL INFO threadThresholds 8/8/64 | 256/8/64 | 512 | 512 ts-cbba87c5e7504a249f5127103d9ce40f-launcher:55262:56184 [1] NCCL INFO 2 coll channels, 2 p2p channels, 2 p2p channels per peer ts-cbba87c5e7504a249f5127103d9ce40f-launcher:55266:56183 [5] NCCL INFO Channel 00/0 : 5[99000] -> 6[cb000] via P2P/IPC/read ts-cbba87c5e7504a249f5127103d9ce40f-launcher:55261:56182 [0] NCCL INFO Channel 00/0 : 0[e000] -> 7[d0000] via P2P/IPC/read ts-cbba87c5e7504a249f5127103d9ce40f-launcher:55267:56189 [6] NCCL INFO Channel 01/0 : 6[cb000] -> 7[d0000] via P2P/IPC/read ts-cbba87c5e7504a249f5127103d9ce40f-launcher:55266:56183 [5] NCCL INFO Channel 01/0 : 5[99000] -> 6[cb000] via P2P/IPC/read ts-cbba87c5e7504a249f5127103d9ce40f-launcher:55261:56182 [0] NCCL INFO Channel 01/0 : 0[e000] -> 7[d0000] via P2P/IPC/read ts-cbba87c5e7504a249f5127103d9ce40f-launcher:55267:56189 [6] NCCL INFO Connected all trees ts-cbba87c5e7504a249f5127103d9ce40f-launcher:55267:56189 [6] NCCL INFO threadThresholds 8/8/64 | 256/8/64 | 512 | 512 ts-cbba87c5e7504a249f5127103d9ce40f-launcher:55267:56189 [6] NCCL INFO 2 coll channels, 2 p2p channels, 2 p2p channels per peer ts-cbba87c5e7504a249f5127103d9ce40f-launcher:55266:56183 [5] NCCL INFO Connected all trees ts-cbba87c5e7504a249f5127103d9ce40f-launcher:55266:56183 [5] NCCL INFO threadThresholds 8/8/64 | 256/8/64 | 512 | 512 ts-cbba87c5e7504a249f5127103d9ce40f-launcher:55266:56183 [5] NCCL INFO 2 coll channels, 2 p2p channels, 2 p2p channels per peer ts-cbba87c5e7504a249f5127103d9ce40f-launcher:55268:56196 [7] NCCL INFO Channel 00/0 : 7[d0000] -> 0[e000] via P2P/IPC/read ts-cbba87c5e7504a249f5127103d9ce40f-launcher:55268:56196 [7] NCCL INFO Channel 01/0 : 7[d0000] -> 0[e000] via P2P/IPC/read ts-cbba87c5e7504a249f5127103d9ce40f-launcher:55261:56182 [0] NCCL INFO Connected all trees ts-cbba87c5e7504a249f5127103d9ce40f-launcher:55261:56182 [0] NCCL INFO threadThresholds 8/8/64 | 256/8/64 | 512 | 512 ts-cbba87c5e7504a249f5127103d9ce40f-launcher:55261:56182 [0] NCCL INFO 2 coll channels, 2 p2p channels, 2 p2p channels per peer ts-cbba87c5e7504a249f5127103d9ce40f-launcher:55268:56196 [7] NCCL INFO Connected all trees ts-cbba87c5e7504a249f5127103d9ce40f-launcher:55268:56196 [7] NCCL INFO threadThresholds 8/8/64 | 256/8/64 | 512 | 512 ts-cbba87c5e7504a249f5127103d9ce40f-launcher:55268:56196 [7] NCCL INFO 2 coll channels, 2 p2p channels, 2 p2p channels per peer ts-cbba87c5e7504a249f5127103d9ce40f-launcher:55264:56194 [3] NCCL INFO Channel 00/0 : 3[51000] -> 10[4b000] [send] via NET/IB/0/GDRDMA ts-cbba87c5e7504a249f5127103d9ce40f-launcher:55263:56186 [2] NCCL INFO Channel 00/0 : 27[51000] -> 2[4b000] [receive] via NET/IB/0/GDRDMA ts-cbba87c5e7504a249f5127103d9ce40f-launcher:55262:56184 [1] NCCL INFO NCCL_NET_GDR_READ set by environment to 1. ts-cbba87c5e7504a249f5127103d9ce40f-launcher:55267:56189 [6] NCCL INFO NCCL_NET_GDR_READ set by environment to 1. ts-cbba87c5e7504a249f5127103d9ce40f-launcher:55266:56183 [5] NCCL INFO NCCL_NET_GDR_READ set by environment to 1. ts-cbba87c5e7504a249f5127103d9ce40f-launcher:55268:56196 [7] NCCL INFO NCCL_NET_GDR_READ set by environment to 1. ts-cbba87c5e7504a249f5127103d9ce40f-launcher:55261:56182 [0] NCCL INFO NCCL_NET_GDR_READ set by environment to 1. ts-cbba87c5e7504a249f5127103d9ce40f-launcher:55263:56186 [2] NCCL INFO Channel 01/0 : 27[51000] -> 2[4b000] [receive] via NET/IB/0/GDRDMA ts-cbba87c5e7504a249f5127103d9ce40f-launcher:55264:56194 [3] NCCL INFO Channel 01/0 : 3[51000] -> 10[4b000] [send] via NET/IB/0/GDRDMA ts-cbba87c5e7504a249f5127103d9ce40f-launcher:55263:56308 [2] NCCL INFO NCCL_IB_GID_INDEX set by environment to 3. ts-cbba87c5e7504a249f5127103d9ce40f-launcher:55263:56308 [2] NCCL INFO NCCL_IB_SL set by environment to 3. ts-cbba87c5e7504a249f5127103d9ce40f-launcher:55264:56309 [3] NCCL INFO NCCL_IB_GID_INDEX set by environment to 3. ts-cbba87c5e7504a249f5127103d9ce40f-launcher:55264:56194 [3] NCCL INFO Connected all rings ts-cbba87c5e7504a249f5127103d9ce40f-launcher:55264:56194 [3] NCCL INFO Channel 00/0 : 3[51000] -> 4[93000] via P2P/IPC/read ts-cbba87c5e7504a249f5127103d9ce40f-launcher:55263:56186 [2] NCCL INFO Connected all rings ts-cbba87c5e7504a249f5127103d9ce40f-launcher:55263:56186 [2] NCCL INFO Channel 00/0 : 2[4b000] -> 3[51000] via P2P/IPC/read ts-cbba87c5e7504a249f5127103d9ce40f-launcher:55264:56194 [3] NCCL INFO Channel 01/0 : 3[51000] -> 4[93000] via P2P/IPC/read ts-cbba87c5e7504a249f5127103d9ce40f-launcher:55263:56186 [2] NCCL INFO Channel 01/0 : 2[4b000] -> 3[51000] via P2P/IPC/read ts-cbba87c5e7504a249f5127103d9ce40f-launcher:55263:56186 [2] NCCL INFO NCCL_NET_GDR_READ set by environment to 1. ts-cbba87c5e7504a249f5127103d9ce40f-launcher:55265:56185 [4] NCCL INFO Channel 00/0 : 4[93000] -> 3[51000] via P2P/IPC/read ts-cbba87c5e7504a249f5127103d9ce40f-launcher:55265:56185 [4] NCCL INFO Channel 01/0 : 4[93000] -> 3[51000] via P2P/IPC/read ts-cbba87c5e7504a249f5127103d9ce40f-launcher:55264:56194 [3] NCCL INFO Channel 00/0 : 3[51000] -> 2[4b000] via P2P/IPC/read ts-cbba87c5e7504a249f5127103d9ce40f-launcher:55264:56194 [3] NCCL INFO Channel 01/0 : 3[51000] -> 2[4b000] via P2P/IPC/read ts-cbba87c5e7504a249f5127103d9ce40f-launcher:55265:56185 [4] NCCL INFO Connected all trees ts-cbba87c5e7504a249f5127103d9ce40f-launcher:55265:56185 [4] NCCL INFO threadThresholds 8/8/64 | 256/8/64 | 512 | 512 ts-cbba87c5e7504a249f5127103d9ce40f-launcher:55265:56185 [4] NCCL INFO 2 coll channels, 2 p2p channels, 2 p2p channels per peer ts-cbba87c5e7504a249f5127103d9ce40f-launcher:55263:56186 [2] NCCL INFO Channel 01/0 : 2[4b000] -> 10[4b000] [send] via NET/IB/0/GDRDMA ts-cbba87c5e7504a249f5127103d9ce40f-launcher:55265:56185 [4] NCCL INFO NCCL_NET_GDR_READ set by environment to 1. ts-cbba87c5e7504a249f5127103d9ce40f-launcher:55263:56186 [2] NCCL INFO Channel 00/0 : 18[4b000] -> 2[4b000] [receive] via NET/IB/0/GDRDMA ts-cbba87c5e7504a249f5127103d9ce40f-launcher:55263:56186 [2] NCCL INFO Channel 00/0 : 2[4b000] -> 18[4b000] [send] via NET/IB/0/GDRDMA ts-cbba87c5e7504a249f5127103d9ce40f-launcher:55263:56186 [2] NCCL INFO Channel 01/0 : 10[4b000] -> 2[4b000] [receive] via NET/IB/0/GDRDMA ts-cbba87c5e7504a249f5127103d9ce40f-launcher:55263:56186 [2] NCCL INFO Connected all trees ts-cbba87c5e7504a249f5127103d9ce40f-launcher:55263:56186 [2] NCCL INFO threadThresholds 8/8/64 | 256/8/64 | 512 | 512 ts-cbba87c5e7504a249f5127103d9ce40f-launcher:55263:56186 [2] NCCL INFO 2 coll channels, 2 p2p channels, 2 p2p channels per peer ts-cbba87c5e7504a249f5127103d9ce40f-launcher:55264:56194 [3] NCCL INFO Connected all trees ts-cbba87c5e7504a249f5127103d9ce40f-launcher:55264:56194 [3] NCCL INFO threadThresholds 8/8/64 | 256/8/64 | 512 | 512 ts-cbba87c5e7504a249f5127103d9ce40f-launcher:55264:56194 [3] NCCL INFO 2 coll channels, 2 p2p channels, 2 p2p channels per peer ts-cbba87c5e7504a249f5127103d9ce40f-launcher:55263:56186 [2] NCCL INFO comm 0x469181d0 rank 2 nranks 32 cudaDev 2 busId 4b000 - Init COMPLETE ts-cbba87c5e7504a249f5127103d9ce40f-launcher:55266:56183 [5] NCCL INFO comm 0x442e5f00 rank 5 nranks 32 cudaDev 5 busId 99000 - Init COMPLETE ts-cbba87c5e7504a249f5127103d9ce40f-launcher:55268:56196 [7] NCCL INFO comm 0x452bcca0 rank 7 nranks 32 cudaDev 7 busId d0000 - Init COMPLETE ts-cbba87c5e7504a249f5127103d9ce40f-launcher:55262:56184 [1] NCCL INFO comm 0x439b1720 rank 1 nranks 32 cudaDev 1 busId 13000 - Init COMPLETE ts-cbba87c5e7504a249f5127103d9ce40f-launcher:55264:56194 [3] NCCL INFO comm 0x45233420 rank 3 nranks 32 cudaDev 3 busId 51000 - Init COMPLETE ts-cbba87c5e7504a249f5127103d9ce40f-launcher:55261:56182 [0] NCCL INFO comm 0x44144920 rank 0 nranks 32 cudaDev 0 busId e000 - Init COMPLETE ts-cbba87c5e7504a249f5127103d9ce40f-launcher:55265:56185 [4] NCCL INFO comm 0x45c38070 rank 4 nranks 32 cudaDev 4 busId 93000 - Init COMPLETE ts-cbba87c5e7504a249f5127103d9ce40f-launcher:55267:56189 [6] NCCL INFO comm 0x42f51010 rank 6 nranks 32 cudaDev 6 busId cb000 - Init COMPLETE ts-cbba87c5e7504a249f5127103d9ce40f-launcher:55264:56312 [3] NCCL INFO NCCL_IB_SL set by environment to 3. [2024-01-28 04:20:25,732] [INFO] [partition_parameters.py:347:__exit__] finished initializing model - num_params = 291, num_elems = 6.74B Loading checkpoint shards: 0%| | 0/2 [00:00 11[51000] [receive] via NET/IB/0/GDRDMA ts-cbba87c5e7504a249f5127103d9ce40f-worker-0:18319:19262 [2] NCCL INFO NCCL_NET_GDR_READ set by environment to 1. ts-cbba87c5e7504a249f5127103d9ce40f-worker-0:18319:19262 [2] NCCL INFO Channel 00/0 : 10[4b000] -> 19[51000] [send] via NET/IB/0/GDRDMA ts-cbba87c5e7504a249f5127103d9ce40f-worker-0:18319:19262 [2] NCCL INFO Channel 01/0 : 26[4b000] -> 10[4b000] [receive] via NET/IB/0/GDRDMA ts-cbba87c5e7504a249f5127103d9ce40f-worker-0:18319:19262 [2] NCCL INFO Channel 01/0 : 10[4b000] -> 26[4b000] [send] via NET/IB/0/GDRDMA ts-cbba87c5e7504a249f5127103d9ce40f-worker-0:18319:19262 [2] NCCL INFO Channel 00/0 : 19[51000] -> 10[4b000] [receive] via NET/IB/0/GDRDMA ts-cbba87c5e7504a249f5127103d9ce40f-worker-0:18319:19262 [2] NCCL INFO Channel 01/0 : 10[4b000] -> 2[4b000] [send] via NET/IB/0/GDRDMA ts-cbba87c5e7504a249f5127103d9ce40f-worker-0:18320:19268 [3] NCCL INFO NCCL_IB_SL set by environment to 3. ts-cbba87c5e7504a249f5127103d9ce40f-worker-0:18320:19257 [3] NCCL INFO Channel 00/0 : 11[51000] -> 10[4b000] via P2P/IPC/read ts-cbba87c5e7504a249f5127103d9ce40f-worker-0:18320:19257 [3] NCCL INFO Channel 01/0 : 11[51000] -> 10[4b000] via P2P/IPC/read ts-cbba87c5e7504a249f5127103d9ce40f-worker-0:18319:19262 [2] NCCL INFO Connected all trees ts-cbba87c5e7504a249f5127103d9ce40f-worker-0:18319:19262 [2] NCCL INFO threadThresholds 8/8/64 | 256/8/64 | 512 | 512 ts-cbba87c5e7504a249f5127103d9ce40f-worker-0:18319:19262 [2] NCCL INFO 2 coll channels, 2 p2p channels, 2 p2p channels per peer ts-cbba87c5e7504a249f5127103d9ce40f-worker-0:18321:19252 [4] NCCL INFO Connected all trees ts-cbba87c5e7504a249f5127103d9ce40f-worker-0:18321:19252 [4] NCCL INFO threadThresholds 8/8/64 | 256/8/64 | 512 | 512 ts-cbba87c5e7504a249f5127103d9ce40f-worker-0:18321:19252 [4] NCCL INFO 2 coll channels, 2 p2p channels, 2 p2p channels per peer ts-cbba87c5e7504a249f5127103d9ce40f-worker-0:18320:19257 [3] NCCL INFO Connected all trees ts-cbba87c5e7504a249f5127103d9ce40f-worker-0:18320:19257 [3] NCCL INFO threadThresholds 8/8/64 | 256/8/64 | 512 | 512 ts-cbba87c5e7504a249f5127103d9ce40f-worker-0:18320:19257 [3] NCCL INFO 2 coll channels, 2 p2p channels, 2 p2p channels per peer ts-cbba87c5e7504a249f5127103d9ce40f-worker-0:18321:19252 [4] NCCL INFO NCCL_NET_GDR_READ set by environment to 1. ts-cbba87c5e7504a249f5127103d9ce40f-worker-0:18324:19250 [7] NCCL INFO comm 0x45360190 rank 15 nranks 32 cudaDev 7 busId d0000 - Init COMPLETE ts-cbba87c5e7504a249f5127103d9ce40f-worker-0:18323:19256 [6] NCCL INFO comm 0x45313e80 rank 14 nranks 32 cudaDev 6 busId cb000 - Init COMPLETE ts-cbba87c5e7504a249f5127103d9ce40f-worker-0:18321:19252 [4] NCCL INFO comm 0x44bdd650 rank 12 nranks 32 cudaDev 4 busId 93000 - Init COMPLETE ts-cbba87c5e7504a249f5127103d9ce40f-worker-0:18322:19248 [5] NCCL INFO comm 0x4392f400 rank 13 nranks 32 cudaDev 5 busId 99000 - Init COMPLETE ts-cbba87c5e7504a249f5127103d9ce40f-worker-0:18320:19257 [3] NCCL INFO comm 0x459099e0 rank 11 nranks 32 cudaDev 3 busId 51000 - Init COMPLETE ts-cbba87c5e7504a249f5127103d9ce40f-worker-0:18319:19262 [2] NCCL INFO comm 0x43864dd0 rank 10 nranks 32 cudaDev 2 busId 4b000 - Init COMPLETE ts-cbba87c5e7504a249f5127103d9ce40f-worker-0:18317:19259 [0] NCCL INFO comm 0x45a6cba0 rank 8 nranks 32 cudts-cbba87c5e7504a249f5127103d9ce40f-worker-2:165568:166514 [3] NCCL INFO NCCL_IB_SL set by environment to 3. Loading checkpoint shards: 0%| | 0/2 [00:00> Using pad_tok[ERROR|tokenization_utils_b Loading checkpoint shards: 100%|██████████| 2/2 [00:39<00:00, 18.25s/it] Loading checkpoint shards: 100%|██████████| 2/2 [00:39<00:00, 19.61s/it] Loading checkpoint shards: 100%|██████████| 2/2 [00:39<00:00, 18.25s/it] Loading checkpoint shards: 100%|██████████| 2/2 [00:39<00:00, 19.61s/it] [ERROR|tokenization_utils_base.py:1042] 2024-01-28 04:21:05,035 >> Using pad_token, but it is not set yet. [ERROR|tokenization_utils_base.py:1042] 2024-01-28 04:21:05,038 >> Using pad_token, but it is not set yet. Loading checkpoint shards: 100%|██████████| 2/2 [00:39<00:00, 18.25s/it] Loading checkpoint shards: 100%|██████████| 2/2 [00:39<00:00, 19.62s/it] [ERROR|tokenization_utils_base.py:1042] 2024-01-28 04:21:05,053 >> Using pad_token, but it is not set yet. Loading checkpoint shards: 100%|██████████| 2/2 [00:39<00:00, 18.27s/it] Loading checkpoint shards: 100%|██████████| 2/2 [00:39<00:00, 19.64s/it] Loading checkpoint shards: 100%|██████████| 2/2 [00:39<00:00, 18.27s/it] Loading checkpoint shards: 100%|██████████| 2/2 [00:39<00:00, 19.64s/it] [ERROR|tokenization_utils_base.py:1042] 2024-01-28 04:21:05,098 >> Using pad_token, but it is not set yet. [ERROR|tokenization_utils_base.py:1042] 2024-01-28 04:21:05,099 >> Using pad_token, but it is not set yet. Loading checkpoint shards: 100%|██████████| 2/2 [00:39<00:00, 18.25s/it] Loading checkpoint shards: 100%|██████████| 2/2 [00:39<00:00, 19.66s/it] [INFO|modeling_utils.py:3029] 2024-01-28 04:21:05,122 >> All model checkpoint weights were used when initializing LlamaForCausalLM. [INFO|modeling_utils.py:3037] 2024-01-28 04:21:05,122 >> All the weights of LlamaForCausalLM were initialized from the model checkpoint at /apdcephfs/share_733425/vinnylywang/jianhuipang/opensourcellms/llama2/Llama-2-7b-hf. If your task is similar to the task the model of the checkpoint was trained on, you can already use LlamaForCausalLM for predictions without further training. [INFO|configuration_utils.py:535] 2024-01-28 04:21:05,131 >> loading configuration file /apdcephfs/share_733425/vinnylywang/jianhuipang/opensourcellms/llama2/Llama-2-7b-hf/generation_config.json [INFO|configuration_utils.py:575] 2024-01-28 04:21:05,131 >> Generate config GenerationConfig { "bos_token_id": 1, "do_sample": true, "eos_token_id": 2, "max_length": 4096, "pad_token_id": 0, "temperature": 0.6, "top_p": 0.9, "transformers_version": "4.28.0.dev0" } [ERROR|tokenization_uti Loading checkpoint shards: 100%|██████████| 2/2 [00:39<00:00, 18.30s/it] Loading checkpoint shards: 100%|██████████| 2/2 [00:39<00:00, 19.67s/it] [ERROR|tokenization_utils_base.py:1042] 2024-01-28 04:21:05,153 >> Using pad_token, but it is not set yet. [INFO|tokenization_utils_base.py:907] 2024-01-28 04:21:07,295 >> Assigning to the eos_token key of the tokenizer [INFO|tokenization_utils_base.py:907] 2024-01-28 04:21:07,295 >> Assigning to the bos_token key of the tokenizer [INFO|tokenization_utils_base.py:907] 2024-01-28 04:21:07,295 >> Assigning to the unk_token key of the tokenizer [INFO|tokenization_utils.py:426] 2024-01-28 04:21:07,395 >> Adding to the vocabulary 01/28/2024 04:21:07 - INFO - __main__ - We have added new 1 token as an anchor Process #0 will write at /apdcephfs/share_733425/vinnylywang/jianhuipang/hf_cache2/datasets/red_pajama-data-1_t-sample/plain_text/1.0.0/6ea3bc8ec2e84ec6d2df1930942e9028ace8c5b9d9143823cf911c50bbd92039/cache-b2bcae33dc91ae3e_00000_of_00032.arrow 01/28/2024 04:21:09 - INFO - datasets.arrow_dataset - Process #0 will write at /apdcephfs/share_733425/vinnylywang/jianhuipang/hf_cache2/datasets/red_pajama-data-1_t-sample/plain_text/1.0.0/6ea3bc8ec2e84ec6d2df1930942e9028ace8c5b9d9143823cf911c50bbd92039/cache-b2bcae33dc91ae3e_00000_of_00032.arrow Process #1 will write at /apdcephfs/share_733425/vinnylywang/jianhuipang/hf_cache2/datasets/red_pajama-data-1_t-sample/plain_text/1.0.0/6ea3bc8ec2e84ec6d2df1930942e9028ace8c5b9d9143823cf911c50bbd92039/cache-b2bcae33dc91ae3e_00001_of_00032.arrow 01/28/2024 04:21:09 - INFO - datasets.arrow_dataset - Process #1 will write at /apdcephfs/share_733425/vinnylywang/jianhuipang/hf_cache2/datasets/red_pajama-data-1_t-sample/plain_text/1.0.0/6ea3bc8ec2e84ec6d2df1930942e9028ace8c5b9d9143823cf911c50bbd92039/cache-b2bcae33dc91ae3e_00001_of_00032.arrow Process #2 will write at /apdcephfs/share_733425/vinnylywang/jianhuipang/hf_cache2/datasets/red_pajama-data-1_t-sample/plain_text/1.0.0/6ea3bc8ec2e84ec6d2df1930942e9028ace8c5b9d9143823cf911c50bbd92039/cache-b2bcae33dc91ae3e_00002_of_00032.arrow 01/28/2024 04:21:09 - INFO - datasets.arrow_dataset - Process #2 will write at /apdcephfs/share_733425/vinnylywang/jianhuipang/hf_cache2/datasets/red_pajama-data-1_t-sample/plain_text/1.0.0/6ea3bc8ec2e84ec6d2df1930942e9028ace8c5b9d9143823cf911c50bbd92039/cache-b2bcae33dc91ae3e_00002_of_00032.arrow Process #3 will write at /apdcephfs/share_733425/vinnylywang/jianhuipang/hf_cache2/datasets/red_pajama-data-1_t-sample/plain_text/1.0.0/6ea3bc8ec2e84ec6d2df1930942e9028ace8c5b9d9143823cf911c50bbd92039/cache-b2bcae33dc91ae3e_00003_of_00032.arrow 01/28/2024 04:21:09 - INFO - datasets.arrow_dataset - Process #3 will write at /apdcephfs/share_733425/vinnylywang/jianhuipang/hf_cache2/datasets/red_pajama-data-1_t-sample/plain_text/1.0.0/6ea3bc8ec2e84ec6d2df1930942e9028ace8c5b9d9143823cf911c50bbd92039/cache-b2bcae33dc91ae3e_00003_of_00032.arrow Process #4 will write at /apdcephfs/share_733425/vinnylywang/jianhuipang/hf_cache2/datasets/red_pajama-data-1_t-sample/plain_text/1.0.0/6ea3bc8ec2e84ec6d2df1930942e9028ace8c5b9d9143823cf911c50bbd92039/cache-b2bcae33dc91ae3e_00004_of_00032.arrow 01/28/2024 04:21:09 - INFO - datasets.arrow_dataset - Process #4 will write at /apdcephfs/share_733425/vinnylywang/jianhuipang/hf_cache2/datasets/red_pajama-data-1_t-sample/plain_text/1.0.0/6ea3bc8ec2e84ec6d2df1930942e9028ace8c5b9d9143823cf911c50bbd92039/cache-b2bcae33dc91ae3e_00004_of_00032.arrow Process #5 will write at /apdcephfs/share_733425/vinnylywang/jianhuipang/hf_cache2/datasets/red_pajama-data-1_t-sample/plain_text/1.0.0/6ea3bc8ec2e84ec6d2df1930942e9028ace8c5b9d9143823cf911c50bbd92039/cache-b2bcae33dc91ae3e_00005_of_00032.arrow 01/28/2024 04:21:09 - INFO - datasets.arrow_dataset - Process #5 will write at /apdcephfs/share_733425/vinnylywang/jianhuipang/hf_cache2/datasets/red_pajama-data-1_t-sample/plain_text/1.0.0/6ea3bc8ec2e84ec6d2df1930942e9028ace8c5b9d9143823cf911c50bbd92039/cache-b2bcae33dc91ae3e_00005_of_00032.arrow Process #6 will write at /apdcephfs/share_733425/vinnylywang/jianhuipang/hf_cache2/datasets/red_pajama-data-1_t-sample/plain_text/1.0.0/6ea3bc8ec2e84ec6d2df1930942e9028ace8c5b9d9143823cf911c50bbd92039/cache-b2bcae33dc91ae3e_00006_of_00032.arrow 01/28/2024 04:21:09 - INFO - datasets.arrow_dataset - Process #6 will write at /apdcephfs/share_733425/vinnylywang/jianhuipang/hf_cache2/datasets/red_pajama-data-1_t-sample/plain_text/1.0.0/6ea3bc8ec2e84ec6d2df1930942e9028ace8c5b9d9143823cf911c50bbd92039/cache-b2bcae33dc91ae3e_00006_of_00032.arrow Process #7 will write at /apdcephfs/share_733425/vinnylywang/jianhuipang/hf_cache2/datasets/red_pajama-data-1_t-sample/plain_text/1.0.0/6ea3bc8ec2e84ec6d2df1930942e9028ace8c5b9d9143823cf911c50bbd92039/cache-b2bcae33dc91ae3e_00007_of_00032.arrow 01/28/2024 04:21:09 - INFO - datasets.arrow_dataset - Process #7 will write at /apdcephfs/share_733425/vinnylywang/jianhuipang/hf_cache2/datasets/red_pajama-data-1_t-sample/plain_text/1.0.0/6ea3bc8ec2e84ec6d2df1930942e9028ace8c5b9d9143823cf911c50bbd92039/cache-b2bcae33dc91ae3e_00007_of_00032.arrow Process #8 will write at /apdcephfs/share_733425/vinnylywang/jianhuipang/hf_cache2/datasets/red_pajama-data-1_t-sample/plain_text/1.0.0/6ea3bc8ec2e84ec6d2df1930942e9028ace8c5b9d9143823cf911c50bbd92039/cache-b2bcae33dc91ae3e_00008_of_00032.arrow 01/28/2024 04:21:09 - INFO - datasets.arrow_dataset - Process #8 will write at /apdcephfs/share_733425/vinnylywang/jianhuipang/hf_cache2/datasets/red_pajama-data-1_t-sample/plain_text/1.0.0/6ea3bc8ec2e84ec6d2df1930942e9028ace8c5b9d9143823cf911c50bbd92039/cache-b2bcae33dc91ae3e_00008_of_00032.arrow Process #9 will write at /apdcephfs/share_733425/vinnylywang/jianhuipang/hf_cache2/datasets/red_pajama-data-1_t-sample/plain_text/1.0.0/6ea3bc8ec2e84ec6d2df1930942e9028ace8c5b9d9143823cf911c50bbd92039/cache-b2bcae33dc91ae3e_00009_of_00032.arrow 01/28/2024 04:21:09 - INFO - datasets.arrow_dataset - Process #9 will write at /apdcephfs/share_733425/vinnylywang/jianhuipang/hf_cache2/datasets/red_pajama-data-1_t-sample/plain_text/1.0.0/6ea3bc8ec2e84ec6d2df1930942e9028ace8c5b9d9143823cf911c50bbd92039/cache-b2bcae33dc91ae3e_00009_of_00032.arrow Process #10 will write at /apdcephfs/share_733425/vinnylywang/jianhuipang/hf_cache2/datasets/red_pajama-data-1_t-sample/plain_text/1.0.0/6ea3bc8ec2e84ec6d2df1930942e9028ace8c5b9d9143823cf911c50bbd92039/cache-b2bcae33dc91ae3e_00010_of_00032.arrow 01/28/2024 04:21:09 - INFO - datasets.arrow_dataset - Process #10 will write at /apdcephfs/share_733425/vinnylywang/jianhuipang/hf_cache2/datasets/red_pajama-data-1_t-sample/plain_text/1.0.0/6ea3bc8ec2e84ec6d2df1930942e9028ace8c5b9d9143823cf911c50bbd92039/cache-b2bcae33dc91ae3e_00010_of_00032.arrow Process #11 will write at /apdcephfs/share_733425/vinnylywang/jianhuipang/hf_cache2/datasets/red_pajama-data-1_t-sample/plain_text/1.0.0/6ea3bc8ec2e84ec6d2df1930942e9028ace8c5b9d9143823cf911c50bbd92039/cache-b2bcae33dc91ae3e_00011_of_00032.arrow 01/28/2024 04:21:09 - INFO - datasets.arrow_dataset - Process #11 will write at /apdcephfs/share_733425/vinnylywang/jianhuipang/hf_cache2/datasets/red_pajama-data-1_t-sample/plain_text/1.0.0/6ea3bc8ec2e84ec6d2df1930942e9028ace8c5b9d9143823cf911c50bbd92039/cache-b2bcae33dc91ae3e_00011_of_00032.arrow Process #12 will write at /apdcephfs/share_733425/vinnylywang/jianhuipang/hf_cache2/datasets/red_pajama-data-1_t-sample/plain_text/1.0.0/6ea3bc8ec2e84ec6d2df1930942e9028ace8c5b9d9143823cf911c50bbd92039/cache-b2bcae33dc91ae3e_00012_of_00032.arrow 01/28/2024 04:21:09 - INFO - datasets.arrow_dataset - Process #12 will write at /apdcephfs/share_733425/vinnylywang/jianhuipang/hf_cache2/datasets/red_pajama-data-1_t-sample/plain_text/1.0.0/6ea3bc8ec2e84ec6d2df1930942e9028ace8c5b9d9143823cf911c50bbd92039/cache-b2bcae33dc91ae3e_00012_of_00032.arrow Process #13 will write at /apdcephfs/share_733425/vinnylywang/jianhuipang/hf_cache2/datasets/red_pajama-data-1_t-sample/plain_text/1.0.0/6ea3bc8ec2e84ec6d2df1930942e9028ace8c5b9d9143823cf911c50bbd92039/cache-b2bcae33dc91ae3e_00013_of_00032.arrow 01/28/2024 04:21:09 - INFO - datasets.arrow_dataset - Process #13 will write at /apdcephfs/share_733425/vinnylywang/jianhuipang/hf_cache2/datasets/red_pajama-data-1_t-sample/plain_text/1.0.0/6ea3bc8ec2e84ec6d2df1930942e9028ace8c5b9d9143823cf911c50bbd92039/cache-b2bcae33dc91ae3e_00013_of_00032.arrow Process #14 will write at /apdcephfs/share_733425/vinnylywang/jianhuipang/hf_cache2/datasets/red_pajama-data-1_t-sample/plain_text/1.0.0/6ea3bc8ec2e84ec6d2df1930942e9028ace8c5b9d9143823cf911c50bbd92039/cache-b2bcae33dc91ae3e_00014_of_00032.arrow 01/28/2024 04:21:09 - INFO - datasets.arrow_dataset - Process #14 will write at /apdcephfs/share_733425/vinnylywang/jianhuipang/hf_cache2/datasets/red_pajama-data-1_t-sample/plain_text/1.0.0/6ea3bc8ec2e84ec6d2df1930942e9028ace8c5b9d9143823cf911c50bbd92039/cache-b2bcae33dc91ae3e_00014_of_00032.arrow Process #15 will write at /apdcephfs/share_733425/vinnylywang/jianhuipang/hf_cache2/datasets/red_pajama-data-1_t-sample/plain_text/1.0.0/6ea3bc8ec2e84ec6d2df1930942e9028ace8c5b9d9143823cf911c50bbd92039/cache-b2bcae33dc91ae3e_00015_of_00032.arrow 01/28/2024 04:21:09 - INFO - datasets.arrow_dataset - Process #15 will write at /apdcephfs/share_733425/vinnylywang/jianhuipang/hf_cache2/datasets/red_pajama-data-1_t-sample/plain_text/1.0.0/6ea3bc8ec2e84ec6d2df1930942e9028ace8c5b9d9143823cf911c50bbd92039/cache-d68a794dd2c2b0b1_00015_of_00032.arrow Process #16 will write at /apdcephfs/share_733425/vinnylywang/jianhuipang/hf_cache2/datasets/red_pajama-data-1_t-sample/plain_text/1.0.0/6ea3bc8ec2e84ec6d2df1930942e9028ace8c5b9d9143823cf911c50bbd92039/cache-d68a794dd2c2b0b1_00016_of_00032.arrow 01/28/2024 04:21:09 - INFO - datasets.arrow_dataset - Process #16 will write at /apdcephfs/share_733425/vinnylywang/jianhuipang/hf_cache2/datasets/red_pajama-data-1_t-sample/plain_text/1.0.0/6ea3bc8ec2e84ec6d2df1930942e9028ace8c5b9d9143823cf911c50bbd92039/cache-d68a794dd2c2b0b1_00016_of_00032.arrow Process #17 will write at /apdcephfs/share_733425/vinnylywang/jianhuipang/hf_cache2/datasets/red_pajama-data-1_t-sample/plain_text/1.0.0/6ea3bc8ec2e84ec6d2df1930942e9028ace8c5b9d9143823cf911c50bbd92039/cache-d68a794dd2c2b0b1_00017_of_00032.arrow 01/28/2024 04:21:09 - INFO - datasets.arrow_dataset - Process #17 will write at /apdcephfs/share_733425/vinnylywang/jianhuipang/hf_cache2/datasets/red_pajama-data-1_t-sample/plain_text/1.0.0/6ea3bc8ec2e84ec6d2df1930942e9028ace8c5b9d9143823cf911c50bbd92039/cache-d68a794dd2c2b0b1_00017_of_00032.arrow Process #18 will write at /apdcephfs/share_733425/vinnylywang/jianhuipang/hf_cache2/datasets/red_pajama-data-1_t-sample/plain_text/1.0.0/6ea3bc8ec2e84ec6d2df1930942e9028ace8c5b9d9143823cf911c50bbd92039/cache-d68a794dd2c2b0b1_00018_of_00032.arrow 01/28/2024 04:21:09 - INFO - datasets.arrow_dataset - Process #18 will write at /apdcephfs/share_733425/vinnylywang/jianhuipang/hf_cache2/datasets/red_pajama-data-1_t-sample/plain_text/1.0.0/6ea3bc8ec2e84ec6d2df1930942e9028ace8c5b9d9143823cf911c50bbd92039/cache-d68a794dd2c2b0b1_00018_of_00032.arrow Process #19 will write at /apdcephfs/share_733425/vinnylywang/jianhuipang/hf_cache2/datasets/red_pajama-data-1_t-sample/plain_text/1.0.0/6ea3bc8ec2e84ec6d2df1930942e9028ace8c5b9d9143823cf911c50bbd92039/cache-d68a794dd2c2b0b1_00019_of_00032.arrow 01/28/2024 04:21:09 - INFO - datasets.arrow_dataset - Process #19 will write at /apdcephfs/share_733425/vinnylywang/jianhuipang/hf_cache2/datasets/red_pajama-data-1_t-sample/plain_text/1.0.0/6ea3bc8ec2e84ec6d2df1930942e9028ace8c5b9d9143823cf911c50bbd92039/cache-d68a794dd2c2b0b1_00019_of_00032.arrow Process #20 will write at /apdcephfs/share_733425/vinnylywang/jianhuipang/hf_cache2/datasets/red_pajama-data-1_t-sample/plain_text/1.0.0/6ea3bc8ec2e84ec6d2df1930942e9028ace8c5b9d9143823cf911c50bbd92039/cache-d68a794dd2c2b0b1_00020_of_00032.arrow 01/28/2024 04:21:09 - INFO - datasets.arrow_dataset - Process #20 will write at /apdcephfs/share_733425/vinnylywang/jianhuipang/hf_cache2/datasets/red_pajama-data-1_t-sample/plain_text/1.0.0/6ea3bc8ec2e84ec6d2df1930942e9028ace8c5b9d9143823cf911c50bbd92039/cache-d68a794dd2c2b0b1_00020_of_00032.arrow Process #21 will write at /apdcephfs/share_733425/vinnylywang/jianhuipang/hf_cache2/datasets/red_pajama-data-1_t-sample/plain_text/1.0.0/6ea3bc8ec2e84ec6d2df1930942e9028ace8c5b9d9143823cf911c50bbd92039/cache-d68a794dd2c2b0b1_00021_of_00032.arrow 01/28/2024 04:21:09 - INFO - datasets.arrow_dataset - Process #21 will write at /apdcephfs/share_733425/vinnylywang/jianhuipang/hf_cache2/datasets/red_pajama-data-1_t-sample/plain_text/1.0.0/6ea3bc8ec2e84ec6d2df1930942e9028ace8c5b9d9143823cf911c50bbd92039/cache-d68a794dd2c2b0b1_00021_of_00032.arrow Process #22 will write at /apdcephfs/share_733425/vinnylywang/jianhuipang/hf_cache2/datasets/red_pajama-data-1_t-sample/plain_text/1.0.0/6ea3bc8ec2e84ec6d2df1930942e9028ace8c5b9d9143823cf911c50bbd92039/cache-d68a794dd2c2b0b1_00022_of_00032.arrow 01/28/2024 04:21:09 - INFO - datasets.arrow_dataset - Process #22 will write at /apdcephfs/share_733425/vinnylywang/jianhuipang/hf_cache2/datasets/red_pajama-data-1_t-sample/plain_text/1.0.0/6ea3bc8ec2e84ec6d2df1930942e9028ace8c5b9d9143823cf911c50bbd92039/cache-d68a794dd2c2b0b1_00022_of_00032.arrow Process #23 will write at /apdcephfs/share_733425/vinnylywang/jianhuipang/hf_cache2/datasets/red_pajama-data-1_t-sample/plain_text/1.0.0/6ea3bc8ec2e84ec6d2df1930942e9028ace8c5b9d9143823cf911c50bbd92039/cache-d68a794dd2c2b0b1_00023_of_00032.arrow 01/28/2024 04:21:09 - INFO - datasets.arrow_dataset - Process #23 will write at /apdcephfs/share_733425/vinnylywang/jianhuipang/hf_cache2/datasets/red_pajama-data-1_t-sample/plain_text/1.0.0/6ea3bc8ec2e84ec6d2df1930942e9028ace8c5b9d9143823cf911c50bbd92039/cache-d68a794dd2c2b0b1_00023_of_00032.arrow Process #24 will write at /apdcephfs/share_733425/vinnylywang/jianhuipang/hf_cache2/datasets/red_pajama-data-1_t-sample/plain_text/1.0.0/6ea3bc8ec2e84ec6d2df1930942e9028ace8c5b9d9143823cf911c50bbd92039/cache-d68a794dd2c2b0b1_00024_of_00032.arrow 01/28/2024 04:21:09 - INFO - datasets.arrow_dataset - Process #24 will write at /apdcephfs/share_733425/vinnylywang/jianhuipang/hf_cache2/datasets/red_pajama-data-1_t-sample/plain_text/1.0.0/6ea3bc8ec2e84ec6d2df1930942e9028ace8c5b9d9143823cf911c50bbd92039/cache-d68a794dd2c2b0b1_00024_of_00032.arrow Process #25 will write at /apdcephfs/share_733425/vinnylywang/jianhuipang/hf_cache2/datasets/red_pajama-data-1_t-sample/plain_text/1.0.0/6ea3bc8ec2e84ec6d2df1930942e9028ace8c5b9d9143823cf911c50bbd92039/cache-d68a794dd2c2b0b1_00025_of_00032.arrow 01/28/2024 04:21:09 - INFO - datasets.arrow_dataset - Process #25 will write at /apdcephfs/share_733425/vinnylywang/jianhuipang/hf_cache2/datasets/red_pajama-data-1_t-sample/plain_text/1.0.0/6ea3bc8ec2e84ec6d2df1930942e9028ace8c5b9d9143823cf911c50bbd92039/cache-d68a794dd2c2b0b1_00025_of_00032.arrow Process #26 will write at /apdcephfs/share_733425/vinnylywang/jianhuipang/hf_cache2/datasets/red_pajama-data-1_t-sample/plain_text/1.0.0/6ea3bc8ec2e84ec6d2df1930942e9028ace8c5b9d9143823cf911c50bbd92039/cache-d68a794dd2c2b0b1_00026_of_00032.arrow 01/28/2024 04:21:09 - INFO - datasets.arrow_dataset - Process #26 will write at /apdcephfs/share_733425/vinnylywang/jianhuipang/hf_cache2/datasets/red_pajama-data-1_t-sample/plain_text/1.0.0/6ea3bc8ec2e84ec6d2df1930942e9028ace8c5b9d9143823cf911c50bbd92039/cache-d68a794dd2c2b0b1_00026_of_00032.arrow Process #27 will write at /apdcephfs/share_733425/vinnylywang/jianhuipang/hf_cache2/datasets/red_pajama-data-1_t-sample/plain_text/1.0.0/6ea3bc8ec2e84ec6d2df1930942e9028ace8c5b9d9143823cf911c50bbd92039/cache-d68a794dd2c2b0b1_00027_of_00032.arrow 01/28/2024 04:21:09 - INFO - datasets.arrow_dataset - Process #27 will write at /apdcephfs/share_733425/vinnylywang/jianhuipang/hf_cache2/datasets/red_pajama-data-1_t-sample/plain_text/1.0.0/6ea3bc8ec2e84ec6d2df1930942e9028ace8c5b9d9143823cf911c50bbd92039/cache-d68a794dd2c2b0b1_00027_of_00032.arrow Process #28 will write at /apdcephfs/share_733425/vinnylywang/jianhuipang/hf_cache2/datasets/red_pajama-data-1_t-sample/plain_text/1.0.0/6ea3bc8ec2e84ec6d2df1930942e9028ace8c5b9d9143823cf911c50bbd92039/cache-d68a794dd2c2b0b1_00028_of_00032.arrow 01/28/2024 04:21:09 - INFO - datasets.arrow_dataset - Process #28 will write at /apdcephfs/share_733425/vinnylywang/jianhuipang/hf_cache2/datasets/red_pajama-data-1_t-sample/plain_text/1.0.0/6ea3bc8ec2e84ec6d2df1930942e9028ace8c5b9d9143823cf911c50bbd92039/cache-d68a794dd2c2b0b1_00028_of_00032.arrow Process #29 will write at /apdcephfs/share_733425/vinnylywang/jianhuipang/hf_cache2/datasets/red_pajama-data-1_t-sample/plain_text/1.0.0/6ea3bc8ec2e84ec6d2df1930942e9028ace8c5b9d9143823cf911c50bbd92039/cache-d68a794dd2c2b0b1_00029_of_00032.arrow 01/28/2024 04:21:09 - INFO - datasets.arrow_dataset - Process #29 will write at /apdcephfs/share_733425/vinnylywang/jianhuipang/hf_cache2/datasets/red_pajama-data-1_t-sample/plain_text/1.0.0/6ea3bc8ec2e84ec6d2df1930942e9028ace8c5b9d9143823cf911c50bbd92039/cache-d68a794dd2c2b0b1_00029_of_00032.arrow Process #30 will write at /apdcephfs/share_733425/vinnylywang/jianhuipang/hf_cache2/datasets/red_pajama-data-1_t-sample/plain_text/1.0.0/6ea3bc8ec2e84ec6d2df1930942e9028ace8c5b9d9143823cf911c50bbd92039/cache-d68a794dd2c2b0b1_00030_of_00032.arrow 01/28/2024 04:21:09 - INFO - datasets.arrow_dataset - Process #30 will write at /apdcephfs/share_733425/vinnylywang/jianhuipang/hf_cache2/datasets/red_pajama-data-1_t-sample/plain_text/1.0.0/6ea3bc8ec2e84ec6d2df1930942e9028ace8c5b9d9143823cf911c50bbd92039/cache-d68a794dd2c2b0b1_00030_of_00032.arrow Process #31 will write at /apdcephfs/share_733425/vinnylywang/jianhuipang/hf_cache2/datasets/red_pajama-data-1_t-sample/plain_text/1.0.0/6ea3bc8ec2e84ec6d2df1930942e9028ace8c5b9d9143823cf911c50bbd92039/cache-d68a794dd2c2b0b1_00031_of_00032.arrow 01/28/2024 04:21:09 - INFO - datasets.arrow_dataset - Process #31 will write at /apdcephfs/share_733425/vinnylywang/jianhuipang/hf_cache2/datasets/red_pajama-data-1_t-sample/plain_text/1.0.0/6ea3bc8ec2e84ec6d2df1930942e9028ace8c5b9d9143823cf911c50bbd92039/cache-d68a794dd2c2b0b1_00031_of_00032.arrow Spawning 32 processes 01/28/2024 04:21:10 - INFO - datasets.arrow_dataset - Spawning 32 processes Map (num_proc=32): 0%| | 0/930514 [00:00> Using cuda_amp half precision backend Map (num_proc=32): 0%| | 0/930514 [00:00 [2024-01-28 04:53:37,879] [INFO] [logging.py:96:log_dist] [Rank 0] Creating fp16 ZeRO stage 3 optimizer, MiCS is enabled False, Hierarchical params gather False [2024-01-28 04:53:37,879] [INFO] [logging.py:96:log_dist] [Rank 0] Creating torch.bfloat16 ZeRO stage 3 optimizer 37, 15717, 310, 970, 368, 3625, 25741, 2041, 515, 13, 29871, 322, 920, 756, 372, 3939, 975, 278, 4940, 29871, 29945, 29900, 2440, 29973, 32001, 320, 1643, 29912, 29878, 29939, 29901, 479, 397, 24974, 29913, 13, 29905, 355, 29912, 690, 2842, 12470, 29913, 13, 4806, 671, 408, 8783, 278, 320, 23066, 29950, 29914, 18871, 2651, 2036, 29912, 29879, 1332, 29875, 4569, 29906, 29900, 29896, 29955, 29913, 322, 27599, 515, 372, 13, 29906, 29889, 29906, 24464, 29905, 13007, 25741, 3190, 2347, 515, 29871, 29896, 29953, 29900, 7284, 29905, 13007, 9279, 322, 4148, 287, 491, 13, 29946, 29941, 7284, 29905, 13007, 15717, 2645, 278, 29871, 29896, 29929, 29955, 29896, 489, 29906, 29900, 29906, 29896, 931, 3785, 29889, 32001, 1334, 1737, 324, 542, 403, 18777, 304, 13, 29905, 14573, 14058, 4597, 1080, 29914, 3186, 12786, 29892, 773, 408, 18470, 4876, 4234, 775, 2246, 29899, 5563, 21904, 313, 617, 29911, 10249, 29879, 29897, 322, 29871, 13, 8921, 313, 4102, 29914, 4230, 29897, 2983, 9401, 411, 1024, 18822, 2820, 278, 3186, 29892, 322, 17998, 1283, 7224, 29871, 13, 1195, 287, 515, 9063, 15562, 29889, 32001, 1334, 1284, 10757, 310, 278, 4688, 8022, 749, 310, 4644, 6813, 297, 1722, 2752, 13, 20415, 29892, 2678, 8772, 491, 4092, 29889, 32001, 2860, 393, 3785, 29892, 278, 1737, 12122, 6894, 537, 29871, 13, 262, 970, 775, 756, 1063, 21003, 10231, 29889, 32001, 1334, 884, 12439, 8018, 15839, 528, 17741, 13, 12817, 304, 278, 1095, 310, 278, 8291, 6415, 29129, 322, 278, 7910, 310, 14137, 4631, 4135, 297, 13, 23369, 1705, 322, 4275, 14325, 29892, 408, 1532, 408, 310, 2545, 1664, 17292, 28342, 763, 25539, 1608, 322, 13, 25719, 10298, 4822, 10916, 313, 6727, 16783, 29914, 331, 16783, 467, 32001, 320, 26956, 29912, 1469, 20847, 3097, 5003, 32001, 319, 1634, 1414, 3577, 363, 445, 5650, 338, 3625, 515, 796, 264, 8144, 472, 13, 29905, 2271, 29912, 991, 597, 1867, 29875, 29889, 990, 29914, 29896, 29900, 29889, 29945, 29906, 29947, 29896, 29914, 2256, 8144, 29889, 29953, 29941, 29929, 29900, 29941, 29945, 29945, 29913, 2651, 2036, 29912, 3445, 1414, 29899, 5113, 1836, 32001, 320, 2042, 29912, 9662, 630, 5244, 29913, 13, 29905, 1643, 29912, 3471, 29901, 12817, 29913, 13, 13, 29933, 720, 4688, 322, 7786, 1736, 2651, 2036, 29912, 12443, 10578, 29906, 29900, 29900, 29945, 5062, 11235, 29892, 13260, 333, 29906, 29900, 29900, 29947, 29888, 2209, 3359, 29879, 29892, 13, 29871, 696, 7586, 29906, 29900, 29896, 29946, 7610, 345, 2941, 271, 24541, 29892, 697, 309, 29906, 29900, 29896, 29953, 311, 5365, 550, 332, 6950, 29913, 505, 2931, 1891, 278, 13, 479, 5275, 310, 12362, 29914, 6585, 7562, 18540, 313, 5800, 1799, 29897, 773, 320, 7278, 29912, 6734, 26946, 952, 1118, 13, 4716, 3867, 1880, 29899, 29567, 6089, 541, 526, 9078, 297, 2159, 313, 29906, 29899, 29945, 5940, 29968, 18777, 29897, 13, 392, 508, 367, 4768, 1463, 491, 5221, 424, 23460, 29889, 32001, 512, 29871, 29906, 29900, 29900, 29947, 2261, 801, 2681, 634, 394, 8770, 2036, 29912, 1646, 801, 2681, 29906, 29900, 29900, 29947, 479, 397, 24974, 29913, 18043, 263, 3031, 979, 13, 16961, 29899, 7052, 313, 1454, 278, 931, 29897, 6559, 373, 18322, 1799, 320, 7278, 29912, 479, 5275, 773, 1375, 292, 7047, 13, 29871, 28914, 313, 4345, 29934, 29897, 13698, 1836, 32001, 2688, 29537, 287, 278, 3978, 310, 29871, 29896, 5940, 29924, 17737, 29560, 13, 4746, 278, 7562, 2831, 479, 1404, 2566, 322, 611, 6504, 1051, 3190, 3145, 975, 278, 13, 29896, 29929, 29929, 29929, 489, 29906, 29900, 29900, 29945, 3785, 29892, 773, 408, 18470, 2472, 2788, 304, 1749, 29879, 29901, 4876, 21904, 13, 392, 17998, 1283, 7224, 29889, 32001, 450, 12399, 3785, 313, 29955, 2440, 29897, 297, 2651, 2036, 29912, 1646, 801, 2681, 29906, 29900, 29900, 29947, 479, 397, 24974, 29913, 338, 20511, 1135, 29871, 13, 5816, 338, 12399, 297, 278, 2198, 5650, 313, 29945, 29900, 2440, 29897, 322, 278, 848, 8974, 526, 29871, 13, 27489, 873, 1422, 29936, 411, 393, 297, 3458, 29892, 1749, 2582, 1510, 263, 10029, 7200, 14978, 310, 29871, 13, 15654, 273, 325, 13217, 29940, 2072, 3082, 20706, 29889, 32001, 7280, 29190, 936, 664, 515, 29871, 29906, 29900, 29896, 29900, 491, 14619, 29882, 371, 4099, 29894, 322, 13, 29950, 309, 1372, 2651, 2036, 29912, 29873, 19426, 371, 4099, 29894, 29906, 29900, 29896, 29900, 2209, 479, 5275, 29913, 4023, 29894, 2868, 1583, 29899, 7099, 433, 1127, 1737, 12122, 13, 2029, 800, 310, 25492, 15303, 8304, 3598, 1494, 1009, 12368, 29892, 13, 15914, 292, 2472, 363, 779, 14850, 4535, 29892, 29955, 29900, 5940, 29968, 25492, 4160, 29889, 32001, 319, 1407, 7786, 13, 1287, 2651, 2036, 29912, 21686, 29879, 29906, 29900, 29906, 29896, 2209, 479, 5275, 29913, 491, 399, 496, 29879, 634, 394, 13217, 5349, 1737, 324, 542, 630, 4203, 263, 7284, 13, 28712, 16046, 4160, 29892, 2534, 26869, 472, 3203, 29871, 29896, 29900, 29900, 25741, 1269, 29892, 322, 1058, 13, 1311, 29899, 7099, 8663, 14354, 373, 1009, 25492, 28723, 29889, 32001, 5806, 278, 6559, 338, 13, 3149, 29899, 262, 29899, 2230, 408, 310, 29871, 29906, 29900, 29906, 29896, 29892, 278, 15717, 7252, 1009, 1284, 886, 13, 351, 475, 303, 2651, 2036, 29912, 1646, 801, 2681, 29906, 29900, 29900, 29947, 479, 397, 24974, 29892, 1850, 29882, 371, 4099, 29894, 29906, 29900, 29896, 29900, 2209, 479, 5275, 29913, 304, 13, 18609, 675, 278, 14675, 310, 18322, 1799, 1737, 5275, 975, 278, 931, 15101, 845, 1862, 4586, 491, 13, 1552, 2211, 11898, 29889, 32001, 3831, 1965, 411, 3517, 29190, 936, 1736, 29892, 1749, 6559, 338, 1568, 7200, 6287, 5634, 29882, 5555, 13, 7054, 12339, 287, 29871, 29946, 29941, 7284, 29905, 13007, 15717, 310, 29871, 29906, 29889, 29906, 24464, 29905, 13007, 25741, 515, 29871, 29896, 29953, 29900, 7284, 29905, 13007, 13, 16418, 5634, 5426, 11267, 979, 975, 29871, 29945, 29900, 2440, 310, 970, 775, 20706, 3265, 1135, 13, 3149, 297, 931, 29892, 322, 884, 901, 2691, 29899, 3874, 1312, 313, 2541, 1629, 29899, 1609, 29899, 6360, 3803, 1070, 537, 975, 13, 1552, 8900, 3785, 467, 32001, 8108, 1189, 1711, 29892, 1749, 6559, 337, 3687, 373, 10079, 11264, 13, 3924, 313, 8257, 29903, 29897, 9063, 848, 3265, 1135, 7481, 29899, 7099, 433, 1127, 4423, 2472, 29889, 32001, 5901, 1736, 5634, 262, 3153, 278, 664, 491, 8432, 2651, 2036, 29912, 18386, 709, 29906, 29900, 29896, 29941, 2209, 29881, 24974, 29913, 13, 392, 29892, 901, 10325, 29892, 390, 579, 468, 29875, 634, 394, 8770, 2036, 29912, 29878, 579, 468, 29875, 29906, 29900, 29896, 29953, 479, 711, 3173, 29892, 13, 29871, 364, 579, 468, 29875, 29906, 29900, 29896, 29947, 479, 711, 3173, 29892, 544, 1648, 29906, 29900, 29906, 29896, 479, 468, 1581, 29881, 24974, 29913, 5634, 17532, 12399, 1737, 12122, 13, 29905, 7278, 29912, 29881, 24974, 322, 24003, 1118, 474, 29889, 29872, 1696, 278, 15834, 304, 607, 278, 3978, 310, 18322, 1799, 13, 17426, 6602, 1009, 11465, 1230, 14137, 14188, 29889, 32001, 512, 445, 664, 591, 2931, 1891, 1737, 12122, 6894, 537, 297, 970, 775, 363, 278, 937, 13, 2230, 472, 445, 6287, 29892, 1716, 297, 4958, 310, 17737, 29560, 322, 15500, 3785, 29889, 32001, 1334, 437, 13, 1333, 22002, 280, 278, 24003, 10696, 29892, 541, 3867, 29190, 936, 848, 322, 1284, 886, 393, 508, 367, 13, 280, 369, 4063, 304, 393, 1095, 408, 5434, 664, 29889, 32001, 320, 7278, 29912, 12756, 7047, 21639, 29913, 2651, 2036, 29912, 2276, 5824, 19982, 29906, 29900, 29900, 29955, 23705, 1338, 29893, 996, 29913, 338, 278, 13, 1491, 29899, 2671, 310, 7047, 21639, 393, 756, 29537, 287, 278, 18066, 267, 310, 21640, 13, 6734, 24771, 13149, 635, 29892, 3704, 278, 2702, 5932, 310, 920, 304, 5376, 13, 2541, 1737, 12122, 6894, 537, 2651, 2036, 29912, 13376, 303, 456, 29906, 29900, 29900, 29953, 10945, 3359, 29892, 1424, 29440, 29906, 29900, 29896, 29946, 23027, 5933, 1836, 32001, 3826, 3076, 2678, 278, 2198, 6559, 8128, 10757, 393, 508, 367, 1304, 29892, 297, 278, 13, 14940, 1206, 310, 970, 775, 322, 472, 263, 1407, 2919, 6287, 29892, 304, 11539, 607, 13, 14032, 4637, 310, 5534, 7047, 21639, 505, 9820, 484, 15774, 29889, 32001, 320, 2042, 29912, 4062, 3002, 29913, 13, 29905, 1643, 29912, 3471, 29901, 5696, 29913, 13, 13, 13, 29905, 1482, 361, 29905, 361, 29887, 798, 386, 1003, 29871, 320, 29887, 798, 386, 1003, 3009, 13, 29905, 361, 29887, 798, 386, 1003, 13, 29905, 463, 29912, 4532, 29913, 13, 29871, 320, 7313, 29961, 2103, 2013, 4914, 2103, 3199, 6360, 368, 29899, 2055, 1169, 29913, 13, 29871, 320, 6671, 29912, 12883, 368, 970, 25741, 975, 931, 313, 1188, 6287, 467, 13, 29913, 32001, 320, 1643, 29912, 1003, 29901, 29887, 798, 386, 29913, 13, 29905, 355, 29912, 4532, 29913, 13, 29905, 7241, 13, 13, 29905, 26956, 29912, 16390, 24541, 29913, 13, 13, 4806, 27387, 515, 320, 23066, 29950, 29914, 2651, 2036, 29912, 29879, 1332, 29899, 1516, 29878, 29906, 29900, 29896, 29929, 29899, 24713, 29913, 599, 25741, 3190, 2347, 2745, 320, 25832, 1964, 579, 1523, 2415, 2539, 6294, 32001, 2688, 5253, 304, 320, 14573, 5261, 1169, 22131, 29914, 25741, 29892, 5412, 491, 317, 15715, 29896, 15882, 29892, 4023, 29894, 2868, 515, 320, 25832, 1299, 7288, 5261, 1169, 797, 7068, 29914, 970, 9279, 6421, 515, 4655, 5849, 363, 2710, 313, 28712, 16046, 29892, 11786, 28632, 29892, 2992, 1846, 32001, 322, 3577, 28914, 313, 10251, 713, 29892, 10772, 2227, 29892, 405, 13427, 29892, 2992, 6250, 32001, 1876, 1169, 297, 278, 8783, 526, 491, 320, 25832, 6344, 2806, 943, 22131, 29914, 15717, 29892, 5412, 491, 779, 6990, 29938, 978, 29892, 4876, 4535, 5854, 29938, 11000, 29889, 32001, 450, 8783, 2996, 408, 1023, 1104, 1288, 6131, 29892, 697, 363, 25741, 322, 697, 363, 15717, 29892, 411, 278, 4642, 29371, 278, 7480, 3025, 263, 9117, 1820, 29889, 32001, 320, 361, 5426, 13, 9760, 1948, 297, 278, 9063, 1591, 3743, 278, 1494, 4235, 29901, 9063, 317, 15715, 29896, 15882, 29892, 4148, 322, 844, 5171, 5335, 342, 15092, 29892, 4148, 322, 844, 5171, 2893, 14903, 313, 20275, 16750, 278, 4148, 1591, 467, 32001, 450, 21578, 1546, 9063, 15717, 322, 9063, 2153, 2041, 515, 11786, 29892, 607, 6511, 304, 9063, 263, 1735, 4148, 287, 491, 4856, 1683, 29889, 32001, 1152, 445, 6559, 591, 21309, 373, 15717, 322, 17262, 9063, 2153, 29892, 408, 278, 4328, 1546, 278, 1023, 338, 451, 8018, 363, 1749, 5925, 5155, 322, 278, 5253, 310, 25741, 411, 263, 844, 5171, 916, 1135, 967, 4148, 338, 3480, 3473, 1821, 29889, 32001, 320, 7241, 13, 2831, 1269, 6251, 297, 278, 4148, 1591, 591, 505, 4148, 2989, 1024, 322, 4876, 408, 1023, 5004, 6031, 310, 10650, 6262, 29889, 32001, 1334, 6206, 2411, 15273, 1821, 470, 18325, 519, 2983, 393, 29901, 526, 451, 1602, 397, 519, 408, 18351, 29899, 29947, 3441, 25832, 6344, 2806, 943, 29934, 29885, 12283, 7099, 397, 519, 29914, 4148, 2983, 6206, 511, 526, 4876, 14157, 2012, 310, 2983, 3441, 25832, 6344, 2806, 943, 29934, 29885, 9823, 29914, 4954, 7039, 4907, 511, 5718, 310, 871, 9654, 4890, 3441, 25832, 6344, 2806, 943, 29934, 29885, 10358, 804, 23201, 1712, 901, 1135, 29871, 29896, 29900, 8958, 1661, 29899, 1026, 2153, 3441, 25832, 6344, 2806, 943, 29934, 29885, 12283, 15670, 23201, 526, 5520, 1135, 29871, 29896, 29900, 29900, 4890, 3441, 25832, 6344, 2806, 943, 29934, 29885, 12229, 549, 12495, 32001, 2860, 21166, 29892, 1048, 320, 25832, 6344, 2806, 943, 29925, 15273, 1821, 2052, 307, 29916, 29914, 15717, 3441, 25832, 6344, 2806, 943, 29925, 15273, 1821, 29925, 312, 29914, 310, 278, 2847, 8783, 29897, 9488, 363, 4340, 7418, 29889, 32001, 3940, 393, 278, 5253, 310, 970, 775, 25741, 313, 392, 15717, 29897, 11122, 297, 278, 13, 11228, 8783, 25088, 18709, 9247, 975, 13, 2230, 2651, 2036, 29912, 29879, 1332, 29899, 771, 854, 749, 29899, 331, 344, 1012, 361, 29887, 798, 386, 1003, 29892, 408, 4318, 363, 25741, 297, 13, 29905, 29907, 999, 29912, 1003, 29901, 29887, 798, 386, 1012, 2870, 29901, 515, 395, 29896, 29900, 29985, 29946, 29938, 25741, 297, 29871, 29896, 29929, 29955, 29896, 29892, 304, 395, 29896, 29900, 29985, 29953, 29938, 297, 29871, 29896, 29929, 29929, 29947, 29892, 304, 13, 284, 3242, 395, 29896, 29900, 29985, 29929, 29938, 297, 29871, 29906, 29900, 29906, 29900, 29905, 7241, 29889, 32001, 1094, 263, 17004, 278, 8900, 534, 1975, 10331, 304, 367, 901, 13, 13844, 297, 7786, 1602, 3076, 1135, 297, 29871, 29946, 29900, 29974, 1629, 29899, 1025, 6743, 29892, 2861, 304, 13964, 4586, 373, 13, 735, 1112, 9247, 7200, 23093, 29889, 32001, 320, 26956, 29912, 7999, 324, 10610, 29913, 13, 13, 29905, 463, 29912, 4532, 29913, 13, 29871, 320, 9525, 13, 29871, 320, 7313, 29961, 24049, 29892, 15450, 29922, 29953, 4912, 29871, 29953, 4912, 29871, 29900, 29871, 29900, 29892, 2103, 2013, 16292, 3199, 1491, 1727, 1080, 29899, 2470, 29913, 13, 29871, 320, 6671, 29912, 1576, 320, 14573, 14058, 4597, 1080, 29914, 3186, 12786, 1304, 408, 1737, 324, 10610, 22525, 5003, 32001, 320, 1643, 29912, 1003, 29901, 11526, 1958, 29913, 13, 29905, 355, 29912, 4532, 29913, 13, 13, 2887, 1737, 324, 10610, 22525, 591, 671, 11758, 3186, 12786, 10723, 515, 278, 3303, 18269, 1737, 359, 305, 2004, 2651, 2036, 29912, 348, 29896, 29929, 29929, 29929, 479, 359, 305, 2004, 1836, 32001, 1763, 4772, 2432, 3381, 491, 2919, 10916, 313, 29872, 29889, 29887, 1696, 7551, 470, 12710, 29897, 2629, 11758, 12786, 29892, 591, 19412, 322, 6219, 777, 12786, 2729, 373, 1737, 12122, 23203, 537, 322, 278, 19383, 310, 758, 331, 8946, 16375, 29769, 5680, 29892, 1316, 408, 19182, 4086, 29889, 32001, 320, 29907, 999, 29912, 1003, 29901, 11526, 1958, 29913, 3697, 278, 2186, 1051, 310, 320, 14573, 14058, 4597, 1080, 29914, 3186, 12786, 1304, 408, 1737, 324, 10610, 22525, 297, 445, 6559, 29889, 32001, 1879, 324, 10610, 310, 9063, 15717, 304, 3186, 12786, 3913, 278, 1023, 19595, 653, 13698, 9129, 297, 2651, 2036, 29912, 293, 344, 29899, 344, 275, 29899, 29906, 29900, 29906, 29906, 29899, 26098, 1118, 23359, 337, 13998, 2400, 29889, 32001, 450, 937, 697, 337, 3687, 373, 278, 4234, 775, 2246, 29899, 5563, 5354, 313, 617, 29911, 10249, 29897, 310, 4876, 14157, 23892, 515, 9063, 15562, 29892, 321, 29889, 29887, 1696, 320, 16266, 26139, 1341, 1118, 320, 16266, 26139, 582, 1118, 320, 16266, 26139, 18038, 1118, 2992, 29889, 32001, 1334, 4687, 515, 278, 306, 2190, 29909, 1051, 310, 13548, 2931, 21759, 29911, 10249, 29879, 2651, 2036, 29912, 6011, 29899, 29883, 312, 430, 29913, 322, 7522, 20545, 1269, 6590, 20123, 304, 263, 3646, 3186, 5120, 29889, 32001, 450, 1473, 1737, 324, 10610, 11043, 3913, 278, 17998, 9210, 310, 9063, 5335, 342, 15092, 313, 29872, 29889, 29887, 1696, 17998, 29899, 29900, 29945, 29901, 29900, 29900, 29897, 322, 4148, 2983, 304, 8161, 278, 1556, 5517, 3186, 5120, 310, 278, 9063, 4148, 29889, 32001, 1152, 1269, 17998, 9210, 591, 8161, 263, 1051, 310, 15878, 7600, 313, 13509, 29892, 2106, 29892, 470, 14278, 20123, 29897, 297, 278, 3186, 393, 29892, 472, 278, 931, 310, 393, 9063, 29892, 750, 393, 17998, 9210, 29936, 9063, 931, 338, 1820, 1244, 29892, 408, 4234, 17998, 1283, 7224, 13100, 975, 931, 2861, 304, 29431, 3620, 29889, 32001, 1763, 1207, 445, 3683, 3381, 591, 671, 278, 306, 2190, 29909, 931, 10640, 2566, 2651, 2036, 29912, 17559, 1272, 1836, 32001, 1987, 591, 3566, 304, 1269, 2058, 263, 8158, 393, 4332, 1973, 278, 4188, 22342, 393, 263, 2183, 4148, 1024, 338, 17443, 310, 372, 29889, 32001, 1763, 445, 1095, 591, 671, 278, 28297, 29890, 15451, 8783, 310, 278, 29511, 310, 278, 1556, 3619, 937, 322, 3942, 2983, 607, 29892, 439, 11427, 515, 2651, 2036, 29912, 1079, 29890, 799, 29899, 7039, 6177, 2802, 277, 12181, 4954, 16123, 2247, 278, 26368, 5528, 5084, 310, 363, 264, 1280, 322, 269, 595, 1280, 7371, 515, 263, 2566, 310, 320, 1949, 29912, 29946, 29871, 29900, 29946, 29946, 29871, 29945, 29946, 29953, 29871, 29929, 29941, 29947, 29913, 2305, 313, 29945, 29945, 29889, 29945, 8958, 310, 8471, 2305, 297, 29871, 29906, 29900, 29896, 29946, 467, 32001, 1094, 310, 3839, 29871, 29906, 29900, 29896, 29929, 372, 18469, 320, 1949, 29912, 29906, 29955, 29871, 29953, 29953, 29906, 29871, 29947, 29900, 29896, 29913, 363, 264, 1280, 322, 320, 1949, 29912, 29906, 29955, 29871, 29906, 29900, 29953, 29871, 29947, 29906, 29896, 29913, 269, 595, 1280, 297, 29871, 29906, 29941, 29953, 24894, 8977, 1080, 29889, 4907, 29913, 32001, 1094, 297, 1749, 8783, 15717, 526, 2989, 1024, 6031, 313, 29878, 1624, 1135, 6219, 491, 937, 29914, 11922, 1024, 511, 591, 937, 5993, 675, 2983, 313, 1609, 1999, 1331, 322, 1206, 3620, 29897, 322, 769, 16280, 5375, 18897, 297, 1716, 937, 322, 3942, 2983, 10868, 8857, 29889, 32001, 1152, 1269, 1543, 1476, 297, 1024, 8857, 591, 22932, 278, 2058, 4665, 29905, 6661, 1333, 24212, 8875, 491, 278, 1024, 10868, 304, 4017, 263, 5645, 393, 338, 29839, 304, 278, 1353, 310, 12407, 24638, 393, 1024, 313, 6979, 29897, 297, 278, 2702, 2058, 29889, 32001, 320, 6661, 1333, 300, 1062, 29912, 1762, 4017, 4665, 2025, 1338, 5634, 294, 278, 17837, 310, 4954, 6689, 4907, 338, 25745, 23724, 29901, 2989, 10916, 325, 13217, 29879, 29399, 310, 2919, 10916, 805, 9450, 2999, 931, 29920, 2873, 5634, 705, 671, 263, 29544, 310, 7601, 8974, 313, 29872, 29889, 29887, 1696, 5874, 28007, 511, 322, 1661, 29899, 16072, 6743, 313, 29872, 29889, 29887, 1696, 14109, 7456, 467, 29913, 32001, 1334, 2533, 445, 4377, 363, 599, 3161, 304, 4017, 263, 2058, 8158, 29892, 17140, 701, 411, 263, 1051, 310, 779, 6990, 29938, 6689, 29892, 8158, 4535, 5854, 29938, 11000, 29889, 32001, 1334, 769, 8877, 445, 1051, 491, 278, 3186, 5120, 393, 263, 2058, 14393, 304, 322, 2533, 278, 8158, 363, 599, 278, 7600, 297, 1269, 5120, 304, 4017, 385, 12463, 8158, 29892, 6590, 304, 278, 4188, 22342, 393, 278, 9063, 14393, 304, 263, 2183, 3186, 5120, 29889, 32001, 1334, 3566, 278, 6257, 9063, 408, 6421, 515, 278, 3186, 5120, 411, 278, 9939, 8158, 29889, 32001, 450, 4876, 29899, 6707, 11043, 9378, 414, 515, 278, 9078, 322, 443, 5521, 8362, 671, 310, 21759, 29911, 10249, 29879, 29901, 1556, 18777, 671, 10035, 323, 10249, 29879, 1316, 408, 320, 16266, 26139, 510, 1118, 320, 16266, 26139, 990, 1118, 470, 320, 16266, 26139, 1212, 1836, 32001, 12808, 445, 947, 451, 3799, 26018, 4822, 20542, 29901, 3148, 29899, 6707, 18777, 29892, 363, 1342, 29892, 671, 278, 320, 16266, 26139, 375, 29913, 21759, 29911, 10249, 1568, 901, 269, 2495, 290, 368, 1135, 1009, 7824, 6795, 20895, 29889, 32001, 1551, 278, 916, 1361, 278, 9210, 29914, 978, 29899, 6707, 11043, 337, 3687, 373, 278, 17998, 9210, 310, 278, 9063, 5335, 342, 15092, 29889, 32001, 16809, 304, 5780, 22920, 373, 13897, 731, 14340, 29892, 263, 2919, 1353, 310, 25741, 297, 278, 8783, 756, 385, 17998, 9210, 5186, 304, 5225, 29889, 32001, 910, 6602, 29879, 3109, 7786, 25741, 3441, 14573, 5261, 1169, 29911, 29999, 29999, 13985, 1349, 681, 392, 27418, 6478, 29914, 310, 29871, 29906, 29900, 29906, 29900, 29879, 25741, 505, 263, 5225, 9210, 29897, 1135, 9642, 6743, 3441, 14573, 5261, 1169, 29911, 29999, 29999, 13985, 1349, 681, 392, 29914, 297, 29871, 29906, 29900, 29900, 29900, 467, 32001, 1094, 263, 1121, 278, 9210, 29914, 978, 29899, 6707, 11043, 1033, 1095, 701, 6459, 292, 263, 2919, 6232, 310, 9642, 25741, 408, 4148, 287, 491, 11715, 18777, 29892, 322, 304, 263, 3109, 261, 15834, 4092, 550, 29889, 32001, 1763, 6795, 1438, 5626, 591, 14405, 278, 1023, 1737, 324, 10610, 13698, 4208, 491, 15399, 278, 9210, 29914, 978, 29899, 6707, 13698, 304, 599, 25741, 411, 263, 1661, 29899, 9171, 17998, 9210, 29892, 322, 278, 4876, 29899, 6707, 373, 304, 599, 916, 25741, 29889, 32001, 320, 2042, 29912, 12191, 322, 8565, 1558, 291, 29913, 13, 29905, 1643, 29912, 3471, 29901, 9902, 29913, 13, 13, 29905, 463, 29912, 4532, 4044, 13, 29871, 320, 9525, 13, 29871, 320, 7313, 29961, 2103, 2013, 16292, 3199, 1429, 287, 29889, 5140, 29913, 13, 29871, 320, 6671, 29912, 29934, 20819, 310, 25741, 313, 27215, 29897, 322, 6136, 15717, 313, 22503, 29897, 491, 3186, 10640, 975, 278, 29871, 29896, 29929, 29955, 29896, 489, 29906, 29900, 29906, 29900, 3785, 5003, 32001, 320, 9868, 29961, 14732, 3199, 7264, 287, 2594, 8727, 6445, 278, 3186, 10640, 364, 2219, 359, 363, 25741, 322, 15717, 975, 278, 29871, 29896, 29929, 29955, 29896, 489, 29906, 29900, 29906, 29900, 3785, 5003, 32001, 320, 1643, 29912, 1003, 29901, 9902, 29913, 13, 29905, 355, 29912, 4532, 4044, 13, 13, 13, 29871, 13, 1762, 1234, 320, 25029, 29912, 29878, 29939, 29901, 479, 397, 24974, 29913, 591, 22229, 278, 1353, 310, 25741, 322, 8359, 15717, 639, 1629, 322, 639, 3186, 10640, 29889, 32001, 1334, 2198, 278, 7625, 2582, 297, 320, 29907, 999, 29912, 1003, 29901, 9902, 29913, 408, 1023, 5096, 287, 2594, 24469, 29892, 6445, 1629, 368, 2867, 3204, 29879, 363, 25741, 322, 15717, 8307, 29889, 32001, 7569, 2594, 11524, 263, 1629, 322, 338, 8877, 287, 297, 269, 29399, 6445, 278, 9063, 29914, 8921, 11959, 363, 1269, 310, 278, 3186, 12786, 310, 320, 29907, 999, 29912, 1003, 29901, 11526, 1958, 29913, 297, 393, 1629, 29889, 32001, 1763, 4772, 714, 27801, 2861, 304, 805, 272, 26538, 17737, 29560, 29892, 297, 278, 4148, 8727, 591, 871, 2050, 15717, 2534, 26869, 472, 3203, 29871, 29945, 25741, 297, 263, 2183, 1629, 29889, 32001, 5806, 5366, 1747, 534, 1975, 297, 278, 24469, 6456, 393, 278, 3001, 3694, 310, 25741, 322, 15717, 6548, 18709, 9247, 975, 931, 29889, 32001, 10133, 363, 278, 937, 2440, 297, 278, 24469, 29892, 278, 1353, 310, 848, 3291, 297, 777, 3186, 12786, 508, 367, 14154, 2319, 29892, 411, 8178, 27721, 373, 278, 25806, 310, 534, 1975, 29889, 32001, 320, 26956, 29912, 7999, 12122, 6894, 537, 975, 931, 29913, 13, 13, 3563, 497, 29892, 278, 2498, 534, 355, 5692, 304, 367, 393, 278, 320, 4534, 29912, 479, 12122, 6894, 537, 297, 970, 775, 338, 10231, 6177, 4644, 6813, 322, 4092, 5136, 630, 1009, 4954, 24130, 749, 4907, 2745, 278, 7256, 310, 278, 29871, 29929, 29900], 'attention_mask': [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1], 'labels': [1, 320, 2042, 29912, 25898, 29913, 13, 29905, 1643, 29912, 3471, 29901, 23333, 29913, 13, 13, 29905, 7278, 29912, 29954, 1581, 6894, 537, 1118, 470, 901, 4049, 967, 10225, 727, 974, 29892, 4249, 27138, 304, 13, 20415, 5849, 14188, 756, 1063, 26606, 12399, 297, 7786, 2440, 29889, 32001, 512, 13, 1595, 16311, 29892, 278, 10122, 310, 29892, 9545, 310, 29892, 322, 6795, 1004, 25414, 363, 320, 7278, 29912, 26098, 13, 29871, 24003, 29913, 297, 12362, 29914, 6585, 7562, 18540, 313, 5800, 1799, 29897, 505, 4520, 263, 3287, 310, 8570, 13, 957, 278, 4940, 316, 6332, 2651, 2036, 29912, 29881, 16093, 29906, 29900, 29900, 29947, 29888, 2209, 3359, 29879, 29892, 3855, 5871, 29906, 29900, 29896, 29900, 29895, 311, 29893, 2770, 29892, 13, 29871, 1055, 29888, 375, 29906, 29900, 29896, 29906, 5041, 267, 29892, 413, 434, 305, 1358, 29906, 29900, 29896, 29906, 26098, 29888, 2209, 29892, 19723, 5475, 4979, 29906, 29900, 29896, 29946, 26098, 29892, 13, 29871, 697, 309, 29906, 29900, 29896, 29953, 311, 5365, 550, 332, 6950, 29892, 696, 7586, 29906, 29900, 29896, 29953, 29893, 2770, 7192, 2209, 29892, 1935, 15044, 29906, 29900, 29896, 29955, 26098, 29892, 13, 29871, 503, 562, 305, 3350, 492, 29906, 29900, 29906, 29896, 26098, 1836, 32001, 320, 7278, 29912, 7999, 12122, 6894, 537, 29913, 338, 373, 278, 916, 1361, 278, 13, 14380, 310, 6894, 537, 393, 380, 1567, 515, 27138, 297, 777, 5534, 6354, 6421, 13, 3166, 1422, 3186, 12786, 322, 4185, 1973, 29889, 32001, 26304, 6894, 537, 297, 18322, 1799, 756, 4520, 13774, 2217, 8570, 297, 21344, 368, 13, 13129, 29889, 32001, 512, 3153, 29892, 1550, 3031, 979, 18994, 29899, 6707, 322, 13, 3149, 29899, 262, 29899, 2230, 18350, 29899, 7052, 11898, 310, 278, 1737, 12122, 1677, 1144, 310, 18322, 1799, 13, 21570, 29560, 1863, 2651, 2036, 29912, 12443, 10578, 29906, 29900, 29900, 29945, 5062, 11235, 29892, 13260, 333, 29906, 29900, 29900, 29947, 29888, 2209, 3359, 29879, 29892, 13, 29871, 2594, 801, 2681, 29906, 29900, 29900, 29947, 479, 397, 24974, 29892, 1850, 29882, 371, 4099, 29894, 29906, 29900, 29896, 29900, 2209, 479, 5275, 29892, 696, 7586, 29906, 29900, 29896, 29946, 7610, 345, 2941, 271, 24541, 29892, 13, 29871, 281, 496, 29879, 29906, 29900, 29906, 29896, 2209, 479, 5275, 1118, 2919, 29899, 7052, 25579, 979, 11898, 310, 278, 1737, 12122, 13, 12574, 310, 18322, 1799, 17737, 29560, 526, 1603, 10225, 292, 29889, 32001, 10506, 263, 4323, 23378, 13, 18609, 2133, 723, 367, 5407, 304, 1871, 1602, 12112, 4475, 304, 5534, 13, 25431, 10907, 2651, 2036, 29912, 2276, 5824, 19982, 29906, 29900, 29900, 29955, 23705, 1338, 29893, 996, 29913, 322, 298, 8491, 16650, 583, 297, 278, 13, 19678, 15483, 313, 1806, 29897, 9999, 29892, 408, 1532, 408, 29126, 2114, 950, 2472, 13, 517, 278, 2553, 1078, 373, 278, 17407, 10879, 322, 5374, 3002, 310, 18322, 1799, 2820, 278, 3186, 29889, 32001, 320, 26956, 29912, 1323, 3224, 29879, 29913, 13, 13, 3047, 445, 664, 591, 29126, 304, 3802, 445, 17261, 491, 7512, 292, 320, 4534, 29912, 1552, 937, 13, 29871, 25579, 979, 6559, 310, 278, 1737, 12122, 3978, 310, 17737, 29560, 304, 970, 775, 13, 29871, 975, 29871, 29945, 29900, 2440, 5003, 32001, 26321, 29892, 591, 3867, 263, 758, 2576, 3821, 1234, 304, 278, 13, 23031, 292, 5925, 1139, 29901, 13, 29905, 463, 29912, 690, 2842, 12470, 29913, 13, 29871, 3645, 607, 3186, 12786, 437, 15717, 310, 970, 368, 3625, 25741, 2041, 515, 13, 29871, 322, 920, 756, 372, 3939, 975, 278, 4940, 29871, 29945, 29900, 2440, 29973, 32001, 320, 1643, 29912, 29878, 29939, 29901, 479, 397, 24974, 29913, 13, 29905, 355, 29912, 690, 2842, 12470, 29913, 13, 4806, 671, 408, 8783, 278, 320, 23066, 29950, 29914, 18871, 2651, 2036, 29912, 29879, 1332, 29875, 4569, 29906, 29900, 29896, 29955, 29913, 322, 27599, 515, 372, 13, 29906, 29889, 29906, 24464, 29905, 13007, 25741, 3190, 2347, 515, 29871, 29896, 29953, 29900, 7284, 29905, 13007, 9279, 322, 4148, 287, 491, 13, 29946, 29941, 7284, 29905, 13007, 15717, 2645, 278, 29871, 29896, 29929, 29955, 29896, 489, 29906, 29900, 29906, 29896, 931, 3785, 29889, 32001, 1334, 1737, 324, 542, 403, 18777, 304, 13, 29905, 14573, 14058, 4597, 1080, 29914, 3186, 12786, 29892, 773, 408, 18470, 4876, 4234, 775, 2246, 29899, 5563, 21904, 313, 617, 29911, 10249, 29879, 29897, 322, 29871, 13, 8921, 313, 4102, 29914, 4230, 29897, 2983, 9401, 411, 1024, 18822, 2820, 278, 3186, 29892, 322, 17998, 1283, 7224, 29871, 13, 1195, 287, 515, 9063, 15562, 29889, 32001, 1334, 1284, 10757, 310, 278, 4688, 8022, 749, 310, 4644, 6813, 297, 1722, 2752, 13, 20415, 29892, 2678, 8772, 491, 4092, 29889, 32001, 2860, 393, 3785, 29892, 278, 1737, 12122, 6894, 537, 29871, 13, 262, 970, 775, 756, 1063, 21003, 10231, 29889, 32001, 1334, 884, 12439, 8018, 15839, 528, 17741, 13, 12817, 304, 278, 1095, 310, 278, 8291, 6415, 29129, 322, 278, 7910, 310, 14137, 4631, 4135, 297, 13, 23369, 1705, 322, 4275, 14325, 29892, 408, 1532, 408, 310, 2545, 1664, 17292, 28342, 763, 25539, 1608, 322, 13, 25719, 10298, 4822, 10916, 313, 6727, 16783, 29914, 331, 16783, 467, 32001, 320, 26956, 29912, 1469, 20847, 3097, 5003, 32001, 319, 1634, 1414, 3577, 363, 445, 5650, 338, 3625, 515, 796, 264, 8144, 472, 13, 29905, 2271, 29912, 991, 597, 1867, 29875, 29889, 990, 29914, 29896, 29900, 29889, 29945, 29906, 29947, 29896, 29914, 2256, 8144, 29889, 29953, 29941, 29929, 29900, 29941, 29945, 29945, 29913, 2651, 2036, 29912, 3445, 1414, 29899, 5113, 1836, 32001, 320, 2042, 29912, 9662, 630, 5244, 29913, 13, 29905, 1643, 29912, 3471, 29901, 12817, 29913, 13, 13, 29933, 720, 4688, 322, 7786, 1736, 2651, 2036, 29912, 12443, 10578, 29906, 29900, 29900, 29945, 5062, 11235, 29892, 13260, 333, 29906, 29900, 29900, 29947, 29888, 2209, 3359, 29879, 29892, 13, 29871, 696, 7586, 29906, 29900, 29896, 29946, 7610, 345, 2941, 271, 24541, 29892, 697, 309, 29906, 29900, 29896, 29953, 311, 5365, 550, 332, 6950, 29913, 505, 2931, 1891, 278, 13, 479, 5275, 310, 12362, 29914, 6585, 7562, 18540, 313, 5800, 1799, 29897, 773, 320, 7278, 29912, 6734, 26946, 952, 1118, 13, 4716, 3867, 1880, 29899, 29567, 6089, 541, 526, 9078, 297, 2159, 313, 29906, 29899, 29945, 5940, 29968, 18777, 29897, 13, 392, 508, 367, 4768, 1463, 491, 5221, 424, 23460, 29889, 32001, 512, 29871, 29906, 29900, 29900, 29947, 2261, 801, 2681, 634, 394, 8770, 2036, 29912, 1646, 801, 2681, 29906, 29900, 29900, 29947, 479, 397, 24974, 29913, 18043, 263, 3031, 979, 13, 16961, 29899, 7052, 313, 1454, 278, 931, 29897, 6559, 373, 18322, 1799, 320, 7278, 29912, 479, 5275, 773, 1375, 292, 7047, 13, 29871, 28914, 313, 4345, 29934, 29897, 13698, 1836, 32001, 2688, 29537, 287, 278, 3978, 310, 29871, 29896, 5940, 29924, 17737, 29560, 13, 4746, 278, 7562, 2831, 479, 1404, 2566, 322, 611, 6504, 1051, 3190, 3145, 975, 278, 13, 29896, 29929, 29929, 29929, 489, 29906, 29900, 29900, 29945, 3785, 29892, 773, 408, 18470, 2472, 2788, 304, 1749, 29879, 29901, 4876, 21904, 13, 392, 17998, 1283, 7224, 29889, 32001, 450, 12399, 3785, 313, 29955, 2440, 29897, 297, 2651, 2036, 29912, 1646, 801, 2681, 29906, 29900, 29900, 29947, 479, 397, 24974, 29913, 338, 20511, 1135, 29871, 13, 5816, 338, 12399, 297, 278, 2198, 5650, 313, 29945, 29900, 2440, 29897, 322, 278, 848, 8974, 526, 29871, 13, 27489, 873, 1422, 29936, 411, 393, 297, 3458, 29892, 1749, 2582, 1510, 263, 10029, 7200, 14978, 310, 29871, 13, 15654, 273, 325, 13217, 29940, 2072, 3082, 20706, 29889, 32001, 7280, 29190, 936, 664, 515, 29871, 29906, 29900, 29896, 29900, 491, 14619, 29882, 371, 4099, 29894, 322, 13, 29950, 309, 1372, 2651, 2036, 29912, 29873, 19426, 371, 4099, 29894, 29906, 29900, 29896, 29900, 2209, 479, 5275, 29913, 4023, 29894, 2868, 1583, 29899, 7099, 433, 1127, 1737, 12122, 13, 2029, 800, 310, 25492, 15303, 8304, 3598, 1494, 1009, 12368, 29892, 13, 15914, 292, 2472, 363, 779, 14850, 4535, 29892, 29955, 29900, 5940, 29968, 25492, 4160, 29889, 32001, 319, 1407, 7786, 13, 1287, 2651, 2036, 29912, 21686, 29879, 29906, 29900, 29906, 29896, 2209, 479, 5275, 29913, 491, 399, 496, 29879, 634, 394, 13217, 5349, 1737, 324, 542, 630, 4203, 263, 7284, 13, 28712, 16046, 4160, 29892, 2534, 26869, 472, 3203, 29871, 29896, 29900, 29900, 25741, 1269, 29892, 322, 1058, 13, 1311, 29899, 7099, 8663, 14354, 373, 1009, 25492, 28723, 29889, 32001, 5806, 278, 6559, 338, 13, 3149, 29899, 262, 29899, 2230, 408, 310, 29871, 29906, 29900, 29906, 29896, 29892, 278, 15717, 7252, 1009, 1284, 886, 13, 351, 475, 303, 2651, 2036, 29912, 1646, 801, 2681, 29906, 29900, 29900, 29947, 479, 397, 24974, 29892, 1850, 29882, 371, 4099, 29894, 29906, 29900, 29896, 29900, 2209, 479, 5275, 29913, 304, 13, 18609, 675, 278, 14675, 310, 18322, 1799, 1737, 5275, 975, 278, 931, 15101, 845, 1862, 4586, 491, 13, 1552, 2211, 11898, 29889, 32001, 3831, 1965, 411, 3517, 29190, 936, 1736, 29892, 1749, 6559, 338, 1568, 7200, 6287, 5634, 29882, 5555, 13, 7054, 12339, 287, 29871, 29946, 29941, 7284, 29905, 13007, 15717, 310, 29871, 29906, 29889, 29906, 24464, 29905, 13007, 25741, 515, 29871, 29896, 29953, 29900, 7284, 29905, 13007, 13, 16418, 5634, 5426, 11267, 979, 975, 29871, 29945, 29900, 2440, 310, 970, 775, 20706, 3265, 1135, 13, 3149, 297, 931, 29892, 322, 884, 901, 2691, 29899, 3874, 1312, 313, 2541, 1629, 29899, 1609, 29899, 6360, 3803, 1070, 537, 975, 13, 1552, 8900, 3785, 467, 32001, 8108, 1189, 1711, 29892, 1749, 6559, 337, 3687, 373, 10079, 11264, 13, 3924, 313, 8257, 29903, 29897, 9063, 848, 3265, 1135, 7481, 29899, 7099, 433, 1127, 4423, 2472, 29889, 32001, 5901, 1736, 5634, 262, 3153, 278, 664, 491, 8432, 2651, 2036, 29912, 18386, 709, 29906, 29900, 29896, 29941, 2209, 29881, 24974, 29913, 13, 392, 29892, 901, 10325, 29892, 390, 579, 468, 29875, 634, 394, 8770, 2036, 29912, 29878, 579, 468, 29875, 29906, 29900, 29896, 29953, 479, 711, 3173, 29892, 13, 29871, 364, 579, 468, 29875, 29906, 29900, 29896, 29947, 479, 711, 3173, 29892, 544, 1648, 29906, 29900, 29906, 29896, 479, 468, 1581, 29881, 24974, 29913, 5634, 17532, 12399, 1737, 12122, 13, 29905, 7278, 29912, 29881, 24974, 322, 24003, 1118, 474, 29889, 29872, 1696, 278, 15834, 304, 607, 278, 3978, 310, 18322, 1799, 13, 17426, 6602, 1009, 11465, 1230, 14137, 14188, 29889, 32001, 512, 445, 664, 591, 2931, 1891, 1737, 12122, 6894, 537, 297, 970, 775, 363, 278, 937, 13, 2230, 472, 445, 6287, 29892, 1716, 297, 4958, 310, 17737, 29560, 322, 15500, 3785, 29889, 32001, 1334, 437, 13, 1333, 22002, 280, 278, 24003, 10696, 29892, 541, 3867, 29190, 936, 848, 322, 1284, 886, 393, 508, 367, 13, 280, 369, 4063, 304, 393, 1095, 408, 5434, 664, 29889, 32001, 320, 7278, 29912, 12756, 7047, 21639, 29913, 2651, 2036, 29912, 2276, 5824, 19982, 29906, 29900, 29900, 29955, 23705, 1338, 29893, 996, 29913, 338, 278, 13, 1491, 29899, 2671, 310, 7047, 21639, 393, 756, 29537, 287, 278, 18066, 267, 310, 21640, 13, 6734, 24771, 13149, 635, 29892, 3704, 278, 2702, 5932, 310, 920, 304, 5376, 13, 2541, 1737, 12122, 6894, 537, 2651, 2036, 29912, 13376, 303, 456, 29906, 29900, 29900, 29953, 10945, 3359, 29892, 1424, 29440, 29906, 29900, 29896, 29946, 23027, 5933, 1836, 32001, 3826, 3076, 2678, 278, 2198, 6559, 8128, 10757, 393, 508, 367, 1304, 29892, 297, 278, 13, 14940, 1206, 310, 970, 775, 322, 472, 263, 1407, 2919, 6287, 29892, 304, 11539, 607, 13, 14032, 4637, 310, 5534, 7047, 21639, 505, 9820, 484, 15774, 29889, 32001, 320, 2042, 29912, 4062, 3002, 29913, 13, 29905, 1643, 29912, 3471, 29901, 5696, 29913, 13, 13, 13, 29905, 1482, 361, 29905, 361, 29887, 798, 386, 1003, 29871, 320, 29887, 798, 386, 1003, 3009, 13, 29905, 361, 29887, 798, 386, 1003, 13, 29905, 463, 29912, 4532, 29913, 13, 29871, 320, 7313, 29961, 2103, 2013, 4914, 2103, 3199, 6360, 368, 29899, 2055, 1169, 29913, 13, 29871, 320, 6671, 29912, 12883, 368, 970, 25741, 975, 931, 313, 1188, 6287, 467, 13, 29913, 32001, 320, 1643, 29912, 1003, 29901, 29887, 798, 386, 29913, 13, 29905, 355, 29912, 4532, 29913, 13, 29905, 7241, 13, 13, 29905, 26956, 29912, 16390, 24541, 29913, 13, 13, 4806, 27387, 515, 320, 23066, 29950, 29914, 2651, 2036, 29912, 29879, 1332, 29899, 1516, 29878, 29906, 29900, 29896, 29929, 29899, 24713, 29913, 599, 25741, 3190, 2347, 2745, 320, 25832, 1964, 579, 1523, 2415, 2539, 6294, 32001, 2688, 5253, 304, 320, 14573, 5261, 1169, 22131, 29914, 25741, 29892, 5412, 491, 317, 15715, 29896, 15882, 29892, 4023, 29894, 2868, 515, 320, 25832, 1299, 7288, 5261, 1169, 797, 7068, 29914, 970, 9279, 6421, 515, 4655, 5849, 363, 2710, 313, 28712, 16046, 29892, 11786, 28632, 29892, 2992, 1846, 32001, 322, 3577, 28914, 313, 10251, 713, 29892, 10772, 2227, 29892, 405, 13427, 29892, 2992, 6250, 32001, 1876, 1169, 297, 278, 8783, 526, 491, 320, 25832, 6344, 2806, 943, 22131, 29914, 15717, 29892, 5412, 491, 779, 6990, 29938, 978, 29892, 4876, 4535, 5854, 29938, 11000, 29889, 32001, 450, 8783, 2996, 408, 1023, 1104, 1288, 6131, 29892, 697, 363, 25741, 322, 697, 363, 15717, 29892, 411, 278, 4642, 29371, 278, 7480, 3025, 263, 9117, 1820, 29889, 32001, 320, 361, 5426, 13, 9760, 1948, 297, 278, 9063, 1591, 3743, 278, 1494, 4235, 29901, 9063, 317, 15715, 29896, 15882, 29892, 4148, 322, 844, 5171, 5335, 342, 15092, 29892, 4148, 322, 844, 5171, 2893, 14903, 313, 20275, 16750, 278, 4148, 1591, 467, 32001, 450, 21578, 1546, 9063, 15717, 322, 9063, 2153, 2041, 515, 11786, 29892, 607, 6511, 304, 9063, 263, 1735, 4148, 287, 491, 4856, 1683, 29889, 32001, 1152, 445, 6559, 591, 21309, 373, 15717, 322, 17262, 9063, 2153, 29892, 408, 278, 4328, 1546, 278, 1023, 338, 451, 8018, 363, 1749, 5925, 5155, 322, 278, 5253, 310, 25741, 411, 263, 844, 5171, 916, 1135, 967, 4148, 338, 3480, 3473, 1821, 29889, 32001, 320, 7241, 13, 2831, 1269, 6251, 297, 278, 4148, 1591, 591, 505, 4148, 2989, 1024, 322, 4876, 408, 1023, 5004, 6031, 310, 10650, 6262, 29889, 32001, 1334, 6206, 2411, 15273, 1821, 470, 18325, 519, 2983, 393, 29901, 526, 451, 1602, 397, 519, 408, 18351, 29899, 29947, 3441, 25832, 6344, 2806, 943, 29934, 29885, 12283, 7099, 397, 519, 29914, 4148, 2983, 6206, 511, 526, 4876, 14157, 2012, 310, 2983, 3441, 25832, 6344, 2806, 943, 29934, 29885, 9823, 29914, 4954, 7039, 4907, 511, 5718, 310, 871, 9654, 4890, 3441, 25832, 6344, 2806, 943, 29934, 29885, 10358, 804, 23201, 1712, 901, 1135, 29871, 29896, 29900, 8958, 1661, 29899, 1026, 2153, 3441, 25832, 6344, 2806, 943, 29934, 29885, 12283, 15670, 23201, 526, 5520, 1135, 29871, 29896, 29900, 29900, 4890, 3441, 25832, 6344, 2806, 943, 29934, 29885, 12229, 549, 12495, 32001, 2860, 21166, 29892, 1048, 320, 25832, 6344, 2806, 943, 29925, 15273, 1821, 2052, 307, 29916, 29914, 15717, 3441, 25832, 6344, 2806, 943, 29925, 15273, 1821, 29925, 312, 29914, 310, 278, 2847, 8783, 29897, 9488, 363, 4340, 7418, 29889, 32001, 3940, 393, 278, 5253, 310, 970, 775, 25741, 313, 392, 15717, 29897, 11122, 297, 278, 13, 11228, 8783, 25088, 18709, 9247, 975, 13, 2230, 2651, 2036, 29912, 29879, 1332, 29899, 771, 854, 749, 29899, 331, 344, 1012, 361, 29887, 798, 386, 1003, 29892, 408, 4318, 363, 25741, 297, 13, 29905, 29907, 999, 29912, 1003, 29901, 29887, 798, 386, 1012, 2870, 29901, 515, 395, 29896, 29900, 29985, 29946, 29938, 25741, 297, 29871, 29896, 29929, 29955, 29896, 29892, 304, 395, 29896, 29900, 29985, 29953, 29938, 297, 29871, 29896, 29929, 29929, 29947, 29892, 304, 13, 284, 3242, 395, 29896, 29900, 29985, 29929, 29938, 297, 29871, 29906, 29900, 29906, 29900, 29905, 7241, 29889, 32001, 1094, 263, 17004, 278, 8900, 534, 1975, 10331, 304, 367, 901, 13, 13844, 297, 7786, 1602, 3076, 1135, 297, 29871, 29946, 29900, 29974, 1629, 29899, 1025, 6743, 29892, 2861, 304, 13964, 4586, 373, 13, 735, 1112, 9247, 7200, 23093, 29889, 32001, 320, 26956, 29912, 7999, 324, 10610, 29913, 13, 13, 29905, 463, 29912, 4532, 29913, 13, 29871, 320, 9525, 13, 29871, 320, 7313, 29961, 24049, 29892, 15450, 29922, 29953, 4912, 29871, 29953, 4912, 29871, 29900, 29871, 29900, 29892, 2103, 2013, 16292, 3199, 1491, 1727, 1080, 29899, 2470, 29913, 13, 29871, 320, 6671, 29912, 1576, 320, 14573, 14058, 4597, 1080, 29914, 3186, 12786, 1304, 408, 1737, 324, 10610, 22525, 5003, 32001, 320, 1643, 29912, 1003, 29901, 11526, 1958, 29913, 13, 29905, 355, 29912, 4532, 29913, 13, 13, 2887, 1737, 324, 10610, 22525, 591, 671, 11758, 3186, 12786, 10723, 515, 278, 3303, 18269, 1737, 359, 305, 2004, 2651, 2036, 29912, 348, 29896, 29929, 29929, 29929, 479, 359, 305, 2004, 1836, 32001, 1763, 4772, 2432, 3381, 491, 2919, 10916, 313, 29872, 29889, 29887, 1696, 7551, 470, 12710, 29897, 2629, 11758, 12786, 29892, 591, 19412, 322, 6219, 777, 12786, 2729, 373, 1737, 12122, 23203, 537, 322, 278, 19383, 310, 758, 331, 8946, 16375, 29769, 5680, 29892, 1316, 408, 19182, 4086, 29889, 32001, 320, 29907, 999, 29912, 1003, 29901, 11526, 1958, 29913, 3697, 278, 2186, 1051, 310, 320, 14573, 14058, 4597, 1080, 29914, 3186, 12786, 1304, 408, 1737, 324, 10610, 22525, 297, 445, 6559, 29889, 32001, 1879, 324, 10610, 310, 9063, 15717, 304, 3186, 12786, 3913, 278, 1023, 19595, 653, 13698, 9129, 297, 2651, 2036, 29912, 293, 344, 29899, 344, 275, 29899, 29906, 29900, 29906, 29906, 29899, 26098, 1118, 23359, 337, 13998, 2400, 29889, 32001, 450, 937, 697, 337, 3687, 373, 278, 4234, 775, 2246, 29899, 5563, 5354, 313, 617, 29911, 10249, 29897, 310, 4876, 14157, 23892, 515, 9063, 15562, 29892, 321, 29889, 29887, 1696, 320, 16266, 26139, 1341, 1118, 320, 16266, 26139, 582, 1118, 320, 16266, 26139, 18038, 1118, 2992, 29889, 32001, 1334, 4687, 515, 278, 306, 2190, 29909, 1051, 310, 13548, 2931, 21759, 29911, 10249, 29879, 2651, 2036, 29912, 6011, 29899, 29883, 312, 430, 29913, 322, 7522, 20545, 1269, 6590, 20123, 304, 263, 3646, 3186, 5120, 29889, 32001, 450, 1473, 1737, 324, 10610, 11043, 3913, 278, 17998, 9210, 310, 9063, 5335, 342, 15092, 313, 29872, 29889, 29887, 1696, 17998, 29899, 29900, 29945, 29901, 29900, 29900, 29897, 322, 4148, 2983, 304, 8161, 278, 1556, 5517, 3186, 5120, 310, 278, 9063, 4148, 29889, 32001, 1152, 1269, 17998, 9210, 591, 8161, 263, 1051, 310, 15878, 7600, 313, 13509, 29892, 2106, 29892, 470, 14278, 20123, 29897, 297, 278, 3186, 393, 29892, 472, 278, 931, 310, 393, 9063, 29892, 750, 393, 17998, 9210, 29936, 9063, 931, 338, 1820, 1244, 29892, 408, 4234, 17998, 1283, 7224, 13100, 975, 931, 2861, 304, 29431, 3620, 29889, 32001, 1763, 1207, 445, 3683, 3381, 591, 671, 278, 306, 2190, 29909, 931, 10640, 2566, 2651, 2036, 29912, 17559, 1272, 1836, 32001, 1987, 591, 3566, 304, 1269, 2058, 263, 8158, 393, 4332, 1973, 278, 4188, 22342, 393, 263, 2183, 4148, 1024, 338, 17443, 310, 372, 29889, 32001, 1763, 445, 1095, 591, 671, 278, 28297, 29890, 15451, 8783, 310, 278, 29511, 310, 278, 1556, 3619, 937, 322, 3942, 2983, 607, 29892, 439, 11427, 515, 2651, 2036, 29912, 1079, 29890, 799, 29899, 7039, 6177, 2802, 277, 12181, 4954, 16123, 2247, 278, 26368, 5528, 5084, 310, 363, 264, 1280, 322, 269, 595, 1280, 7371, 515, 263, 2566, 310, 320, 1949, 29912, 29946, 29871, 29900, 29946, 29946, 29871, 29945, 29946, 29953, 29871, 29929, 29941, 29947, 29913, 2305, 313, 29945, 29945, 29889, 29945, 8958, 310, 8471, 2305, 297, 29871, 29906, 29900, 29896, 29946, 467, 32001, 1094, 310, 3839, 29871, 29906, 29900, 29896, 29929, 372, 18469, 320, 1949, 29912, 29906, 29955, 29871, 29953, 29953, 29906, 29871, 29947, 29900, 29896, 29913, 363, 264, 1280, 322, 320, 1949, 29912, 29906, 29955, 29871, 29906, 29900, 29953, 29871, 29947, 29906, 29896, 29913, 269, 595, 1280, 297, 29871, 29906, 29941, 29953, 24894, 8977, 1080, 29889, 4907, 29913, 32001, 1094, 297, 1749, 8783, 15717, 526, 2989, 1024, 6031, 313, 29878, 1624, 1135, 6219, 491, 937, 29914, 11922, 1024, 511, 591, 937, 5993, 675, 2983, 313, 1609, 1999, 1331, 322, 1206, 3620, 29897, 322, 769, 16280, 5375, 18897, 297, 1716, 937, 322, 3942, 2983, 10868, 8857, 29889, 32001, 1152, 1269, 1543, 1476, 297, 1024, 8857, 591, 22932, 278, 2058, 4665, 29905, 6661, 1333, 24212, 8875, 491, 278, 1024, 10868, 304, 4017, 263, 5645, 393, 338, 29839, 304, 278, 1353, 310, 12407, 24638, 393, 1024, 313, 6979, 29897, 297, 278, 2702, 2058, 29889, 32001, 320, 6661, 1333, 300, 1062, 29912, 1762, 4017, 4665, 2025, 1338, 5634, 294, 278, 17837, 310, 4954, 6689, 4907, 338, 25745, 23724, 29901, 2989, 10916, 325, 13217, 29879, 29399, 310, 2919, 10916, 805, 9450, 2999, 931, 29920, 2873, 5634, 705, 671, 263, 29544, 310, 7601, 8974, 313, 29872, 29889, 29887, 1696, 5874, 28007, 511, 322, 1661, 29899, 16072, 6743, 313, 29872, 29889, 29887, 1696, 14109, 7456, 467, 29913, 32001, 1334, 2533, 445, 4377, 363, 599, 3161, 304, 4017, 263, 2058, 8158, 29892, 17140, 701, 411, 263, 1051, 310, 779, 6990, 29938, 6689, 29892, 8158, 4535, 5854, 29938, 11000, 29889, 32001, 1334, 769, 8877, 445, 1051, 491, 278, 3186, 5120, 393, 263, 2058, 14393, 304, 322, 2533, 278, 8158, 363, 599, 278, 7600, 297, 1269, 5120, 304, 4017, 385, 12463, 8158, 29892, 6590, 304, 278, 4188, 22342, 393, 278, 9063, 14393, 304, 263, 2183, 3186, 5120, 29889, 32001, 1334, 3566, 278, 6257, 9063, 408, 6421, 515, 278, 3186, 5120, 411, 278, 9939, 8158, 29889, 32001, 450, 4876, 29899, 6707, 11043, 9378, 414, 515, 278, 9078, 322, 443, 5521, 8362, 671, 310, 21759, 29911, 10249, 29879, 29901, 1556, 18777, 671, 10035, 323, 10249, 29879, 1316, 408, 320, 16266, 26139, 510, 1118, 320, 16266, 26139, 990, 1118, 470, 320, 16266, 26139, 1212, 1836, 32001, 12808, 445, 947, 451, 3799, 26018, 4822, 20542, 29901, 3148, 29899, 6707, 18777, 29892, 363, 1342, 29892, 671, 278, 320, 16266, 26139, 375, 29913, 21759, 29911, 10249, 1568, 901, 269, 2495, 290, 368, 1135, 1009, 7824, 6795, 20895, 29889, 32001, 1551, 278, 916, 1361, 278, 9210, 29914, 978, 29899, 6707, 11043, 337, 3687, 373, 278, 17998, 9210, 310, 278, 9063, 5335, 342, 15092, 29889, 32001, 16809, 304, 5780, 22920, 373, 13897, 731, 14340, 29892, 263, 2919, 1353, 310, 25741, 297, 278, 8783, 756, 385, 17998, 9210, 5186, 304, 5225, 29889, 32001, 910, 6602, 29879, 3109, 7786, 25741, 3441, 14573, 5261, 1169, 29911, 29999, 29999, 13985, 1349, 681, 392, 27418, 6478, 29914, 310, 29871, 29906, 29900, 29906, 29900, 29879, 25741, 505, 263, 5225, 9210, 29897, 1135, 9642, 6743, 3441, 14573, 5261, 1169, 29911, 29999, 29999, 13985, 1349, 681, 392, 29914, 297, 29871, 29906, 29900, 29900, 29900, 467, 32001, 1094, 263, 1121, 278, 9210, 29914, 978, 29899, 6707, 11043, 1033, 1095, 701, 6459, 292, 263, 2919, 6232, 310, 9642, 25741, 408, 4148, 287, 491, 11715, 18777, 29892, 322, 304, 263, 3109, 261, 15834, 4092, 550, 29889, 32001, 1763, 6795, 1438, 5626, 591, 14405, 278, 1023, 1737, 324, 10610, 13698, 4208, 491, 15399, 278, 9210, 29914, 978, 29899, 6707, 13698, 304, 599, 25741, 411, 263, 1661, 29899, 9171, 17998, 9210, 29892, 322, 278, 4876, 29899, 6707, 373, 304, 599, 916, 25741, 29889, 32001, 320, 2042, 29912, 12191, 322, 8565, 1558, 291, 29913, 13, 29905, 1643, 29912, 3471, 29901, 9902, 29913, 13, 13, 29905, 463, 29912, 4532, 4044, 13, 29871, 320, 9525, 13, 29871, 320, 7313, 29961, 2103, 2013, 16292, 3199, 1429, 287, 29889, 5140, 29913, 13, 29871, 320, 6671, 29912, 29934, 20819, 310, 25741, 313, 27215, 29897, 322, 6136, 15717, 313, 22503, 29897, 491, 3186, 10640, 975, 278, 29871, 29896, 29929, 29955, 29896, 489, 29906, 29900, 29906, 29900, 3785, 5003, 32001, 320, 9868, 29961, 14732, 3199, 7264, 287, 2594, 8727, 6445, 278, 3186, 10640, 364, 2219, 359, 363, 25741, 322, 15717, 975, 278, 29871, 29896, 29929, 29955, 29896, 489, 29906, 29900, 29906, 29900, 3785, 5003, 32001, 320, 1643, 29912, 1003, 29901, 9902, 29913, 13, 29905, 355, 29912, 4532, 4044, 13, 13, 13, 29871, 13, 1762, 1234, 320, 25029, 29912, 29878, 29939, 29901, 479, 397, 24974, 29913, 591, 22229, 278, 1353, 310, 25741, 322, 8359, 15717, 639, 1629, 322, 639, 3186, 10640, 29889, 32001, 1334, 2198, 278, 7625, 2582, 297, 320, 29907, 999, 29912, 1003, 29901, 9902, 29913, 408, 1023, 5096, 287, 2594, 24469, 29892, 6445, 1629, 368, 2867, 3204, 29879, 363, 25741, 322, 15717, 8307, 29889, 32001, 7569, 2594, 11524, 263, 1629, 322, 338, 8877, 287, 297, 269, 29399, 6445, 278, 9063, 29914, 8921, 11959, 363, 1269, 310, 278, 3186, 12786, 310, 320, 29907, 999, 29912, 1003, 29901, 11526, 1958, 29913, 297, 393, 1629, 29889, 32001, 1763, 4772, 714, 27801, 2861, 304, 805, 272, 26538, 17737, 29560, 29892, 297, 278, 4148, 8727, 591, 871, 2050, 15717, 2534, 26869, 472, 3203, 29871, 29945, 25741, 297, 263, 2183, 1629, 29889, 32001, 5806, 5366, 1747, 534, 1975, 297, 278, 24469, 6456, 393, 278, 3001, 3694, 310, 25741, 322, 15717, 6548, 18709, 9247, 975, 931, 29889, 32001, 10133, 363, 278, 937, 2440, 297, 278, 24469, 29892, 278, 1353, 310, 848, 3291, 297, 777, 3186, 12786, 508, 367, 14154, 2319, 29892, 411, 8178, 27721, 373, 278, 25806, 310, 534, 1975, 29889, 32001, 320, 26956, 29912, 7999, 12122, 6894, 537, 975, 931, 29913, 13, 13, 3563, 497, 29892, 278, 2498, 534, 355, 5692, 304, 367, 393, 278, 320, 4534, 29912, 479, 12122, 6894, 537, 297, 970, 775, 338, 10231, 6177, 4644, 6813, 322, 4092, 5136, 630, 1009, 4954, 24130, 749, 4907, 2745, 278, 7256, 310, 278, 29871, 29929, 29900]} {'input_ids': [1, 320, 2042, 29912, 25898, 29913, 13, 29905, 1643, 29912, 3471, 29901, 23333, 29913, 13, 13, 29905, 7278, 29912, 29954, 1581, 6894, 537, 1118, 470, 901, 4049, 967, 10225, 727, 974, 29892, 4249, 27138, 304, 13, 20415, 5849, 14188, 756, 1063, 26606, 12399, 297, 7786, 2440, 29889, 32001, 512, 13, 1595, 16311, 29892, 278, 10122, 310, 29892, 9545, 310, 29892, 322, 6795, 1004, 25414, 363, 320, 7278, 29912, 26098, 13, 29871, 24003, 29913, 297, 12362, 29914, 6585, 7562, 18540, 313, 5800, 1799, 29897, 505, 4520, 263, 3287, 310, 8570, 13, 957, 278, 4940, 316, 6332, 2651, 2036, 29912, 29881, 16093, 29906, 29900, 29900, 29947, 29888, 2209, 3359, 29879, 29892, 3855, 5871, 29906, 29900, 29896, 29900, 29895, 311, 29893, 2770, 29892, 13, 29871, 1055, 29888, 375, 29906, 29900, 29896, 29906, 5041, 267, 29892, 413, 434, 305, 1358, 29906, 29900, 29896, 29906, 26098, 29888, 2209, 29892, 19723, 5475, 4979, 29906, 29900, 29896, 29946, 26098, 29892, 13, 29871, 697, 309, 29906, 29900, 29896, 29953, 311, 5365, 550, 332, 6950, 29892, 696, 7586, 29906, 29900, 29896, 29953, 29893, 2770, 7192, 2209, 29892, 1935, 15044, 29906, 29900, 29896, 29955, 26098, 29892, 13, 29871, 503, 562, 305, 3350, 492, 29906, 29900, 29906, 29896, 26098, 1836, 32001, 320, 7278, 29912, 7999, 12122, 6894, 537, 29913, 338, 373, 278, 916, 1361, 278, 13, 14380, 310, 6894, 537, 393, 380, 1567, 515, 27138, 297, 777, 5534, 6354, 6421, 13, 3166, 1422, 3186, 12786, 322, 4185, 1973, 29889, 32001, 26304, 6894, 537, 297, 18322, 1799, 756, 4520, 13774, 2217, 8570, 297, 21344, 368, 13, 13129, 29889, 32001, 512, 3153, 29892, 1550, 3031, 979, 18994, 29899, 6707, 322, 13, 3149, 29899, 262, 29899, 2230, 18350, 29899, 7052, 11898, 310, 278, 1737, 12122, 1677, 1144, 310, 18322, 1799, 13, 21570, 29560, 1863, 2651, 2036, 29912, 12443, 10578, 29906, 29900, 29900, 29945, 5062, 11235, 29892, 13260, 333, 29906, 29900, 29900, 29947, 29888, 2209, 3359, 29879, 29892, 13, 29871, 2594, 801, 2681, 29906, 29900, 29900, 29947, 479, 397, 24974, 29892, 1850, 29882, 371, 4099, 29894, 29906, 29900, 29896, 29900, 2209, 479, 5275, 29892, 696, 7586, 29906, 29900, 29896, 29946, 7610, 345, 2941, 271, 24541, 29892, 13, 29871, 281, 496, 29879, 29906, 29900, 29906, 29896, 2209, 479, 5275, 1118, 2919, 29899, 7052, 25579, 979, 11898, 310, 278, 1737, 12122, 13, 12574, 310, 18322, 1799, 17737, 29560, 526, 1603, 10225, 292, 29889, 32001, 10506, 263, 4323, 23378, 13, 18609, 2133, 723, 367, 5407, 304, 1871, 1602, 12112, 4475, 304, 5534, 13, 25431, 10907, 2651, 2036, 29912, 2276, 5824, 19982, 29906, 29900, 29900, 29955, 23705, 1338, 29893, 996, 29913, 322, 298, 8491, 16650, 583, 297, 278, 13, 19678, 15483, 313, 1806, 29897, 9999, 29892, 408, 1532, 408, 29126, 2114, 950, 2472, 13, 517, 278, 2553, 1078, 373, 278, 17407, 10879, 322, 5374, 3002, 310, 18322, 1799, 2820, 278, 3186, 29889, 32001, 320, 26956, 29912, 1323, 3224, 29879, 29913, 13, 13, 3047, 445, 664, 591, 29126, 304, 3802, 445, 17261, 491, 7512, 292, 320, 4534, 29912, 1552, 937, 13, 29871, 25579, 979, 6559, 310, 278, 1737, 12122, 3978, 310, 17737, 29560, 304, 970, 775, 13, 29871, 975, 29871, 29945, 29900, 2440, 5003, 32001, 26321, 29892, 591, 3867, 263, 758, 2576, 3821, 1234, 304, 278, 13, 23031, 292, 5925, 1139, 29901, 13, 29905, 463, 29912, 690, 2842, 12470, 29913, 13, 29871, 3645, 607, 3186, 12786, 437, 15717, 310, 970, 368, 3625, 25741, 2041, 515, 13, 29871, 322, 920, 756, 372, 3939, 975, 278, 4940, 29871, 29945, 29900, 2440, 29973, 32001, 320, 1643, 29912, 29878, 29939, 29901, 479, 397, 24974, 29913, 13, 29905, 355, 29912, 690, 2842, 12470, 29913, 13, 4806, 671, 408, 8783, 278, 320, 23066, 29950, 29914, 18871, 2651, 2036, 29912, 29879, 1332, 29875, 4569, 29906, 29900, 29896, 29955, 29913, 322, 27599, 515, 372, 13, 29906, 29889, 29906, 24464, 29905, 13007, 25741, 3190, 2347, 515, 29871, 29896, 29953, 29900, 7284, 29905, 13007, 9279, 322, 4148, 287, 491, 13, 29946, 29941, 7284, 29905, 13007, 15717, 2645, 278, 29871, 29896, 29929, 29955, 29896, 489, 29906, 29900, 29906, 29896, 931, 3785, 29889, 32001, 1334, 1737, 324, 542, 403, 18777, 304, 13, 29905, 14573, 14058, 4597, 1080, 29914, 3186, 12786, 29892, 773, 408, 18470, 4876, 4234, 775, 2246, 29899, 5563, 21904, 313, 617, 29911, 10249, 29879, 29897, 322, 29871, 13, 8921, 313, 4102, 29914, 4230, 29897, 2983, 9401, 411, 1024, 18822, 2820, 278, 3186, 29892, 322, 17998, 1283, 7224, 29871, 13, 1195, 287, 515, 9063, 15562, 29889, 32001, 1334, 1284, 10757, 310, 278, 4688, 8022, 749, 310, 4644, 6813, 297, 1722, 2752, 13, 20415, 29892, 2678, 8772, 491, 4092, 29889, 32001, 2860, 393, 3785, 29892, 278, 1737, 12122, 6894, 537, 29871, 13, 262, 970, 775, 756, 1063, 21003, 10231, 29889, 32001, 1334, 884, 12439, 8018, 15839, 528, 17741, 13, 12817, 304, 278, 1095, 310, 278, 8291, 6415, 29129, 322, 278, 7910, 310, 14137, 4631, 4135, 297, 13, 23369, 1705, 322, 4275, 14325, 29892, 408, 1532, 408, 310, 2545, 1664, 17292, 28342, 763, 25539, 1608, 322, 13, 25719, 10298, 4822, 10916, 313, 6727, 16783, 29914, 331, 16783, 467, 32001, 320, 26956, 29912, 1469, 20847, 3097, 5003, 32001, 319, 1634, 1414, 3577, 363, 445, 5650, 338, 3625, 515, 796, 264, 8144, 472, 13, 29905, 2271, 29912, 991, 597, 1867, 29875, 29889, 990, 29914, 29896, 29900, 29889, 29945, 29906, 29947, 29896, 29914, 2256, 8144, 29889, 29953, 29941, 29929, 29900, 29941, 29945, 29945, 29913, 2651, 2036, 29912, 3445, 1414, 29899, 5113, 1836, 32001, 320, 2042, 29912, 9662, 630, 5244, 29913, 13, 29905, 1643, 29912, 3471, 29901, 12817, 29913, 13, 13, 29933, 720, 4688, 322, 7786, 1736, 2651, 2036, 29912, 12443, 10578, 29906, 29900, 29900, 29945, 5062, 11235, 29892, 13260, 333, 29906, 29900, 29900, 29947, 29888, 2209, 3359, 29879, 29892, 13, 29871, 696, 7586, 29906, 29900, 29896, 29946, 7610, 345, 2941, 271, 24541, 29892, 697, 309, 29906, 29900, 29896, 29953, 311, 5365, 550, 332, 6950, 29913, 505, 2931, 1891, 278, 13, 479, 5275, 310, 12362, 29914, 6585, 7562, 18540, 313, 5800, 1799, 29897, 773, 320, 7278, 29912, 6734, 26946, 952, 1118, 13, 4716, 3867, 1880, 29899, 29567, 6089, 541, 526, 9078, 297, 2159, 313, 29906, 29899, 29945, 5940, 29968, 18777, 29897, 13, 392, 508, 367, 4768, 1463, 491, 5221, 424, 23460, 29889, 32001, 512, 29871, 29906, 29900, 29900, 29947, 2261, 801, 2681, 634, 394, 8770, 2036, 29912, 1646, 801, 2681, 29906, 29900, 29900, 29947, 479, 397, 24974, 29913, 18043, 263, 3031, 979, 13, 16961, 29899, 7052, 313, 1454, 278, 931, 29897, 6559, 373, 18322, 1799, 320, 7278, 29912, 479, 5275, 773, 1375, 292, 7047, 13, 29871, 28914, 313, 4345, 29934, 29897, 13698, 1836, 32001, 2688, 29537, 287, 278, 3978, 310, 29871, 29896, 5940, 29924, 17737, 29560, 13, 4746, 278, 7562, 2831, 479, 1404, 2566, 322, 611, 6504, 1051, 3190, 3145, 975, 278, 13, 29896, 29929, 29929, 29929, 489, 29906, 29900, 29900, 29945, 3785, 29892, 773, 408, 18470, 2472, 2788, 304, 1749, 29879, 29901, 4876, 21904, 13, 392, 17998, 1283, 7224, 29889, 32001, 450, 12399, 3785, 313, 29955, 2440, 29897, 297, 2651, 2036, 29912, 1646, 801, 2681, 29906, 29900, 29900, 29947, 479, 397, 24974, 29913, 338, 20511, 1135, 29871, 13, 5816, 338, 12399, 297, 278, 2198, 5650, 313, 29945, 29900, 2440, 29897, 322, 278, 848, 8974, 526, 29871, 13, 27489, 873, 1422, 29936, 411, 393, 297, 3458, 29892, 1749, 2582, 1510, 263, 10029, 7200, 14978, 310, 29871, 13, 15654, 273, 325, 13217, 29940, 2072, 3082, 20706, 29889, 32001, 7280, 29190, 936, 664, 515, 29871, 29906, 29900, 29896, 29900, 491, 14619, 29882, 371, 4099, 29894, 322, 13, 29950, 309, 1372, 2651, 2036, 29912, 29873, 19426, 371, 4099, 29894, 29906, 29900, 29896, 29900, 2209, 479, 5275, 29913, 4023, 29894, 2868, 1583, 29899, 7099, 433, 1127, 1737, 12122, 13, 2029, 800, 310, 25492, 15303, 8304, 3598, 1494, 1009, 12368, 29892, 13, 15914, 292, 2472, 363, 779, 14850, 4535, 29892, 29955, 29900, 5940, 29968, 25492, 4160, 29889, 32001, 319, 1407, 7786, 13, 1287, 2651, 2036, 29912, 21686, 29879, 29906, 29900, 29906, 29896, 2209, 479, 5275, 29913, 491, 399, 496, 29879, 634, 394, 13217, 5349, 1737, 324, 542, 630, 4203, 263, 7284, 13, 28712, 16046, 4160, 29892, 2534, 26869, 472, 3203, 29871, 29896, 29900, 29900, 25741, 1269, 29892, 322, 1058, 13, 1311, 29899, 7099, 8663, 14354, 373, 1009, 25492, 28723, 29889, 32001, 5806, 278, 6559, 338, 13, 3149, 29899, 262, 29899, 2230, 408, 310, 29871, 29906, 29900, 29906, 29896, 29892, 278, 15717, 7252, 1009, 1284, 886, 13, 351, 475, 303, 2651, 2036, 29912, 1646, 801, 2681, 29906, 29900, 29900, 29947, 479, 397, 24974, 29892, 1850, 29882, 371, 4099, 29894, 29906, 29900, 29896, 29900, 2209, 479, 5275, 29913, 304, 13, 18609, 675, 278, 14675, 310, 18322, 1799, 1737, 5275, 975, 278, 931, 15101, 845, 1862, 4586, 491, 13, 1552, 2211, 11898, 29889, 32001, 3831, 1965, 411, 3517, 29190, 936, 1736, 29892, 1749, 6559, 338, 1568, 7200, 6287, 5634, 29882, 5555, 13, 7054, 12339, 287, 29871, 29946, 29941, 7284, 29905, 13007, 15717, 310, 29871, 29906, 29889, 29906, 24464, 29905, 13007, 25741, 515, 29871, 29896, 29953, 29900, 7284, 29905, 13007, 13, 16418, 5634, 5426, 11267, 979, 975, 29871, 29945, 29900, 2440, 310, 970, 775, 20706, 3265, 1135, 13, 3149, 297, 931, 29892, 322, 884, 901, 2691, 29899, 3874, 1312, 313, 2541, 1629, 29899, 1609, 29899, 6360, 3803, 1070, 537, 975, 13, 1552, 8900, 3785, 467, 32001, 8108, 1189, 1711, 29892, 1749, 6559, 337, 3687, 373, 10079, 11264, 13, 3924, 313, 8257, 29903, 29897, 9063, 848, 3265, 1135, 7481, 29899, 7099, 433, 1127, 4423, 2472, 29889, 32001, 5901, 1736, 5634, 262, 3153, 278, 664, 491, 8432, 2651, 2036, 29912, 18386, 709, 29906, 29900, 29896, 29941, 2209, 29881, 24974, 29913, 13, 392, 29892, 901, 10325, 29892, 390, 579, 468, 29875, 634, 394, 8770, 2036, 29912, 29878, 579, 468, 29875, 29906, 29900, 29896, 29953, 479, 711, 3173, 29892, 13, 29871, 364, 579, 468, 29875, 29906, 29900, 29896, 29947, 479, 711, 3173, 29892, 544, 1648, 29906, 29900, 29906, 29896, 479, 468, 1581, 29881, 24974, 29913, 5634, 17532, 12399, 1737, 12122, 13, 29905, 7278, 29912, 29881, 24974, 322, 24003, 1118, 474, 29889, 29872, 1696, 278, 15834, 304, 607, 278, 3978, 310, 18322, 1799, 13, 17426, 6602, 1009, 11465, 1230, 14137, 14188, 29889, 32001, 512, 445, 664, 591, 2931, 1891, 1737, 12122, 6894, 537, 297, 970, 775, 363, 278, 937, 13, 2230, 472, 445, 6287, 29892, 1716, 297, 4958, 310, 17737, 29560, 322, 15500, 3785, 29889, 32001, 1334, 437, 13, 1333, 22002, 280, 278, 24003, 10696, 29892, 541, 3867, 29190, 936, 848, 322, 1284, 886, 393, 508, 367, 13, 280, 369, 4063, 304, 393, 1095, 408, 5434, 664, 29889, 32001, 320, 7278, 29912, 12756, 7047, 21639, 29913, 2651, 2036, 29912, 2276, 5824, 19982, 29906, 29900, 29900, 29955, 23705, 1338, 29893, 996, 29913, 338, 278, 13, 1491, 29899, 2671, 310, 7047, 21639, 393, 756, 29537, 287, 278, 18066, 267, 310, 21640, 13, 6734, 24771, 13149, 635, 29892, 3704, 278, 2702, 5932, 310, 920, 304, 5376, 13, 2541, 1737, 12122, 6894, 537, 2651, 2036, 29912, 13376, 303, 456, 29906, 29900, 29900, 29953, 10945, 3359, 29892, 1424, 29440, 29906, 29900, 29896, 29946, 23027, 5933, 1836, 32001, 3826, 3076, 2678, 278, 2198, 6559, 8128, 10757, 393, 508, 367, 1304, 29892, 297, 278, 13, 14940, 1206, 310, 970, 775, 322, 472, 263, 1407, 2919, 6287, 29892, 304, 11539, 607, 13, 14032, 4637, 310, 5534, 7047, 21639, 505, 9820, 484, 15774, 29889, 32001, 320, 2042, 29912, 4062, 3002, 29913, 13, 29905, 1643, 29912, 3471, 29901, 5696, 29913, 13, 13, 13, 29905, 1482, 361, 29905, 361, 29887, 798, 386, 1003, 29871, 320, 29887, 798, 386, 1003, 3009, 13, 29905, 361, 29887, 798, 386, 1003, 13, 29905, 463, 29912, 4532, 29913, 13, 29871, 320, 7313, 29961, 2103, 2013, 4914, 2103, 3199, 6360, 368, 29899, 2055, 1169, 29913, 13, 29871, 320, 6671, 29912, 12883, 368, 970, 25741, 975, 931, 313, 1188, 6287, 467, 13, 29913, 32001, 320, 1643, 29912, 1003, 29901, 29887, 798, 386, 29913, 13, 29905, 355, 29912, 4532, 29913, 13, 29905, 7241, 13, 13, 29905, 26956, 29912, 16390, 24541, 29913, 13, 13, 4806, 27387, 515, 320, 23066, 29950, 29914, 2651, 2036, 29912, 29879, 1332, 29899, 1516, 29878, 29906, 29900, 29896, 29929, 29899, 24713, 29913, 599, 25741, 3190, 2347, 2745, 320, 25832, 1964, 579, 1523, 2415, 2539, 6294, 32001, 2688, 5253, 304, 320, 14573, 5261, 1169, 22131, 29914, 25741, 29892, 5412, 491, 317, 15715, 29896, 15882, 29892, 4023, 29894, 2868, 515, 320, 25832, 1299, 7288, 5261, 1169, 797, 7068, 29914, 970, 9279, 6421, 515, 4655, 5849, 363, 2710, 313, 28712, 16046, 29892, 11786, 28632, 29892, 2992, 1846, 32001, 322, 3577, 28914, 313, 10251, 713, 29892, 10772, 2227, 29892, 405, 13427, 29892, 2992, 6250, 32001, 1876, 1169, 297, 278, 8783, 526, 491, 320, 25832, 6344, 2806, 943, 22131, 29914, 15717, 29892, 5412, 491, 779, 6990, 29938, 978, 29892, 4876, 4535, 5854, 29938, 11000, 29889, 32001, 450, 8783, 2996, 408, 1023, 1104, 1288, 6131, 29892, 697, 363, 25741, 322, 697, 363, 15717, 29892, 411, 278, 4642, 29371, 278, 7480, 3025, 263, 9117, 1820, 29889, 32001, 320, 361, 5426, 13, 9760, 1948, 297, 278, 9063, 1591, 3743, 278, 1494, 4235, 29901, 9063, 317, 15715, 29896, 15882, 29892, 4148, 322, 844, 5171, 5335, 342, 15092, 29892, 4148, 322, 844, 5171, 2893, 14903, 313, 20275, 16750, 278, 4148, 1591, 467, 32001, 450, 21578, 1546, 9063, 15717, 322, 9063, 2153, 2041, 515, 11786, 29892, 607, 6511, 304, 9063, 263, 1735, 4148, 287, 491, 4856, 1683, 29889, 32001, 1152, 445, 6559, 591, 21309, 373, 15717, 322, 17262, 9063, 2153, 29892, 408, 278, 4328, 1546, 278, 1023, 338, 451, 8018, 363, 1749, 5925, 5155, 322, 278, 5253, 310, 25741, 411, 263, 844, 5171, 916, 1135, 967, 4148, 338, 3480, 3473, 1821, 29889, 32001, 320, 7241, 13, 2831, 1269, 6251, 297, 278, 4148, 1591, 591, 505, 4148, 2989, 1024, 322, 4876, 408, 1023, 5004, 6031, 310, 10650, 6262, 29889, 32001, 1334, 6206, 2411, 15273, 1821, 470, 18325, 519, 2983, 393, 29901, 526, 451, 1602, 397, 519, 408, 18351, 29899, 29947, 3441, 25832, 6344, 2806, 943, 29934, 29885, 12283, 7099, 397, 519, 29914, 4148, 2983, 6206, 511, 526, 4876, 14157, 2012, 310, 2983, 3441, 25832, 6344, 2806, 943, 29934, 29885, 9823, 29914, 4954, 7039, 4907, 511, 5718, 310, 871, 9654, 4890, 3441, 25832, 6344, 2806, 943, 29934, 29885, 10358, 804, 23201, 1712, 901, 1135, 29871, 29896, 29900, 8958, 1661, 29899, 1026, 2153, 3441, 25832, 6344, 2806, 943, 29934, 29885, 12283, 15670, 23201, 526, 5520, 1135, 29871, 29896, 29900, 29900, 4890, 3441, 25832, 6344, 2806, 943, 29934, 29885, 12229, 549, 12495, 32001, 2860, 21166, 29892, 1048, 320, 25832, 6344, 2806, 943, 29925, 15273, 1821, 2052, 307, 29916, 29914, 15717, 3441, 25832, 6344, 2806, 943, 29925, 15273, 1821, 29925, 312, 29914, 310, 278, 2847, 8783, 29897, 9488, 363, 4340, 7418, 29889, 32001, 3940, 393, 278, 5253, 310, 970, 775, 25741, 313, 392, 15717, 29897, 11122, 297, 278, 13, 11228, 8783, 25088, 18709, 9247, 975, 13, 2230, 2651, 2036, 29912, 29879, 1332, 29899, 771, 854, 749, 29899, 331, 344, 1012, 361, 29887, 798, 386, 1003, 29892, 408, 4318, 363, 25741, 297, 13, 29905, 29907, 999, 29912, 1003, 29901, 29887, 798, 386, 1012, 2870, 29901, 515, 395, 29896, 29900, 29985, 29946, 29938, 25741, 297, 29871, 29896, 29929, 29955, 29896, 29892, 304, 395, 29896, 29900, 29985, 29953, 29938, 297, 29871, 29896, 29929, 29929, 29947, 29892, 304, 13, 284, 3242, 395, 29896, 29900, 29985, 29929, 29938, 297, 29871, 29906, 29900, 29906, 29900, 29905, 7241, 29889, 32001, 1094, 263, 17004, 278, 8900, 534, 1975, 10331, 304, 367, 901, 13, 13844, 297, 7786, 1602, 3076, 1135, 297, 29871, 29946, 29900, 29974, 1629, 29899, 1025, 6743, 29892, 2861, 304, 13964, 4586, 373, 13, 735, 1112, 9247, 7200, 23093, 29889, 32001, 320, 26956, 29912, 7999, 324, 10610, 29913, 13, 13, 29905, 463, 29912, 4532, 29913, 13, 29871, 320, 9525, 13, 29871, 320, 7313, 29961, 24049, 29892, 15450, 29922, 29953, 4912, 29871, 29953, 4912, 29871, 29900, 29871, 29900, 29892, 2103, 2013, 16292, 3199, 1491, 1727, 1080, 29899, 2470, 29913, 13, 29871, 320, 6671, 29912, 1576, 320, 14573, 14058, 4597, 1080, 29914, 3186, 12786, 1304, 408, 1737, 324, 10610, 22525, 5003, 32001, 320, 1643, 29912, 1003, 29901, 11526, 1958, 29913, 13, 29905, 355, 29912, 4532, 29913, 13, 13, 2887, 1737, 324, 10610, 22525, 591, 671, 11758, 3186, 12786, 10723, 515, 278, 3303, 18269, 1737, 359, 305, 2004, 2651, 2036, 29912, 348, 29896, 29929, 29929, 29929, 479, 359, 305, 2004, 1836, 32001, 1763, 4772, 2432, 3381, 491, 2919, 10916, 313, 29872, 29889, 29887, 1696, 7551, 470, 12710, 29897, 2629, 11758, 12786, 29892, 591, 19412, 322, 6219, 777, 12786, 2729, 373, 1737, 12122, 23203, 537, 322, 278, 19383, 310, 758, 331, 8946, 16375, 29769, 5680, 29892, 1316, 408, 19182, 4086, 29889, 32001, 320, 29907, 999, 29912, 1003, 29901, 11526, 1958, 29913, 3697, 278, 2186, 1051, 310, 320, 14573, 14058, 4597, 1080, 29914, 3186, 12786, 1304, 408, 1737, 324, 10610, 22525, 297, 445, 6559, 29889, 32001, 1879, 324, 10610, 310, 9063, 15717, 304, 3186, 12786, 3913, 278, 1023, 19595, 653, 13698, 9129, 297, 2651, 2036, 29912, 293, 344, 29899, 344, 275, 29899, 29906, 29900, 29906, 29906, 29899, 26098, 1118, 23359, 337, 13998, 2400, 29889, 32001, 450, 937, 697, 337, 3687, 373, 278, 4234, 775, 2246, 29899, 5563, 5354, 313, 617, 29911, 10249, 29897, 310, 4876, 14157, 23892, 515, 9063, 15562, 29892, 321, 29889, 29887, 1696, 320, 16266, 26139, 1341, 1118, 320, 16266, 26139, 582, 1118, 320, 16266, 26139, 18038, 1118, 2992, 29889, 32001, 1334, 4687, 515, 278, 306, 2190, 29909, 1051, 310, 13548, 2931, 21759, 29911, 10249, 29879, 2651, 2036, 29912, 6011, 29899, 29883, 312, 430, 29913, 322, 7522, 20545, 1269, 6590, 20123, 304, 263, 3646, 3186, 5120, 29889, 32001, 450, 1473, 1737, 324, 10610, 11043, 3913, 278, 17998, 9210, 310, 9063, 5335, 342, 15092, 313, 29872, 29889, 29887, 1696, 17998, 29899, 29900, 29945, 29901, 29900, 29900, 29897, 322, 4148, 2983, 304, 8161, 278, 1556, 5517, 3186, 5120, 310, 278, 9063, 4148, 29889, 32001, 1152, 1269, 17998, 9210, 591, 8161, 263, 1051, 310, 15878, 7600, 313, 13509, 29892, 2106, 29892, 470, 14278, 20123, 29897, 297, 278, 3186, 393, 29892, 472, 278, 931, 310, 393, 9063, 29892, 750, 393, 17998, 9210, 29936, 9063, 931, 338, 1820, 1244, 29892, 408, 4234, 17998, 1283, 7224, 13100, 975, 931, 2861, 304, 29431, 3620, 29889, 32001, 1763, 1207, 445, 3683, 3381, 591, 671, 278, 306, 2190, 29909, 931, 10640, 2566, 2651, 2036, 29912, 17559, 1272, 1836, 32001, 1987, 591, 3566, 304, 1269, 2058, 263, 8158, 393, 4332, 1973, 278, 4188, 22342, 393, 263, 2183, 4148, 1024, 338, 17443, 310, 372, 29889, 32001, 1763, 445, 1095, 591, 671, 278, 28297, 29890, 15451, 8783, 310, 278, 29511, 310, 278, 1556, 3619, 937, 322, 3942, 2983, 607, 29892, 439, 11427, 515, 2651, 2036, 29912, 1079, 29890, 799, 29899, 7039, 6177, 2802, 277, 12181, 4954, 16123, 2247, 278, 26368, 5528, 5084, 310, 363, 264, 1280, 322, 269, 595, 1280, 7371, 515, 263, 2566, 310, 320, 1949, 29912, 29946, 29871, 29900, 29946, 29946, 29871, 29945, 29946, 29953, 29871, 29929, 29941, 29947, 29913, 2305, 313, 29945, 29945, 29889, 29945, 8958, 310, 8471, 2305, 297, 29871, 29906, 29900, 29896, 29946, 467, 32001, 1094, 310, 3839, 29871, 29906, 29900, 29896, 29929, 372, 18469, 320, 1949, 29912, 29906, 29955, 29871, 29953, 29953, 29906, 29871, 29947, 29900, 29896, 29913, 363, 264, 1280, 322, 320, 1949, 29912, 29906, 29955, 29871, 29906, 29900, 29953, 29871, 29947, 29906, 29896, 29913, 269, 595, 1280, 297, 29871, 29906, 29941, 29953, 24894, 8977, 1080, 29889, 4907, 29913, 32001, 1094, 297, 1749, 8783, 15717, 526, 2989, 1024, 6031, 313, 29878, 1624, 1135, 6219, 491, 937, 29914, 11922, 1024, 511, 591, 937, 5993, 675, 2983, 313, 1609, 1999, 1331, 322, 1206, 3620, 29897, 322, 769, 16280, 5375, 18897, 297, 1716, 937, 322, 3942, 2983, 10868, 8857, 29889, 32001, 1152, 1269, 1543, 1476, 297, 1024, 8857, 591, 22932, 278, 2058, 4665, 29905, 6661, 1333, 24212, 8875, 491, 278, 1024, 10868, 304, 4017, 263, 5645, 393, 338, 29839, 304, 278, 1353, 310, 12407, 24638, 393, 1024, 313, 6979, 29897, 297, 278, 2702, 2058, 29889, 32001, 320, 6661, 1333, 300, 1062, 29912, 1762, 4017, 4665, 2025, 1338, 5634, 294, 278, 17837, 310, 4954, 6689, 4907, 338, 25745, 23724, 29901, 2989, 10916, 325, 13217, 29879, 29399, 310, 2919, 10916, 805, 9450, 2999, 931, 29920, 2873, 5634, 705, 671, 263, 29544, 310, 7601, 8974, 313, 29872, 29889, 29887, 1696, 5874, 28007, 511, 322, 1661, 29899, 16072, 6743, 313, 29872, 29889, 29887, 1696, 14109, 7456, 467, 29913, 32001, 1334, 2533, 445, 4377, 363, 599, 3161, 304, 4017, 263, 2058, 8158, 29892, 17140, 701, 411, 263, 1051, 310, 779, 6990, 29938, 6689, 29892, 8158, 4535, 5854, 29938, 11000, 29889, 32001, 1334, 769, 8877, 445, 1051, 491, 278, 3186, 5120, 393, 263, 2058, 14393, 304, 322, 2533, 278, 8158, 363, 599, 278, 7600, 297, 1269, 5120, 304, 4017, 385, 12463, 8158, 29892, 6590, 304, 278, 4188, 22342, 393, 278, 9063, 14393, 304, 263, 2183, 3186, 5120, 29889, 32001, 1334, 3566, 278, 6257, 9063, 408, 6421, 515, 278, 3186, 5120, 411, 278, 9939, 8158, 29889, 32001, 450, 4876, 29899, 6707, 11043, 9378, 414, 515, 278, 9078, 322, 443, 5521, 8362, 671, 310, 21759, 29911, 10249, 29879, 29901, 1556, 18777, 671, 10035, 323, 10249, 29879, 1316, 408, 320, 16266, 26139, 510, 1118, 320, 16266, 26139, 990, 1118, 470, 320, 16266, 26139, 1212, 1836, 32001, 12808, 445, 947, 451, 3799, 26018, 4822, 20542, 29901, 3148, 29899, 6707, 18777, 29892, 363, 1342, 29892, 671, 278, 320, 16266, 26139, 375, 29913, 21759, 29911, 10249, 1568, 901, 269, 2495, 290, 368, 1135, 1009, 7824, 6795, 20895, 29889, 32001, 1551, 278, 916, 1361, 278, 9210, 29914, 978, 29899, 6707, 11043, 337, 3687, 373, 278, 17998, 9210, 310, 278, 9063, 5335, 342, 15092, 29889, 32001, 16809, 304, 5780, 22920, 373, 13897, 731, 14340, 29892, 263, 2919, 1353, 310, 25741, 297, 278, 8783, 756, 385, 17998, 9210, 5186, 304, 5225, 29889, 32001, 910, 6602, 29879, 3109, 7786, 25741, 3441, 14573, 5261, 1169, 29911, 29999, 29999, 13985, 1349, 681, 392, 27418, 6478, 29914, 310, 29871, 29906, 29900, 29906, 29900, 29879, 25741, 505, 263, 5225, 9210, 29897, 1135, 9642, 6743, 3441, 14573, 5261, 1169, 29911, 29999, 29999, 13985, 1349, 681, 392, 29914, 297, 29871, 29906, 29900, 29900, 29900, 467, 32001, 1094, 263, 1121, 278, 9210, 29914, 978, 29899, 6707, 11043, 1033, 1095, 701, 6459, 292, 263, 2919, 6232, 310, 9642, 25741, 408, 4148, 287, 491, 11715, 18777, 29892, 322, 304, 263, 3109, 261, 15834, 4092, 550, 29889, 32001, 1763, 6795, 1438, 5626, 591, 14405, 278, 1023, 1737, 324, 10610, 13698, 4208, 491, 15399, 278, 9210, 29914, 978, 29899, 6707, 13698, 304, 599, 25741, 411, 263, 1661, 29899, 9171, 17998, 9210, 29892, 322, 278, 4876, 29899, 6707, 373, 304, 599, 916, 25741, 29889, 32001, 320, 2042, 29912, 12191, 322, 8565, 1558, 291, 29913, 13, 29905, 1643, 29912, 3471, 29901, 9902, 29913, 13, 13, 29905, 463, 29912, 4532, 4044, 13, 29871, 320, 9525, 13, 29871, 320, 7313, 29961, 2103, 2013, 16292, 3199, 1429, 287, 29889, 5140, 29913, 13, 29871, 320, 6671, 29912, 29934, 20819, 310, 25741, 313, 27215, 29897, 322, 6136, 15717, 313, 22503, 29897, 491, 3186, 10640, 975, 278, 29871, 29896, 29929, 29955, 29896, 489, 29906, 29900, 29906, 29900, 3785, 5003, 32001, 320, 9868, 29961, 14732, 3199, 7264, 287, 2594, 8727, 6445, 278, 3186, 10640, 364, 2219, 359, 363, 25741, 322, 15717, 975, 278, 29871, 29896, 29929, 29955, 29896, 489, 29906, 29900, 29906, 29900, 3785, 5003, 32001, 320, 1643, 29912, 1003, 29901, 9902, 29913, 13, 29905, 355, 29912, 4532, 4044, 13, 13, 13, 29871, 13, 1762, 1234, 320, 25029, 29912, 29878, 29939, 29901, 479, 397, 24974, 29913, 591, 22229, 278, 1353, 310, 25741, 322, 8359, 15717, 639, 1629, 322, 639, 3186, 10640, 29889, 32001, 1334, 2198, 278, 7625, 2582, 297, 320, 29907, 999, 29912, 1003, 29901, 9902, 29913, 408, 1023, 5096, 287, 2594, 24469, 29892, 6445, 1629, 368, 2867, 3204, 29879, 363, 25741, 322, 15717, 8307, 29889, 32001, 7569, 2594, 11524, 263, 1629, 322, 338, 8877, 287, 297, 269, 29399, 6445, 278, 9063, 29914, 8921, 11959, 363, 1269, 310, 278, 3186, 12786, 310, 320, 29907, 999, 29912, 1003, 29901, 11526, 1958, 29913, 297, 393, 1629, 29889, 32001, 1763, 4772, 714, 27801, 2861, 304, 805, 272, 26538, 17737, 29560, 29892, 297, 278, 4148, 8727, 591, 871, 2050, 15717, 2534, 26869, 472, 3203, 29871, 29945, 25741, 297, 263, 2183, 1629, 29889, 32001, 5806, 5366, 1747, 534, 1975, 297, 278, 24469, 6456, 393, 278, 3001, 3694, 310, 25741, 322, 15717, 6548, 18709, 9247, 975, 931, 29889, 32001, 10133, 363, 278, 937, 2440, 297, 278, 24469, 29892, 278, 1353, 310, 848, 3291, 297, 777, 3186, 12786, 508, 367, 14154, 2319, 29892, 411, 8178, 27721, 373, 278, 25806, 310, 534, 1975, 29889, 32001, 320, 26956, 29912, 7999, 12122, 6894, 537, 975, 931, 29913, 13, 13, 3563, 497, 29892, 278, 2498, 534, 355, 5692, 304, 367, 393, 278, 320, 4534, 29912, 479, 12122, 6894, 537, 297, 970, 775, 338, 10231, 6177, 4644, 6813, 322, 4092, 5136, 630, 1009, 4954, 24130, 749, 4907, 2745, 278, 7256, 310, 278, 29871, 29929, 29900], 'attention_mask': [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1], 'labels': [1, 320, 2042, 29912, 25898, 29913, 13, 29905, 1643, 29912, 3471, 29901, 23333, 29913, 13, 13, 29905, 7278, 29912, 29954, 1581, 6894, 537, 1118, 470, 901, 4049, 967, 10225, 727, 974, 29892, 4249, 27138, 304, 13, 20415, 5849, 14188, 756, 1063, 26606, 12399, 297, 7786, 2440, 29889, 32001, 512, 13, 1595, 16311, 29892, 278, 10122, 310, 29892, 9545, 310, 29892, 322, 6795, 1004, 25414, 363, 320, 7278, 29912, 26098, 13, 29871, 24003, 29913, 297, 12362, 29914, 6585, 7562, 18540, 313, 5800, 1799, 29897, 505, 4520, 263, 3287, 310, 8570, 13, 957, 278, 4940, 316, 6332, 2651, 2036, 29912, 29881, 16093, 29906, 29900, 29900, 29947, 29888, 2209, 3359, 29879, 29892, 3855, 5871, 29906, 29900, 29896, 29900, 29895, 311, 29893, 2770, 29892, 13, 29871, 1055, 29888, 375, 29906, 29900, 29896, 29906, 5041, 267, 29892, 413, 434, 305, 1358, 29906, 29900, 29896, 29906, 26098, 29888, 2209, 29892, 19723, 5475, 4979, 29906, 29900, 29896, 29946, 26098, 29892, 13, 29871, 697, 309, 29906, 29900, 29896, 29953, 311, 5365, 550, 332, 6950, 29892, 696, 7586, 29906, 29900, 29896, 29953, 29893, 2770, 7192, 2209, 29892, 1935, 15044, 29906, 29900, 29896, 29955, 26098, 29892, 13, 29871, 503, 562, 305, 3350, 492, 29906, 29900, 29906, 29896, 26098, 1836, 32001, 320, 7278, 29912, 7999, 12122, 6894, 537, 29913, 338, 373, 278, 916, 1361, 278, 13, 14380, 310, 6894, 537, 393, 380, 1567, 515, 27138, 297, 777, 5534, 6354, 6421, 13, 3166, 1422, 3186, 12786, 322, 4185, 1973, 29889, 32001, 26304, 6894, 537, 297, 18322, 1799, 756, 4520, 13774, 2217, 8570, 297, 21344, 368, 13, 13129, 29889, 32001, 512, 3153, 29892, 1550, 3031, 979, 18994, 29899, 6707, 322, 13, 3149, 29899, 262, 29899, 2230, 18350, 29899, 7052, 11898, 310, 278, 1737, 12122, 1677, 1144, 310, 18322, 1799, 13, 21570, 29560, 1863, 2651, 2036, 29912, 12443, 10578, 29906, 29900, 29900, 29945, 5062, 11235, 29892, 13260, 333, 29906, 29900, 29900, 29947, 29888, 2209, 3359, 29879, 29892, 13, 29871, 2594, 801, 2681, 29906, 29900, 29900, 29947, 479, 397, 24974, 29892, 1850, 29882, 371, 4099, 29894, 29906, 29900, 29896, 29900, 2209, 479, 5275, 29892, 696, 7586, 29906, 29900, 29896, 29946, 7610, 345, 2941, 271, 24541, 29892, 13, 29871, 281, 496, 29879, 29906, 29900, 29906, 29896, 2209, 479, 5275, 1118, 2919, 29899, 7052, 25579, 979, 11898, 310, 278, 1737, 12122, 13, 12574, 310, 18322, 1799, 17737, 29560, 526, 1603, 10225, 292, 29889, 32001, 10506, 263, 4323, 23378, 13, 18609, 2133, 723, 367, 5407, 304, 1871, 1602, 12112, 4475, 304, 5534, 13, 25431, 10907, 2651, 2036, 29912, 2276, 5824, 19982, 29906, 29900, 29900, 29955, 23705, 1338, 29893, 996, 29913, 322, 298, 8491, 16650, 583, 297, 278, 13, 19678, 15483, 313, 1806, 29897, 9999, 29892, 408, 1532, 408, 29126, 2114, 950, 2472, 13, 517, 278, 2553, 1078, 373, 278, 17407, 10879, 322, 5374, 3002, 310, 18322, 1799, 2820, 278, 3186, 29889, 32001, 320, 26956, 29912, 1323, 3224, 29879, 29913, 13, 13, 3047, 445, 664, 591, 29126, 304, 3802, 445, 17261, 491, 7512, 292, 320, 4534, 29912, 1552, 937, 13, 29871, 25579, 979, 6559, 310, 278, 1737, 12122, 3978, 310, 17737, 29560, 304, 970, 775, 13, 29871, 975, 29871, 29945, 29900, 2440, 5003, 32001, 26321, 29892, 591, 3867, 263, 758, 2576, 3821, 1234, 304, 278, 13, 23031, 292, 5925, 1139, 29901, 13, 29905, 463, 29912, 690, 2842, 12470, 29913, 13, 29871, 3645, 607, 3186, 12786, 437, 15717, 310, 970, 368, 3625, 25741, 2041, 515, 13, 29871, 322, 920, 756, 372, 3939, 975, 278, 4940, 29871, 29945, 29900, 2440, 29973, 32001, 320, 1643, 29912, 29878, 29939, 29901, 479, 397, 24974, 29913, 13, 29905, 355, 29912, 690, 2842, 12470, 29913, 13, 4806, 671, 408, 8783, 278, 320, 23066, 29950, 29914, 18871, 2651, 2036, 29912, 29879, 1332, 29875, 4569, 29906, 29900, 29896, 29955, 29913, 322, 27599, 515, 372, 13, 29906, 29889, 29906, 24464, 29905, 13007, 25741, 3190, 2347, 515, 29871, 29896, 29953, 29900, 7284, 29905, 13007, 9279, 322, 4148, 287, 491, 13, 29946, 29941, 7284, 29905, 13007, 15717, 2645, 278, 29871, 29896, 29929, 29955, 29896, 489, 29906, 29900, 29906, 29896, 931, 3785, 29889, 32001, 1334, 1737, 324, 542, 403, 18777, 304, 13, 29905, 14573, 14058, 4597, 1080, 29914, 3186, 12786, 29892, 773, 408, 18470, 4876, 4234, 775, 2246, 29899, 5563, 21904, 313, 617, 29911, 10249, 29879, 29897, 322, 29871, 13, 8921, 313, 4102, 29914, 4230, 29897, 2983, 9401, 411, 1024, 18822, 2820, 278, 3186, 29892, 322, 17998, 1283, 7224, 29871, 13, 1195, 287, 515, 9063, 15562, 29889, 32001, 1334, 1284, 10757, 310, 278, 4688, 8022, 749, 310, 4644, 6813, 297, 1722, 2752, 13, 20415, 29892, 2678, 8772, 491, 4092, 29889, 32001, 2860, 393, 3785, 29892, 278, 1737, 12122, 6894, 537, 29871, 13, 262, 970, 775, 756, 1063, 21003, 10231, 29889, 32001, 1334, 884, 12439, 8018, 15839, 528, 17741, 13, 12817, 304, 278, 1095, 310, 278, 8291, 6415, 29129, 322, 278, 7910, 310, 14137, 4631, 4135, 297, 13, 23369, 1705, 322, 4275, 14325, 29892, 408, 1532, 408, 310, 2545, 1664, 17292, 28342, 763, 25539, 1608, 322, 13, 25719, 10298, 4822, 10916, 313, 6727, 16783, 29914, 331, 16783, 467, 32001, 320, 26956, 29912, 1469, 20847, 3097, 5003, 32001, 319, 1634, 1414, 3577, 363, 445, 5650, 338, 3625, 515, 796, 264, 8144, 472, 13, 29905, 2271, 29912, 991, 597, 1867, 29875, 29889, 990, 29914, 29896, 29900, 29889, 29945, 29906, 29947, 29896, 29914, 2256, 8144, 29889, 29953, 29941, 29929, 29900, 29941, 29945, 29945, 29913, 2651, 2036, 29912, 3445, 1414, 29899, 5113, 1836, 32001, 320, 2042, 29912, 9662, 630, 5244, 29913, 13, 29905, 1643, 29912, 3471, 29901, 12817, 29913, 13, 13, 29933, 720, 4688, 322, 7786, 1736, 2651, 2036, 29912, 12443, 10578, 29906, 29900, 29900, 29945, 5062, 11235, 29892, 13260, 333, 29906, 29900, 29900, 29947, 29888, 2209, 3359, 29879, 29892, 13, 29871, 696, 7586, 29906, 29900, 29896, 29946, 7610, 345, 2941, 271, 24541, 29892, 697, 309, 29906, 29900, 29896, 29953, 311, 5365, 550, 332, 6950, 29913, 505, 2931, 1891, 278, 13, 479, 5275, 310, 12362, 29914, 6585, 7562, 18540, 313, 5800, 1799, 29897, 773, 320, 7278, 29912, 6734, 26946, 952, 1118, 13, 4716, 3867, 1880, 29899, 29567, 6089, 541, 526, 9078, 297, 2159, 313, 29906, 29899, 29945, 5940, 29968, 18777, 29897, 13, 392, 508, 367, 4768, 1463, 491, 5221, 424, 23460, 29889, 32001, 512, 29871, 29906, 29900, 29900, 29947, 2261, 801, 2681, 634, 394, 8770, 2036, 29912, 1646, 801, 2681, 29906, 29900, 29900, 29947, 479, 397, 24974, 29913, 18043, 263, 3031, 979, 13, 16961, 29899, 7052, 313, 1454, 278, 931, 29897, 6559, 373, 18322, 1799, 320, 7278, 29912, 479, 5275, 773, 1375, 292, 7047, 13, 29871, 28914, 313, 4345, 29934, 29897, 13698, 1836, 32001, 2688, 29537, 287, 278, 3978, 310, 29871, 29896, 5940, 29924, 17737, 29560, 13, 4746, 278, 7562, 2831, 479, 1404, 2566, 322, 611, 6504, 1051, 3190, 3145, 975, 278, 13, 29896, 29929, 29929, 29929, 489, 29906, 29900, 29900, 29945, 3785, 29892, 773, 408, 18470, 2472, 2788, 304, 1749, 29879, 29901, 4876, 21904, 13, 392, 17998, 1283, 7224, 29889, 32001, 450, 12399, 3785, 313, 29955, 2440, 29897, 297, 2651, 2036, 29912, 1646, 801, 2681, 29906, 29900, 29900, 29947, 479, 397, 24974, 29913, 338, 20511, 1135, 29871, 13, 5816, 338, 12399, 297, 278, 2198, 5650, 313, 29945, 29900, 2440, 29897, 322, 278, 848, 8974, 526, 29871, 13, 27489, 873, 1422, 29936, 411, 393, 297, 3458, 29892, 1749, 2582, 1510, 263, 10029, 7200, 14978, 310, 29871, 13, 15654, 273, 325, 13217, 29940, 2072, 3082, 20706, 29889, 32001, 7280, 29190, 936, 664, 515, 29871, 29906, 29900, 29896, 29900, 491, 14619, 29882, 371, 4099, 29894, 322, 13, 29950, 309, 1372, 2651, 2036, 29912, 29873, 19426, 371, 4099, 29894, 29906, 29900, 29896, 29900, 2209, 479, 5275, 29913, 4023, 29894, 2868, 1583, 29899, 7099, 433, 1127, 1737, 12122, 13, 2029, 800, 310, 25492, 15303, 8304, 3598, 1494, 1009, 12368, 29892, 13, 15914, 292, 2472, 363, 779, 14850, 4535, 29892, 29955, 29900, 5940, 29968, 25492, 4160, 29889, 32001, 319, 1407, 7786, 13, 1287, 2651, 2036, 29912, 21686, 29879, 29906, 29900, 29906, 29896, 2209, 479, 5275, 29913, 491, 399, 496, 29879, 634, 394, 13217, 5349, 1737, 324, 542, 630, 4203, 263, 7284, 13, 28712, 16046, 4160, 29892, 2534, 26869, 472, 3203, 29871, 29896, 29900, 29900, 25741, 1269, 29892, 322, 1058, 13, 1311, 29899, 7099, 8663, 14354, 373, 1009, 25492, 28723, 29889, 32001, 5806, 278, 6559, 338, 13, 3149, 29899, 262, 29899, 2230, 408, 310, 29871, 29906, 29900, 29906, 29896, 29892, 278, 15717, 7252, 1009, 1284, 886, 13, 351, 475, 303, 2651, 2036, 29912, 1646, 801, 2681, 29906, 29900, 29900, 29947, 479, 397, 24974, 29892, 1850, 29882, 371, 4099, 29894, 29906, 29900, 29896, 29900, 2209, 479, 5275, 29913, 304, 13, 18609, 675, 278, 14675, 310, 18322, 1799, 1737, 5275, 975, 278, 931, 15101, 845, 1862, 4586, 491, 13, 1552, 2211, 11898, 29889, 32001, 3831, 1965, 411, 3517, 29190, 936, 1736, 29892, 1749, 6559, 338, 1568, 7200, 6287, 5634, 29882, 5555, 13, 7054, 12339, 287, 29871, 29946, 29941, 7284, 29905, 13007, 15717, 310, 29871, 29906, 29889, 29906, 24464, 29905, 13007, 25741, 515, 29871, 29896, 29953, 29900, 7284, 29905, 13007, 13, 16418, 5634, 5426, 11267, 979, 975, 29871, 29945, 29900, 2440, 310, 970, 775, 20706, 3265, 1135, 13, 3149, 297, 931, 29892, 322, 884, 901, 2691, 29899, 3874, 1312, 313, 2541, 1629, 29899, 1609, 29899, 6360, 3803, 1070, 537, 975, 13, 1552, 8900, 3785, 467, 32001, 8108, 1189, 1711, 29892, 1749, 6559, 337, 3687, 373, 10079, 11264, 13, 3924, 313, 8257, 29903, 29897, 9063, 848, 3265, 1135, 7481, 29899, 7099, 433, 1127, 4423, 2472, 29889, 32001, 5901, 1736, 5634, 262, 3153, 278, 664, 491, 8432, 2651, 2036, 29912, 18386, 709, 29906, 29900, 29896, 29941, 2209, 29881, 24974, 29913, 13, 392, 29892, 901, 10325, 29892, 390, 579, 468, 29875, 634, 394, 8770, 2036, 29912, 29878, 579, 468, 29875, 29906, 29900, 29896, 29953, 479, 711, 3173, 29892, 13, 29871, 364, 579, 468, 29875, 29906, 29900, 29896, 29947, 479, 711, 3173, 29892, 544, 1648, 29906, 29900, 29906, 29896, 479, 468, 1581, 29881, 24974, 29913, 5634, 17532, 12399, 1737, 12122, 13, 29905, 7278, 29912, 29881, 24974, 322, 24003, 1118, 474, 29889, 29872, 1696, 278, 15834, 304, 607, 278, 3978, 310, 18322, 1799, 13, 17426, 6602, 1009, 11465, 1230, 14137, 14188, 29889, 32001, 512, 445, 664, 591, 2931, 1891, 1737, 12122, 6894, 537, 297, 970, 775, 363, 278, 937, 13, 2230, 472, 445, 6287, 29892, 1716, 297, 4958, 310, 17737, 29560, 322, 15500, 3785, 29889, 32001, 1334, 437, 13, 1333, 22002, 280, 278, 24003, 10696, 29892, 541, 3867, 29190, 936, 848, 322, 1284, 886, 393, 508, 367, 13, 280, 369, 4063, 304, 393, 1095, 408, 5434, 664, 29889, 32001, 320, 7278, 29912, 12756, 7047, 21639, 29913, 2651, 2036, 29912, 2276, 5824, 19982, 29906, 29900, 29900, 29955, 23705, 1338, 29893, 996, 29913, 338, 278, 13, 1491, 29899, 2671, 310, 7047, 21639, 393, 756, 29537, 287, 278, 18066, 267, 310, 21640, 13, 6734, 24771, 13149, 635, 29892, 3704, 278, 2702, 5932, 310, 920, 304, 5376, 13, 2541, 1737, 12122, 6894, 537, 2651, 2036, 29912, 13376, 303, 456, 29906, 29900, 29900, 29953, 10945, 3359, 29892, 1424, 29440, 29906, 29900, 29896, 29946, 23027, 5933, 1836, 32001, 3826, 3076, 2678, 278, 2198, 6559, 8128, 10757, 393, 508, 367, 1304, 29892, 297, 278, 13, 14940, 1206, 310, 970, 775, 322, 472, 263, 1407, 2919, 6287, 29892, 304, 11539, 607, 13, 14032, 4637, 310, 5534, 7047, 21639, 505, 9820, 484, 15774, 29889, 32001, 320, 2042, 29912, 4062, 3002, 29913, 13, 29905, 1643, 29912, 3471, 29901, 5696, 29913, 13, 13, 13, 29905, 1482, 361, 29905, 361, 29887, 798, 386, 1003, 29871, 320, 29887, 798, 386, 1003, 3009, 13, 29905, 361, 29887, 798, 386, 1003, 13, 29905, 463, 29912, 4532, 29913, 13, 29871, 320, 7313, 29961, 2103, 2013, 4914, 2103, 3199, 6360, 368, 29899, 2055, 1169, 29913, 13, 29871, 320, 6671, 29912, 12883, 368, 970, 25741, 975, 931, 313, 1188, 6287, 467, 13, 29913, 32001, 320, 1643, 29912, 1003, 29901, 29887, 798, 386, 29913, 13, 29905, 355, 29912, 4532, 29913, 13, 29905, 7241, 13, 13, 29905, 26956, 29912, 16390, 24541, 29913, 13, 13, 4806, 27387, 515, 320, 23066, 29950, 29914, 2651, 2036, 29912, 29879, 1332, 29899, 1516, 29878, 29906, 29900, 29896, 29929, 29899, 24713, 29913, 599, 25741, 3190, 2347, 2745, 320, 25832, 1964, 579, 1523, 2415, 2539, 6294, 32001, 2688, 5253, 304, 320, 14573, 5261, 1169, 22131, 29914, 25741, 29892, 5412, 491, 317, 15715, 29896, 15882, 29892, 4023, 29894, 2868, 515, 320, 25832, 1299, 7288, 5261, 1169, 797, 7068, 29914, 970, 9279, 6421, 515, 4655, 5849, 363, 2710, 313, 28712, 16046, 29892, 11786, 28632, 29892, 2992, 1846, 32001, 322, 3577, 28914, 313, 10251, 713, 29892, 10772, 2227, 29892, 405, 13427, 29892, 2992, 6250, 32001, 1876, 1169, 297, 278, 8783, 526, 491, 320, 25832, 6344, 2806, 943, 22131, 29914, 15717, 29892, 5412, 491, 779, 6990, 29938, 978, 29892, 4876, 4535, 5854, 29938, 11000, 29889, 32001, 450, 8783, 2996, 408, 1023, 1104, 1288, 6131, 29892, 697, 363, 25741, 322, 697, 363, 15717, 29892, 411, 278, 4642, 29371, 278, 7480, 3025, 263, 9117, 1820, 29889, 32001, 320, 361, 5426, 13, 9760, 1948, 297, 278, 9063, 1591, 3743, 278, 1494, 4235, 29901, 9063, 317, 15715, 29896, 15882, 29892, 4148, 322, 844, 5171, 5335, 342, 15092, 29892, 4148, 322, 844, 5171, 2893, 14903, 313, 20275, 16750, 278, 4148, 1591, 467, 32001, 450, 21578, 1546, 9063, 15717, 322, 9063, 2153, 2041, 515, 11786, 29892, 607, 6511, 304, 9063, 263, 1735, 4148, 287, 491, 4856, 1683, 29889, 32001, 1152, 445, 6559, 591, 21309, 373, 15717, 322, 17262, 9063, 2153, 29892, 408, 278, 4328, 1546, 278, 1023, 338, 451, 8018, 363, 1749, 5925, 5155, 322, 278, 5253, 310, 25741, 411, 263, 844, 5171, 916, 1135, 967, 4148, 338, 3480, 3473, 1821, 29889, 32001, 320, 7241, 13, 2831, 1269, 6251, 297, 278, 4148, 1591, 591, 505, 4148, 2989, 1024, 322, 4876, 408, 1023, 5004, 6031, 310, 10650, 6262, 29889, 32001, 1334, 6206, 2411, 15273, 1821, 470, 18325, 519, 2983, 393, 29901, 526, 451, 1602, 397, 519, 408, 18351, 29899, 29947, 3441, 25832, 6344, 2806, 943, 29934, 29885, 12283, 7099, 397, 519, 29914, 4148, 2983, 6206, 511, 526, 4876, 14157, 2012, 310, 2983, 3441, 25832, 6344, 2806, 943, 29934, 29885, 9823, 29914, 4954, 7039, 4907, 511, 5718, 310, 871, 9654, 4890, 3441, 25832, 6344, 2806, 943, 29934, 29885, 10358, 804, 23201, 1712, 901, 1135, 29871, 29896, 29900, 8958, 1661, 29899, 1026, 2153, 3441, 25832, 6344, 2806, 943, 29934, 29885, 12283, 15670, 23201, 526, 5520, 1135, 29871, 29896, 29900, 29900, 4890, 3441, 25832, 6344, 2806, 943, 29934, 29885, 12229, 549, 12495, 32001, 2860, 21166, 29892, 1048, 320, 25832, 6344, 2806, 943, 29925, 15273, 1821, 2052, 307, 29916, 29914, 15717, 3441, 25832, 6344, 2806, 943, 29925, 15273, 1821, 29925, 312, 29914, 310, 278, 2847, 8783, 29897, 9488, 363, 4340, 7418, 29889, 32001, 3940, 393, 278, 5253, 310, 970, 775, 25741, 313, 392, 15717, 29897, 11122, 297, 278, 13, 11228, 8783, 25088, 18709, 9247, 975, 13, 2230, 2651, 2036, 29912, 29879, 1332, 29899, 771, 854, 749, 29899, 331, 344, 1012, 361, 29887, 798, 386, 1003, 29892, 408, 4318, 363, 25741, 297, 13, 29905, 29907, 999, 29912, 1003, 29901, 29887, 798, 386, 1012, 2870, 29901, 515, 395, 29896, 29900, 29985, 29946, 29938, 25741, 297, 29871, 29896, 29929, 29955, 29896, 29892, 304, 395, 29896, 29900, 29985, 29953, 29938, 297, 29871, 29896, 29929, 29929, 29947, 29892, 304, 13, 284, 3242, 395, 29896, 29900, 29985, 29929, 29938, 297, 29871, 29906, 29900, 29906, 29900, 29905, 7241, 29889, 32001, 1094, 263, 17004, 278, 8900, 534, 1975, 10331, 304, 367, 901, 13, 13844, 297, 7786, 1602, 3076, 1135, 297, 29871, 29946, 29900, 29974, 1629, 29899, 1025, 6743, 29892, 2861, 304, 13964, 4586, 373, 13, 735, 1112, 9247, 7200, 23093, 29889, 32001, 320, 26956, 29912, 7999, 324, 10610, 29913, 13, 13, 29905, 463, 29912, 4532, 29913, 13, 29871, 320, 9525, 13, 29871, 320, 7313, 29961, 24049, 29892, 15450, 29922, 29953, 4912, 29871, 29953, 4912, 29871, 29900, 29871, 29900, 29892, 2103, 2013, 16292, 3199, 1491, 1727, 1080, 29899, 2470, 29913, 13, 29871, 320, 6671, 29912, 1576, 320, 14573, 14058, 4597, 1080, 29914, 3186, 12786, 1304, 408, 1737, 324, 10610, 22525, 5003, 32001, 320, 1643, 29912, 1003, 29901, 11526, 1958, 29913, 13, 29905, 355, 29912, 4532, 29913, 13, 13, 2887, 1737, 324, 10610, 22525, 591, 671, 11758, 3186, 12786, 10723, 515, 278, 3303, 18269, 1737, 359, 305, 2004, 2651, 2036, 29912, 348, 29896, 29929, 29929, 29929, 479, 359, 305, 2004, 1836, 32001, 1763, 4772, 2432, 3381, 491, 2919, 10916, 313, 29872, 29889, 29887, 1696, 7551, 470, 12710, 29897, 2629, 11758, 12786, 29892, 591, 19412, 322, 6219, 777, 12786, 2729, 373, 1737, 12122, 23203, 537, 322, 278, 19383, 310, 758, 331, 8946, 16375, 29769, 5680, 29892, 1316, 408, 19182, 4086, 29889, 32001, 320, 29907, 999, 29912, 1003, 29901, 11526, 1958, 29913, 3697, 278, 2186, 1051, 310, 320, 14573, 14058, 4597, 1080, 29914, 3186, 12786, 1304, 408, 1737, 324, 10610, 22525, 297, 445, 6559, 29889, 32001, 1879, 324, 10610, 310, 9063, 15717, 304, 3186, 12786, 3913, 278, 1023, 19595, 653, 13698, 9129, 297, 2651, 2036, 29912, 293, 344, 29899, 344, 275, 29899, 29906, 29900, 29906, 29906, 29899, 26098, 1118, 23359, 337, 13998, 2400, 29889, 32001, 450, 937, 697, 337, 3687, 373, 278, 4234, 775, 2246, 29899, 5563, 5354, 313, 617, 29911, 10249, 29897, 310, 4876, 14157, 23892, 515, 9063, 15562, 29892, 321, 29889, 29887, 1696, 320, 16266, 26139, 1341, 1118, 320, 16266, 26139, 582, 1118, 320, 16266, 26139, 18038, 1118, 2992, 29889, 32001, 1334, 4687, 515, 278, 306, 2190, 29909, 1051, 310, 13548, 2931, 21759, 29911, 10249, 29879, 2651, 2036, 29912, 6011, 29899, 29883, 312, 430, 29913, 322, 7522, 20545, 1269, 6590, 20123, 304, 263, 3646, 3186, 5120, 29889, 32001, 450, 1473, 1737, 324, 10610, 11043, 3913, 278, 17998, 9210, 310, 9063, 5335, 342, 15092, 313, 29872, 29889, 29887, 1696, 17998, 29899, 29900, 29945, 29901, 29900, 29900, 29897, 322, 4148, 2983, 304, 8161, 278, 1556, 5517, 3186, 5120, 310, 278, 9063, 4148, 29889, 32001, 1152, 1269, 17998, 9210, 591, 8161, 263, 1051, 310, 15878, 7600, 313, 13509, 29892, 2106, 29892, 470, 14278, 20123, 29897, 297, 278, 3186, 393, 29892, 472, 278, 931, 310, 393, 9063, 29892, 750, 393, 17998, 9210, 29936, 9063, 931, 338, 1820, 1244, 29892, 408, 4234, 17998, 1283, 7224, 13100, 975, 931, 2861, 304, 29431, 3620, 29889, 32001, 1763, 1207, 445, 3683, 3381, 591, 671, 278, 306, 2190, 29909, 931, 10640, 2566, 2651, 2036, 29912, 17559, 1272, 1836, 32001, 1987, 591, 3566, 304, 1269, 2058, 263, 8158, 393, 4332, 1973, 278, 4188, 22342, 393, 263, 2183, 4148, 1024, 338, 17443, 310, 372, 29889, 32001, 1763, 445, 1095, 591, 671, 278, 28297, 29890, 15451, 8783, 310, 278, 29511, 310, 278, 1556, 3619, 937, 322, 3942, 2983, 607, 29892, 439, 11427, 515, 2651, 2036, 29912, 1079, 29890, 799, 29899, 7039, 6177, 2802, 277, 12181, 4954, 16123, 2247, 278, 26368, 5528, 5084, 310, 363, 264, 1280, 322, 269, 595, 1280, 7371, 515, 263, 2566, 310, 320, 1949, 29912, 29946, 29871, 29900, 29946, 29946, 29871, 29945, 29946, 29953, 29871, 29929, 29941, 29947, 29913, 2305, 313, 29945, 29945, 29889, 29945, 8958, 310, 8471, 2305, 297, 29871, 29906, 29900, 29896, 29946, 467, 32001, 1094, 310, 3839, 29871, 29906, 29900, 29896, 29929, 372, 18469, 320, 1949, 29912, 29906, 29955, 29871, 29953, 29953, 29906, 29871, 29947, 29900, 29896, 29913, 363, 264, 1280, 322, 320, 1949, 29912, 29906, 29955, 29871, 29906, 29900, 29953, 29871, 29947, 29906, 29896, 29913, 269, 595, 1280, 297, 29871, 29906, 29941, 29953, 24894, 8977, 1080, 29889, 4907, 29913, 32001, 1094, 297, 1749, 8783, 15717, 526, 2989, 1024, 6031, 313, 29878, 1624, 1135, 6219, 491, 937, 29914, 11922, 1024, 511, 591, 937, 5993, 675, 2983, 313, 1609, 1999, 1331, 322, 1206, 3620, 29897, 322, 769, 16280, 5375, 18897, 297, 1716, 937, 322, 3942, 2983, 10868, 8857, 29889, 32001, 1152, 1269, 1543, 1476, 297, 1024, 8857, 591, 22932, 278, 2058, 4665, 29905, 6661, 1333, 24212, 8875, 491, 278, 1024, 10868, 304, 4017, 263, 5645, 393, 338, 29839, 304, 278, 1353, 310, 12407, 24638, 393, 1024, 313, 6979, 29897, 297, 278, 2702, 2058, 29889, 32001, 320, 6661, 1333, 300, 1062, 29912, 1762, 4017, 4665, 2025, 1338, 5634, 294, 278, 17837, 310, 4954, 6689, 4907, 338, 25745, 23724, 29901, 2989, 10916, 325, 13217, 29879, 29399, 310, 2919, 10916, 805, 9450, 2999, 931, 29920, 2873, 5634, 705, 671, 263, 29544, 310, 7601, 8974, 313, 29872, 29889, 29887, 1696, 5874, 28007, 511, 322, 1661, 29899, 16072, 6743, 313, 29872, 29889, 29887, 1696, 14109, 7456, 467, 29913, 32001, 1334, 2533, 445, 4377, 363, 599, 3161, 304, 4017, 263, 2058, 8158, 29892, 17140, 701, 411, 263, 1051, 310, 779, 6990, 29938, 6689, 29892, 8158, 4535, 5854, 29938, 11000, 29889, 32001, 1334, 769, 8877, 445, 1051, 491, 278, 3186, 5120, 393, 263, 2058, 14393, 304, 322, 2533, 278, 8158, 363, 599, 278, 7600, 297, 1269, 5120, 304, 4017, 385, 12463, 8158, 29892, 6590, 304, 278, 4188, 22342, 393, 278, 9063, 14393, 304, 263, 2183, 3186, 5120, 29889, 32001, 1334, 3566, 278, 6257, 9063, 408, 6421, 515, 278, 3186, 5120, 411, 278, 9939, 8158, 29889, 32001, 450, 4876, 29899, 6707, 11043, 9378, 414, 515, 278, 9078, 322, 443, 5521, 8362, 671, 310, 21759, 29911, 10249, 29879, 29901, 1556, 18777, 671, 10035, 323, 10249, 29879, 1316, 408, 320, 16266, 26139, 510, 1118, 320, 16266, 26139, 990, 1118, 470, 320, 16266, 26139, 1212, 1836, 32001, 12808, 445, 947, 451, 3799, 26018, 4822, 20542, 29901, 3148, 29899, 6707, 18777, 29892, 363, 1342, 29892, 671, 278, 320, 16266, 26139, 375, 29913, 21759, 29911, 10249, 1568, 901, 269, 2495, 290, 368, 1135, 1009, 7824, 6795, 20895, 29889, 32001, 1551, 278, 916, 1361, 278, 9210, 29914, 978, 29899, 6707, 11043, 337, 3687, 373, 278, 17998, 9210, 310, 278, 9063, 5335, 342, 15092, 29889, 32001, 16809, 304, 5780, 22920, 373, 13897, 731, 14340, 29892, 263, 2919, 1353, 310, 25741, 297, 278, 8783, 756, 385, 17998, 9210, 5186, 304, 5225, 29889, 32001, 910, 6602, 29879, 3109, 7786, 25741, 3441, 14573, 5261, 1169, 29911, 29999, 29999, 13985, 1349, 681, 392, 27418, 6478, 29914, 310, 29871, 29906, 29900, 29906, 29900, 29879, 25741, 505, 263, 5225, 9210, 29897, 1135, 9642, 6743, 3441, 14573, 5261, 1169, 29911, 29999, 29999, 13985, 1349, 681, 392, 29914, 297, 29871, 29906, 29900, 29900, 29900, 467, 32001, 1094, 263, 1121, 278, 9210, 29914, 978, 29899, 6707, 11043, 1033, 1095, 701, 6459, 292, 263, 2919, 6232, 310, 9642, 25741, 408, 4148, 287, 491, 11715, 18777, 29892, 322, 304, 263, 3109, 261, 15834, 4092, 550, 29889, 32001, 1763, 6795, 1438, 5626, 591, 14405, 278, 1023, 1737, 324, 10610, 13698, 4208, 491, 15399, 278, 9210, 29914, 978, 29899, 6707, 13698, 304, 599, 25741, 411, 263, 1661, 29899, 9171, 17998, 9210, 29892, 322, 278, 4876, 29899, 6707, 373, 304, 599, 916, 25741, 29889, 32001, 320, 2042, 29912, 12191, 322, 8565, 1558, 291, 29913, 13, 29905, 1643, 29912, 3471, 29901, 9902, 29913, 13, 13, 29905, 463, 29912, 4532, 4044, 13, 29871, 320, 9525, 13, 29871, 320, 7313, 29961, 2103, 2013, 16292, 3199, 1429, 287, 29889, 5140, 29913, 13, 29871, 320, 6671, 29912, 29934, 20819, 310, 25741, 313, 27215, 29897, 322, 6136, 15717, 313, 22503, 29897, 491, 3186, 10640, 975, 278, 29871, 29896, 29929, 29955, 29896, 489, 29906, 29900, 29906, 29900, 3785, 5003, 32001, 320, 9868, 29961, 14732, 3199, 7264, 287, 2594, 8727, 6445, 278, 3186, 10640, 364, 2219, 359, 363, 25741, 322, 15717, 975, 278, 29871, 29896, 29929, 29955, 29896, 489, 29906, 29900, 29906, 29900, 3785, 5003, 32001, 320, 1643, 29912, 1003, 29901, 9902, 29913, 13, 29905, 355, 29912, 4532, 4044, 13, 13, 13, 29871, 13, 1762, 1234, 320, 25029, 29912, 29878, 29939, 29901, 479, 397, 24974, 29913, 591, 22229, 278, 1353, 310, 25741, 322, 8359, 15717, 639, 1629, 322, 639, 3186, 10640, 29889, 32001, 1334, 2198, 278, 7625, 2582, 297, 320, 29907, 999, 29912, 1003, 29901, 9902, 29913, 408, 1023, 5096, 287, 2594, 24469, 29892, 6445, 1629, 368, 2867, 3204, 29879, 363, 25741, 322, 15717, 8307, 29889, 32001, 7569, 2594, 11524, 263, 1629, 322, 338, 8877, 287, 297, 269, 29399, 6445, 278, 9063, 29914, 8921, 11959, 363, 1269, 310, 278, 3186, 12786, 310, 320, 29907, 999, 29912, 1003, 29901, 11526, 1958, 29913, 297, 393, 1629, 29889, 32001, 1763, 4772, 714, 27801, 2861, 304, 805, 272, 26538, 17737, 29560, 29892, 297, 278, 4148, 8727, 591, 871, 2050, 15717, 2534, 26869, 472, 3203, 29871, 29945, 25741, 297, 263, 2183, 1629, 29889, 32001, 5806, 5366, 1747, 534, 1975, 297, 278, 24469, 6456, 393, 278, 3001, 3694, 310, 25741, 322, 15717, 6548, 18709, 9247, 975, 931, 29889, 32001, 10133, 363, 278, 937, 2440, 297, 278, 24469, 29892, 278, 1353, 310, 848, 3291, 297, 777, 3186, 12786, 508, 367, 14154, 2319, 29892, 411, 8178, 27721, 373, 278, 25806, 310, 534, 1975, 29889, 32001, 320, 26956, 29912, 7999, 12122, 6894, 537, 975, 931, 29913, 13, 13, 3563, 497, 29892, 278, 2498, 534, 355, 5692, 304, 367, 393, 278, 320, 4534, 29912, 479, 12122, 6894, 537, 297, 970, 775, 338, 10231, 6177, 4644, 6813, 322, 4092, 5136, 630, 1009, 4954, 24130, 749, 4907, 2745, 278, 7256, 310, 278, 29871, 29929, 29900]} {'input_ids': [1, 320, 2042, 29912, 25898, 29913, 13, 29905, 1643, 29912, 3471, 29901, 23333, 29913, 13, 13, 29905, 7278, 29912, 29954, 1581, 6894, 537, 1118, 470, 901, 4049, 967, 10225, 727, 974, 29892, 4249, 27138, 304, 13, 20415, 5849, 14188, 756, 1063, 26606, 12399, 297, 7786, 2440, 29889, 32001, 512, 13, 1595, 16311, 29892, 278, 10122, 310, 29892, 9545, 310, 29892, 322, 6795, 1004, 25414, 363, 320, 7278, 29912, 26098, 13, 29871, 24003, 29913, 297, 12362, 29914, 6585, 7562, 18540, 313, 5800, 1799, 29897, 505, 4520, 263, 3287, 310, 8570, 13, 957, 278, 4940, 316, 6332, 2651, 2036, 29912, 29881, 16093, 29906, 29900, 29900, 29947, 29888, 2209, 3359, 29879, 29892, 3855, 5871, 29906, 29900, 29896, 29900, 29895, 311, 29893, 2770, 29892, 13, 29871, 1055, 29888, 375, 29906, 29900, 29896, 29906, 5041, 267, 29892, 413, 434, 305, 1358, 29906, 29900, 29896, 29906, 26098, 29888, 2209, 29892, 19723, 5475, 4979, 29906, 29900, 29896, 29946, 26098, 29892, 13, 29871, 697, 309, 29906, 29900, 29896, 29953, 311, 5365, 550, 332, 6950, 29892, 696, 7586, 29906, 29900, 29896, 29953, 29893, 2770, 7192, 2209, 29892, 1935, 15044, 29906, 29900, 29896, 29955, 26098, 29892, 13, 29871, 503, 562, 305, 3350, 492, 29906, 29900, 29906, 29896, 26098, 1836, 32001, 320, 7278, 29912, 7999, 12122, 6894, 537, 29913, 338, 373, 278, 916, 1361, 278, 13, 14380, 310, 6894, 537, 393, 380, 1567, 515, 27138, 297, 777, 5534, 6354, 6421, 13, 3166, 1422, 3186, 12786, 322, 4185, 1973, 29889, 32001, 26304, 6894, 537, 297, 18322, 1799, 756, 4520, 13774, 2217, 8570, 297, 21344, 368, 13, 13129, 29889, 32001, 512, 3153, 29892, 1550, 3031, 979, 18994, 29899, 6707, 322, 13, 3149, 29899, 262, 29899, 2230, 18350, 29899, 7052, 11898, 310, 278, 1737, 12122, 1677, 1144, 310, 18322, 1799, 13, 21570, 29560, 1863, 2651, 2036, 29912, 12443, 10578, 29906, 29900, 29900, 29945, 5062, 11235, 29892, 13260, 333, 29906, 29900, 29900, 29947, 29888, 2209, 3359, 29879, 29892, 13, 29871, 2594, 801, 2681, 29906, 29900, 29900, 29947, 479, 397, 24974, 29892, 1850, 29882, 371, 4099, 29894, 29906, 29900, 29896, 29900, 2209, 479, 5275, 29892, 696, 7586, 29906, 29900, 29896, 29946, 7610, 345, 2941, 271, 24541, 29892, 13, 29871, 281, 496, 29879, 29906, 29900, 29906, 29896, 2209, 479, 5275, 1118, 2919, 29899, 7052, 25579, 979, 11898, 310, 278, 1737, 12122, 13, 12574, 310, 18322, 1799, 17737, 29560, 526, 1603, 10225, 292, 29889, 32001, 10506, 263, 4323, 23378, 13, 18609, 2133, 723, 367, 5407, 304, 1871, 1602, 12112, 4475, 304, 5534, 13, 25431, 10907, 2651, 2036, 29912, 2276, 5824, 19982, 29906, 29900, 29900, 29955, 23705, 1338, 29893, 996, 29913, 322, 298, 8491, 16650, 583, 297, 278, 13, 19678, 15483, 313, 1806, 29897, 9999, 29892, 408, 1532, 408, 29126, 2114, 950, 2472, 13, 517, 278, 2553, 1078, 373, 278, 17407, 10879, 322, 5374, 3002, 310, 18322, 1799, 2820, 278, 3186, 29889, 32001, 320, 26956, 29912, 1323, 3224, 29879, 29913, 13, 13, 3047, 445, 664, 591, 29126, 304, 3802, 445, 17261, 491, 7512, 292, 320, 4534, 29912, 1552, 937, 13, 29871, 25579, 979, 6559, 310, 278, 1737, 12122, 3978, 310, 17737, 29560, 304, 970, 775, 13, 29871, 975, 29871, 29945, 29900, 2440, 5003, 32001, 26321, 29892, 591, 3867, 263, 758, 2576, 3821, 1234, 304, 278, 13, 23031, 292, 5925, 1139, 29901, 13, 29905, 463, 29912, 690, 2842, 12470, 29913, 13, 29871, 3645, 607, 3186, 12786, 437, 15717, 310, 970, 368, 3625, 25741, 2041, 515, 13, 29871, 322, 920, 756, 372, 3939, 975, 278, 4940, 29871, 29945, 29900, 2440, 29973, 32001, 320, 1643, 29912, 29878, 29939, 29901, 479, 397, 24974, 29913, 13, 29905, 355, 29912, 690, 2842, 12470, 29913, 13, 4806, 671, 408, 8783, 278, 320, 23066, 29950, 29914, 18871, 2651, 2036, 29912, 29879, 1332, 29875, 4569, 29906, 29900, 29896, 29955, 29913, 322, 27599, 515, 372, 13, 29906, 29889, 29906, 24464, 29905, 13007, 25741, 3190, 2347, 515, 29871, 29896, 29953, 29900, 7284, 29905, 13007, 9279, 322, 4148, 287, 491, 13, 29946, 29941, 7284, 29905, 13007, 15717, 2645, 278, 29871, 29896, 29929, 29955, 29896, 489, 29906, 29900, 29906, 29896, 931, 3785, 29889, 32001, 1334, 1737, 324, 542, 403, 18777, 304, 13, 29905, 14573, 14058, 4597, 1080, 29914, 3186, 12786, 29892, 773, 408, 18470, 4876, 4234, 775, 2246, 29899, 5563, 21904, 313, 617, 29911, 10249, 29879, 29897, 322, 29871, 13, 8921, 313, 4102, 29914, 4230, 29897, 2983, 9401, 411, 1024, 18822, 2820, 278, 3186, 29892, 322, 17998, 1283, 7224, 29871, 13, 1195, 287, 515, 9063, 15562, 29889, 32001, 1334, 1284, 10757, 310, 278, 4688, 8022, 749, 310, 4644, 6813, 297, 1722, 2752, 13, 20415, 29892, 2678, 8772, 491, 4092, 29889, 32001, 2860, 393, 3785, 29892, 278, 1737, 12122, 6894, 537, 29871, 13, 262, 970, 775, 756, 1063, 21003, 10231, 29889, 32001, 1334, 884, 12439, 8018, 15839, 528, 17741, 13, 12817, 304, 278, 1095, 310, 278, 8291, 6415, 29129, 322, 278, 7910, 310, 14137, 4631, 4135, 297, 13, 23369, 1705, 322, 4275, 14325, 29892, 408, 1532, 408, 310, 2545, 1664, 17292, 28342, 763, 25539, 1608, 322, 13, 25719, 10298, 4822, 10916, 313, 6727, 16783, 29914, 331, 16783, 467, 32001, 320, 26956, 29912, 1469, 20847, 3097, 5003, 32001, 319, 1634, 1414, 3577, 363, 445, 5650, 338, 3625, 515, 796, 264, 8144, 472, 13, 29905, 2271, 29912, 991, 597, 1867, 29875, 29889, 990, 29914, 29896, 29900, 29889, 29945, 29906, 29947, 29896, 29914, 2256, 8144, 29889, 29953, 29941, 29929, 29900, 29941, 29945, 29945, 29913, 2651, 2036, 29912, 3445, 1414, 29899, 5113, 1836, 32001, 320, 2042, 29912, 9662, 630, 5244, 29913, 13, 29905, 1643, 29912, 3471, 29901, 12817, 29913, 13, 13, 29933, 720, 4688, 322, 7786, 1736, 2651, 2036, 29912, 12443, 10578, 29906, 29900, 29900, 29945, 5062, 11235, 29892, 13260, 333, 29906, 29900, 29900, 29947, 29888, 2209, 3359, 29879, 29892, 13, 29871, 696, 7586, 29906, 29900, 29896, 29946, 7610, 345, 2941, 271, 24541, 29892, 697, 309, 29906, 29900, 29896, 29953, 311, 5365, 550, 332, 6950, 29913, 505, 2931, 1891, 278, 13, 479, 5275, 310, 12362, 29914, 6585, 7562, 18540, 313, 5800, 1799, 29897, 773, 320, 7278, 29912, 6734, 26946, 952, 1118, 13, 4716, 3867, 1880, 29899, 29567, 6089, 541, 526, 9078, 297, 2159, 313, 29906, 29899, 29945, 5940, 29968, 18777, 29897, 13, 392, 508, 367, 4768, 1463, 491, 5221, 424, 23460, 29889, 32001, 512, 29871, 29906, 29900, 29900, 29947, 2261, 801, 2681, 634, 394, 8770, 2036, 29912, 1646, 801, 2681, 29906, 29900, 29900, 29947, 479, 397, 24974, 29913, 18043, 263, 3031, 979, 13, 16961, 29899, 7052, 313, 1454, 278, 931, 29897, 6559, 373, 18322, 1799, 320, 7278, 29912, 479, 5275, 773, 1375, 292, 7047, 13, 29871, 28914, 313, 4345, 29934, 29897, 13698, 1836, 32001, 2688, 29537, 287, 278, 3978, 310, 29871, 29896, 5940, 29924, 17737, 29560, 13, 4746, 278, 7562, 2831, 479, 1404, 2566, 322, 611, 6504, 1051, 3190, 3145, 975, 278, 13, 29896, 29929, 29929, 29929, 489, 29906, 29900, 29900, 29945, 3785, 29892, 773, 408, 18470, 2472, 2788, 304, 1749, 29879, 29901, 4876, 21904, 13, 392, 17998, 1283, 7224, 29889, 32001, 450, 12399, 3785, 313, 29955, 2440, 29897, 297, 2651, 2036, 29912, 1646, 801, 2681, 29906, 29900, 29900, 29947, 479, 397, 24974, 29913, 338, 20511, 1135, 29871, 13, 5816, 338, 12399, 297, 278, 2198, 5650, 313, 29945, 29900, 2440, 29897, 322, 278, 848, 8974, 526, 29871, 13, 27489, 873, 1422, 29936, 411, 393, 297, 3458, 29892, 1749, 2582, 1510, 263, 10029, 7200, 14978, 310, 29871, 13, 15654, 273, 325, 13217, 29940, 2072, 3082, 20706, 29889, 32001, 7280, 29190, 936, 664, 515, 29871, 29906, 29900, 29896, 29900, 491, 14619, 29882, 371, 4099, 29894, 322, 13, 29950, 309, 1372, 2651, 2036, 29912, 29873, 19426, 371, 4099, 29894, 29906, 29900, 29896, 29900, 2209, 479, 5275, 29913, 4023, 29894, 2868, 1583, 29899, 7099, 433, 1127, 1737, 12122, 13, 2029, 800, 310, 25492, 15303, 8304, 3598, 1494, 1009, 12368, 29892, 13, 15914, 292, 2472, 363, 779, 14850, 4535, 29892, 29955, 29900, 5940, 29968, 25492, 4160, 29889, 32001, 319, 1407, 7786, 13, 1287, 2651, 2036, 29912, 21686, 29879, 29906, 29900, 29906, 29896, 2209, 479, 5275, 29913, 491, 399, 496, 29879, 634, 394, 13217, 5349, 1737, 324, 542, 630, 4203, 263, 7284, 13, 28712, 16046, 4160, 29892, 2534, 26869, 472, 3203, 29871, 29896, 29900, 29900, 25741, 1269, 29892, 322, 1058, 13, 1311, 29899, 7099, 8663, 14354, 373, 1009, 25492, 28723, 29889, 32001, 5806, 278, 6559, 338, 13, 3149, 29899, 262, 29899, 2230, 408, 310, 29871, 29906, 29900, 29906, 29896, 29892, 278, 15717, 7252, 1009, 1284, 886, 13, 351, 475, 303, 2651, 2036, 29912, 1646, 801, 2681, 29906, 29900, 29900, 29947, 479, 397, 24974, 29892, 1850, 29882, 371, 4099, 29894, 29906, 29900, 29896, 29900, 2209, 479, 5275, 29913, 304, 13, 18609, 675, 278, 14675, 310, 18322, 1799, 1737, 5275, 975, 278, 931, 15101, 845, 1862, 4586, 491, 13, 1552, 2211, 11898, 29889, 32001, 3831, 1965, 411, 3517, 29190, 936, 1736, 29892, 1749, 6559, 338, 1568, 7200, 6287, 5634, 29882, 5555, 13, 7054, 12339, 287, 29871, 29946, 29941, 7284, 29905, 13007, 15717, 310, 29871, 29906, 29889, 29906, 24464, 29905, 13007, 25741, 515, 29871, 29896, 29953, 29900, 7284, 29905, 13007, 13, 16418, 5634, 5426, 11267, 979, 975, 29871, 29945, 29900, 2440, 310, 970, 775, 20706, 3265, 1135, 13, 3149, 297, 931, 29892, 322, 884, 901, 2691, 29899, 3874, 1312, 313, 2541, 1629, 29899, 1609, 29899, 6360, 3803, 1070, 537, 975, 13, 1552, 8900, 3785, 467, 32001, 8108, 1189, 1711, 29892, 1749, 6559, 337, 3687, 373, 10079, 11264, 13, 3924, 313, 8257, 29903, 29897, 9063, 848, 3265, 1135, 7481, 29899, 7099, 433, 1127, 4423, 2472, 29889, 32001, 5901, 1736, 5634, 262, 3153, 278, 664, 491, 8432, 2651, 2036, 29912, 18386, 709, 29906, 29900, 29896, 29941, 2209, 29881, 24974, 29913, 13, 392, 29892, 901, 10325, 29892, 390, 579, 468, 29875, 634, 394, 8770, 2036, 29912, 29878, 579, 468, 29875, 29906, 29900, 29896, 29953, 479, 711, 3173, 29892, 13, 29871, 364, 579, 468, 29875, 29906, 29900, 29896, 29947, 479, 711, 3173, 29892, 544, 1648, 29906, 29900, 29906, 29896, 479, 468, 1581, 29881, 24974, 29913, 5634, 17532, 12399, 1737, 12122, 13, 29905, 7278, 29912, 29881, 24974, 322, 24003, 1118, 474, 29889, 29872, 1696, 278, 15834, 304, 607, 278, 3978, 310, 18322, 1799, 13, 17426, 6602, 1009, 11465, 1230, 14137, 14188, 29889, 32001, 512, 445, 664, 591, 2931, 1891, 1737, 12122, 6894, 537, 297, 970, 775, 363, 278, 937, 13, 2230, 472, 445, 6287, 29892, 1716, 297, 4958, 310, 17737, 29560, 322, 15500, 3785, 29889, 32001, 1334, 437, 13, 1333, 22002, 280, 278, 24003, 10696, 29892, 541, 3867, 29190, 936, 848, 322, 1284, 886, 393, 508, 367, 13, 280, 369, 4063, 304, 393, 1095, 408, 5434, 664, 29889, 32001, 320, 7278, 29912, 12756, 7047, 21639, 29913, 2651, 2036, 29912, 2276, 5824, 19982, 29906, 29900, 29900, 29955, 23705, 1338, 29893, 996, 29913, 338, 278, 13, 1491, 29899, 2671, 310, 7047, 21639, 393, 756, 29537, 287, 278, 18066, 267, 310, 21640, 13, 6734, 24771, 13149, 635, 29892, 3704, 278, 2702, 5932, 310, 920, 304, 5376, 13, 2541, 1737, 12122, 6894, 537, 2651, 2036, 29912, 13376, 303, 456, 29906, 29900, 29900, 29953, 10945, 3359, 29892, 1424, 29440, 29906, 29900, 29896, 29946, 23027, 5933, 1836, 32001, 3826, 3076, 2678, 278, 2198, 6559, 8128, 10757, 393, 508, 367, 1304, 29892, 297, 278, 13, 14940, 1206, 310, 970, 775, 322, 472, 263, 1407, 2919, 6287, 29892, 304, 11539, 607, 13, 14032, 4637, 310, 5534, 7047, 21639, 505, 9820, 484, 15774, 29889, 32001, 320, 2042, 29912, 4062, 3002, 29913, 13, 29905, 1643, 29912, 3471, 29901, 5696, 29913, 13, 13, 13, 29905, 1482, 361, 29905, 361, 29887, 798, 386, 1003, 29871, 320, 29887, 798, 386, 1003, 3009, 13, 29905, 361, 29887, 798, 386, 1003, 13, 29905, 463, 29912, 4532, 29913, 13, 29871, 320, 7313, 29961, 2103, 2013, 4914, 2103, 3199, 6360, 368, 29899, 2055, 1169, 29913, 13, 29871, 320, 6671, 29912, 12883, 368, 970, 25741, 975, 931, 313, 1188, 6287, 467, 13, 29913, 32001, 320, 1643, 29912, 1003, 29901, 29887, 798, 386, 29913, 13, 29905, 355, 29912, 4532, 29913, 13, 29905, 7241, 13, 13, 29905, 26956, 29912, 16390, 24541, 29913, 13, 13, 4806, 27387, 515, 320, 23066, 29950, 29914, 2651, 2036, 29912, 29879, 1332, 29899, 1516, 29878, 29906, 29900, 29896, 29929, 29899, 24713, 29913, 599, 25741, 3190, 2347, 2745, 320, 25832, 1964, 579, 1523, 2415, 2539, 6294, 32001, 2688, 5253, 304, 320, 14573, 5261, 1169, 22131, 29914, 25741, 29892, 5412, 491, 317, 15715, 29896, 15882, 29892, 4023, 29894, 2868, 515, 320, 25832, 1299, 7288, 5261, 1169, 797, 7068, 29914, 970, 9279, 6421, 515, 4655, 5849, 363, 2710, 313, 28712, 16046, 29892, 11786, 28632, 29892, 2992, 1846, 32001, 322, 3577, 28914, 313, 10251, 713, 29892, 10772, 2227, 29892, 405, 13427, 29892, 2992, 6250, 32001, 1876, 1169, 297, 278, 8783, 526, 491, 320, 25832, 6344, 2806, 943, 22131, 29914, 15717, 29892, 5412, 491, 779, 6990, 29938, 978, 29892, 4876, 4535, 5854, 29938, 11000, 29889, 32001, 450, 8783, 2996, 408, 1023, 1104, 1288, 6131, 29892, 697, 363, 25741, 322, 697, 363, 15717, 29892, 411, 278, 4642, 29371, 278, 7480, 3025, 263, 9117, 1820, 29889, 32001, 320, 361, 5426, 13, 9760, 1948, 297, 278, 9063, 1591, 3743, 278, 1494, 4235, 29901, 9063, 317, 15715, 29896, 15882, 29892, 4148, 322, 844, 5171, 5335, 342, 15092, 29892, 4148, 322, 844, 5171, 2893, 14903, 313, 20275, 16750, 278, 4148, 1591, 467, 32001, 450, 21578, 1546, 9063, 15717, 322, 9063, 2153, 2041, 515, 11786, 29892, 607, 6511, 304, 9063, 263, 1735, 4148, 287, 491, 4856, 1683, 29889, 32001, 1152, 445, 6559, 591, 21309, 373, 15717, 322, 17262, 9063, 2153, 29892, 408, 278, 4328, 1546, 278, 1023, 338, 451, 8018, 363, 1749, 5925, 5155, 322, 278, 5253, 310, 25741, 411, 263, 844, 5171, 916, 1135, 967, 4148, 338, 3480, 3473, 1821, 29889, 32001, 320, 7241, 13, 2831, 1269, 6251, 297, 278, 4148, 1591, 591, 505, 4148, 2989, 1024, 322, 4876, 408, 1023, 5004, 6031, 310, 10650, 6262, 29889, 32001, 1334, 6206, 2411, 15273, 1821, 470, 18325, 519, 2983, 393, 29901, 526, 451, 1602, 397, 519, 408, 18351, 29899, 29947, 3441, 25832, 6344, 2806, 943, 29934, 29885, 12283, 7099, 397, 519, 29914, 4148, 2983, 6206, 511, 526, 4876, 14157, 2012, 310, 2983, 3441, 25832, 6344, 2806, 943, 29934, 29885, 9823, 29914, 4954, 7039, 4907, 511, 5718, 310, 871, 9654, 4890, 3441, 25832, 6344, 2806, 943, 29934, 29885, 10358, 804, 23201, 1712, 901, 1135, 29871, 29896, 29900, 8958, 1661, 29899, 1026, 2153, 3441, 25832, 6344, 2806, 943, 29934, 29885, 12283, 15670, 23201, 526, 5520, 1135, 29871, 29896, 29900, 29900, 4890, 3441, 25832, 6344, 2806, 943, 29934, 29885, 12229, 549, 12495, 32001, 2860, 21166, 29892, 1048, 320, 25832, 6344, 2806, 943, 29925, 15273, 1821, 2052, 307, 29916, 29914, 15717, 3441, 25832, 6344, 2806, 943, 29925, 15273, 1821, 29925, 312, 29914, 310, 278, 2847, 8783, 29897, 9488, 363, 4340, 7418, 29889, 32001, 3940, 393, 278, 5253, 310, 970, 775, 25741, 313, 392, 15717, 29897, 11122, 297, 278, 13, 11228, 8783, 25088, 18709, 9247, 975, 13, 2230, 2651, 2036, 29912, 29879, 1332, 29899, 771, 854, 749, 29899, 331, 344, 1012, 361, 29887, 798, 386, 1003, 29892, 408, 4318, 363, 25741, 297, 13, 29905, 29907, 999, 29912, 1003, 29901, 29887, 798, 386, 1012, 2870, 29901, 515, 395, 29896, 29900, 29985, 29946, 29938, 25741, 297, 29871, 29896, 29929, 29955, 29896, 29892, 304, 395, 29896, 29900, 29985, 29953, 29938, 297, 29871, 29896, 29929, 29929, 29947, 29892, 304, 13, 284, 3242, 395, 29896, 29900, 29985, 29929, 29938, 297, 29871, 29906, 29900, 29906, 29900, 29905, 7241, 29889, 32001, 1094, 263, 17004, 278, 8900, 534, 1975, 10331, 304, 367, 901, 13, 13844, 297, 7786, 1602, 3076, 1135, 297, 29871, 29946, 29900, 29974, 1629, 29899, 1025, 6743, 29892, 2861, 304, 13964, 4586, 373, 13, 735, 1112, 9247, 7200, 23093, 29889, 32001, 320, 26956, 29912, 7999, 324, 10610, 29913, 13, 13, 29905, 463, 29912, 4532, 29913, 13, 29871, 320, 9525, 13, 29871, 320, 7313, 29961, 24049, 29892, 15450, 29922, 29953, 4912, 29871, 29953, 4912, 29871, 29900, 29871, 29900, 29892, 2103, 2013, 16292, 3199, 1491, 1727, 1080, 29899, 2470, 29913, 13, 29871, 320, 6671, 29912, 1576, 320, 14573, 14058, 4597, 1080, 29914, 3186, 12786, 1304, 408, 1737, 324, 10610, 22525, 5003, 32001, 320, 1643, 29912, 1003, 29901, 11526, 1958, 29913, 13, 29905, 355, 29912, 4532, 29913, 13, 13, 2887, 1737, 324, 10610, 22525, 591, 671, 11758, 3186, 12786, 10723, 515, 278, 3303, 18269, 1737, 359, 305, 2004, 2651, 2036, 29912, 348, 29896, 29929, 29929, 29929, 479, 359, 305, 2004, 1836, 32001, 1763, 4772, 2432, 3381, 491, 2919, 10916, 313, 29872, 29889, 29887, 1696, 7551, 470, 12710, 29897, 2629, 11758, 12786, 29892, 591, 19412, 322, 6219, 777, 12786, 2729, 373, 1737, 12122, 23203, 537, 322, 278, 19383, 310, 758, 331, 8946, 16375, 29769, 5680, 29892, 1316, 408, 19182, 4086, 29889, 32001, 320, 29907, 999, 29912, 1003, 29901, 11526, 1958, 29913, 3697, 278, 2186, 1051, 310, 320, 14573, 14058, 4597, 1080, 29914, 3186, 12786, 1304, 408, 1737, 324, 10610, 22525, 297, 445, 6559, 29889, 32001, 1879, 324, 10610, 310, 9063, 15717, 304, 3186, 12786, 3913, 278, 1023, 19595, 653, 13698, 9129, 297, 2651, 2036, 29912, 293, 344, 29899, 344, 275, 29899, 29906, 29900, 29906, 29906, 29899, 26098, 1118, 23359, 337, 13998, 2400, 29889, 32001, 450, 937, 697, 337, 3687, 373, 278, 4234, 775, 2246, 29899, 5563, 5354, 313, 617, 29911, 10249, 29897, 310, 4876, 14157, 23892, 515, 9063, 15562, 29892, 321, 29889, 29887, 1696, 320, 16266, 26139, 1341, 1118, 320, 16266, 26139, 582, 1118, 320, 16266, 26139, 18038, 1118, 2992, 29889, 32001, 1334, 4687, 515, 278, 306, 2190, 29909, 1051, 310, 13548, 2931, 21759, 29911, 10249, 29879, 2651, 2036, 29912, 6011, 29899, 29883, 312, 430, 29913, 322, 7522, 20545, 1269, 6590, 20123, 304, 263, 3646, 3186, 5120, 29889, 32001, 450, 1473, 1737, 324, 10610, 11043, 3913, 278, 17998, 9210, 310, 9063, 5335, 342, 15092, 313, 29872, 29889, 29887, 1696, 17998, 29899, 29900, 29945, 29901, 29900, 29900, 29897, 322, 4148, 2983, 304, 8161, 278, 1556, 5517, 3186, 5120, 310, 278, 9063, 4148, 29889, 32001, 1152, 1269, 17998, 9210, 591, 8161, 263, 1051, 310, 15878, 7600, 313, 13509, 29892, 2106, 29892, 470, 14278, 20123, 29897, 297, 278, 3186, 393, 29892, 472, 278, 931, 310, 393, 9063, 29892, 750, 393, 17998, 9210, 29936, 9063, 931, 338, 1820, 1244, 29892, 408, 4234, 17998, 1283, 7224, 13100, 975, 931, 2861, 304, 29431, 3620, 29889, 32001, 1763, 1207, 445, 3683, 3381, 591, 671, 278, 306, 2190, 29909, 931, 10640, 2566, 2651, 2036, 29912, 17559, 1272, 1836, 32001, 1987, 591, 3566, 304, 1269, 2058, 263, 8158, 393, 4332, 1973, 278, 4188, 22342, 393, 263, 2183, 4148, 1024, 338, 17443, 310, 372, 29889, 32001, 1763, 445, 1095, 591, 671, 278, 28297, 29890, 15451, 8783, 310, 278, 29511, 310, 278, 1556, 3619, 937, 322, 3942, 2983, 607, 29892, 439, 11427, 515, 2651, 2036, 29912, 1079, 29890, 799, 29899, 7039, 6177, 2802, 277, 12181, 4954, 16123, 2247, 278, 26368, 5528, 5084, 310, 363, 264, 1280, 322, 269, 595, 1280, 7371, 515, 263, 2566, 310, 320, 1949, 29912, 29946, 29871, 29900, 29946, 29946, 29871, 29945, 29946, 29953, 29871, 29929, 29941, 29947, 29913, 2305, 313, 29945, 29945, 29889, 29945, 8958, 310, 8471, 2305, 297, 29871, 29906, 29900, 29896, 29946, 467, 32001, 1094, 310, 3839, 29871, 29906, 29900, 29896, 29929, 372, 18469, 320, 1949, 29912, 29906, 29955, 29871, 29953, 29953, 29906, 29871, 29947, 29900, 29896, 29913, 363, 264, 1280, 322, 320, 1949, 29912, 29906, 29955, 29871, 29906, 29900, 29953, 29871, 29947, 29906, 29896, 29913, 269, 595, 1280, 297, 29871, 29906, 29941, 29953, 24894, 8977, 1080, 29889, 4907, 29913, 32001, 1094, 297, 1749, 8783, 15717, 526, 2989, 1024, 6031, 313, 29878, 1624, 1135, 6219, 491, 937, 29914, 11922, 1024, 511, 591, 937, 5993, 675, 2983, 313, 1609, 1999, 1331, 322, 1206, 3620, 29897, 322, 769, 16280, 5375, 18897, 297, 1716, 937, 322, 3942, 2983, 10868, 8857, 29889, 32001, 1152, 1269, 1543, 1476, 297, 1024, 8857, 591, 22932, 278, 2058, 4665, 29905, 6661, 1333, 24212, 8875, 491, 278, 1024, 10868, 304, 4017, 263, 5645, 393, 338, 29839, 304, 278, 1353, 310, 12407, 24638, 393, 1024, 313, 6979, 29897, 297, 278, 2702, 2058, 29889, 32001, 320, 6661, 1333, 300, 1062, 29912, 1762, 4017, 4665, 2025, 1338, 5634, 294, 278, 17837, 310, 4954, 6689, 4907, 338, 25745, 23724, 29901, 2989, 10916, 325, 13217, 29879, 29399, 310, 2919, 10916, 805, 9450, 2999, 931, 29920, 2873, 5634, 705, 671, 263, 29544, 310, 7601, 8974, 313, 29872, 29889, 29887, 1696, 5874, 28007, 511, 322, 1661, 29899, 16072, 6743, 313, 29872, 29889, 29887, 1696, 14109, 7456, 467, 29913, 32001, 1334, 2533, 445, 4377, 363, 599, 3161, 304, 4017, 263, 2058, 8158, 29892, 17140, 701, 411, 263, 1051, 310, 779, 6990, 29938, 6689, 29892, 8158, 4535, 5854, 29938, 11000, 29889, 32001, 1334, 769, 8877, 445, 1051, 491, 278, 3186, 5120, 393, 263, 2058, 14393, 304, 322, 2533, 278, 8158, 363, 599, 278, 7600, 297, 1269, 5120, 304, 4017, 385, 12463, 8158, 29892, 6590, 304, 278, 4188, 22342, 393, 278, 9063, 14393, 304, 263, 2183, 3186, 5120, 29889, 32001, 1334, 3566, 278, 6257, 9063, 408, 6421, 515, 278, 3186, 5120, 411, 278, 9939, 8158, 29889, 32001, 450, 4876, 29899, 6707, 11043, 9378, 414, 515, 278, 9078, 322, 443, 5521, 8362, 671, 310, 21759, 29911, 10249, 29879, 29901, 1556, 18777, 671, 10035, 323, 10249, 29879, 1316, 408, 320, 16266, 26139, 510, 1118, 320, 16266, 26139, 990, 1118, 470, 320, 16266, 26139, 1212, 1836, 32001, 12808, 445, 947, 451, 3799, 26018, 4822, 20542, 29901, 3148, 29899, 6707, 18777, 29892, 363, 1342, 29892, 671, 278, 320, 16266, 26139, 375, 29913, 21759, 29911, 10249, 1568, 901, 269, 2495, 290, 368, 1135, 1009, 7824, 6795, 20895, 29889, 32001, 1551, 278, 916, 1361, 278, 9210, 29914, 978, 29899, 6707, 11043, 337, 3687, 373, 278, 17998, 9210, 310, 278, 9063, 5335, 342, 15092, 29889, 32001, 16809, 304, 5780, 22920, 373, 13897, 731, 14340, 29892, 263, 2919, 1353, 310, 25741, 297, 278, 8783, 756, 385, 17998, 9210, 5186, 304, 5225, 29889, 32001, 910, 6602, 29879, 3109, 7786, 25741, 3441, 14573, 5261, 1169, 29911, 29999, 29999, 13985, 1349, 681, 392, 27418, 6478, 29914, 310, 29871, 29906, 29900, 29906, 29900, 29879, 25741, 505, 263, 5225, 9210, 29897, 1135, 9642, 6743, 3441, 14573, 5261, 1169, 29911, 29999, 29999, 13985, 1349, 681, 392, 29914, 297, 29871, 29906, 29900, 29900, 29900, 467, 32001, 1094, 263, 1121, 278, 9210, 29914, 978, 29899, 6707, 11043, 1033, 1095, 701, 6459, 292, 263, 2919, 6232, 310, 9642, 25741, 408, 4148, 287, 491, 11715, 18777, 29892, 322, 304, 263, 3109, 261, 15834, 4092, 550, 29889, 32001, 1763, 6795, 1438, 5626, 591, 14405, 278, 1023, 1737, 324, 10610, 13698, 4208, 491, 15399, 278, 9210, 29914, 978, 29899, 6707, 13698, 304, 599, 25741, 411, 263, 1661, 29899, 9171, 17998, 9210, 29892, 322, 278, 4876, 29899, 6707, 373, 304, 599, 916, 25741, 29889, 32001, 320, 2042, 29912, 12191, 322, 8565, 1558, 291, 29913, 13, 29905, 1643, 29912, 3471, 29901, 9902, 29913, 13, 13, 29905, 463, 29912, 4532, 4044, 13, 29871, 320, 9525, 13, 29871, 320, 7313, 29961, 2103, 2013, 16292, 3199, 1429, 287, 29889, 5140, 29913, 13, 29871, 320, 6671, 29912, 29934, 20819, 310, 25741, 313, 27215, 29897, 322, 6136, 15717, 313, 22503, 29897, 491, 3186, 10640, 975, 278, 29871, 29896, 29929, 29955, 29896, 489, 29906, 29900, 29906, 29900, 3785, 5003, 32001, 320, 9868, 29961, 14732, 3199, 7264, 287, 2594, 8727, 6445, 278, 3186, 10640, 364, 2219, 359, 363, 25741, 322, 15717, 975, 278, 29871, 29896, 29929, 29955, 29896, 489, 29906, 29900, 29906, 29900, 3785, 5003, 32001, 320, 1643, 29912, 1003, 29901, 9902, 29913, 13, 29905, 355, 29912, 4532, 4044, 13, 13, 13, 29871, 13, 1762, 1234, 320, 25029, 29912, 29878, 29939, 29901, 479, 397, 24974, 29913, 591, 22229, 278, 1353, 310, 25741, 322, 8359, 15717, 639, 1629, 322, 639, 3186, 10640, 29889, 32001, 1334, 2198, 278, 7625, 2582, 297, 320, 29907, 999, 29912, 1003, 29901, 9902, 29913, 408, 1023, 5096, 287, 2594, 24469, 29892, 6445, 1629, 368, 2867, 3204, 29879, 363, 25741, 322, 15717, 8307, 29889, 32001, 7569, 2594, 11524, 263, 1629, 322, 338, 8877, 287, 297, 269, 29399, 6445, 278, 9063, 29914, 8921, 11959, 363, 1269, 310, 278, 3186, 12786, 310, 320, 29907, 999, 29912, 1003, 29901, 11526, 1958, 29913, 297, 393, 1629, 29889, 32001, 1763, 4772, 714, 27801, 2861, 304, 805, 272, 26538, 17737, 29560, 29892, 297, 278, 4148, 8727, 591, 871, 2050, 15717, 2534, 26869, 472, 3203, 29871, 29945, 25741, 297, 263, 2183, 1629, 29889, 32001, 5806, 5366, 1747, 534, 1975, 297, 278, 24469, 6456, 393, 278, 3001, 3694, 310, 25741, 322, 15717, 6548, 18709, 9247, 975, 931, 29889, 32001, 10133, 363, 278, 937, 2440, 297, 278, 24469, 29892, 278, 1353, 310, 848, 3291, 297, 777, 3186, 12786, 508, 367, 14154, 2319, 29892, 411, 8178, 27721, 373, 278, 25806, 310, 534, 1975, 29889, 32001, 320, 26956, 29912, 7999, 12122, 6894, 537, 975, 931, 29913, 13, 13, 3563, 497, 29892, 278, 2498, 534, 355, 5692, 304, 367, 393, 278, 320, 4534, 29912, 479, 12122, 6894, 537, 297, 970, 775, 338, 10231, 6177, 4644, 6813, 322, 4092, 5136, 630, 1009, 4954, 24130, 749, 4907, 2745, 278, 7256, 310, 278, 29871, 29929, 29900], 'attention_mask': [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1], 'labels': [1, 320, 2042, 29912, 25898, 29913, 13, 29905, 1643, 29912, 3471, 29901, 23333, 29913, 13, 13, 29905, 7278, 29912, 29954, 1581, 6894, 537, 1118, 470, 901, 4049, 967, 10225, 727, 974, 29892, 4249, 27138, 304, 13, 20415, 5849, 14188, 756, 1063, 26606, 12399, 297, 7786, 2440, 29889, 32001, 512, 13, 1595, 16311, 29892, 278, 10122, 310, 29892, 9545, 310, 29892, 322, 6795, 1004, 25414, 363, 320, 7278, 29912, 26098, 13, 29871, 24003, 29913, 297, 12362, 29914, 6585, 7562, 18540, 313, 5800, 1799, 29897, 505, 4520, 263, 3287, 310, 8570, 13, 957, 278, 4940, 316, 6332, 2651, 2036, 29912, 29881, 16093, 29906, 29900, 29900, 29947, 29888, 2209, 3359, 29879, 29892, 3855, 5871, 29906, 29900, 29896, 29900, 29895, 311, 29893, 2770, 29892, 13, 29871, 1055, 29888, 375, 29906, 29900, 29896, 29906, 5041, 267, 29892, 413, 434, 305, 1358, 29906, 29900, 29896, 29906, 26098, 29888, 2209, 29892, 19723, 5475, 4979, 29906, 29900, 29896, 29946, 26098, 29892, 13, 29871, 697, 309, 29906, 29900, 29896, 29953, 311, 5365, 550, 332, 6950, 29892, 696, 7586, 29906, 29900, 29896, 29953, 29893, 2770, 7192, 2209, 29892, 1935, 15044, 29906, 29900, 29896, 29955, 26098, 29892, 13, 29871, 503, 562, 305, 3350, 492, 29906, 29900, 29906, 29896, 26098, 1836, 32001, 320, 7278, 29912, 7999, 12122, 6894, 537, 29913, 338, 373, 278, 916, 1361, 278, 13, 14380, 310, 6894, 537, 393, 380, 1567, 515, 27138, 297, 777, 5534, 6354, 6421, 13, 3166, 1422, 3186, 12786, 322, 4185, 1973, 29889, 32001, 26304, 6894, 537, 297, 18322, 1799, 756, 4520, 13774, 2217, 8570, 297, 21344, 368, 13, 13129, 29889, 32001, 512, 3153, 29892, 1550, 3031, 979, 18994, 29899, 6707, 322, 13, 3149, 29899, 262, 29899, 2230, 18350, 29899, 7052, 11898, 310, 278, 1737, 12122, 1677, 1144, 310, 18322, 1799, 13, 21570, 29560, 1863, 2651, 2036, 29912, 12443, 10578, 29906, 29900, 29900, 29945, 5062, 11235, 29892, 13260, 333, 29906, 29900, 29900, 29947, 29888, 2209, 3359, 29879, 29892, 13, 29871, 2594, 801, 2681, 29906, 29900, 29900, 29947, 479, 397, 24974, 29892, 1850, 29882, 371, 4099, 29894, 29906, 29900, 29896, 29900, 2209, 479, 5275, 29892, 696, 7586, 29906, 29900, 29896, 29946, 7610, 345, 2941, 271, 24541, 29892, 13, 29871, 281, 496, 29879, 29906, 29900, 29906, 29896, 2209, 479, 5275, 1118, 2919, 29899, 7052, 25579, 979, 11898, 310, 278, 1737, 12122, 13, 12574, 310, 18322, 1799, 17737, 29560, 526, 1603, 10225, 292, 29889, 32001, 10506, 263, 4323, 23378, 13, 18609, 2133, 723, 367, 5407, 304, 1871, 1602, 12112, 4475, 304, 5534, 13, 25431, 10907, 2651, 2036, 29912, 2276, 5824, 19982, 29906, 29900, 29900, 29955, 23705, 1338, 29893, 996, 29913, 322, 298, 8491, 16650, 583, 297, 278, 13, 19678, 15483, 313, 1806, 29897, 9999, 29892, 408, 1532, 408, 29126, 2114, 950, 2472, 13, 517, 278, 2553, 1078, 373, 278, 17407, 10879, 322, 5374, 3002, 310, 18322, 1799, 2820, 278, 3186, 29889, 32001, 320, 26956, 29912, 1323, 3224, 29879, 29913, 13, 13, 3047, 445, 664, 591, 29126, 304, 3802, 445, 17261, 491, 7512, 292, 320, 4534, 29912, 1552, 937, 13, 29871, 25579, 979, 6559, 310, 278, 1737, 12122, 3978, 310, 17737, 29560, 304, 970, 775, 13, 29871, 975, 29871, 29945, 29900, 2440, 5003, 32001, 26321, 29892, 591, 3867, 263, 758, 2576, 3821, 1234, 304, 278, 13, 23031, 292, 5925, 1139, 29901, 13, 29905, 463, 29912, 690, 2842, 12470, 29913, 13, 29871, 3645, 607, 3186, 12786, 437, 15717, 310, 970, 368, 3625, 25741, 2041, 515, 13, 29871, 322, 920, 756, 372, 3939, 975, 278, 4940, 29871, 29945, 29900, 2440, 29973, 32001, 320, 1643, 29912, 29878, 29939, 29901, 479, 397, 24974, 29913, 13, 29905, 355, 29912, 690, 2842, 12470, 29913, 13, 4806, 671, 408, 8783, 278, 320, 23066, 29950, 29914, 18871, 2651, 2036, 29912, 29879, 1332, 29875, 4569, 29906, 29900, 29896, 29955, 29913, 322, 27599, 515, 372, 13, 29906, 29889, 29906, 24464, 29905, 13007, 25741, 3190, 2347, 515, 29871, 29896, 29953, 29900, 7284, 29905, 13007, 9279, 322, 4148, 287, 491, 13, 29946, 29941, 7284, 29905, 13007, 15717, 2645, 278, 29871, 29896, 29929, 29955, 29896, 489, 29906, 29900, 29906, 29896, 931, 3785, 29889, 32001, 1334, 1737, 324, 542, 403, 18777, 304, 13, 29905, 14573, 14058, 4597, 1080, 29914, 3186, 12786, 29892, 773, 408, 18470, 4876, 4234, 775, 2246, 29899, 5563, 21904, 313, 617, 29911, 10249, 29879, 29897, 322, 29871, 13, 8921, 313, 4102, 29914, 4230, 29897, 2983, 9401, 411, 1024, 18822, 2820, 278, 3186, 29892, 322, 17998, 1283, 7224, 29871, 13, 1195, 287, 515, 9063, 15562, 29889, 32001, 1334, 1284, 10757, 310, 278, 4688, 8022, 749, 310, 4644, 6813, 297, 1722, 2752, 13, 20415, 29892, 2678, 8772, 491, 4092, 29889, 32001, 2860, 393, 3785, 29892, 278, 1737, 12122, 6894, 537, 29871, 13, 262, 970, 775, 756, 1063, 21003, 10231, 29889, 32001, 1334, 884, 12439, 8018, 15839, 528, 17741, 13, 12817, 304, 278, 1095, 310, 278, 8291, 6415, 29129, 322, 278, 7910, 310, 14137, 4631, 4135, 297, 13, 23369, 1705, 322, 4275, 14325, 29892, 408, 1532, 408, 310, 2545, 1664, 17292, 28342, 763, 25539, 1608, 322, 13, 25719, 10298, 4822, 10916, 313, 6727, 16783, 29914, 331, 16783, 467, 32001, 320, 26956, 29912, 1469, 20847, 3097, 5003, 32001, 319, 1634, 1414, 3577, 363, 445, 5650, 338, 3625, 515, 796, 264, 8144, 472, 13, 29905, 2271, 29912, 991, 597, 1867, 29875, 29889, 990, 29914, 29896, 29900, 29889, 29945, 29906, 29947, 29896, 29914, 2256, 8144, 29889, 29953, 29941, 29929, 29900, 29941, 29945, 29945, 29913, 2651, 2036, 29912, 3445, 1414, 29899, 5113, 1836, 32001, 320, 2042, 29912, 9662, 630, 5244, 29913, 13, 29905, 1643, 29912, 3471, 29901, 12817, 29913, 13, 13, 29933, 720, 4688, 322, 7786, 1736, 2651, 2036, 29912, 12443, 10578, 29906, 29900, 29900, 29945, 5062, 11235, 29892, 13260, 333, 29906, 29900, 29900, 29947, 29888, 2209, 3359, 29879, 29892, 13, 29871, 696, 7586, 29906, 29900, 29896, 29946, 7610, 345, 2941, 271, 24541, 29892, 697, 309, 29906, 29900, 29896, 29953, 311, 5365, 550, 332, 6950, 29913, 505, 2931, 1891, 278, 13, 479, 5275, 310, 12362, 29914, 6585, 7562, 18540, 313, 5800, 1799, 29897, 773, 320, 7278, 29912, 6734, 26946, 952, 1118, 13, 4716, 3867, 1880, 29899, 29567, 6089, 541, 526, 9078, 297, 2159, 313, 29906, 29899, 29945, 5940, 29968, 18777, 29897, 13, 392, 508, 367, 4768, 1463, 491, 5221, 424, 23460, 29889, 32001, 512, 29871, 29906, 29900, 29900, 29947, 2261, 801, 2681, 634, 394, 8770, 2036, 29912, 1646, 801, 2681, 29906, 29900, 29900, 29947, 479, 397, 24974, 29913, 18043, 263, 3031, 979, 13, 16961, 29899, 7052, 313, 1454, 278, 931, 29897, 6559, 373, 18322, 1799, 320, 7278, 29912, 479, 5275, 773, 1375, 292, 7047, 13, 29871, 28914, 313, 4345, 29934, 29897, 13698, 1836, 32001, 2688, 29537, 287, 278, 3978, 310, 29871, 29896, 5940, 29924, 17737, 29560, 13, 4746, 278, 7562, 2831, 479, 1404, 2566, 322, 611, 6504, 1051, 3190, 3145, 975, 278, 13, 29896, 29929, 29929, 29929, 489, 29906, 29900, 29900, 29945, 3785, 29892, 773, 408, 18470, 2472, 2788, 304, 1749, 29879, 29901, 4876, 21904, 13, 392, 17998, 1283, 7224, 29889, 32001, 450, 12399, 3785, 313, 29955, 2440, 29897, 297, 2651, 2036, 29912, 1646, 801, 2681, 29906, 29900, 29900, 29947, 479, 397, 24974, 29913, 338, 20511, 1135, 29871, 13, 5816, 338, 12399, 297, 278, 2198, 5650, 313, 29945, 29900, 2440, 29897, 322, 278, 848, 8974, 526, 29871, 13, 27489, 873, 1422, 29936, 411, 393, 297, 3458, 29892, 1749, 2582, 1510, 263, 10029, 7200, 14978, 310, 29871, 13, 15654, 273, 325, 13217, 29940, 2072, 3082, 20706, 29889, 32001, 7280, 29190, 936, 664, 515, 29871, 29906, 29900, 29896, 29900, 491, 14619, 29882, 371, 4099, 29894, 322, 13, 29950, 309, 1372, 2651, 2036, 29912, 29873, 19426, 371, 4099, 29894, 29906, 29900, 29896, 29900, 2209, 479, 5275, 29913, 4023, 29894, 2868, 1583, 29899, 7099, 433, 1127, 1737, 12122, 13, 2029, 800, 310, 25492, 15303, 8304, 3598, 1494, 1009, 12368, 29892, 13, 15914, 292, 2472, 363, 779, 14850, 4535, 29892, 29955, 29900, 5940, 29968, 25492, 4160, 29889, 32001, 319, 1407, 7786, 13, 1287, 2651, 2036, 29912, 21686, 29879, 29906, 29900, 29906, 29896, 2209, 479, 5275, 29913, 491, 399, 496, 29879, 634, 394, 13217, 5349, 1737, 324, 542, 630, 4203, 263, 7284, 13, 28712, 16046, 4160, 29892, 2534, 26869, 472, 3203, 29871, 29896, 29900, 29900, 25741, 1269, 29892, 322, 1058, 13, 1311, 29899, 7099, 8663, 14354, 373, 1009, 25492, 28723, 29889, 32001, 5806, 278, 6559, 338, 13, 3149, 29899, 262, 29899, 2230, 408, 310, 29871, 29906, 29900, 29906, 29896, 29892, 278, 15717, 7252, 1009, 1284, 886, 13, 351, 475, 303, 2651, 2036, 29912, 1646, 801, 2681, 29906, 29900, 29900, 29947, 479, 397, 24974, 29892, 1850, 29882, 371, 4099, 29894, 29906, 29900, 29896, 29900, 2209, 479, 5275, 29913, 304, 13, 18609, 675, 278, 14675, 310, 18322, 1799, 1737, 5275, 975, 278, 931, 15101, 845, 1862, 4586, 491, 13, 1552, 2211, 11898, 29889, 32001, 3831, 1965, 411, 3517, 29190, 936, 1736, 29892, 1749, 6559, 338, 1568, 7200, 6287, 5634, 29882, 5555, 13, 7054, 12339, 287, 29871, 29946, 29941, 7284, 29905, 13007, 15717, 310, 29871, 29906, 29889, 29906, 24464, 29905, 13007, 25741, 515, 29871, 29896, 29953, 29900, 7284, 29905, 13007, 13, 16418, 5634, 5426, 11267, 979, 975, 29871, 29945, 29900, 2440, 310, 970, 775, 20706, 3265, 1135, 13, 3149, 297, 931, 29892, 322, 884, 901, 2691, 29899, 3874, 1312, 313, 2541, 1629, 29899, 1609, 29899, 6360, 3803, 1070, 537, 975, 13, 1552, 8900, 3785, 467, 32001, 8108, 1189, 1711, 29892, 1749, 6559, 337, 3687, 373, 10079, 11264, 13, 3924, 313, 8257, 29903, 29897, 9063, 848, 3265, 1135, 7481, 29899, 7099, 433, 1127, 4423, 2472, 29889, 32001, 5901, 1736, 5634, 262, 3153, 278, 664, 491, 8432, 2651, 2036, 29912, 18386, 709, 29906, 29900, 29896, 29941, 2209, 29881, 24974, 29913, 13, 392, 29892, 901, 10325, 29892, 390, 579, 468, 29875, 634, 394, 8770, 2036, 29912, 29878, 579, 468, 29875, 29906, 29900, 29896, 29953, 479, 711, 3173, 29892, 13, 29871, 364, 579, 468, 29875, 29906, 29900, 29896, 29947, 479, 711, 3173, 29892, 544, 1648, 29906, 29900, 29906, 29896, 479, 468, 1581, 29881, 24974, 29913, 5634, 17532, 12399, 1737, 12122, 13, 29905, 7278, 29912, 29881, 24974, 322, 24003, 1118, 474, 29889, 29872, 1696, 278, 15834, 304, 607, 278, 3978, 310, 18322, 1799, 13, 17426, 6602, 1009, 11465, 1230, 14137, 14188, 29889, 32001, 512, 445, 664, 591, 2931, 1891, 1737, 12122, 6894, 537, 297, 970, 775, 363, 278, 937, 13, 2230, 472, 445, 6287, 29892, 1716, 297, 4958, 310, 17737, 29560, 322, 15500, 3785, 29889, 32001, 1334, 437, 13, 1333, 22002, 280, 278, 24003, 10696, 29892, 541, 3867, 29190, 936, 848, 322, 1284, 886, 393, 508, 367, 13, 280, 369, 4063, 304, 393, 1095, 408, 5434, 664, 29889, 32001, 320, 7278, 29912, 12756, 7047, 21639, 29913, 2651, 2036, 29912, 2276, 5824, 19982, 29906, 29900, 29900, 29955, 23705, 1338, 29893, 996, 29913, 338, 278, 13, 1491, 29899, 2671, 310, 7047, 21639, 393, 756, 29537, 287, 278, 18066, 267, 310, 21640, 13, 6734, 24771, 13149, 635, 29892, 3704, 278, 2702, 5932, 310, 920, 304, 5376, 13, 2541, 1737, 12122, 6894, 537, 2651, 2036, 29912, 13376, 303, 456, 29906, 29900, 29900, 29953, 10945, 3359, 29892, 1424, 29440, 29906, 29900, 29896, 29946, 23027, 5933, 1836, 32001, 3826, 3076, 2678, 278, 2198, 6559, 8128, 10757, 393, 508, 367, 1304, 29892, 297, 278, 13, 14940, 1206, 310, 970, 775, 322, 472, 263, 1407, 2919, 6287, 29892, 304, 11539, 607, 13, 14032, 4637, 310, 5534, 7047, 21639, 505, 9820, 484, 15774, 29889, 32001, 320, 2042, 29912, 4062, 3002, 29913, 13, 29905, 1643, 29912, 3471, 29901, 5696, 29913, 13, 13, 13, 29905, 1482, 361, 29905, 361, 29887, 798, 386, 1003, 29871, 320, 29887, 798, 386, 1003, 3009, 13, 29905, 361, 29887, 798, 386, 1003, 13, 29905, 463, 29912, 4532, 29913, 13, 29871, 320, 7313, 29961, 2103, 2013, 4914, 2103, 3199, 6360, 368, 29899, 2055, 1169, 29913, 13, 29871, 320, 6671, 29912, 12883, 368, 970, 25741, 975, 931, 313, 1188, 6287, 467, 13, 29913, 32001, 320, 1643, 29912, 1003, 29901, 29887, 798, 386, 29913, 13, 29905, 355, 29912, 4532, 29913, 13, 29905, 7241, 13, 13, 29905, 26956, 29912, 16390, 24541, 29913, 13, 13, 4806, 27387, 515, 320, 23066, 29950, 29914, 2651, 2036, 29912, 29879, 1332, 29899, 1516, 29878, 29906, 29900, 29896, 29929, 29899, 24713, 29913, 599, 25741, 3190, 2347, 2745, 320, 25832, 1964, 579, 1523, 2415, 2539, 6294, 32001, 2688, 5253, 304, 320, 14573, 5261, 1169, 22131, 29914, 25741, 29892, 5412, 491, 317, 15715, 29896, 15882, 29892, 4023, 29894, 2868, 515, 320, 25832, 1299, 7288, 5261, 1169, 797, 7068, 29914, 970, 9279, 6421, 515, 4655, 5849, 363, 2710, 313, 28712, 16046, 29892, 11786, 28632, 29892, 2992, 1846, 32001, 322, 3577, 28914, 313, 10251, 713, 29892, 10772, 2227, 29892, 405, 13427, 29892, 2992, 6250, 32001, 1876, 1169, 297, 278, 8783, 526, 491, 320, 25832, 6344, 2806, 943, 22131, 29914, 15717, 29892, 5412, 491, 779, 6990, 29938, 978, 29892, 4876, 4535, 5854, 29938, 11000, 29889, 32001, 450, 8783, 2996, 408, 1023, 1104, 1288, 6131, 29892, 697, 363, 25741, 322, 697, 363, 15717, 29892, 411, 278, 4642, 29371, 278, 7480, 3025, 263, 9117, 1820, 29889, 32001, 320, 361, 5426, 13, 9760, 1948, 297, 278, 9063, 1591, 3743, 278, 1494, 4235, 29901, 9063, 317, 15715, 29896, 15882, 29892, 4148, 322, 844, 5171, 5335, 342, 15092, 29892, 4148, 322, 844, 5171, 2893, 14903, 313, 20275, 16750, 278, 4148, 1591, 467, 32001, 450, 21578, 1546, 9063, 15717, 322, 9063, 2153, 2041, 515, 11786, 29892, 607, 6511, 304, 9063, 263, 1735, 4148, 287, 491, 4856, 1683, 29889, 32001, 1152, 445, 6559, 591, 21309, 373, 15717, 322, 17262, 9063, 2153, 29892, 408, 278, 4328, 1546, 278, 1023, 338, 451, 8018, 363, 1749, 5925, 5155, 322, 278, 5253, 310, 25741, 411, 263, 844, 5171, 916, 1135, 967, 4148, 338, 3480, 3473, 1821, 29889, 32001, 320, 7241, 13, 2831, 1269, 6251, 297, 278, 4148, 1591, 591, 505, 4148, 2989, 1024, 322, 4876, 408, 1023, 5004, 6031, 310, 10650, 6262, 29889, 32001, 1334, 6206, 2411, 15273, 1821, 470, 18325, 519, 2983, 393, 29901, 526, 451, 1602, 397, 519, 408, 18351, 29899, 29947, 3441, 25832, 6344, 2806, 943, 29934, 29885, 12283, 7099, 397, 519, 29914, 4148, 2983, 6206, 511, 526, 4876, 14157, 2012, 310, 2983, 3441, 25832, 6344, 2806, 943, 29934, 29885, 9823, 29914, 4954, 7039, 4907, 511, 5718, 310, 871, 9654, 4890, 3441, 25832, 6344, 2806, 943, 29934, 29885, 10358, 804, 23201, 1712, 901, 1135, 29871, 29896, 29900, 8958, 1661, 29899, 1026, 2153, 3441, 25832, 6344, 2806, 943, 29934, 29885, 12283, 15670, 23201, 526, 5520, 1135, 29871, 29896, 29900, 29900, 4890, 3441, 25832, 6344, 2806, 943, 29934, 29885, 12229, 549, 12495, 32001, 2860, 21166, 29892, 1048, 320, 25832, 6344, 2806, 943, 29925, 15273, 1821, 2052, 307, 29916, 29914, 15717, 3441, 25832, 6344, 2806, 943, 29925, 15273, 1821, 29925, 312, 29914, 310, 278, 2847, 8783, 29897, 9488, 363, 4340, 7418, 29889, 32001, 3940, 393, 278, 5253, 310, 970, 775, 25741, 313, 392, 15717, 29897, 11122, 297, 278, 13, 11228, 8783, 25088, 18709, 9247, 975, 13, 2230, 2651, 2036, 29912, 29879, 1332, 29899, 771, 854, 749, 29899, 331, 344, 1012, 361, 29887, 798, 386, 1003, 29892, 408, 4318, 363, 25741, 297, 13, 29905, 29907, 999, 29912, 1003, 29901, 29887, 798, 386, 1012, 2870, 29901, 515, 395, 29896, 29900, 29985, 29946, 29938, 25741, 297, 29871, 29896, 29929, 29955, 29896, 29892, 304, 395, 29896, 29900, 29985, 29953, 29938, 297, 29871, 29896, 29929, 29929, 29947, 29892, 304, 13, 284, 3242, 395, 29896, 29900, 29985, 29929, 29938, 297, 29871, 29906, 29900, 29906, 29900, 29905, 7241, 29889, 32001, 1094, 263, 17004, 278, 8900, 534, 1975, 10331, 304, 367, 901, 13, 13844, 297, 7786, 1602, 3076, 1135, 297, 29871, 29946, 29900, 29974, 1629, 29899, 1025, 6743, 29892, 2861, 304, 13964, 4586, 373, 13, 735, 1112, 9247, 7200, 23093, 29889, 32001, 320, 26956, 29912, 7999, 324, 10610, 29913, 13, 13, 29905, 463, 29912, 4532, 29913, 13, 29871, 320, 9525, 13, 29871, 320, 7313, 29961, 24049, 29892, 15450, 29922, 29953, 4912, 29871, 29953, 4912, 29871, 29900, 29871, 29900, 29892, 2103, 2013, 16292, 3199, 1491, 1727, 1080, 29899, 2470, 29913, 13, 29871, 320, 6671, 29912, 1576, 320, 14573, 14058, 4597, 1080, 29914, 3186, 12786, 1304, 408, 1737, 324, 10610, 22525, 5003, 32001, 320, 1643, 29912, 1003, 29901, 11526, 1958, 29913, 13, 29905, 355, 29912, 4532, 29913, 13, 13, 2887, 1737, 324, 10610, 22525, 591, 671, 11758, 3186, 12786, 10723, 515, 278, 3303, 18269, 1737, 359, 305, 2004, 2651, 2036, 29912, 348, 29896, 29929, 29929, 29929, 479, 359, 305, 2004, 1836, 32001, 1763, 4772, 2432, 3381, 491, 2919, 10916, 313, 29872, 29889, 29887, 1696, 7551, 470, 12710, 29897, 2629, 11758, 12786, 29892, 591, 19412, 322, 6219, 777, 12786, 2729, 373, 1737, 12122, 23203, 537, 322, 278, 19383, 310, 758, 331, 8946, 16375, 29769, 5680, 29892, 1316, 408, 19182, 4086, 29889, 32001, 320, 29907, 999, 29912, 1003, 29901, 11526, 1958, 29913, 3697, 278, 2186, 1051, 310, 320, 14573, 14058, 4597, 1080, 29914, 3186, 12786, 1304, 408, 1737, 324, 10610, 22525, 297, 445, 6559, 29889, 32001, 1879, 324, 10610, 310, 9063, 15717, 304, 3186, 12786, 3913, 278, 1023, 19595, 653, 13698, 9129, 297, 2651, 2036, 29912, 293, 344, 29899, 344, 275, 29899, 29906, 29900, 29906, 29906, 29899, 26098, 1118, 23359, 337, 13998, 2400, 29889, 32001, 450, 937, 697, 337, 3687, 373, 278, 4234, 775, 2246, 29899, 5563, 5354, 313, 617, 29911, 10249, 29897, 310, 4876, 14157, 23892, 515, 9063, 15562, 29892, 321, 29889, 29887, 1696, 320, 16266, 26139, 1341, 1118, 320, 16266, 26139, 582, 1118, 320, 16266, 26139, 18038, 1118, 2992, 29889, 32001, 1334, 4687, 515, 278, 306, 2190, 29909, 1051, 310, 13548, 2931, 21759, 29911, 10249, 29879, 2651, 2036, 29912, 6011, 29899, 29883, 312, 430, 29913, 322, 7522, 20545, 1269, 6590, 20123, 304, 263, 3646, 3186, 5120, 29889, 32001, 450, 1473, 1737, 324, 10610, 11043, 3913, 278, 17998, 9210, 310, 9063, 5335, 342, 15092, 313, 29872, 29889, 29887, 1696, 17998, 29899, 29900, 29945, 29901, 29900, 29900, 29897, 322, 4148, 2983, 304, 8161, 278, 1556, 5517, 3186, 5120, 310, 278, 9063, 4148, 29889, 32001, 1152, 1269, 17998, 9210, 591, 8161, 263, 1051, 310, 15878, 7600, 313, 13509, 29892, 2106, 29892, 470, 14278, 20123, 29897, 297, 278, 3186, 393, 29892, 472, 278, 931, 310, 393, 9063, 29892, 750, 393, 17998, 9210, 29936, 9063, 931, 338, 1820, 1244, 29892, 408, 4234, 17998, 1283, 7224, 13100, 975, 931, 2861, 304, 29431, 3620, 29889, 32001, 1763, 1207, 445, 3683, 3381, 591, 671, 278, 306, 2190, 29909, 931, 10640, 2566, 2651, 2036, 29912, 17559, 1272, 1836, 32001, 1987, 591, 3566, 304, 1269, 2058, 263, 8158, 393, 4332, 1973, 278, 4188, 22342, 393, 263, 2183, 4148, 1024, 338, 17443, 310, 372, 29889, 32001, 1763, 445, 1095, 591, 671, 278, 28297, 29890, 15451, 8783, 310, 278, 29511, 310, 278, 1556, 3619, 937, 322, 3942, 2983, 607, 29892, 439, 11427, 515, 2651, 2036, 29912, 1079, 29890, 799, 29899, 7039, 6177, 2802, 277, 12181, 4954, 16123, 2247, 278, 26368, 5528, 5084, 310, 363, 264, 1280, 322, 269, 595, 1280, 7371, 515, 263, 2566, 310, 320, 1949, 29912, 29946, 29871, 29900, 29946, 29946, 29871, 29945, 29946, 29953, 29871, 29929, 29941, 29947, 29913, 2305, 313, 29945, 29945, 29889, 29945, 8958, 310, 8471, 2305, 297, 29871, 29906, 29900, 29896, 29946, 467, 32001, 1094, 310, 3839, 29871, 29906, 29900, 29896, 29929, 372, 18469, 320, 1949, 29912, 29906, 29955, 29871, 29953, 29953, 29906, 29871, 29947, 29900, 29896, 29913, 363, 264, 1280, 322, 320, 1949, 29912, 29906, 29955, 29871, 29906, 29900, 29953, 29871, 29947, 29906, 29896, 29913, 269, 595, 1280, 297, 29871, 29906, 29941, 29953, 24894, 8977, 1080, 29889, 4907, 29913, 32001, 1094, 297, 1749, 8783, 15717, 526, 2989, 1024, 6031, 313, 29878, 1624, 1135, 6219, 491, 937, 29914, 11922, 1024, 511, 591, 937, 5993, 675, 2983, 313, 1609, 1999, 1331, 322, 1206, 3620, 29897, 322, 769, 16280, 5375, 18897, 297, 1716, 937, 322, 3942, 2983, 10868, 8857, 29889, 32001, 1152, 1269, 1543, 1476, 297, 1024, 8857, 591, 22932, 278, 2058, 4665, 29905, 6661, 1333, 24212, 8875, 491, 278, 1024, 10868, 304, 4017, 263, 5645, 393, 338, 29839, 304, 278, 1353, 310, 12407, 24638, 393, 1024, 313, 6979, 29897, 297, 278, 2702, 2058, 29889, 32001, 320, 6661, 1333, 300, 1062, 29912, 1762, 4017, 4665, 2025, 1338, 5634, 294, 278, 17837, 310, 4954, 6689, 4907, 338, 25745, 23724, 29901, 2989, 10916, 325, 13217, 29879, 29399, 310, 2919, 10916, 805, 9450, 2999, 931, 29920, 2873, 5634, 705, 671, 263, 29544, 310, 7601, 8974, 313, 29872, 29889, 29887, 1696, 5874, 28007, 511, 322, 1661, 29899, 16072, 6743, 313, 29872, 29889, 29887, 1696, 14109, 7456, 467, 29913, 32001, 1334, 2533, 445, 4377, 363, 599, 3161, 304, 4017, 263, 2058, 8158, 29892, 17140, 701, 411, 263, 1051, 310, 779, 6990, 29938, 6689, 29892, 8158, 4535, 5854, 29938, 11000, 29889, 32001, 1334, 769, 8877, 445, 1051, 491, 278, 3186, 5120, 393, 263, 2058, 14393, 304, 322, 2533, 278, 8158, 363, 599, 278, 7600, 297, 1269, 5120, 304, 4017, 385, 12463, 8158, 29892, 6590, 304, 278, 4188, 22342, 393, 278, 9063, 14393, 304, 263, 2183, 3186, 5120, 29889, 32001, 1334, 3566, 278, 6257, 9063, 408, 6421, 515, 278, 3186, 5120, 411, 278, 9939, 8158, 29889, 32001, 450, 4876, 29899, 6707, 11043, 9378, 414, 515, 278, 9078, 322, 443, 5521, 8362, 671, 310, 21759, 29911, 10249, 29879, 29901, 1556, 18777, 671, 10035, 323, 10249, 29879, 1316, 408, 320, 16266, 26139, 510, 1118, 320, 16266, 26139, 990, 1118, 470, 320, 16266, 26139, 1212, 1836, 32001, 12808, 445, 947, 451, 3799, 26018, 4822, 20542, 29901, 3148, 29899, 6707, 18777, 29892, 363, 1342, 29892, 671, 278, 320, 16266, 26139, 375, 29913, 21759, 29911, 10249, 1568, 901, 269, 2495, 290, 368, 1135, 1009, 7824, 6795, 20895, 29889, 32001, 1551, 278, 916, 1361, 278, 9210, 29914, 978, 29899, 6707, 11043, 337, 3687, 373, 278, 17998, 9210, 310, 278, 9063, 5335, 342, 15092, 29889, 32001, 16809, 304, 5780, 22920, 373, 13897, 731, 14340, 29892, 263, 2919, 1353, 310, 25741, 297, 278, 8783, 756, 385, 17998, 9210, 5186, 304, 5225, 29889, 32001, 910, 6602, 29879, 3109, 7786, 25741, 3441, 14573, 5261, 1169, 29911, 29999, 29999, 13985, 1349, 681, 392, 27418, 6478, 29914, 310, 29871, 29906, 29900, 29906, 29900, 29879, 25741, 505, 263, 5225, 9210, 29897, 1135, 9642, 6743, 3441, 14573, 5261, 1169, 29911, 29999, 29999, 13985, 1349, 681, 392, 29914, 297, 29871, 29906, 29900, 29900, 29900, 467, 32001, 1094, 263, 1121, 278, 9210, 29914, 978, 29899, 6707, 11043, 1033, 1095, 701, 6459, 292, 263, 2919, 6232, 310, 9642, 25741, 408, 4148, 287, 491, 11715, 18777, 29892, 322, 304, 263, 3109, 261, 15834, 4092, 550, 29889, 32001, 1763, 6795, 1438, 5626, 591, 14405, 278, 1023, 1737, 324, 10610, 13698, 4208, 491, 15399, 278, 9210, 29914, 978, 29899, 6707, 13698, 304, 599, 25741, 411, 263, 1661, 29899, 9171, 17998, 9210, 29892, 322, 278, 4876, 29899, 6707, 373, 304, 599, 916, 25741, 29889, 32001, 320, 2042, 29912, 12191, 322, 8565, 1558, 291, 29913, 13, 29905, 1643, 29912, 3471, 29901, 9902, 29913, 13, 13, 29905, 463, 29912, 4532, 4044, 13, 29871, 320, 9525, 13, 29871, 320, 7313, 29961, 2103, 2013, 16292, 3199, 1429, 287, 29889, 5140, 29913, 13, 29871, 320, 6671, 29912, 29934, 20819, 310, 25741, 313, 27215, 29897, 322, 6136, 15717, 313, 22503, 29897, 491, 3186, 10640, 975, 278, 29871, 29896, 29929, 29955, 29896, 489, 29906, 29900, 29906, 29900, 3785, 5003, 32001, 320, 9868, 29961, 14732, 3199, 7264, 287, 2594, 8727, 6445, 278, 3186, 10640, 364, 2219, 359, 363, 25741, 322, 15717, 975, 278, 29871, 29896, 29929, 29955, 29896, 489, 29906, 29900, 29906, 29900, 3785, 5003, 32001, 320, 1643, 29912, 1003, 29901, 9902, 29913, 13, 29905, 355, 29912, 4532, 4044, 13, 13, 13, 29871, 13, 1762, 1234, 320, 25029, 29912, 29878, 29939, 29901, 479, 397, 24974, 29913, 591, 22229, 278, 1353, 310, 25741, 322, 8359, 15717, 639, 1629, 322, 639, 3186, 10640, 29889, 32001, 1334, 2198, 278, 7625, 2582, 297, 320, 29907, 999, 29912, 1003, 29901, 9902, 29913, 408, 1023, 5096, 287, 2594, 24469, 29892, 6445, 1629, 368, 2867, 3204, 29879, 363, 25741, 322, 15717, 8307, 29889, 32001, 7569, 2594, 11524, 263, 1629, 322, 338, 8877, 287, 297, 269, 29399, 6445, 278, 9063, 29914, 8921, 11959, 363, 1269, 310, 278, 3186, 12786, 310, 320, 29907, 999, 29912, 1003, 29901, 11526, 1958, 29913, 297, 393, 1629, 29889, 32001, 1763, 4772, 714, 27801, 2861, 304, 805, 272, 26538, 17737, 29560, 29892, 297, 278, 4148, 8727, 591, 871, 2050, 15717, 2534, 26869, 472, 3203, 29871, 29945, 25741, 297, 263, 2183, 1629, 29889, 32001, 5806, 5366, 1747, 534, 1975, 297, 278, 24469, 6456, 393, 278, 3001, 3694, 310, 25741, 322, 15717, 6548, 18709, 9247, 975, 931, 29889, 32001, 10133, 363, 278, 937, 2440, 297, 278, 24469, 29892, 278, 1353, 310, 848, 3291, 297, 777, 3186, 12786, 508, 367, 14154, 2319, 29892, 411, 8178, 27721, 373, 278, 25806, 310, 534, 1975, 29889, 32001, 320, 26956, 29912, 7999, 12122, 6894, 537, 975, 931, 29913, 13, 13, 3563, 497, 29892, 278, 2498, 534, 355, 5692, 304, 367, 393, 278, 320, 4534, 29912, 479, 12122, 6894, 537, 297, 970, 775, 338, 10231, 6177, 4644, 6813, 322, 4092, 5136, 630, 1009, 4954, 24130, 749, 4907, 2745, 278, 7256, 310, 278, 29871, 29929, 29900]} Using /root/.cache/torch_extensions/py38_cu117 as PyTorch extensions root... Using /root/.cache/torch_extensions/py38_cu117 as PyTorch extensions root... Using /root/.cache/torch_extensions/py38_cu117 as PyTorch extensions root... Using /root/.cache/torch_extensions/py38_cu117 as PyTorch extensions root... Using /root/.cache/torch_extensions/py38_cu117 as PyTorch extensions root... Using /root/.cache/torch_extensions/py38_cu117 as PyTorch extensions root... Using /root/.cache/torch_extensions/py38_cu117 as PyTorch extensions root... Detected CUDA files, patching ldflags Emitting ninja build file /root/.cache/torch_extensions/py38_cu117/cpu_adam/build.ninja... Building extension module cpu_adam... Allowing ninja to set a default number of workers... (overridable by setting the environment variable MAX_JOBS=N) Using /root/.cache/torch_extensions/py38_cu117 as PyTorch extensions root... ninja: no work to do. Loading extension module cpu_adam... Time to load cpu_adam op: 1.1635229587554932 seconds Loading extension module cpu_adam... Time to load cpu_adam op: 1.0293986797332764 seconds Loading extension module cpu_adam... Time to load cpu_adam op: 1.1567466259002686 seconds Loading extension module cpu_adam... Time to load cpu_adam op: 1.0295722484588623 seconds Loading extension module cpu_adam... Time to load cpu_adam op: 1.1684362888336182 seconds Loading extension module cpu_adamLoadTime to load cpu_adam op: 1.047499179840088 seconds Loading extension module cpu_adam... Time to load cpu_adam op: 0.9710671901702881 seconds Loading extension module cpu_adam... Time to load cpu_adam op: 1.137798547744751 seconds Adam Optimizer #0 is created with AVX2 arithmetic capability. Config: alpha=0.000020, betas=(0.900000, 0.999000), weight_decay=0.000000, adam_w=1 3552246 seconds Time to load cpu_adam op: 1.1131987571716309 seconds Loading extension module cpu_adam... Time to load cpu_adam op: 1.167346715927124 seconds sion module cpu_adam... Time to load cpu_adam op: 1.107062578201294 seconds Adam Optimizer #0 is created with AVX2 arithmetic capability. Config: alpha=0.000020, betas=(0.900000, 0.999000), weight_decay=0.000000, adam_w=1 [INFO|trainer.py:1755] 2024-01-28 04:53:43,971 >> ***** Running training ***** [INFO|trainer.py:1756] 2024-01-28 04:53:43,971 >> Num examples = 930514 [INFO|trainer.py:1757] 2024-01-28 04:53:43,971 >> Num Epochs = 1 [INFO|trainer.py:1758] 2024-01-28 04:53:43,971 >> Instantaneous batch size per device = 2 [INFO|trainer.py:1759] 2024-01-28 04:53:43,971 >> Total train batch size (w. parallel, distributed & accumulation) = 512 [INFO|trainer.py:1760] 2024-01-28 04:53:43,971 >> Gradient Accumulation steps = 8 [INFO|trainer.py:1761] 2024-01-28 04:53:43,971 >> Total optimization steps = 1817 [INFO|trainer.py:1762] 2024-01-28 04:53:43,973 >> Number of trainable parameters = 6738432000 0%| | 0/1817 [00:00> ***** Running training ***** [INFO|trainer.py:1756] 2024-01-28 04:53:43,959 >> Num examples = 930514 [INFO|trainer.py:1757] 2024-01-28 04:53:43,959 >> Num Epochs = 1 [INFO|trainer.py:1758] 2024-01-28 04:53:43,959 >> Instantaneous batch size per device = 2 [INFO|trainer.py:1759] 2024-01-28 04:53:43,959 >> Total train batch size (w. parallel, distributed & accumulation) = 512 [INFO|trainer.py:1760] 2024-01-28 04:53:43,959 >> Gradient Accumulation steps = 8 [INFO|trainer.py:1761] 2024-01-28 04:53:43,959 >> Total optimization steps = 1817 [INFO|trainer.py:1762] 2024-01-28 04:53:43,961 >> Number of trainable parameters = 6738432000 0%| | 0/1817 [00:0016->23 [1] 17/-1/-1->16->23 ts-cbba87c5e7504a249f5127103d9ce40f-worker-1:97052:7432 [1] NCCL INFO Trees [0] -1/-1/-1->17->16 [1] -1/-1/-1->17->16 ts-cbba87c5e7504a249f5127103d9ce40f-worker-1:97053:7437 [2] NCCL INFO Trees [0] 19/26/-1->18->2 [1] 19/-1/-1->18->11 ts-cbba87c5e7504a249f5127103d9ce40f-worker-1:97058:7435 [7] NCCL INFO Trees [0] 16/-1/-1->23->22 [1] 16/-1/-1->23->22 ts-cbba87c5e7504a249f5127103d9ce40f-worker-1:97057:7433 [6] NCCL INFO Trees [0] 23/-1/-1->22->21 [1] 23/-1/-1->22->21 ts-cbba87c5e7504a249f5127103d9ce40f-worker-1:97055:7430 [4] NCCL INFO Trees [0] 21/-1/-1->20->19 [1] 21/-1/-1->20->19 ts-cbba87c5e7504a249f5127103d9ce40f-worker-1:97054:7431 [3] NCCL INFO Trees [0] 20/10/-1->19->18 [1] 20/-1/-1->19->18 ts-cbba87c5e7504a249f5127103d9ce40f-worker-1:97056:7434 [5] NCCL INFO Trees [0] 22/-1/-1->21->20 [1] 22/-1/-1->21->20 ts-cbba87c5e7504a249f5127103d9ce40f-worker-1:97053:7437 [2] NCCL INFO Channel 00/0 : 18[4b000] -> 23[d0000] via P2P/IPC/read ts-cbba87c5e7504a249f5127103d9ce40f-worker-1:97055:7430 [4] NCCL INFO Channel 00/0 : 20[93000] -> 17[13000] via P2P/IPC/read ts-cbba87c5e7504a249f5127103d9ce40f-worker-1:97051:7436 [0] NCCL INFO Channel 00/0 : 16[e000] -> 19[51000] via P2P/IPC/read ts-cbba87c5e7504a249f5127103d9ce40f-worker-1:97053:7437 [2] NCCL INFO Channel 01/0 : 18[4b000] -> 23[d0000] via P2P/IPC/read ts-cbba87c5e7504a249f5127103d9ce40f-worker-1:97051:7436 [0] NCCL INFO Channel 01/0 : 16[e000] -> 19[51000] via P2P/IPC/read ts-cbba87c5e7504a249f5127103d9ce40f-worker-1:97055:7430 [4] NCCL INFO Channel 01/0 : 20[93000] -> 17[13000] via P2P/IPC/read ts-cbba87c5e7504a249f5127103d9ce40f-worker-1:97057:7433 [6] NCCL INFO Channel 00/0 : 22[cb000] -> 21[99000] via P2P/IPC/read ts-cbba87c5e7504a249f5127103d9ce40f-worker-1:97056:7434 [5] NCCL INFO Channel 00/0 : 21[99000] -> 20[93000] via P2P/IPC/read ts-cbba87c5e7504a249f5127103d9ce40f-worker-1:97057:7433 [6] NCCL INFO Channel 01/0 : 22[cb000] -> 21[99000] via P2P/IPC/read ts-cbba87c5e7504a249f5127103d9ce40f-worker-1:97058:7435 [7] NCCL INFO Channel 00/0 : 23[d0000] -> 22[cb000] via P2P/IPC/read ts-cbba87c5e7504a249f5127103d9ce40f-worker-1:97056:7434 [5] NCCL INFO Channel 01/0 : 21[99000] -> 20[93000] via P2P/IPC/read ts-cbba87c5e7504a249f5127103d9ce40f-worker-1:97052:7432 [1] NCCL INFO Channel 00/0 : 17[13000] -> 16[e000] via P2P/IPC/read ts-cbba87c5e7504a249f5127103d9ce40f-worker-1:97058:7435 [7] NCCL INFO Channel 01/0 : 23[d0000] -> 22[cb000] via P2P/IPC/read ts-cbba87c5e7504a249f5127103d9ce40f-worker-1:97052:7432 [1] NCCL INFO Channel 01/0 : 17[13000] -> 16[e000] via P2P/IPC/read ts-cbba87c5e7504a249f5127103d9ce40f-worker-1:97058:7435 [7] NCCL INFO Connected all rings ts-cbba87c5e7504a249f5127103d9ce40f-worker-1:97055:7430 [4] NCCL INFO Connected all rings ts-cbba87c5e7504a249f5127103d9ce40f-worker-1:97051:7436 [0] NCCL INFO Connected all rings ts-cbba87c5e7504a249f5127103d9ce40f-worker-1:97051:7436 [0] NCCL INFO Channel 00/0 : 16[e000] -> 17[13000] via P2P/IPC/read ts-cbba87c5e7504a249f5127103d9ce40f-worker-1:97052:7432 [1] NCCL INFO Connected all rings ts-cbba87c5e7504a249f5127103d9ce40f-worker-1:97051:7436 [0] NCCL INFO Channel 01/0 : 16[e000] -> 17[13000] via P2P/IPC/read ts-cbba87c5e7504a249f5127103d9ce40f-worker-1:97057:7433 [6] NCCL INFO Connected all rings ts-cbba87c5e7504a249f5127103d9ce40f-worker-1:97056:7434 [5] NCCL INFO Connected all rings ts-cbba87c5e7504a249f5127103d9ce40f-worker-1:97055:7430 [4] NCCL INFO Channel 00/0 : 20[93000] -> 21[99000] via P2P/IPC/read ts-cbba87c5e7504a249f5127103d9ce40f-worker-1:97055:7430 [4] NCCL INFO Channel 01/0 : 20[93000] -> 21[99000] via P2P/IPC/read ts-cbba87c5e7504a249f5127103d9ce40f-worker-1:97057:7433 [6] NCCL INFO Channel 00/0 : 22[cb000] -> 23[d0000] via P2P/IPC/read ts-cbba87c5e7504a249f5127103d9ce40f-worker-1:97056:7434 [5] NCCL INFO Channel 00/0 : 21[99000] -> 22[cb000] via P2P/IPC/read ts-cbba87c5e7504a249f5127103d9ce40f-worker-1:97051:7436 [0] NCCL INFO Channel 00/0 : 16[e000] -> 23[d0000] via P2P/IPC/read ts-cbba87c5e7504a249f5127103d9ce40f-worker-1:97052:7432 [1] NCCL INFO Connected all trees ts-cbba87c5e7504a249f5127103d9ce40f-worker-1:97052:7432 [1] NCCL INFO threadThresholds 8/8/64 | 256/8/64 | 512 | 512 ts-cbba87c5e7504a249f5127103d9ce40f-worker-1:97052:7432 [1] NCCL INFO 2 coll channels, 2 p2p channels, 2 p2p channels per peer ts-cbba87c5e7504a249f5127103d9ce40f-worker-1:97057:7433 [6] NCCL INFO Channel 01/0 : 22[cb000] -> 23[d0000] via P2P/IPC/read ts-cbba87c5e7504a249f5127103d9ce40f-worker-1:97056:7434 [5] NCCL INFO Channel 01/0 : 21[99000] -> 22[cb000] via P2P/IPC/read ts-cbba87c5e7504a249f5127103d9ce40f-worker-1:97051:7436 [0] NCCL INFO Channel 01/0 : 16[e000] -> 23[d0000] via P2P/IPC/read ts-cbba87c5e7504a249f5127103d9ce40f-worker-1:97056:7434 [5] NCCL INFO Connected all trees ts-cbba87c5e7504a249f5127103d9ce40f-worker-1:97056:7434 [5] NCCL INFO threadThresholds 8/8/64 | 256/8/64 | 512 | 512 ts-cbba87c5e7504a249f5127103d9ce40f-worker-1:97056:7434 [5] NCCL INFO 2 coll channels, 2 p2p channels, 2 p2p channels per peer ts-cbba87c5e7504a249f5127103d9ce40f-worker-1:97057:7433 [6] NCCL INFO Connected all trees ts-cbba87c5e7504a249f5127103d9ce40f-worker-1:97057:7433 [6] NCCL INFO threadThresholds 8/8/64 | 256/8/64 | 512 | 512 ts-cbba87c5e7504a249f5127103d9ce40f-worker-1:97057:7433 [6] NCCL INFO 2 coll channels, 2 p2p channels, 2 p2p channels per peer ts-cbba87c5e7504a249f5127103d9ce40f-worker-1:97058:7435 [7] NCCL INFO Channel 00/0 : 23[d0000] -> 16[e000] via P2P/IPC/read ts-cbba87c5e7504a249f5127103d9ce40f-worker-1:97058:7435 [7] NCCL INFO Channel 01/0 : 23[d0000] -> 16[e000] via P2P/IPC/read ts-cbba87c5e7504a249f5127103d9ce40f-worker-1:97053:7437 [2] NCCL INFO Channel 00/0 : 11[51000] -> 18[4b000] [receive] via NET/IB/0/GDRDMA ts-cbba87c5e7504a249f5127103d9ce40f-worker-1:97054:7431 [3] NCCL INFO Channel 00/0 : 19[51000] -> 26[4b000] [send] via NET/IB/0/GDRDMA ts-cbba87c5e7504a249f5127103d9ce40f-worker-1:97058:7435 [7] NCCL INFO Connected all trees ts-cbba87c5e7504a249f5127103d9ce40f-worker-1:97058:7435 [7] NCCL INFO threadThresholds 8/8/64 | 256/8/64 | 512 | 512 ts-cbba87c5e7504a249f5127103d9ce40f-worker-1:97058:7435 [7] NCCL INFO 2 coll channels, 2 p2p channels, 2 p2p channels per peer ts-cbba87c5e7504a249f5127103d9ce40f-worker-1:97051:7436 [0] NCCL INFO Connected all trees ts-cbba87c5e7504a249f5127103d9ce40f-worker-1:97051:7436 [0] NCCL INFO threadThresholds 8/8/64 | 256/8/64 | 512 | 512 ts-cbba87c5e7504a249f5127103d9ce40f-worker-1:97051:7436 [0] NCCL INFO 2 coll channels, 2 p2p channels, 2 p2p channels per peer ts-cbba87c5e7504a249f5127103d9ce40f-worker-1:97053:7437 [2] NCCL INFO Channel 01/0 : 11[51000] -> 18[4b000] [receive] via NET/IB/0/GDRDMA ts-cbba87c5e7504a249f5127103d9ce40f-worker-1:97054:7431 [3] NCCL INFO Channel 01/0 : 19[51000] -> 26[4b000] [send] via NET/IB/0/GDRDMA ts-cbba87c5e7504a249f5127103d9ce40f-worker-1:97054:7431 [3] NCCL INFO Connected all rings ts-cbba87c5e7504a249f5127103d9ce40f-worker-1:97054:7431 [3] NCCL INFO Channel 00/0 : 19[51000] -> 20[93000] via P2P/IPC/read ts-cbba87c5e7504a249f5127103d9ce40f-worker-1:97054:7431 [3] NCCL INFO Channel 01/0 : 19[51000] -> 20[93000] via P2P/IPC/read ts-cbba87c5e7504a249f5127103d9ce40f-worker-1:97053:7437 [2] NCCL INFO Connected all rings ts-cbba87c5e7504a249f5127103d9ce40f-worker-1:97053:7437 [2] NCCL INFO Channel 00/0 : 18[4b000] -> 19[51000] via P2P/IPC/read ts-cbba87c5e7504a249f5127103d9ce40f-worker-1:97053:7437 [2] NCCL INFO Channel 01/0 : 18[4b000] -> 19[51000] via P2P/IPC/read ts-cbba87c5e7504a249f5127103d9ce40f-worker-1:97055:7430 [4] NCCL INFO Channel 00/0 : 20[93000] -> 19[51000] via P2P/IPC/read ts-cbba87c5e7504a249f5127103d9ce40f-worker-1:97055:7430 [4] NCCL INFO Channel 01/0 : 20[93000] -> 19[51000] via P2P/IPC/read ts-cbba87c5e7504a249f5127103d9ce40f-worker-1:97053:7437 [2] NCCL INFO Channel 00/0 : 18[4b000] -> 26[4b000] [send] via NET/IB/0/GDRDMA ts-cbba87c5e7504a249f5127103d9ce40f-worker-1:97054:7431 [3] NCCL INFO Channel 00/0 : 10[4b000] -> 19[51000] [receive] via NET/IB/0/GDRDMA ts-cbba87c5e7504a249f5127103d9ce40f-worker-1:97053:7437 [2] NCCL INFO Channel 00/0 : 2[4b000] -> 18[4b000] [receive] via NET/IB/0/GDRDMA ts-cbba87c5e7504a249f5127103d9ce40f-worker-1:97053:7437 [2] NCCL INFO Channel 00/0 : 18[4b000] -> 2[4b000] [send] via NET/IB/0/GDRDMA ts-cbba87c5e7504a249f5127103d9ce40f-worker-1:97054:7431 [3] NCCL INFO Channel 00/0 : 19[51000] -> 10[4b000] [send] via NET/IB/0/GDRDMA ts-cbba87c5e7504a249f5127103d9ce40f-worker-1:97053:7437 [2] NCCL INFO Channel 00/0 : 26[4b000] -> 18[4b000] [receive] via NET/IB/0/GDRDMA ts-cbba87c5e7504a249f5127103d9ce40f-worker-1:97054:7431 [3] NCCL INFO Channel 00/0 : 19[51000] -> 18[4b000] via P2P/IPC/read ts-cbba87c5e7504a249f5127103d9ce40f-worker-1:97054:7431 [3] NCCL INFO Channel 01/0 : 19[51000] -> 18[4b000] via P2P/IPC/read ts-cbba87c5e7504a249f5127103d9ce40f-worker-1:97055:7430 [4] NCCL INFO Connected all trees ts-cbba87c5e7504a249f5127103d9ce40f-worker-1:97055:7430 [4] NCCL INFO threadThresholds 8/8/64 | 256/8/64 | 512 | 512 ts-cbba87c5e7504a249f5127103d9ce40f-worker-1:97055:7430 [4] NCCL INFO 2 coll channels, 2 p2p channels, 2 p2p channels per peer ts-cbba87c5e7504a249f5127103d9ce40f-worker-1:97053:7437 [2] NCCL INFO Channel 01/0 : 18[4b000] -> 11[51000] [send] via NET/IB/0/GDRDMA ts-cbba87c5e7504a249f5127103d9ce40f-worker-1:97053:7437 [2] NCCL INFO Connected all trees ts-cbba87c5e7504a249f5127103d9ce40f-worker-1:97053:7437 [2] NCCL INFO threadThresholds 8/8/64 | 256/8/64 | 512 | 512 ts-cbba87c5e7504a249f5127103d9ce40f-worker-1:97053:7437 [2] NCCL INFO 2 coll channels, 2 p2p channels, 2 p2p channels per peer ts-cbba87c5e7504a249f5127103d9ce40f-worker-1:97054:7431 [3] NCCL INFO Connected all trees ts-cbba87c5e7504a249f5127103d9ce40f-worker-1:97054:7431 [3] NCCL INFO threadThresholds 8/8/64 | 256/8/64 | 512 | 512 ts-cbba87c5e7504a249f5127103d9ce40f-worker-1:97054:7431 [3] NCCL INFO 2 coll channels, 2 p2p channels, 2 p2p channels per peer ts-cbba87c5e7504a249f5127103d9ce40f-worker-1:97056:7434 [5] NCCL INFO comm 0x7ef52002a770 rank 21 nranks 32 cudaDev 5 busId 99000 - Init COMPLETE ts-cbba87c5e7504a249f5127103d9ce40f-worker-1:97058:7435 [7] NCCL INFO comm 0x7fc8a802a5f0 rank 23 nranks 32 cudaDev 7 busId d0000 - Init COMPLETE ts-cbba87c5e7504a249f5127103d9ce40f-worker-1:97055:7430 [4] NCCL INFO comm 0x7f261802a750 rank 20 nranks 32 cudaDev 4 busId 93000 - Init COMPLETE ts-cbba87c5e7504a249f5127103d9ce40f-worker-1:97057:7433 [6] NCCL INFO comm 0x7f77e802a890 rank 22 nranks 32 cudaDev 6 busId cb000 - Init COMPLETE ts-cbba87c5e7504a249f5127103d9ce40f-worker-1:97054:7431 [3] NCCL INFO comm 0x7f37e402a7b0 rank 19 nranks 32 cudaDev 3 busId 51000 - Init COMPLETE ts-cbba87c5e7504a249f5127103d9ce40f-worker-1:97052:7432 [1] NCCL INFO comm 0x7f351c02aa30 rank 17 nranks 32 cudaDev 1 busId 13000 - Init COMPLETE ts-cbba87c5e7504a249f5127103d9ce40f-worker-1:97051:7436 [0] NCCL INFO comm 0x7fb80c02a8e0 rank 16 nranks 32 cudaDev 0 busId e000 - Init COMPLETE ts-cbba87c5e7504a249f5127103d9ce40f-worker-1:97053:7437 [2] NCCL INFO comm 0x7f9f8c02a810 rank 18 nranks 32 cudaDev 2 busId 4b000 - Init COMPLETE 32 cudaDev 5 busId 99000 - Init COMPLETE ts-cbba87c5e7504a249f5127103d9ce40f-worker-2:165569:178031 [4] NCCL INFO comm 0x7f3ec002a730 rank 28 nranks 32 cudaDev 4 busId 93000 - Init COMPLETE ts-cbba87c5e7504a249f5127103d9ce40f-worker-2:165572:178027 [7] NCCL INFO comm 0x7ef28802a810 rank 31 nranks 32 cudaDev 7 busId d0000 - Init COMPLETE ts-cbba87c5e7504a249f5127103d9ce40f-worker-2:165571:178033 [6] NCCL INFO comm 0x7f43bc02a860 rank 30 nranks 32 cudaDev 6 busId cb000 - Init COMPLETE ts-cbba87c5e7504a249f5127103d9ce40f-worker-2:165568:178034 [3] NCCL INFO comm 0x7fee2002a8b0 rank 27 nranks 32 cudaDev 3 busId 51000 - Init COMPLETE ts-cbba87c5e7504a249f5127103d9ce40f-worker-2:165567:178032 [2] NCCL INFO comm 0x7f396c02a800 rank 26 nranks 32 cudaDev 2 busId 4b000 - Init COMPLETE ts-cbba87c5e7504a249f5127103d9ce40f-worker-2:165565:178030 [0] NCCL INFO comm 0x7fdee002a7e0 rank 24 nranks 32 cudaDev 0 busId e000 - Init COMPLETE ts-cbba87c5e7504a249f5127103d9ce40f-worker-2:165566:178028 [1] NCCL INFO comm 0x7ef39402a6c0 rank 25 nranks 32 cudaDev 1 busId 13000 - Init COMPLETE rward_thisversion_new - `use_cache=True` is incompatible with gradient checkpointing. Setting `use_cache=False`... 01/28/2024 04:53:45 - WARNING - llama_sft_forward_thisversion_new - `use_cache=True` is incompatible with gradient checkpointing. Setting `use_cache=False`... ts-cbba87c5e7504a249f5127103d9ce40f-worker-0:18322:30611 [5] NCCL INFO Using network IB ts-cbba87c5e7504a249f5127103d9ce40f-worker-0:18319:30613 [2] NCCL INFO Using network IB ts-cbba87c5e7504a249f5127103d9ce40f-worker-0:18323:30614 [6] NCCL INFO Using network IB ts-cbba87c5e7504a249f5127103d9ce40f-worker-0:18317:30615 [0] NCCL INFO Using network IB ts-cbba87c5e7504a249f5127103d9ce40f-worker-0:18321:30616 [4] NCCL INFO Using network IB ts-cbba87c5e7504a249f5127103d9ce40f-worker-0:18318:30612 [1] NCCL INFO Using network IB ts-cbba87c5e7504a249f5127103d9ce40f-worker-0:18324:30617 [7] NCCL INFO Using network IB ts-cbba87c5e7504a249f5127103d9ce40f-worker-0:18320:30618 [3] NCCL INFO Using network IB ts-cbba87c5e7504a249f5127103d9ce40f-worker-0:18317:30615 [0] NCCL INFO Setting affinity for GPU 0 to ffff,ffffffff,00000000,0000ffff,ffffffff ts-cbba87c5e7504a249f5127103d9ce40f-worker-0:18324:30617 [7] NCCL INFO Setting affinity for GPU 7 to ffffffff,ffff0000,00000000,ffffffff,ffff0000,00000000 ts-cbba87c5e7504a249f5127103d9ce40f-worker-0:18323:30614 [6] NCCL INFO Setting affinity for GPU 6 to ffffffff,ffff0000,00000000,ffffffff,ffff0000,00000000 ts-cbba87c5e7504a249f5127103d9ce40f-worker-0:18322:30611 [5] NCCL INFO Setting affinity for GPU 5 to ffffffff,ffff0000,00000000,ffffffff,ffff0000,00000000 ts-cbba87c5e7504a249f5127103d9ce40f-worker-0:18318:30612 [1] NCCL INFO Setting affinity for GPU 1 to ffff,ffffffff,00000000,0000ffff,ffffffff ts-cbba87c5e7504a249f5127103d9ce40f-worker-0:18319:30613 [2] NCCL INFO Setting affinity for GPU 2 to ffff,ffffffff,00000000,0000ffff,ffffffff ts-cbba87c5e7504a249f5127103d9ce40f-worker-0:18320:30618 [3] NCCL INFO Setting affinity for GPU 3 to ffff,ffffffff,00000000,0000ffff,ffffffff ts-cbba87c5e7504a249f5127103d9ce40f-worker-0:18321:30616 [4] NCCL INFO Setting affinity for GPU 4 to ffffffff,ffff0000,00000000,ffffffff,ffff0000,00000000 ts-cbba87c5e7504a249f5127103d9ce40f-worker-0:18322:30611 [5] NCCL INFO Trees [0] 14/-1/-1->13->12 [1] 14/-1/-1->13->12 ts-cbba87c5e7504a249f5127103d9ce40f-worker-0:18324:30617 [7] NCCL INFO Trees [0] 8/-1/-1->15->14 [1] 8/-1/-1->15->14 ts-cbba87c5e7504a249f5127103d9ce40f-worker-0:18317:30615 [0] NCCL INFO Trees [0] 9/-1/-1->8->15 [1] 9/-1/-1->8->15 ts-cbba87c5e7504a249f5127103d9ce40f-worker-0:18323:30614 [6] NCCL INFO Trees [0] 15/-1/-1->14->13 [1] 15/-1/-1->14->13 ts-cbba87c5e7504a249f5127103d9ce40f-worker-0:18320:30618 [3] NCCL INFO Trees [0] 12/-1/-1->11->10 [1] 12/18/-1->11->10 ts-cbba87c5e7504a249f5127103d9ce40f-worker-0:18321:30616 [4] NCCL INFO Trees [0] 13/-1/-1->12->11 [1] 13/-1/-1->12->11 ts-cbba87c5e7504a249f5127103d9ce40f-worker-0:18318:30612 [1] NCCL INFO Trees [0] -1/-1/-1->9->8 [1] -1/-1/-1->9->8 ts-cbba87c5e7504a249f5127103d9ce40f-worker-0:18319:30613 [2] NCCL INFO Trees [0] 11/-1/-1->10->19 [1] 11/2/-1->10->26 ts-cbba87c5e7504a249f5127103d9ce40f-worker-0:18317:30615 [0] NCCL INFO Channel 00/0 : 8[e000] -> 11[51000] via P2P/IPC/read ts-cbba87c5e7504a249f5127103d9ce40f-worker-0:18321:30616 [4] NCCL INFO Channel 00/0 : 12[93000] -> 9[13000] via P2P/IPC/read ts-cbba87c5e7504a249f5127103d9ce40f-worker-0:18319:30613 [2] NCCL INFO Channel 00/0 : 10[4b000] -> 15[d0000] via P2P/IPC/read ts-cbba87c5e7504a249f5127103d9ce40f-worker-0:18317:30615 [0] NCCL INFO Channel 01/0 : 8[e000] -> 11[51000] via P2P/IPC/read ts-cbba87c5e7504a249f5127103d9ce40f-worker-0:18321:30616 [4] NCCL INFO Channel 01/0 : 12[93000] -> 9[13000] via P2P/IPC/read ts-cbba87c5e7504a249f5127103d9ce40f-worker-0:18319:30613 [2] NCCL INFO Channel 01/0 : 10[4b000] -> 15[d0000] via P2P/IPC/read ts-cbba87c5e7504a249f5127103d9ce40f-worker-0:18322:30611 [5] NCCL INFO Channel 00/0 : 13[99000] -> 12[93000] via P2P/IPC/read ts-cbba87c5e7504a249f5127103d9ce40f-worker-0:18323:30614 [6] NCCL INFO Channel 00/0 : 14[cb000] -> 13[99000] via P2P/IPC/read ts-cbba87c5e7504a249f5127103d9ce40f-worker-0:18322:30611 [5] NCCL INFO Channel 01/0 : 13[99000] -> 12[93000] via P2P/IPC/read ts-cbba87c5e7504a249f5127103d9ce40f-worker-0:18323:30614 [6] NCCL INFO Channel 01/0 : 14[cb000] -> 13[99000] via P2P/IPC/read ts-cbba87c5e7504a249f5127103d9ce40f-worker-0:18318:30612 [1] NCCL INFO Channel 00/0 : 9[13000] -> 8[e000] via P2P/IPC/read ts-cbba87c5e7504a249f5127103d9ce40f-worker-0:18324:30617 [7] NCCL INFO Channel 00/0 : 15[d0000] -> 14[cb000] via P2P/IPC/read ts-cbba87c5e7504a249f5127103d9ce40f-worker-0:18318:30612 [1] NCCL INFO Channel 01/0 : 9[13000] -> 8[e000] via P2P/IPC/read ts-cbba87c5e7504a249f5127103d9ce40f-worker-0:18324:30617 [7] NCCL INFO Channel 01/0 : 15[d0000] -> 14[cb000] via P2P/IPC/read ts-cbba87c5e7504a249f5127103d9ce40f-worker-0:18321:30616 [4] NCCL INFO Connected all rings ts-cbba87c5e7504a249f5127103d9ce40f-worker-0:18317:30615 [0] NCCL INFO Connected all rings ts-cbba87c5e7504a249f5127103d9ce40f-worker-0:18317:30615 [0] NCCL INFO Channel 00/0 : 8[e000] -> 9[13000] via P2P/IPC/read ts-cbba87c5e7504a249f5127103d9ce40f-worker-0:18318:30612 [1] NCCL INFO Connected all rings ts-cbba87c5e7504a249f5127103d9ce40f-worker-0:18324:30617 [7] NCCL INFO Connected all rings ts-cbba87c5e7504a249f5127103d9ce40f-worker-0:18317:30615 [0] NCCL INFO Channel 01/0 : 8[e000] -> 9[13000] via P2P/IPC/read ts-cbba87c5e7504a249f5127103d9ce40f-worker-0:18322:30611 [5] NCCL INFO Connected all rings ts-cbba87c5e7504a249f5127103d9ce40f-worker-0:18323:30614 [6] NCCL INFO Connected all rings ts-cbba87c5e7504a249f5127103d9ce40f-worker-0:18321:30616 [4] NCCL INFO Channel 00/0 : 12[93000] -> 13[99000] via P2P/IPC/read ts-cbba87c5e7504a249f5127103d9ce40f-worker-0:18321:30616 [4] NCCL INFO Channel 01/0 : 12[93000] -> 13[99000] via P2P/IPC/read ts-cbba87c5e7504a249f5127103d9ce40f-worker-0:18322:30611 [5] NCCL INFO Channel 00/0 : 13[99000] -> 14[cb000] via P2P/IPC/read ts-cbba87c5e7504a249f5127103d9ce40f-worker-0:18317:30615 [0] NCCL INFO Channel 00/0 : 8[e000] -> 15[d0000] via P2P/IPC/read ts-cbba87c5e7504a249f5127103d9ce40f-worker-0:18318:30612 [1] NCCL INFO Connected all trees ts-cbba87c5e7504a249f5127103d9ce40f-worker-0:18323:30614 [6] NCCL INFO Channel 00/0 : 14[cb000] -> 15[d0000] via P2P/IPC/read ts-cbba87c5e7504a249f5127103d9ce40f-worker-0:18318:30612 [1] NCCL INFO threadThresholds 8/8/64 | 256/8/64 | 512 | 512 ts-cbba87c5e7504a249f5127103d9ce40f-worker-0:18318:30612 [1] NCCL INFO 2 coll channels, 2 p2p channels, 2 p2p channels per peer ts-cbba87c5e7504a249f5127103d9ce40f-worker-0:18322:30611 [5] NCCL INFO Channel 01/0 : 13[99000] -> 14[cb000] via P2P/IPC/read ts-cbba87c5e7504a249f5127103d9ce40f-worker-0:18317:30615 [0] NCCL INFO Channel 01/0 : 8[e000] -> 15[d0000] via P2P/IPC/read ts-cbba87c5e7504a249f5127103d9ce40f-worker-0:18323:30614 [6] NCCL INFO Channel 01/0 : 14[cb000] -> 15[d0000] via P2P/IPC/read ts-cbba87c5e7504a249f5127103d9ce40f-worker-0:18322:30611 [5] NCCL INFO Connected all trees ts-cbba87c5e7504a249f5127103d9ce40f-worker-0:18322:30611 [5] NCCL INFO threadThresholds 8/8/64 | 256/8/64 | 512 | 512 ts-cbba87c5e7504a249f5127103d9ce40f-worker-0:18322:30611 [5] NCCL INFO 2 coll channels, 2 p2p channels, 2 p2p channels per peer ts-cbba87c5e7504a249f5127103d9ce40f-worker-0:18323:30614 [6] NCCL INFO Connected all trees ts-cbba87c5e7504a249f5127103d9ce40f-worker-0:18323:30614 [6] NCCL INFO threadThresholds 8/8/64 | 256/8/64 | 512 | 512 ts-cbba87c5e7504a249f5127103d9ce40f-worker-0:18323:30614 [6] NCCL INFO 2 coll channels, 2 p2p channels, 2 p2p channels per peer ts-cbba87c5e7504a249f5127103d9ce40f-worker-0:18324:30617 [7] NCCL INFO Channel 00/0 : 15[d0000] -> 8[e000] via P2P/IPC/read ts-cbba87c5e7504a249f5127103d9ce40f-worker-0:18324:30617 [7] NCCL INFO Channel 01/0 : 15[d0000] -> 8[e000] via P2P/IPC/read ts-cbba87c5e7504a249f5127103d9ce40f-worker-0:18317:30615 [0] NCCL INFO Connected all trees ts-cbba87c5e7504a249f5127103d9ce40f-worker-0:18317:30615 [0] NCCL INFO threadThresholds 8/8/64 | 256/8/64 | 512 | 512 ts-cbba87c5e7504a249f5127103d9ce40f-worker-0:18317:30615 [0] NCCL INFO 2 coll channels, 2 p2p channels, 2 p2p channels per peer ts-cbba87c5e7504a249f5127103d9ce40f-worker-0:18324:30617 [7] NCCL INFO Connected all trees ts-cbba87c5e7504a249f5127103d9ce40f-worker-0:18324:30617 [7] NCCL INFO threadThresholds 8/8/64 | 256/8/64 | 512 | 512 ts-cbba87c5e7504a249f5127103d9ce40f-worker-0:18324:30617 [7] NCCL INFO 2 coll channels, 2 p2p channels, 2 p2p channels per peer ts-cbba87c5e7504a249f5127103d9ce40f-worker-0:18320:30618 [3] NCCL INFO Channel 00/0 : 11[51000] -> 18[4b000] [send] via NET/IB/0/GDRDMA ts-cbba87c5e7504a249f5127103d9ce40f-worker-0:18319:30613 [2] NCCL INFO Channel 00/0 : 3[51000] -> 10[4b000] [receive] via NET/IB/0/GDRDMA ts-cbba87c5e7504a249f5127103d9ce40f-worker-0:18319:30613 [2] NCCL INFO Channel 01/0 : 3[51000] -> 10[4b000] [receive] via NET/IB/0/GDRDMA ts-cbba87c5e7504a249f5127103d9ce40f-worker-0:18320:30618 [3] NCCL INFO Channel 01/0 : 11[51000] -> 18[4b000] [send] via NET/IB/0/GDRDMA ts-cbba87c5e7504a249f5127103d9ce40f-worker-0:18320:30618 [3] NCCL INFO Connected all rings ts-cbba87c5e7504a249f5127103d9ce40f-worker-0:18320:30618 [3] NCCL INFO Channel 00/0 : 11[51000] -> 12[93000] via P2P/IPC/read ts-cbba87c5e7504a249f5127103d9ce40f-worker-0:18320:30618 [3] NCCL INFO Channel 01/0 : 11[51000] -> 12[93000] via P2P/IPC/read ts-cbba87c5e7504a249f5127103d9ce40f-worker-0:18319:30613 [2] NCCL INFO Connected all rings ts-cbba87c5e7504a249f5127103d9ce40f-worker-0:18319:30613 [2] NCCL INFO Channel 00/0 : 10[4b000] -> 11[51000] via P2P/IPC/read ts-cbba87c5e7504a249f5127103d9ce40f-worker-0:18319:30613 [2] NCCL INFO Channel 01/0 : 10[4b000] -> 11[51000] via P2P/IPC/read ts-cbba87c5e7504a249f5127103d9ce40f-worker-0:18321:30616 [4] NCCL INFO Channel 00/0 : 12[93000] -> 11[51000] via P2P/IPC/read ts-cbba87c5e7504a249f5127103d9ce40f-worker-0:18321:30616 [4] NCCL INFO Channel 01/0 : 12[93000] -> 11[51000] via P2P/IPC/read ts-cbba87c5e7504a249f5127103d9ce40f-worker-0:18319:30613 [2] NCCL INFO Channel 01/0 : 2[4b000] -> 10[4b000] [receive] via NET/IB/0/GDRDMA ts-cbba87c5e7504a249f5127103d9ce40f-worker-0:18320:30618 [3] NCCL INFO Channel 01/0 : 18[4b000] -> 11[51000] [receive] via NET/IB/0/GDRDMA ts-cbba87c5e7504a249f5127103d9ce40f-worker-0:18319:30613 [2] NCCL INFO Channel 00/0 : 10[4b000] -> 19[51000] [send] via NET/IB/0/GDRDMA ts-cbba87c5e7504a249f5127103d9ce40f-worker-0:18319:30613 [2] NCCL INFO Channel 01/0 : 26[4b000] -> 10[4b000] [receive] via NET/IB/0/GDRDMA ts-cbba87c5e7504a249f5127103d9ce40f-worker-0:18319:30613 [2] NCCL INFO Channel 01/0 : 10[4b000] -> 26[4b000] [send] via NET/IB/0/GDRDMA ts-cbba87c5e7504a249f5127103d9ce40f-worker-0:18319:30613 [2] NCCL INFO Channel 00/0 : 19[51000] -> 10[4b000] [receive] via NET/IB/0/GDRDMA ts-cbba87c5e7504a249f5127103d9ce40f-worker-0:18319:30613 [2] NCCL INFO Channel 01/0 : 10[4b000] -> 2[4b000] [send] via NET/IB/0/GDRDMA ts-cbba87c5e7504a249f5127103d9ce40f-worker-0:18320:30618 [3] NCCL INFO Channel 00/0 : 11[51000] -> 10[4b000] via P2P/IPC/read ts-cbba87c5e7504a249f5127103d9ce40f-worker-0:18320:30618 [3] NCCL INFO Channel 01/0 : 11[51000] -> 10[4b000] via P2P/IPC/read ts-cbba87c5e7504a249f5127103d9ce40f-worker-0:18319:30613 [2] NCCL INFO Connected all trees ts-cbba87c5e7504a249f5127103d9ce40f-worker-0:18319:30613 [2] NCCL INFO threadThresholds 8/8/64 | 256/8/64 | 512 | 512 ts-cbba87c5e7504a249f5127103d9ce40f-worker-0:18319:30613 [2] NCCL INFO 2 coll channels, 2 p2p channels, 2 p2p channels per peer ts-cbba87c5e7504a249f5127103d9ce40f-worker-0:18321:30616 [4] NCCL INFO Connected all trees ts-cbba87c5e7504a249f5127103d9ce40f-worker-0:18321:30616 [4] NCCL INFO threadThresholds 8/8/64 | 256/8/64 | 512 | 512 ts-cbba87c5e7504a249f5127103d9ce40f-worker-0:18321:30616 [4] NCCL INFO 2 coll channels, 2 p2p channels, 2 p2p channels per peer ts-cbba87c5e7504a249f5127103d9ce40f-worker-0:18320:30618 [3] NCCL INFO Connected all trees ts-cbba87c5e7504a249f5127103d9ce40f-worker-0:18320:30618 [3] NCCL INFO threadThresholds 8/8/64 | 256/8/64 | 512 | 512 ts-cbba87c5e7504a249f5127103d9ce40f-worker-0:18320:30618 [3] NCCL INFO 2 coll channels, 2 p2p channels, 2 p2p channels per peer ts-cbba87c5e7504a249f5127103d9ce40f-worker-0:18324:30617 [7] NCCL INFO comm 0x7f01e002a910 rank 15 nranks 32 cudaDev 7 busId d0000 - Init COMPLETE ts-cbba87c5e7504a249f5127103d9ce40f-worker-0:18322:30611 [5] NCCL INFO comm 0x7f552402a980 rank 13 nranks 32 cudaDev 5 busId 99000 - Init COMPLETE ts-cbba87c5e7504a249f5127103d9ce40f-worker-0:18323:30614 [6] NCCL INFO comm 0x7fa11002a750 rank 14 nranks 32 cudaDev 6 busId cb000 - Init COMPLETE ts-cbba87c5e7504a249f5127103d9ce40f-worker-0:18321:30616 [4] NCCL INFO comm 0x7fa66002a670 rank 12 nranks 32 cudaDev 4 busId 93000 - Init COMPLETE ts-cbba87c5e7504a249f5127103d9ce40f-worker-0:18319:30613 [2] NCCL INFO comm 0x7fc71802a800 rank 10 nranks 32 cudaDev 2 busId 4b000 - Init COMPLETE ts-cbba87c5e7504a249f5127103d9ce40f-worker-0:18318:30612 [1] NCCL INFO comm 0x7f995402a8b0 rank 9 nranks 32 cudaDev 1 busId 13000 - Init COMPLETE ts-cbba87c5e7504a249f5127103d9ce40f-worker-0:18320:30618 [3] NCCL INFO comm 0x7f4ac402a470 rank 11 nranks 32 cudaDev 3 busId 51000 - Init COMPLETE ts-cbba87c5e7504a249f5127103d9ce40f-worker-0:18317:30615 [0] NCCL INFO comm 0x7fb02402aa10 rank 8 nranks 32 cudaDev 0 busId e000 - Init COMPLETE a87c5e7504a249f5127103d9ce40f-launcher:55268:69552 [7] NCCL INFO comm 0x7f846402ab30 rank 7 nranks 32 cudaDev 7 busId d0000 - Init COMPLETE ts-cbba87c5e7504a249f5127103d9ce40f-launcher:55265:69556 [4] NCCL INFO comm 0x7f3d8c02a970 rank 4 nranks 32 cudaDev 4 busId 93000 - Init COMPLETE ts-cbba87c5e7504a249f5127103d9ce40f-launcher:55266:69553 [5] NCCL INFO comm 0x7fb32002a8d0 rank 5 nranks 32 cudaDev 5 busId 99000 - Init COMPLETE ts-cbba87c5e7504a249f5127103d9ce40f-launcher:55267:69555 [6] NCCL INFO comm 0x7f467002a8b0 rank 6 nranks 32 cudaDev 6 busId cb000 - Init COMPLETE ts-cbba87c5e7504a249f5127103d9ce40f-launcher:55264:69558 [3] NCCL INFO comm 0x7f34c802a830 rank 3 nranks 32 cudaDev 3 busId 51000 - Init COMPLETE ts-cbba87c5e7504a249f5127103d9ce40f-launcher:55261:69551 [0] NCCL INFO comm 0x7f876402abc0 rank 0 nranks 32 cudaDev 0 busId e000 - Init COMPLETE ts-cbba87c5e7504a249f5127103d9ce40f-launcher:55262:69554 [1] NCCL INFO comm 0x7fdb5c02a4b0 rank 1 nranks 32 cudaDev 1 busId 13000 - Init COMPLETE 0%| | 1/1817 [01:05<32:56:56, 65.32s/it] {'loss': 3.8636, 'learning_rate': 1.0000000000000002e-06, 'epoch': 0.0} 0%| | 1/1817 [01:05<32:56:56, 65.32s/it] 0%| | 2/1817 [02:05<31:27:33, 62.40s/it] {'loss': 3.7703, 'learning_rate': 2.0000000000000003e-06, 'epoch': 0.0} 0%| | 2/1817 [02:05<31:27:33, 62.40s/it] 0%| | 3/1817 [03:02<30:06:18, 59.75s/it] {'loss': 3.8113, 'learning_rate': 3e-06, 'epoch': 0.0} 0%| | 3/1817 [03:02<30:06:18, 59.75s/it] 0%| | 4/1817 [03:58<29:25:11, 58.42s/it] {'loss': 3.8136, 'learning_rate': 4.000000000000001e-06, 'epoch': 0.0} 0%| | 4/1817 [03:58<29:24:00, 58.38s/it] 0%| | 5/1817 [04:57<29:30:29, 58.63s/it] {'loss': 3.7263, 'learning_rate': 5e-06, 'epoch': 0.0} 0%| | 5/1817 [04:57<29:30:29, 58.63s/it] 0%| | 6/1817 [05:53<29:04:44, 57.81s/it] {'loss': 3.5733, 'learning_rate': 6e-06, 'epoch': 0.0} 0%| | 6/1817 [05:53<29:04:44, 57.81s/it] 0%| | 7/1817 [06:50<28:49:34, 57.33s/it] {'loss': 3.4134, 'learning_rate': 7e-06, 'epoch': 0.0} 0%| | 7/1817 [06:50<28:49:06, 57.32s/it] 0%| | 8/1817 [07:46<28:36:03, 56.92s/it] {'loss': 3.2471, 'learning_rate': 8.000000000000001e-06, 'epoch': 0.0} 0%| | 8/1817 [07:46<28:35:57, 56.91s/it] 0%| | 9/1817 [08:43<28:40:10, 57.09s/it] {'loss': 3.1673, 'learning_rate': 9e-06, 'epoch': 0.0} 0%| | 9/1817 [08:43<28:40:15, 57.09s/it] 1%| | 10/1817 [09:40<28:33:01, 56.88s/it] {'loss': 2.9928, 'learning_rate': 1e-05, 'epoch': 0.01} 1%| | 10/1817 [09:40<28:33:05, 56.88s/it] 1%| | 11/1817 [10:36<28:29:11, 56.78s/it] {'loss': 2.8702, 'learning_rate': 1.1000000000000001e-05, 'epoch': 0.01} 1%| | 11/1817 [10:36<28:29:05, 56.78s/it] 1%| | 12/1817 [11:35<28:40:43, 57.20s/it] {'loss': 2.8749, 'learning_rate': 1.2e-05, 'epoch': 0.01} 1%| | 12/1817 [11:34<28:40:40, 57.20s/it] 1%| | 13/1817 [12:31<28:29:34, 56.86s/it] {'loss': 2.747, 'learning_rate': 1.3000000000000001e-05, 'epoch': 0.01} 1%| | 13/1817 [12:30<28:29:37, 56.86s/it] 1%| | 14/1817 [13:28<28:36:00, 57.11s/it] {'loss': 2.679, 'learning_rate': 1.4e-05, 'epoch': 0.01} 1%| | 14/1817 [13:28<28:36:00, 57.11s/it] 1%| | 15/1817 [14:26<28:44:10, 57.41s/it] {'loss': 2.5937, 'learning_rate': 1.5000000000000002e-05, 'epoch': 0.01} 1%| | 15/1817 [14:26<28:44:12, 57.41s/it] 1%| | 16/1817 [15:24<28:47:42, 57.56s/it] {'loss': 2.5901, 'learning_rate': 1.6000000000000003e-05, 'epoch': 0.01} 1%| | 16/1817 [15:24<28:47:44, 57.56s/it] 1%| | 17/1817 [16:21<28:39:17, 57.31s/it] {'loss': 2.5661, 'learning_rate': 1.7e-05, 'epoch': 0.01} 1%| | 17/1817 [16:21<28:39:19, 57.31s/it] 1%| | 18/1817 [17:19<28:41:16, 57.41s/it] {'loss': 2.5105, 'learning_rate': 1.8e-05, 'epoch': 0.01} 1%| | 18/1817 [17:18<28:41:14, 57.41s/it] 1%| | 19/1817 [18:15<28:28:47, 57.02s/it] {'loss': 2.4969, 'learning_rate': 1.9e-05, 'epoch': 0.01} 1%| | 19/1817 [18:15<28:28:48, 57.02s/it] 1%| | 20/1817 [19:11<28:19:43, 56.75s/it] {'loss': 2.4041, 'learning_rate': 2e-05, 'epoch': 0.01} 1%| | 20/1817 [19:11<28:19:45, 56.75s/it] 1%| | 21/1817 [20:09<28:33:06, 57.23s/it] {'loss': 2.396, 'learning_rate': 2e-05, 'epoch': 0.01} 1%| | 21/1817 [20:09<28:33:08, 57.23s/it] 1%| | 22/1817 [21:06<28:22:38, 56.91s/it] {'loss': 2.3917, 'learning_rate': 2e-05, 'epoch': 0.01} 1%| | 22/1817 [21:05<28:22:40, 56.91s/it] 1%|▏ | 23/1817 [22:02<28:16:05, 56.73s/it] {'loss': 2.386, 'learning_rate': 2e-05, 'epoch': 0.01} 1%|▏ | 23/1817 [22:01<28:16:06, 56.73s/it] 1%|▏ | 24/1817 [22:58<28:07:49, 56.48s/it] {'loss': 2.3783, 'learning_rate': 2e-05, 'epoch': 0.01} 1%|▏ | 24/1817 [22:57<28:07:51, 56.48s/it] 1%|▏ | 25/1817 [23:58<28:41:46, 57.65s/it] {'loss': 2.3728, 'learning_rate': 2e-05, 'epoch': 0.01} 1%|▏ | 25/1817 [23:58<28:41:48, 57.65s/it] 1%|▏ | 26/1817 [24:54<28:27:59, 57.22s/it] {'loss': 2.3195, 'learning_rate': 2e-05, 'epoch': 0.01} 1%|▏ | 26/1817 [24:54<28:28:00, 57.22s/it] 1%|▏ | 27/1817 [25:50<28:17:41, 56.91s/it] {'loss': 2.2962, 'learning_rate': 2e-05, 'epoch': 0.01} 1%|▏ | 27/1817 [25:50<28:17:43, 56.91s/it] 2%|▏ | 28/1817 [26:50<28:40:02, 57.69s/it] {'loss': 2.2905, 'learning_rate': 2e-05, 'epoch': 0.02} 2%|▏ | 28/1817 [26:50<28:40:04, 57.69s/it] 2%|▏ | 29/1817 [27:46<28:24:22, 57.19s/it] {'loss': 2.2932, 'learning_rate': 2e-05, 'epoch': 0.02} 2%|▏ | 29/1817 [27:46<28:24:23, 57.19s/it] 2%|▏ | 30/1817 [28:47<28:53:18, 58.20s/it] {'loss': 2.2593, 'learning_rate': 2e-05, 'epoch': 0.02} 2%|▏ | 30/1817 [28:46<28:53:19, 58.20s/it] 2%|▏ | 31/1817 [29:42<28:31:35, 57.50s/it] {'loss': 2.2366, 'learning_rate': 2e-05, 'epoch': 0.02} 2%|▏ | 31/1817 [29:42<28:31:36, 57.50s/it] 2%|▏ | 32/1817 [30:38<28:11:50, 56.87s/it] {'loss': 2.28, 'learning_rate': 2e-05, 'epoch': 0.02} 2%|▏ | 32/1817 [30:38<28:11:52, 56.87s/it] 2%|▏ | 33/1817 [31:34<28:07:00, 56.74s/it] {'loss': 2.2601, 'learning_rate': 2e-05, 'epoch': 0.02} 2%|▏ | 33/1817 [31:34<28:07:02, 56.74s/it] 2%|▏ | 34/1817 [32:31<28:01:42, 56.59s/it] {'loss': 2.2557, 'learning_rate': 2e-05, 'epoch': 0.02} 2%|▏ | 34/1817 [32:30<28:01:44, 56.59s/it] 2%|▏ | 35/1817 [33:27<27:58:00, 56.50s/it] {'loss': 2.2057, 'learning_rate': 2e-05, 'epoch': 0.02} 2%|▏ | 35/1817 [33:27<27:58:02, 56.50s/it] 2%|▏ | 36/1817 [34:26<28:21:45, 57.33s/it] {'loss': 2.2578, 'learning_rate': 2e-05, 'epoch': 0.02} 2%|▏ | 36/1817 [34:26<28:21:47, 57.33s/it] 2%|▏ | 37/1817 [35:24<28:26:22, 57.52s/it] {'loss': 2.2087, 'learning_rate': 2e-05, 'epoch': 0.02} 2%|▏ | 37/1817 [35:24<28:26:24, 57.52s/it] 2%|▏ | 38/1817 [36:20<28:13:14, 57.11s/it] {'loss': 2.1957, 'learning_rate': 2e-05, 'epoch': 0.02} 2%|▏ | 38/1817 [36:20<28:13:15, 57.11s/it] 2%|▏ | 39/1817 [37:16<28:02:49, 56.79s/it] {'loss': 2.1652, 'learning_rate': 2e-05, 'epoch': 0.02} 2%|▏ | 39/1817 [37:16<28:02:51, 56.79s/it] 2%|▏ | 40/1817 [38:13<27:57:34, 56.64s/it] {'loss': 2.1536, 'learning_rate': 2e-05, 'epoch': 0.02} 2%|▏ | 40/1817 [38:12<27:57:36, 56.64s/it] 2%|▏ | 41/1817 [39:10<28:06:13, 56.97s/it] {'loss': 2.1599, 'learning_rate': 2e-05, 'epoch': 0.02} 2%|▏ | 41/1817 [39:10<28:06:15, 56.97s/it] 2%|▏ | 42/1817 [40:08<28:15:55, 57.33s/it] {'loss': 2.1407, 'learning_rate': 2e-05, 'epoch': 0.02} 2%|▏ | 42/1817 [40:08<28:15:55, 57.33s/it] 2%|▏ | 43/1817 [41:05<28:05:09, 57.00s/it] {'loss': 2.1391, 'learning_rate': 2e-05, 'epoch': 0.02} 2%|▏ | 43/1817 [41:04<28:05:11, 57.00s/it] 2%|▏ | 44/1817 [42:05<28:33:52, 58.00s/it] {'loss': 2.1557, 'learning_rate': 2e-05, 'epoch': 0.02} 2%|▏ | 44/1817 [42:05<28:33:54, 58.00s/it] 2%|▏ | 45/1817 [43:03<28:33:49, 58.03s/it] {'loss': 2.1357, 'learning_rate': 2e-05, 'epoch': 0.02} 2%|▏ | 45/1817 [43:03<28:33:51, 58.03s/it] 3%|▎ | 46/1817 [44:01<28:37:01, 58.17s/it] {'loss': 2.1604, 'learning_rate': 2e-05, 'epoch': 0.03} 3%|▎ | 46/1817 [44:01<28:37:01, 58.17s/it] 3%|▎ | 47/1817 [44:58<28:19:36, 57.61s/it] {'loss': 2.1395, 'learning_rate': 2e-05, 'epoch': 0.03} 3%|▎ | 47/1817 [44:58<28:19:38, 57.61s/it] 3%|▎ | 48/1817 [45:55<28:17:40, 57.58s/it] {'loss': 2.1455, 'learning_rate': 2e-05, 'epoch': 0.03} 3%|▎ | 48/1817 [45:55<28:17:42, 57.58s/it] 3%|▎ | 49/1817 [46:52<28:04:09, 57.15s/it] {'loss': 2.1665, 'learning_rate': 2e-05, 'epoch': 0.03} 3%|▎ | 49/1817 [46:51<28:04:11, 57.16s/it] 3%|▎ | 50/1817 [47:51<28:20:09, 57.73s/it] {'loss': 2.1306, 'learning_rate': 2e-05, 'epoch': 0.03} 3%|▎ | 50/1817 [47:50<28:20:11, 57.73s/it] 3%|▎ | 51/1817 [48:48<28:16:33, 57.64s/it] {'loss': 2.1156, 'learning_rate': 2e-05, 'epoch': 0.03} 3%|▎ | 51/1817 [48:48<28:16:34, 57.64s/it] 3%|▎ | 52/1817 [49:44<28:04:18, 57.26s/it] {'loss': 2.0731, 'learning_rate': 2e-05, 'epoch': 0.03} 3%|▎ | 52/1817 [49:44<28:04:20, 57.26s/it] 3%|▎ | 53/1817 [50:40<27:47:56, 56.73s/it] {'loss': 2.0844, 'learning_rate': 2e-05, 'epoch': 0.03} 3%|▎ | 53/1817 [50:40<27:47:56, 56.73s/it] 3%|▎ | 54/1817 [51:36<27:42:44, 56.59s/it] {'loss': 2.1258, 'learning_rate': 2e-05, 'epoch': 0.03} 3%|▎ | 54/1817 [51:36<27:42:45, 56.59s/it] 3%|▎ | 55/1817 [52:33<27:43:21, 56.64s/it] {'loss': 2.0568, 'learning_rate': 2e-05, 'epoch': 0.03} 3%|▎ | 55/1817 [52:33<27:43:21, 56.64s/it] 3%|▎ | 56/1817 [53:29<27:38:59, 56.52s/it] {'loss': 2.0904, 'learning_rate': 2e-05, 'epoch': 0.03} 3%|▎ | 56/1817 [53:29<27:39:01, 56.53s/it] 3%|▎ | 57/1817 [54:26<27:37:10, 56.49s/it] {'loss': 2.0861, 'learning_rate': 2e-05, 'epoch': 0.03} 3%|▎ | 57/1817 [54:25<27:37:12, 56.50s/it] 3%|▎ | 58/1817 [55:25<28:01:26, 57.35s/it] {'loss': 2.107, 'learning_rate': 2e-05, 'epoch': 0.03} 3%|▎ | 58/1817 [55:25<28:01:26, 57.35s/it] 3%|▎ | 59/1817 [56:23<28:02:12, 57.41s/it] {'loss': 2.0821, 'learning_rate': 2e-05, 'epoch': 0.03} 3%|▎ | 59/1817 [56:22<28:02:14, 57.41s/it] 3%|▎ | 60/1817 [57:19<27:52:06, 57.10s/it] {'loss': 2.0627, 'learning_rate': 2e-05, 'epoch': 0.03} 3%|▎ | 60/1817 [57:19<27:52:08, 57.10s/it] 3%|▎ | 61/1817 [58:15<27:44:02, 56.86s/it] {'loss': 2.0347, 'learning_rate': 2e-05, 'epoch': 0.03} 3%|▎ | 61/1817 [58:15<27:44:04, 56.86s/it] 3%|▎ | 62/1817 [59:12<27:39:01, 56.72s/it] {'loss': 2.0778, 'learning_rate': 2e-05, 'epoch': 0.03} 3%|▎ | 62/1817 [59:11<27:39:03, 56.72s/it] 3%|▎ | 63/1817 [1:00:07<27:30:08, 56.45s/it] {'loss': 2.0441, 'learning_rate': 2e-05, 'epoch': 0.03} 3%|▎ | 63/1817 [1:00:07<27:29:59, 56.44s/it] 4%|▎ | 64/1817 [1:01:06<27:44:28, 56.97s/it] {'loss': 2.0633, 'learning_rate': 2e-05, 'epoch': 0.04} 4%|▎ | 64/1817 [1:01:05<27:44:06, 56.96s/it] 4%|▎ | 65/1817 [1:02:02<27:37:56, 56.78s/it] {'loss': 2.0964, 'learning_rate': 2e-05, 'epoch': 0.04} 4%|▎ | 65/1817 [1:02:02<27:37:27, 56.76s/it] 4%|▎ | 66/1817 [1:02:58<27:32:36, 56.63s/it] {'loss': 2.0493, 'learning_rate': 2e-05, 'epoch': 0.04} 4%|▎ | 66/1817 [1:02:58<27:32:36, 56.63s/it] 4%|▎ | 67/1817 [1:03:55<27:30:47, 56.60s/it] {'loss': 2.0396, 'learning_rate': 2e-05, 'epoch': 0.04} 4%|▎ | 67/1817 [1:03:54<27:30:27, 56.59s/it] 4%|▎ | 68/1817 [1:04:50<27:24:13, 56.41s/it] {'loss': 2.0578, 'learning_rate': 2e-05, 'epoch': 0.04} 4%|▎ | 68/1817 [1:04:50<27:24:13, 56.41s/it] 4%|▍ | 69/1817 [1:05:47<27:21:20, 56.34s/it] {'loss': 2.0523, 'learning_rate': 2e-05, 'epoch': 0.04} 4%|▍ | 69/1817 [1:05:47<27:21:20, 56.34s/it] 4%|▍ | 70/1817 [1:06:45<27:32:57, 56.77s/it] {'loss': 2.0805, 'learning_rate': 2e-05, 'epoch': 0.04} 4%|▍ | 70/1817 [1:06:44<27:32:57, 56.77s/it] 4%|▍ | 71/1817 [1:07:41<27:28:22, 56.65s/it] {'loss': 2.0711, 'learning_rate': 2e-05, 'epoch': 0.04} 4%|▍ | 71/1817 [1:07:41<27:28:22, 56.65s/it] 4%|▍ | 72/1817 [1:08:37<27:23:34, 56.51s/it] {'loss': 2.0318, 'learning_rate': 2e-05, 'epoch': 0.04} 4%|▍ | 72/1817 [1:08:37<27:23:34, 56.51s/it] 4%|▍ | 73/1817 [1:09:35<27:33:11, 56.88s/it] {'loss': 2.063, 'learning_rate': 2e-05, 'epoch': 0.04} 4%|▍ | 73/1817 [1:09:35<27:33:11, 56.88s/it] 4%|▍ | 74/1817 [1:10:35<28:00:14, 57.84s/it] {'loss': 2.07, 'learning_rate': 2e-05, 'epoch': 0.04} 4%|▍ | 74/1817 [1:10:35<28:00:14, 57.84s/it] 4%|▍ | 75/1817 [1:11:31<27:46:44, 57.41s/it] {'loss': 2.0351, 'learning_rate': 2e-05, 'epoch': 0.04} 4%|▍ | 75/1817 [1:11:31<27:46:44, 57.41s/it] 4%|▍ | 76/1817 [1:12:26<27:27:39, 56.78s/it] {'loss': 2.048, 'learning_rate': 2e-05, 'epoch': 0.04} 4%|▍ | 76/1817 [1:12:26<27:27:39, 56.78s/it] 4%|▍ | 77/1817 [1:13:22<27:12:06, 56.28s/it] {'loss': 2.0764, 'learning_rate': 2e-05, 'epoch': 0.04} 4%|▍ | 77/1817 [1:13:22<27:12:06, 56.28s/it] 4%|▍ | 78/1817 [1:14:18<27:10:46, 56.27s/it] {'loss': 2.0467, 'learning_rate': 2e-05, 'epoch': 0.04} 4%|▍ | 78/1817 [1:14:18<27:10:46, 56.27s/it] 4%|▍ | 79/1817 [1:15:16<27:29:43, 56.95s/it] {'loss': 2.0354, 'learning_rate': 2e-05, 'epoch': 0.04} 4%|▍ | 79/1817 [1:15:16<27:29:43, 56.95s/it] 4%|▍ | 80/1817 [1:16:13<27:22:41, 56.74s/it] {'loss': 2.062, 'learning_rate': 2e-05, 'epoch': 0.04} 4%|▍ | 80/1817 [1:16:13<27:22:41, 56.74s/it] 4%|▍ | 81/1817 [1:17:09<27:15:23, 56.52s/it] {'loss': 2.0688, 'learning_rate': 2e-05, 'epoch': 0.04} 4%|▍ | 81/1817 [1:17:09<27:15:23, 56.52s/it] 5%|▍ | 82/1817 [1:18:07<27:27:28, 56.97s/it] {'loss': 2.0161, 'learning_rate': 2e-05, 'epoch': 0.05} 5%|▍ | 82/1817 [1:18:07<27:27:28, 56.97s/it] 5%|▍ | 83/1817 [1:19:03<27:20:08, 56.75s/it] {'loss': 2.028, 'learning_rate': 2e-05, 'epoch': 0.05} 5%|▍ | 83/1817 [1:19:03<27:20:08, 56.75s/it] 5%|▍ | 84/1817 [1:20:01<27:28:18, 57.07s/it] {'loss': 2.0174, 'learning_rate': 2e-05, 'epoch': 0.05} 5%|▍ | 84/1817 [1:20:01<27:28:18, 57.07s/it] 5%|▍ | 85/1817 [1:20:57<27:20:45, 56.84s/it] {'loss': 2.0403, 'learning_rate': 2e-05, 'epoch': 0.05} 5%|▍ | 85/1817 [1:20:57<27:20:45, 56.84s/it] 5%|▍ | 86/1817 [1:21:55<27:32:47, 57.29s/it] {'loss': 2.0512, 'learning_rate': 2e-05, 'epoch': 0.05} 5%|▍ | 86/1817 [1:21:55<27:32:47, 57.29s/it] 5%|▍ | 87/1817 [1:22:52<27:23:28, 57.00s/it] {'loss': 2.0794, 'learning_rate': 2e-05, 'epoch': 0.05} 5%|▍ | 87/1817 [1:22:52<27:23:28, 57.00s/it] 5%|▍ | 88/1817 [1:23:48<27:16:01, 56.77s/it] {'loss': 2.0693, 'learning_rate': 2e-05, 'epoch': 0.05} 5%|▍ | 88/1817 [1:23:48<27:16:01, 56.77s/it] 5%|▍ | 89/1817 [1:24:44<27:10:36, 56.62s/it] {'loss': 2.0453, 'learning_rate': 2e-05, 'epoch': 0.05} 5%|▍ | 89/1817 [1:24:44<27:10:36, 56.62s/it] 5%|▍ | 90/1817 [1:25:40<27:03:24, 56.40s/it] {'loss': 2.0052, 'learning_rate': 2e-05, 'epoch': 0.05} 5%|▍ | 90/1817 [1:25:40<27:03:24, 56.40s/it] 5%|▌ | 91/1817 [1:26:36<27:00:37, 56.34s/it] {'loss': 2.0306, 'learning_rate': 2e-05, 'epoch': 0.05} 5%|▌ | 91/1817 [1:26:36<27:00:37, 56.34s/it] 5%|▌ | 92/1817 [1:27:34<27:08:40, 56.65s/it] {'loss': 2.0471, 'learning_rate': 2e-05, 'epoch': 0.05} 5%|▌ | 92/1817 [1:27:34<27:08:40, 56.65s/it] 5%|▌ | 93/1817 [1:28:31<27:15:19, 56.91s/it] {'loss': 1.9901, 'learning_rate': 2e-05, 'epoch': 0.05} 5%|▌ | 93/1817 [1:28:31<27:15:19, 56.91s/it] 5%|▌ | 94/1817 [1:29:27<27:08:26, 56.71s/it] {'loss': 1.9872, 'learning_rate': 2e-05, 'epoch': 0.05} 5%|▌ | 94/1817 [1:29:27<27:08:26, 56.71s/it] 5%|▌ | 95/1817 [1:30:23<27:02:39, 56.54s/it] {'loss': 2.0345, 'learning_rate': 2e-05, 'epoch': 0.05} 5%|▌ | 95/1817 [1:30:23<27:02:39, 56.54s/it] 5%|▌ | 96/1817 [1:31:22<27:21:51, 57.24s/it] {'loss': 2.0245, 'learning_rate': 2e-05, 'epoch': 0.05} 5%|▌ | 96/1817 [1:31:22<27:21:51, 57.24s/it] 5%|▌ | 97/1817 [1:32:19<27:15:02, 57.04s/it] {'loss': 2.0409, 'learning_rate': 2e-05, 'epoch': 0.05} 5%|▌ | 97/1817 [1:32:19<27:15:02, 57.04s/it] 5%|▌ | 98/1817 [1:33:15<27:07:35, 56.81s/it] {'loss': 2.03, 'learning_rate': 2e-05, 'epoch': 0.05} 5%|▌ | 98/1817 [1:33:15<27:07:35, 56.81s/it] 5%|▌ | 99/1817 [1:34:11<27:02:10, 56.65s/it] {'loss': 2.0382, 'learning_rate': 2e-05, 'epoch': 0.05} 5%|▌ | 99/1817 [1:34:11<27:02:10, 56.65s/it] 6%|▌ | 100/1817 [1:35:08<26:59:27, 56.59s/it] {'loss': 2.0107, 'learning_rate': 2e-05, 'epoch': 0.06} 6%|▌ | 100/1817 [1:35:08<26:59:27, 56.59s/it] 6%|▌ | 101/1817 [1:36:08<27:24:14, 57.49s/it] {'loss': 2.0322, 'learning_rate': 2e-05, 'epoch': 0.06} 6%|▌ | 101/1817 [1:36:08<27:24:14, 57.49s/it] 6%|▌ | 102/1817 [1:37:04<27:12:02, 57.10s/it] {'loss': 2.0571, 'learning_rate': 2e-05, 'epoch': 0.06} 6%|▌ | 102/1817 [1:37:04<27:12:02, 57.10s/it] 6%|▌ | 103/1817 [1:38:03<27:31:12, 57.80s/it] {'loss': 2.0138, 'learning_rate': 2e-05, 'epoch': 0.06} 6%|▌ | 103/1817 [1:38:03<27:31:12, 57.80s/it] 6%|▌ | 104/1817 [1:39:04<27:56:55, 58.74s/it] {'loss': 1.9975, 'learning_rate': 2e-05, 'epoch': 0.06} 6%|▌ | 104/1817 [1:39:04<27:56:55, 58.74s/it] 6%|▌ | 105/1817 [1:40:01<27:44:39, 58.34s/it] {'loss': 1.9912, 'learning_rate': 2e-05, 'epoch': 0.06} 6%|▌ | 105/1817 [1:40:01<27:44:39, 58.34s/it] 6%|▌ | 106/1817 [1:40:58<27:25:07, 57.69s/it] {'loss': 2.0275, 'learning_rate': 2e-05, 'epoch': 0.06} 6%|▌ | 106/1817 [1:40:58<27:25:07, 57.69s/it] 6%|▌ | 107/1817 [1:41:58<27:50:09, 58.60s/it] {'loss': 2.0311, 'learning_rate': 2e-05, 'epoch': 0.06} 6%|▌ | 107/1817 [1:41:58<27:50:09, 58.60s/it] 6%|▌ | 108/1817 [1:42:55<27:28:51, 57.89s/it] {'loss': 2.0406, 'learning_rate': 2e-05, 'epoch': 0.06} 6%|▌ | 108/1817 [1:42:55<27:28:51, 57.89s/it] 6%|▌ | 109/1817 [1:43:51<27:13:27, 57.38s/it] {'loss': 2.0054, 'learning_rate': 2e-05, 'epoch': 0.06} 6%|▌ | 109/1817 [1:43:51<27:13:27, 57.38s/it] 6%|▌ | 110/1817 [1:44:47<27:03:44, 57.07s/it] {'loss': 2.0398, 'learning_rate': 2e-05, 'epoch': 0.06} 6%|▌ | 110/1817 [1:44:47<27:03:44, 57.07s/it] 6%|▌ | 111/1817 [1:45:45<27:06:29, 57.20s/it] {'loss': 2.0129, 'learning_rate': 2e-05, 'epoch': 0.06} 6%|▌ | 111/1817 [1:45:45<27:06:29, 57.20s/it] 6%|▌ | 112/1817 [1:46:41<26:57:40, 56.93s/it] {'loss': 2.0557, 'learning_rate': 2e-05, 'epoch': 0.06} 6%|▌ | 112/1817 [1:46:41<26:57:40, 56.93s/it] 6%|▌ | 113/1817 [1:47:38<26:53:29, 56.81s/it] {'loss': 2.0321, 'learning_rate': 2e-05, 'epoch': 0.06} 6%|▌ | 113/1817 [1:47:37<26:53:29, 56.81s/it] 6%|▋ | 114/1817 [1:48:36<27:04:28, 57.23s/it] {'loss': 2.0349, 'learning_rate': 2e-05, 'epoch': 0.06} 6%|▋ | 114/1817 [1:48:36<27:04:28, 57.23s/it] 6%|▋ | 115/1817 [1:49:34<27:12:25, 57.55s/it] {'loss': 2.0087, 'learning_rate': 2e-05, 'epoch': 0.06} 6%|▋ | 115/1817 [1:49:34<27:12:25, 57.55s/it] 6%|▋ | 116/1817 [1:50:32<27:17:52, 57.77s/it] {'loss': 2.0371, 'learning_rate': 2e-05, 'epoch': 0.06} 6%|▋ | 116/1817 [1:50:32<27:17:52, 57.77s/it] 6%|▋ | 117/1817 [1:51:29<27:04:01, 57.32s/it] {'loss': 1.9876, 'learning_rate': 2e-05, 'epoch': 0.06} 6%|▋ | 117/1817 [1:51:29<27:04:01, 57.32s/it] 6%|▋ | 118/1817 [1:52:27<27:10:22, 57.58s/it] {'loss': 2.0365, 'learning_rate': 2e-05, 'epoch': 0.06} 6%|▋ | 118/1817 [1:52:27<27:10:22, 57.58s/it] 7%|▋ | 119/1817 [1:53:23<26:57:59, 57.17s/it] {'loss': 2.0673, 'learning_rate': 2e-05, 'epoch': 0.07} 7%|▋ | 119/1817 [1:53:23<26:57:59, 57.17s/it] 7%|▋ | 120/1817 [1:54:19<26:48:30, 56.87s/it] {'loss': 2.066, 'learning_rate': 2e-05, 'epoch': 0.07} 7%|▋ | 120/1817 [1:54:19<26:48:30, 56.87s/it] 7%|▋ | 121/1817 [1:55:17<26:56:14, 57.18s/it] {'loss': 1.983, 'learning_rate': 2e-05, 'epoch': 0.07} 7%|▋ | 121/1817 [1:55:17<26:56:14, 57.18s/it] 7%|▋ | 122/1817 [1:56:13<26:47:04, 56.89s/it] {'loss': 1.9835, 'learning_rate': 2e-05, 'epoch': 0.07} 7%|▋ | 122/1817 [1:56:13<26:47:04, 56.89s/it] 7%|▋ | 123/1817 [1:57:09<26:39:55, 56.67s/it] {'loss': 2.0407, 'learning_rate': 2e-05, 'epoch': 0.07} 7%|▋ | 123/1817 [1:57:09<26:39:55, 56.67s/it] 7%|▋ | 124/1817 [1:58:05<26:34:12, 56.50s/it] {'loss': 2.0092, 'learning_rate': 2e-05, 'epoch': 0.07} 7%|▋ | 124/1817 [1:58:05<26:34:12, 56.50s/it] 7%|▋ | 125/1817 [1:59:02<26:30:12, 56.39s/it] {'loss': 2.0376, 'learning_rate': 2e-05, 'epoch': 0.07} 7%|▋ | 125/1817 [1:59:02<26:30:12, 56.39s/it] 7%|▋ | 126/1817 [2:00:00<26:46:51, 57.01s/it] {'loss': 2.0607, 'learning_rate': 2e-05, 'epoch': 0.07} 7%|▋ | 126/1817 [2:00:00<26:46:44, 57.01s/it] 7%|▋ | 127/1817 [2:00:57<26:40:02, 56.81s/it] {'loss': 2.0141, 'learning_rate': 2e-05, 'epoch': 0.07} 7%|▋ | 127/1817 [2:00:56<26:39:43, 56.80s/it] 7%|▋ | 128/1817 [2:01:52<26:23:10, 56.24s/it] {'loss': 2.0374, 'learning_rate': 2e-05, 'epoch': 0.07} 7%|▋ | 128/1817 [2:01:51<26:22:43, 56.22s/it] 7%|▋ | 129/1817 [2:02:48<26:24:09, 56.31s/it] {'loss': 2.0031, 'learning_rate': 2e-05, 'epoch': 0.07} 7%|▋ | 129/1817 [2:02:48<26:23:38, 56.29s/it] 7%|▋ | 130/1817 [2:03:44<26:23:30, 56.32s/it] {'loss': 1.9915, 'learning_rate': 2e-05, 'epoch': 0.07} 7%|▋ | 130/1817 [2:03:44<26:23:09, 56.31s/it] 7%|▋ | 131/1817 [2:04:43<26:41:46, 57.00s/it] {'loss': 2.0102, 'learning_rate': 2e-05, 'epoch': 0.07} 7%|▋ | 131/1817 [2:04:43<26:41:32, 56.99s/it] 7%|▋ | 132/1817 [2:05:40<26:41:29, 57.03s/it] {'loss': 1.9827, 'learning_rate': 2e-05, 'epoch': 0.07} 7%|▋ | 132/1817 [2:05:40<26:41:20, 57.02s/it] 7%|▋ | 133/1817 [2:06:38<26:49:04, 57.33s/it] {'loss': 2.0121, 'learning_rate': 2e-05, 'epoch': 0.07} 7%|▋ | 133/1817 [2:06:38<26:48:58, 57.33s/it] 7%|▋ | 134/1817 [2:07:38<27:07:15, 58.01s/it] {'loss': 2.0329, 'learning_rate': 2e-05, 'epoch': 0.07} 7%|▋ | 134/1817 [2:07:37<27:07:20, 58.02s/it] 7%|▋ | 135/1817 [2:08:35<27:02:16, 57.87s/it] {'loss': 2.0099, 'learning_rate': 2e-05, 'epoch': 0.07} 7%|▋ | 135/1817 [2:08:35<27:02:16, 57.87s/it] 7%|▋ | 136/1817 [2:09:32<26:48:02, 57.40s/it] {'loss': 2.0402, 'learning_rate': 2e-05, 'epoch': 0.07} 7%|▋ | 136/1817 [2:09:31<26:48:12, 57.40s/it] 8%|▊ | 137/1817 [2:10:27<26:34:16, 56.94s/it] {'loss': 2.0251, 'learning_rate': 2e-05, 'epoch': 0.08} 8%|▊ | 137/1817 [2:10:27<26:34:23, 56.94s/it] 8%|▊ | 138/1817 [2:11:24<26:27:48, 56.74s/it] {'loss': 1.9765, 'learning_rate': 2e-05, 'epoch': 0.08} 8%|▊ | 138/1817 [2:11:23<26:27:53, 56.74s/it] 8%|▊ | 139/1817 [2:12:20<26:24:05, 56.64s/it] {'loss': 1.9729, 'learning_rate': 2e-05, 'epoch': 0.08} 8%|▊ | 139/1817 [2:12:20<26:24:10, 56.64s/it] 8%|▊ | 140/1817 [2:13:16<26:20:50, 56.56s/it] {'loss': 1.9933, 'learning_rate': 2e-05, 'epoch': 0.08} 8%|▊ | 140/1817 [2:13:16<26:20:54, 56.56s/it] 8%|▊ | 141/1817 [2:14:15<26:42:39, 57.37s/it] {'loss': 1.9915, 'learning_rate': 2e-05, 'epoch': 0.08} 8%|▊ | 141/1817 [2:14:15<26:42:39, 57.37s/it] 8%|▊ | 142/1817 [2:15:14<26:50:49, 57.70s/it] {'loss': 2.0224, 'learning_rate': 2e-05, 'epoch': 0.08} 8%|▊ | 142/1817 [2:15:14<26:50:51, 57.70s/it] 8%|▊ | 143/1817 [2:16:11<26:40:28, 57.36s/it] {'loss': 2.0296, 'learning_rate': 2e-05, 'epoch': 0.08} 8%|▊ | 143/1817 [2:16:10<26:40:30, 57.37s/it] 8%|▊ | 144/1817 [2:17:07<26:32:22, 57.11s/it] {'loss': 1.9872, 'learning_rate': 2e-05, 'epoch': 0.08} 8%|▊ | 144/1817 [2:17:07<26:32:25, 57.11s/it] 8%|▊ | 145/1817 [2:18:04<26:26:04, 56.92s/it] {'loss': 1.9899, 'learning_rate': 2e-05, 'epoch': 0.08} 8%|▊ | 145/1817 [2:18:03<26:26:06, 56.92s/it] 8%|▊ | 146/1817 [2:19:00<26:20:35, 56.75s/it] {'loss': 2.0396, 'learning_rate': 2e-05, 'epoch': 0.08} 8%|▊ | 146/1817 [2:19:00<26:20:37, 56.76s/it] 8%|▊ | 147/1817 [2:19:59<26:37:12, 57.38s/it] {'loss': 1.9966, 'learning_rate': 2e-05, 'epoch': 0.08} 8%|▊ | 147/1817 [2:19:59<26:37:13, 57.39s/it] 8%|▊ | 148/1817 [2:20:55<26:27:46, 57.08s/it] {'loss': 2.0093, 'learning_rate': 2e-05, 'epoch': 0.08} 8%|▊ | 148/1817 [2:20:55<26:27:46, 57.08s/it] 8%|▊ | 149/1817 [2:21:54<26:37:47, 57.47s/it] {'loss': 1.97, 'learning_rate': 2e-05, 'epoch': 0.08} 8%|▊ | 149/1817 [2:21:53<26:37:49, 57.48s/it] 8%|▊ | 150/1817 [2:22:50<26:27:00, 57.12s/it] {'loss': 2.0071, 'learning_rate': 2e-05, 'epoch': 0.08} 8%|▊ | 150/1817 [2:22:50<26:27:02, 57.12s/it] 8%|▊ | 151/1817 [2:23:46<26:17:59, 56.83s/it] {'loss': 1.9676, 'learning_rate': 2e-05, 'epoch': 0.08} 8%|▊ | 151/1817 [2:23:46<26:18:00, 56.83s/it] 8%|▊ | 152/1817 [2:24:42<26:12:50, 56.68s/it] {'loss': 2.0816, 'learning_rate': 2e-05, 'epoch': 0.08} 8%|▊ | 152/1817 [2:24:42<26:12:50, 56.68s/it] 8%|▊ | 153/1817 [2:25:39<26:07:30, 56.52s/it] {'loss': 2.0419, 'learning_rate': 2e-05, 'epoch': 0.08} 8%|▊ | 153/1817 [2:25:38<26:07:32, 56.52s/it] 8%|▊ | 154/1817 [2:26:34<25:58:50, 56.24s/it] {'loss': 2.0079, 'learning_rate': 2e-05, 'epoch': 0.08} 8%|▊ | 154/1817 [2:26:34<25:58:52, 56.24s/it] 9%|▊ | 155/1817 [2:27:30<25:57:31, 56.23s/it] {'loss': 2.0014, 'learning_rate': 2e-05, 'epoch': 0.09} 9%|▊ | 155/1817 [2:27:30<25:57:32, 56.23s/it] 9%|▊ | 156/1817 [2:28:33<26:46:52, 58.04s/it] {'loss': 2.0343, 'learning_rate': 2e-05, 'epoch': 0.09} 9%|▊ | 156/1817 [2:28:32<26:46:53, 58.05s/it] 9%|▊ | 157/1817 [2:29:31<26:46:10, 58.05s/it] {'loss': 2.0198, 'learning_rate': 2e-05, 'epoch': 0.09} 9%|▊ | 157/1817 [2:29:31<26:46:11, 58.06s/it] 9%|▊ | 158/1817 [2:30:27<26:30:57, 57.54s/it] {'loss': 1.9926, 'learning_rate': 2e-05, 'epoch': 0.09} 9%|▊ | 158/1817 [2:30:27<26:30:57, 57.54s/it] 9%|▉ | 159/1817 [2:31:23<26:19:04, 57.14s/it] {'loss': 2.006, 'learning_rate': 2e-05, 'epoch': 0.09} 9%|▉ | 159/1817 [2:31:23<26:19:05, 57.14s/it] 9%|▉ | 160/1817 [2:32:21<26:21:01, 57.25s/it] {'loss': 1.9852, 'learning_rate': 2e-05, 'epoch': 0.09} 9%|▉ | 160/1817 [2:32:21<26:21:02, 57.25s/it] 9%|▉ | 161/1817 [2:33:17<26:15:00, 57.07s/it] {'loss': 2.0274, 'learning_rate': 2e-05, 'epoch': 0.09} 9%|▉ | 161/1817 [2:33:17<26:15:00, 57.07s/it] 9%|▉ | 162/1817 [2:34:14<26:08:41, 56.87s/it] {'loss': 1.975, 'learning_rate': 2e-05, 'epoch': 0.09} 9%|▉ | 162/1817 [2:34:14<26:08:43, 56.87s/it] 9%|▉ | 163/1817 [2:35:12<26:16:30, 57.19s/it] {'loss': 2.0356, 'learning_rate': 2e-05, 'epoch': 0.09} 9%|▉ | 163/1817 [2:35:12<26:16:32, 57.19s/it] 9%|▉ | 164/1817 [2:36:10<26:25:57, 57.57s/it] {'loss': 2.0225, 'learning_rate': 2e-05, 'epoch': 0.09} 9%|▉ | 164/1817 [2:36:10<26:25:57, 57.57s/it] 9%|▉ | 165/1817 [2:37:05<26:03:38, 56.79s/it] {'loss': 1.958, 'learning_rate': 2e-05, 'epoch': 0.09} 9%|▉ | 165/1817 [2:37:05<26:03:38, 56.79s/it] 9%|▉ | 166/1817 [2:38:04<26:14:33, 57.22s/it] {'loss': 2.0483, 'learning_rate': 2e-05, 'epoch': 0.09} 9%|▉ | 166/1817 [2:38:03<26:14:35, 57.22s/it] 9%|▉ | 167/1817 [2:39:00<26:05:32, 56.93s/it] {'loss': 2.0199, 'learning_rate': 2e-05, 'epoch': 0.09} 9%|▉ | 167/1817 [2:39:00<26:05:34, 56.93s/it] 9%|▉ | 168/1817 [2:39:59<26:27:16, 57.75s/it] {'loss': 1.9999, 'learning_rate': 2e-05, 'epoch': 0.09} 9%|▉ | 168/1817 [2:39:59<26:27:18, 57.76s/it] 9%|▉ | 169/1817 [2:40:57<26:28:05, 57.82s/it] {'loss': 1.9878, 'learning_rate': 2e-05, 'epoch': 0.09} 9%|▉ | 169/1817 [2:40:57<26:28:05, 57.82s/it] 9%|▉ | 170/1817 [2:41:54<26:14:48, 57.37s/it] {'loss': 2.0169, 'learning_rate': 2e-05, 'epoch': 0.09} 9%|▉ | 170/1817 [2:41:53<26:14:50, 57.37s/it] 9%|▉ | 171/1817 [2:42:52<26:24:01, 57.74s/it] {'loss': 2.0459, 'learning_rate': 2e-05, 'epoch': 0.09} 9%|▉ | 171/1817 [2:42:52<26:24:03, 57.74s/it] 9%|▉ | 172/1817 [2:43:50<26:20:51, 57.66s/it] {'loss': 1.9969, 'learning_rate': 2e-05, 'epoch': 0.09} 9%|▉ | 172/1817 [2:43:50<26:20:53, 57.66s/it] 10%|▉ | 173/1817 [2:44:46<26:06:21, 57.17s/it] {'loss': 1.9726, 'learning_rate': 2e-05, 'epoch': 0.1} 10%|▉ | 173/1817 [2:44:46<26:06:22, 57.17s/it] 10%|▉ | 174/1817 [2:45:42<25:57:38, 56.88s/it] {'loss': 1.9738, 'learning_rate': 2e-05, 'epoch': 0.1} 10%|▉ | 174/1817 [2:45:42<25:57:40, 56.88s/it] 10%|▉ | 175/1817 [2:46:41<26:10:52, 57.40s/it] {'loss': 1.9442, 'learning_rate': 2e-05, 'epoch': 0.1} 10%|▉ | 175/1817 [2:46:40<26:10:54, 57.40s/it] 10%|▉ | 176/1817 [2:47:37<25:59:42, 57.03s/it] {'loss': 2.0094, 'learning_rate': 2e-05, 'epoch': 0.1} 10%|▉ | 176/1817 [2:47:37<25:59:44, 57.03s/it] 10%|▉ | 177/1817 [2:48:33<25:51:09, 56.75s/it] {'loss': 1.956, 'learning_rate': 2e-05, 'epoch': 0.1} 10%|▉ | 177/1817 [2:48:33<25:51:11, 56.75s/it] 10%|▉ | 178/1817 [2:49:29<25:45:43, 56.59s/it] {'loss': 1.9616, 'learning_rate': 2e-05, 'epoch': 0.1} 10%|▉ | 178/1817 [2:49:29<25:45:45, 56.59s/it] 10%|▉ | 179/1817 [2:50:25<25:39:51, 56.41s/it] {'loss': 1.988, 'learning_rate': 2e-05, 'epoch': 0.1} 10%|▉ | 179/1817 [2:50:25<25:39:53, 56.41s/it] 10%|▉ | 180/1817 [2:51:21<25:35:49, 56.29s/it] {'loss': 1.9787, 'learning_rate': 2e-05, 'epoch': 0.1} 10%|▉ | 180/1817 [2:51:21<25:35:51, 56.29s/it] 10%|▉ | 181/1817 [2:52:16<25:20:04, 55.75s/it] {'loss': 2.0211, 'learning_rate': 2e-05, 'epoch': 0.1} 10%|▉ | 181/1817 [2:52:15<25:20:06, 55.75s/it] 10%|█ | 182/1817 [2:53:12<25:23:26, 55.91s/it] {'loss': 1.9992, 'learning_rate': 2e-05, 'epoch': 0.1} 10%|█ | 182/1817 [2:53:12<25:23:26, 55.91s/it] 10%|█ | 183/1817 [2:54:09<25:38:21, 56.49s/it] {'loss': 2.0028, 'learning_rate': 2e-05, 'epoch': 0.1} 10%|█ | 183/1817 [2:54:10<25:38:21, 56.49s/it] 10%|█ | 184/1817 [2:55:06<25:33:46, 56.35s/it] {'loss': 2.0113, 'learning_rate': 2e-05, 'epoch': 0.1} 10%|█ | 184/1817 [2:55:06<25:33:46, 56.35s/it] 10%|█ | 185/1817 [2:56:03<25:39:52, 56.61s/it] {'loss': 1.9877, 'learning_rate': 2e-05, 'epoch': 0.1} 10%|█ | 185/1817 [2:56:03<25:39:52, 56.61s/it] 10%|█ | 186/1817 [2:56:59<25:37:11, 56.55s/it] {'loss': 2.008, 'learning_rate': 2e-05, 'epoch': 0.1} 10%|█ | 186/1817 [2:56:59<25:37:11, 56.55s/it] 10%|█ | 187/1817 [2:57:57<25:49:30, 57.04s/it] {'loss': 1.9948, 'learning_rate': 2e-05, 'epoch': 0.1} 10%|█ | 187/1817 [2:57:57<25:49:30, 57.04s/it] 10%|█ | 188/1817 [2:58:54<25:42:16, 56.81s/it] {'loss': 1.9963, 'learning_rate': 2e-05, 'epoch': 0.1} 10%|█ | 188/1817 [2:58:54<25:42:16, 56.81s/it] 10%|█ | 189/1817 [2:59:50<25:37:14, 56.65s/it] {'loss': 1.9975, 'learning_rate': 2e-05, 'epoch': 0.1} 10%|█ | 189/1817 [2:59:50<25:37:14, 56.65s/it] 10%|█ | 190/1817 [3:00:47<25:34:19, 56.58s/it] {'loss': 1.9719, 'learning_rate': 2e-05, 'epoch': 0.1} 10%|█ | 190/1817 [3:00:46<25:34:19, 56.58s/it] 11%|█ | 191/1817 [3:01:43<25:31:49, 56.53s/it] {'loss': 1.9874, 'learning_rate': 2e-05, 'epoch': 0.11} 11%|█ | 191/1817 [3:01:43<25:31:49, 56.53s/it] 11%|█ | 192/1817 [3:02:40<25:35:37, 56.70s/it] {'loss': 1.9598, 'learning_rate': 2e-05, 'epoch': 0.11} 11%|█ | 192/1817 [3:02:40<25:35:37, 56.70s/it] 11%|█ | 193/1817 [3:03:38<25:44:18, 57.06s/it] {'loss': 1.9973, 'learning_rate': 2e-05, 'epoch': 0.11} 11%|█ | 193/1817 [3:03:38<25:44:18, 57.06s/it] 11%|█ | 194/1817 [3:04:33<25:29:09, 56.53s/it] {'loss': 2.0411, 'learning_rate': 2e-05, 'epoch': 0.11} 11%|█ | 194/1817 [3:04:33<25:29:09, 56.53s/it] 11%|█ | 195/1817 [3:05:31<25:41:35, 57.03s/it] {'loss': 1.9802, 'learning_rate': 2e-05, 'epoch': 0.11} 11%|█ | 195/1817 [3:05:31<25:41:35, 57.03s/it] 11%|█ | 196/1817 [3:06:27<25:33:15, 56.75s/it] {'loss': 1.9525, 'learning_rate': 2e-05, 'epoch': 0.11} 11%|█ | 196/1817 [3:06:27<25:33:15, 56.75s/it] 11%|█ | 197/1817 [3:07:21<25:06:16, 55.79s/it] {'loss': 1.985, 'learning_rate': 2e-05, 'epoch': 0.11} 11%|█ | 197/1817 [3:07:21<25:06:16, 55.79s/it] 11%|█ | 198/1817 [3:08:17<25:08:55, 55.92s/it] {'loss': 1.9857, 'learning_rate': 2e-05, 'epoch': 0.11} 11%|█ | 198/1817 [3:08:17<25:08:57, 55.92s/it] 11%|█ | 199/1817 [3:09:15<25:21:53, 56.44s/it] {'loss': 1.9854, 'learning_rate': 2e-05, 'epoch': 0.11} 11%|█ | 199/1817 [3:09:15<25:21:53, 56.44s/it] 11%|█ | 200/1817 [3:10:11<25:18:33, 56.35s/it] {'loss': 2.0151, 'learning_rate': 2e-05, 'epoch': 0.11} 11%|█ | 200/1817 [3:10:11<25:18:33, 56.35s/it] 11%|█ | 201/1817 [3:11:07<25:16:24, 56.30s/it] {'loss': 2.0209, 'learning_rate': 2e-05, 'epoch': 0.11} 11%|█ | 201/1817 [3:11:07<25:16:24, 56.30s/it] 11%|█ | 202/1817 [3:12:03<25:14:43, 56.27s/it] {'loss': 1.9691, 'learning_rate': 2e-05, 'epoch': 0.11} 11%|█ | 202/1817 [3:12:03<25:14:43, 56.27s/it] 11%|█ | 203/1817 [3:13:00<25:12:52, 56.24s/it] {'loss': 1.9715, 'learning_rate': 2e-05, 'epoch': 0.11} 11%|█ | 203/1817 [3:12:59<25:12:52, 56.24s/it] 11%|█ | 204/1817 [3:13:57<25:25:53, 56.76s/it] {'loss': 2.0055, 'learning_rate': 2e-05, 'epoch': 0.11} 11%|█ | 204/1817 [3:13:57<25:25:53, 56.76s/it] 11%|█▏ | 205/1817 [3:14:54<25:20:01, 56.58s/it] {'loss': 1.9726, 'learning_rate': 2e-05, 'epoch': 0.11} 11%|█▏ | 205/1817 [3:14:54<25:20:01, 56.58s/it] 11%|█▏ | 206/1817 [3:15:51<25:25:22, 56.81s/it] {'loss': 1.9819, 'learning_rate': 2e-05, 'epoch': 0.11} 11%|█▏ | 206/1817 [3:15:51<25:25:22, 56.81s/it] 11%|█▏ | 207/1817 [3:16:47<25:19:14, 56.62s/it] {'loss': 1.9832, 'learning_rate': 2e-05, 'epoch': 0.11} 11%|█▏ | 207/1817 [3:16:47<25:19:14, 56.62s/it] 11%|█▏ | 208/1817 [3:17:43<25:14:58, 56.49s/it] {'loss': 1.9795, 'learning_rate': 2e-05, 'epoch': 0.11} 11%|█▏ | 208/1817 [3:17:43<25:14:58, 56.49s/it] 12%|█▏ | 209/1817 [3:18:42<25:29:15, 57.06s/it] {'loss': 2.0146, 'learning_rate': 2e-05, 'epoch': 0.11} 12%|█▏ | 209/1817 [3:18:42<25:29:15, 57.06s/it] 12%|█▏ | 210/1817 [3:19:39<25:33:27, 57.25s/it] {'loss': 1.9724, 'learning_rate': 2e-05, 'epoch': 0.12} 12%|█▏ | 210/1817 [3:19:39<25:33:27, 57.25s/it] 12%|█▏ | 211/1817 [3:20:38<25:39:50, 57.53s/it] {'loss': 1.9986, 'learning_rate': 2e-05, 'epoch': 0.12} 12%|█▏ | 211/1817 [3:20:38<25:39:50, 57.53s/it] 12%|█▏ | 212/1817 [3:21:34<25:28:14, 57.13s/it] {'loss': 1.9972, 'learning_rate': 2e-05, 'epoch': 0.12} 12%|█▏ | 212/1817 [3:21:34<25:28:14, 57.13s/it] 12%|█▏ | 213/1817 [3:22:32<25:34:03, 57.38s/it] {'loss': 1.9739, 'learning_rate': 2e-05, 'epoch': 0.12} 12%|█▏ | 213/1817 [3:22:32<25:34:03, 57.38s/it] 12%|█▏ | 214/1817 [3:23:27<25:18:38, 56.84s/it] {'loss': 1.9707, 'learning_rate': 2e-05, 'epoch': 0.12} 12%|█▏ | 214/1817 [3:23:27<25:18:38, 56.84s/it] 12%|█▏ | 215/1817 [3:24:24<25:13:34, 56.69s/it] {'loss': 1.9854, 'learning_rate': 2e-05, 'epoch': 0.12} 12%|█▏ | 215/1817 [3:24:24<25:13:34, 56.69s/it] 12%|█▏ | 216/1817 [3:25:22<25:23:21, 57.09s/it] {'loss': 1.995, 'learning_rate': 2e-05, 'epoch': 0.12} 12%|█▏ | 216/1817 [3:25:22<25:23:21, 57.09s/it] 12%|█▏ | 217/1817 [3:26:24<26:08:10, 58.81s/it] {'loss': 1.979, 'learning_rate': 2e-05, 'epoch': 0.12} 12%|█▏ | 217/1817 [3:26:24<26:08:10, 58.81s/it] 12%|█▏ | 218/1817 [3:27:24<26:15:34, 59.12s/it] {'loss': 2.017, 'learning_rate': 2e-05, 'epoch': 0.12} 12%|█▏ | 218/1817 [3:27:24<26:15:34, 59.12s/it] 12%|█▏ | 219/1817 [3:28:21<25:52:02, 58.27s/it] {'loss': 1.9547, 'learning_rate': 2e-05, 'epoch': 0.12} 12%|█▏ | 219/1817 [3:28:21<25:52:02, 58.27s/it] 12%|█▏ | 220/1817 [3:29:17<25:33:37, 57.62s/it] {'loss': 2.0069, 'learning_rate': 2e-05, 'epoch': 0.12} 12%|█▏ | 220/1817 [3:29:17<25:33:37, 57.62s/it] 12%|█▏ | 221/1817 [3:30:14<25:30:48, 57.55s/it] {'loss': 1.983, 'learning_rate': 2e-05, 'epoch': 0.12} 12%|█▏ | 221/1817 [3:30:14<25:30:48, 57.55s/it] 12%|█▏ | 222/1817 [3:31:11<25:19:06, 57.15s/it] {'loss': 1.987, 'learning_rate': 2e-05, 'epoch': 0.12} 12%|█▏ | 222/1817 [3:31:10<25:19:06, 57.15s/it] 12%|█▏ | 223/1817 [3:32:09<25:31:32, 57.65s/it] {'loss': 2.002, 'learning_rate': 2e-05, 'epoch': 0.12} 12%|█▏ | 223/1817 [3:32:09<25:31:32, 57.65s/it] 12%|█▏ | 224/1817 [3:33:05<25:18:06, 57.18s/it] {'loss': 1.9487, 'learning_rate': 2e-05, 'epoch': 0.12} 12%|█▏ | 224/1817 [3:33:05<25:18:06, 57.18s/it] 12%|█▏ | 225/1817 [3:34:00<24:56:32, 56.40s/it] {'loss': 1.9567, 'learning_rate': 2e-05, 'epoch': 0.12} 12%|█▏ | 225/1817 [3:34:00<24:56:32, 56.40s/it] 12%|█▏ | 226/1817 [3:34:57<25:05:23, 56.77s/it] {'loss': 1.983, 'learning_rate': 2e-05, 'epoch': 0.12} 12%|█▏ | 226/1817 [3:34:57<25:05:23, 56.77s/it] 12%|█▏ | 227/1817 [3:35:54<25:02:01, 56.68s/it] {'loss': 2.0049, 'learning_rate': 2e-05, 'epoch': 0.12} 12%|█▏ | 227/1817 [3:35:54<25:02:01, 56.68s/it] 13%|█▎ | 228/1817 [3:36:51<24:59:45, 56.63s/it] {'loss': 1.9532, 'learning_rate': 2e-05, 'epoch': 0.13} 13%|█▎ | 228/1817 [3:36:50<24:59:45, 56.63s/it] 13%|█▎ | 229/1817 [3:37:47<24:54:53, 56.48s/it] {'loss': 1.9665, 'learning_rate': 2e-05, 'epoch': 0.13} 13%|█▎ | 229/1817 [3:37:47<24:54:53, 56.48s/it] 13%|█▎ | 230/1817 [3:38:44<25:02:27, 56.80s/it] {'loss': 2.0141, 'learning_rate': 2e-05, 'epoch': 0.13} 13%|█▎ | 230/1817 [3:38:44<25:02:27, 56.80s/it] 13%|█▎ | 231/1817 [3:39:42<25:07:17, 57.02s/it] {'loss': 2.013, 'learning_rate': 2e-05, 'epoch': 0.13} 13%|█▎ | 231/1817 [3:39:42<25:07:17, 57.02s/it] 13%|█▎ | 232/1817 [3:40:38<25:00:23, 56.80s/it] {'loss': 2.0475, 'learning_rate': 2e-05, 'epoch': 0.13} 13%|█▎ | 232/1817 [3:40:38<25:00:23, 56.80s/it] 13%|█▎ | 233/1817 [3:41:34<24:53:33, 56.57s/it] {'loss': 1.9374, 'learning_rate': 2e-05, 'epoch': 0.13} 13%|█▎ | 233/1817 [3:41:34<24:53:33, 56.57s/it] 13%|█▎ | 234/1817 [3:42:31<24:50:22, 56.49s/it] {'loss': 1.9688, 'learning_rate': 2e-05, 'epoch': 0.13} 13%|█▎ | 234/1817 [3:42:30<24:50:23, 56.49s/it] 13%|█▎ | 235/1817 [3:43:27<24:49:22, 56.49s/it] {'loss': 1.9867, 'learning_rate': 2e-05, 'epoch': 0.13} 13%|█▎ | 235/1817 [3:43:27<24:49:22, 56.49s/it] 13%|█▎ | 236/1817 [3:44:26<25:07:24, 57.21s/it] {'loss': 1.9704, 'learning_rate': 2e-05, 'epoch': 0.13} 13%|█▎ | 236/1817 [3:44:26<25:07:24, 57.21s/it] 13%|█▎ | 237/1817 [3:45:26<25:28:38, 58.05s/it] {'loss': 1.9948, 'learning_rate': 2e-05, 'epoch': 0.13} 13%|█▎ | 237/1817 [3:45:26<25:28:40, 58.05s/it] 13%|█▎ | 238/1817 [3:46:23<25:19:36, 57.74s/it] {'loss': 1.9752, 'learning_rate': 2e-05, 'epoch': 0.13} 13%|█▎ | 238/1817 [3:46:23<25:19:36, 57.74s/it] 13%|█▎ | 239/1817 [3:47:20<25:17:11, 57.69s/it] {'loss': 1.9864, 'learning_rate': 2e-05, 'epoch': 0.13} 13%|█▎ | 239/1817 [3:47:20<25:17:11, 57.69s/it] 13%|█▎ | 240/1817 [3:48:17<25:04:52, 57.26s/it] {'loss': 2.0121, 'learning_rate': 2e-05, 'epoch': 0.13} 13%|█▎ | 240/1817 [3:48:16<25:04:52, 57.26s/it] 13%|█▎ | 241/1817 [3:49:13<24:55:26, 56.93s/it] {'loss': 2.0434, 'learning_rate': 2e-05, 'epoch': 0.13} 13%|█▎ | 241/1817 [3:49:13<24:55:26, 56.93s/it] 13%|█▎ | 242/1817 [3:50:09<24:48:01, 56.69s/it] {'loss': 1.9096, 'learning_rate': 2e-05, 'epoch': 0.13} 13%|█▎ | 242/1817 [3:50:09<24:48:01, 56.69s/it] 13%|█▎ | 243/1817 [3:51:05<24:44:53, 56.60s/it] {'loss': 1.9723, 'learning_rate': 2e-05, 'epoch': 0.13} 13%|█▎ | 243/1817 [3:51:05<24:44:53, 56.60s/it] 13%|█▎ | 244/1817 [3:52:02<24:43:15, 56.58s/it] {'loss': 1.9664, 'learning_rate': 2e-05, 'epoch': 0.13} 13%|█▎ | 244/1817 [3:52:02<24:43:16, 56.58s/it] 13%|█▎ | 245/1817 [3:52:58<24:39:59, 56.49s/it] {'loss': 1.9572, 'learning_rate': 2e-05, 'epoch': 0.13} 13%|█▎ | 245/1817 [3:52:58<24:39:59, 56.49s/it] 14%|█▎ | 246/1817 [3:53:54<24:35:37, 56.36s/it] {'loss': 1.9521, 'learning_rate': 2e-05, 'epoch': 0.14} 14%|█▎ | 246/1817 [3:53:54<24:35:37, 56.36s/it] 14%|█▎ | 247/1817 [3:54:50<24:32:56, 56.29s/it] {'loss': 1.9972, 'learning_rate': 2e-05, 'epoch': 0.14} 14%|█▎ | 247/1817 [3:54:50<24:32:56, 56.29s/it] 14%|█▎ | 248/1817 [3:55:48<24:41:02, 56.64s/it] {'loss': 2.0013, 'learning_rate': 2e-05, 'epoch': 0.14} 14%|█▎ | 248/1817 [3:55:48<24:41:02, 56.64s/it] 14%|█▎ | 249/1817 [3:56:44<24:36:23, 56.49s/it] {'loss': 1.9783, 'learning_rate': 2e-05, 'epoch': 0.14} 14%|█▎ | 249/1817 [3:56:44<24:36:23, 56.49s/it] 14%|█▍ | 250/1817 [3:57:41<24:41:41, 56.73s/it] {'loss': 2.0187, 'learning_rate': 2e-05, 'epoch': 0.14} 14%|█▍ | 250/1817 [3:57:41<24:41:41, 56.73s/it] 14%|█▍ | 251/1817 [3:58:36<24:22:53, 56.05s/it] {'loss': 1.987, 'learning_rate': 2e-05, 'epoch': 0.14} 14%|█▍ | 251/1817 [3:58:35<24:22:52, 56.05s/it] 14%|█▍ | 252/1817 [3:59:32<24:22:06, 56.06s/it] {'loss': 1.9713, 'learning_rate': 2e-05, 'epoch': 0.14} 14%|█▍ | 252/1817 [3:59:32<24:22:06, 56.06s/it] 14%|█▍ | 253/1817 [4:00:28<24:21:07, 56.05s/it] {'loss': 2.0136, 'learning_rate': 2e-05, 'epoch': 0.14} 14%|█▍ | 253/1817 [4:00:28<24:21:07, 56.05s/it] 14%|█▍ | 254/1817 [4:01:25<24:27:59, 56.35s/it] {'loss': 1.9547, 'learning_rate': 2e-05, 'epoch': 0.14} 14%|█▍ | 254/1817 [4:01:25<24:27:59, 56.35s/it] 14%|█▍ | 255/1817 [4:02:22<24:33:50, 56.61s/it] {'loss': 1.999, 'learning_rate': 2e-05, 'epoch': 0.14} 14%|█▍ | 255/1817 [4:02:22<24:33:49, 56.61s/it] 14%|█▍ | 256/1817 [4:03:19<24:34:36, 56.68s/it] {'loss': 1.979, 'learning_rate': 2e-05, 'epoch': 0.14} 14%|█▍ | 256/1817 [4:03:19<24:34:37, 56.68s/it] 14%|█▍ | 257/1817 [4:04:15<24:30:17, 56.55s/it] {'loss': 1.9883, 'learning_rate': 2e-05, 'epoch': 0.14} 14%|█▍ | 257/1817 [4:04:15<24:30:17, 56.55s/it] 14%|█▍ | 258/1817 [4:05:12<24:28:41, 56.52s/it] {'loss': 1.9951, 'learning_rate': 2e-05, 'epoch': 0.14} 14%|█▍ | 258/1817 [4:05:11<24:28:41, 56.52s/it] 14%|█▍ | 259/1817 [4:06:12<24:53:55, 57.53s/it] {'loss': 2.0044, 'learning_rate': 2e-05, 'epoch': 0.14} 14%|█▍ | 259/1817 [4:06:11<24:53:55, 57.53s/it] 14%|█▍ | 260/1817 [4:07:08<24:46:07, 57.27s/it] {'loss': 1.9718, 'learning_rate': 2e-05, 'epoch': 0.14} 14%|█▍ | 260/1817 [4:07:08<24:46:07, 57.27s/it] 14%|█▍ | 261/1817 [4:08:05<24:43:22, 57.20s/it] {'loss': 2.0178, 'learning_rate': 2e-05, 'epoch': 0.14} 14%|█▍ | 261/1817 [4:08:05<24:43:22, 57.20s/it] 14%|█▍ | 262/1817 [4:09:02<24:41:48, 57.18s/it] {'loss': 1.9783, 'learning_rate': 2e-05, 'epoch': 0.14} 14%|█▍ | 262/1817 [4:09:02<24:41:48, 57.18s/it] 14%|█▍ | 263/1817 [4:10:01<24:51:16, 57.58s/it] {'loss': 1.9472, 'learning_rate': 2e-05, 'epoch': 0.14} 14%|█▍ | 263/1817 [4:10:01<24:51:16, 57.58s/it] 15%|█▍ | 264/1817 [4:10:57<24:38:58, 57.14s/it] {'loss': 2.0069, 'learning_rate': 2e-05, 'epoch': 0.15} 15%|█▍ | 264/1817 [4:10:57<24:38:58, 57.14s/it] 15%|█▍ | 265/1817 [4:11:54<24:38:27, 57.16s/it] {'loss': 2.0164, 'learning_rate': 2e-05, 'epoch': 0.15} 15%|█▍ | 265/1817 [4:11:54<24:38:27, 57.16s/it] 15%|█▍ | 266/1817 [4:12:50<24:30:09, 56.87s/it] {'loss': 2.0058, 'learning_rate': 2e-05, 'epoch': 0.15} 15%|█▍ | 266/1817 [4:12:50<24:30:09, 56.87s/it] 15%|█▍ | 267/1817 [4:13:44<24:02:53, 55.85s/it] {'loss': 1.9583, 'learning_rate': 2e-05, 'epoch': 0.15} 15%|█▍ | 267/1817 [4:13:44<24:02:53, 55.85s/it] 15%|█▍ | 268/1817 [4:14:40<24:03:57, 55.93s/it] {'loss': 1.9386, 'learning_rate': 2e-05, 'epoch': 0.15} 15%|█▍ | 268/1817 [4:14:40<24:03:57, 55.93s/it] 15%|█▍ | 269/1817 [4:15:38<24:17:18, 56.49s/it] {'loss': 1.9522, 'learning_rate': 2e-05, 'epoch': 0.15} 15%|█▍ | 269/1817 [4:15:38<24:17:18, 56.49s/it] 15%|█▍ | 270/1817 [4:16:34<24:14:56, 56.43s/it] {'loss': 1.9751, 'learning_rate': 2e-05, 'epoch': 0.15} 15%|█▍ | 270/1817 [4:16:34<24:14:56, 56.43s/it] 15%|█▍ | 271/1817 [4:17:30<24:11:36, 56.34s/it] {'loss': 2.0115, 'learning_rate': 2e-05, 'epoch': 0.15} 15%|█▍ | 271/1817 [4:17:30<24:11:36, 56.34s/it] 15%|█▍ | 272/1817 [4:18:26<24:08:54, 56.27s/it] {'loss': 1.9574, 'learning_rate': 2e-05, 'epoch': 0.15} 15%|█▍ | 272/1817 [4:18:26<24:08:54, 56.27s/it] 15%|█▌ | 273/1817 [4:19:23<24:08:35, 56.29s/it] {'loss': 1.971, 'learning_rate': 2e-05, 'epoch': 0.15} 15%|█▌ | 273/1817 [4:19:22<24:08:35, 56.29s/it] 15%|█▌ | 274/1817 [4:20:19<24:09:04, 56.35s/it] {'loss': 1.9887, 'learning_rate': 2e-05, 'epoch': 0.15} 15%|█▌ | 274/1817 [4:20:19<24:09:04, 56.35s/it] 15%|█▌ | 275/1817 [4:21:17<24:24:07, 56.97s/it] {'loss': 1.9788, 'learning_rate': 2e-05, 'epoch': 0.15} 15%|█▌ | 275/1817 [4:21:17<24:24:07, 56.97s/it] 15%|█▌ | 276/1817 [4:22:12<24:02:41, 56.17s/it] {'loss': 1.9287, 'learning_rate': 2e-05, 'epoch': 0.15} 15%|█▌ | 276/1817 [4:22:12<24:02:41, 56.17s/it] 15%|█▌ | 277/1817 [4:23:06<23:45:26, 55.54s/it] {'loss': 1.9569, 'learning_rate': 2e-05, 'epoch': 0.15} 15%|█▌ | 277/1817 [4:23:06<23:45:26, 55.54s/it] 15%|█▌ | 278/1817 [4:24:06<24:17:36, 56.83s/it] {'loss': 1.9489, 'learning_rate': 2e-05, 'epoch': 0.15} 15%|█▌ | 278/1817 [4:24:06<24:17:36, 56.83s/it] 15%|█▌ | 279/1817 [4:25:03<24:21:13, 57.00s/it] {'loss': 1.9782, 'learning_rate': 2e-05, 'epoch': 0.15} 15%|█▌ | 279/1817 [4:25:03<24:21:14, 57.01s/it] 15%|█▌ | 280/1817 [4:26:01<24:28:28, 57.32s/it] {'loss': 1.9509, 'learning_rate': 2e-05, 'epoch': 0.15} 15%|█▌ | 280/1817 [4:26:01<24:28:28, 57.32s/it] 15%|█▌ | 281/1817 [4:26:59<24:32:30, 57.52s/it] {'loss': 1.9702, 'learning_rate': 2e-05, 'epoch': 0.15} 15%|█▌ | 281/1817 [4:26:59<24:32:30, 57.52s/it] 16%|█▌ | 282/1817 [4:27:57<24:34:30, 57.64s/it] {'loss': 1.9787, 'learning_rate': 2e-05, 'epoch': 0.16} 16%|█▌ | 282/1817 [4:27:57<24:34:29, 57.64s/it] 16%|█▌ | 283/1817 [4:28:57<24:47:43, 58.19s/it] {'loss': 1.9604, 'learning_rate': 2e-05, 'epoch': 0.16} 16%|█▌ | 283/1817 [4:28:56<24:47:43, 58.19s/it] 16%|█▌ | 284/1817 [4:29:53<24:29:58, 57.53s/it] {'loss': 2.0174, 'learning_rate': 2e-05, 'epoch': 0.16} 16%|█▌ | 284/1817 [4:29:52<24:29:58, 57.53s/it] 16%|█▌ | 285/1817 [4:30:49<24:19:15, 57.15s/it] {'loss': 2.0036, 'learning_rate': 2e-05, 'epoch': 0.16} 16%|█▌ | 285/1817 [4:30:49<24:19:15, 57.15s/it] 16%|█▌ | 286/1817 [4:31:45<24:11:45, 56.89s/it] {'loss': 1.9348, 'learning_rate': 2e-05, 'epoch': 0.16} 16%|█▌ | 286/1817 [4:31:45<24:11:45, 56.89s/it] 16%|█▌ | 287/1817 [4:32:40<23:55:25, 56.29s/it] {'loss': 1.9926, 'learning_rate': 2e-05, 'epoch': 0.16} 16%|█▌ | 287/1817 [4:32:40<23:55:25, 56.29s/it] 16%|█▌ | 288/1817 [4:33:36<23:53:37, 56.26s/it] {'loss': 1.9835, 'learning_rate': 2e-05, 'epoch': 0.16} 16%|█▌ | 288/1817 [4:33:36<23:53:37, 56.26s/it] 16%|█▌ | 289/1817 [4:34:33<23:52:56, 56.27s/it] {'loss': 1.9626, 'learning_rate': 2e-05, 'epoch': 0.16} 16%|█▌ | 289/1817 [4:34:32<23:52:56, 56.27s/it] 16%|█▌ | 290/1817 [4:35:30<24:01:02, 56.62s/it] {'loss': 1.9729, 'learning_rate': 2e-05, 'epoch': 0.16} 16%|█▌ | 290/1817 [4:35:30<24:01:02, 56.62s/it] 16%|█▌ | 291/1817 [4:36:26<23:58:48, 56.57s/it] {'loss': 1.9641, 'learning_rate': 2e-05, 'epoch': 0.16} 16%|█▌ | 291/1817 [4:36:26<23:58:48, 56.57s/it] 16%|█▌ | 292/1817 [4:37:23<23:56:55, 56.54s/it] {'loss': 1.9649, 'learning_rate': 2e-05, 'epoch': 0.16} 16%|█▌ | 292/1817 [4:37:23<23:56:56, 56.54s/it] 16%|█▌ | 293/1817 [4:38:19<23:54:41, 56.48s/it] {'loss': 1.9842, 'learning_rate': 2e-05, 'epoch': 0.16} 16%|█▌ | 293/1817 [4:38:19<23:54:41, 56.48s/it] 16%|█▌ | 294/1817 [4:39:16<23:53:10, 56.46s/it] {'loss': 1.9644, 'learning_rate': 2e-05, 'epoch': 0.16} 16%|█▌ | 294/1817 [4:39:15<23:53:10, 56.46s/it] 16%|█▌ | 295/1817 [4:40:10<23:35:17, 55.79s/it] {'loss': 1.9793, 'learning_rate': 2e-05, 'epoch': 0.16} 16%|█▌ | 295/1817 [4:40:10<23:35:17, 55.79s/it] 16%|█▋ | 296/1817 [4:41:06<23:33:18, 55.75s/it] {'loss': 1.9672, 'learning_rate': 2e-05, 'epoch': 0.16} 16%|█▋ | 296/1817 [4:41:05<23:33:18, 55.75s/it] 16%|█▋ | 297/1817 [4:42:02<23:38:06, 55.98s/it] {'loss': 1.9651, 'learning_rate': 2e-05, 'epoch': 0.16} 16%|█▋ | 297/1817 [4:42:02<23:38:06, 55.98s/it] 16%|█▋ | 298/1817 [4:42:58<23:39:52, 56.08s/it] {'loss': 1.9608, 'learning_rate': 2e-05, 'epoch': 0.16} 16%|█▋ | 298/1817 [4:42:58<23:39:52, 56.08s/it] 16%|█▋ | 299/1817 [4:43:56<23:52:26, 56.62s/it] {'loss': 1.9416, 'learning_rate': 2e-05, 'epoch': 0.16} 16%|█▋ | 299/1817 [4:43:56<23:52:26, 56.62s/it] 17%|█▋ | 300/1817 [4:44:52<23:50:19, 56.57s/it] {'loss': 1.9738, 'learning_rate': 2e-05, 'epoch': 0.17} 17%|█▋ | 300/1817 [4:44:52<23:50:19, 56.57s/it] 17%|█▋ | 301/1817 [4:45:49<23:47:34, 56.50s/it] {'loss': 1.986, 'learning_rate': 2e-05, 'epoch': 0.17} 17%|█▋ | 301/1817 [4:45:49<23:47:34, 56.50s/it] 17%|█▋ | 302/1817 [4:46:47<23:54:23, 56.81s/it] {'loss': 1.9302, 'learning_rate': 2e-05, 'epoch': 0.17} 17%|█▋ | 302/1817 [4:46:46<23:54:23, 56.81s/it] 17%|█▋ | 303/1817 [4:47:45<24:05:30, 57.29s/it] {'loss': 1.9641, 'learning_rate': 2e-05, 'epoch': 0.17} 17%|█▋ | 303/1817 [4:47:45<24:05:30, 57.29s/it] 17%|█▋ | 304/1817 [4:48:41<23:57:44, 57.02s/it] {'loss': 1.9934, 'learning_rate': 2e-05, 'epoch': 0.17} 17%|█▋ | 304/1817 [4:48:41<23:57:44, 57.02s/it] 17%|█▋ | 305/1817 [4:49:39<24:01:51, 57.22s/it] {'loss': 1.9558, 'learning_rate': 2e-05, 'epoch': 0.17} 17%|█▋ | 305/1817 [4:49:39<24:01:51, 57.22s/it] 17%|█▋ | 306/1817 [4:50:36<23:56:14, 57.03s/it] {'loss': 1.9362, 'learning_rate': 2e-05, 'epoch': 0.17} 17%|█▋ | 306/1817 [4:50:35<23:56:14, 57.03s/it] 17%|█▋ | 307/1817 [4:51:32<23:50:23, 56.84s/it] {'loss': 1.9559, 'learning_rate': 2e-05, 'epoch': 0.17} 17%|█▋ | 307/1817 [4:51:32<23:50:23, 56.84s/it] 17%|█▋ | 308/1817 [4:52:28<23:47:04, 56.74s/it] {'loss': 1.9508, 'learning_rate': 2e-05, 'epoch': 0.17} 17%|█▋ | 308/1817 [4:52:28<23:47:04, 56.74s/it] 17%|█▋ | 309/1817 [4:53:25<23:43:59, 56.66s/it] {'loss': 1.9171, 'learning_rate': 2e-05, 'epoch': 0.17} 17%|█▋ | 309/1817 [4:53:25<23:43:58, 56.66s/it] 17%|█▋ | 310/1817 [4:54:23<23:50:21, 56.95s/it] {'loss': 1.9584, 'learning_rate': 2e-05, 'epoch': 0.17} 17%|█▋ | 310/1817 [4:54:22<23:50:20, 56.95s/it] 17%|█▋ | 311/1817 [4:55:20<23:53:21, 57.11s/it] {'loss': 1.9805, 'learning_rate': 2e-05, 'epoch': 0.17} 17%|█▋ | 311/1817 [4:55:20<23:53:21, 57.11s/it] 17%|█▋ | 312/1817 [4:56:15<23:34:27, 56.39s/it] {'loss': 1.9882, 'learning_rate': 2e-05, 'epoch': 0.17} 17%|█▋ | 312/1817 [4:56:15<23:34:27, 56.39s/it] 17%|█▋ | 313/1817 [4:57:11<23:31:44, 56.32s/it] {'loss': 1.9692, 'learning_rate': 2e-05, 'epoch': 0.17} 17%|█▋ | 313/1817 [4:57:11<23:31:44, 56.32s/it] 17%|█▋ | 314/1817 [4:58:07<23:29:23, 56.26s/it] {'loss': 2.0299, 'learning_rate': 2e-05, 'epoch': 0.17} 17%|█▋ | 314/1817 [4:58:07<23:29:23, 56.26s/it] 17%|█▋ | 315/1817 [4:59:03<23:27:23, 56.22s/it] {'loss': 1.965, 'learning_rate': 2e-05, 'epoch': 0.17} 17%|█▋ | 315/1817 [4:59:03<23:27:23, 56.22s/it] 17%|█▋ | 316/1817 [5:00:01<23:35:56, 56.60s/it] {'loss': 1.9698, 'learning_rate': 2e-05, 'epoch': 0.17} 17%|█▋ | 316/1817 [5:00:00<23:35:56, 56.60s/it] 17%|█▋ | 317/1817 [5:00:58<23:42:05, 56.88s/it] {'loss': 1.9847, 'learning_rate': 2e-05, 'epoch': 0.17} 17%|█▋ | 317/1817 [5:00:58<23:42:05, 56.88s/it] 18%|█▊ | 318/1817 [5:01:54<23:35:58, 56.68s/it] {'loss': 2.0106, 'learning_rate': 2e-05, 'epoch': 0.17} 18%|█▊ | 318/1817 [5:01:54<23:35:58, 56.68s/it] 18%|█▊ | 319/1817 [5:02:49<23:20:15, 56.09s/it] {'loss': 1.9598, 'learning_rate': 2e-05, 'epoch': 0.18} 18%|█▊ | 319/1817 [5:02:49<23:20:15, 56.08s/it] 18%|█▊ | 320/1817 [5:03:47<23:31:24, 56.57s/it] {'loss': 1.9901, 'learning_rate': 2e-05, 'epoch': 0.18} 18%|█▊ | 320/1817 [5:03:47<23:31:24, 56.57s/it] 18%|█▊ | 321/1817 [5:04:45<23:43:07, 57.08s/it] {'loss': 2.022, 'learning_rate': 2e-05, 'epoch': 0.18} 18%|█▊ | 321/1817 [5:04:45<23:43:07, 57.08s/it] 18%|█▊ | 322/1817 [5:05:41<23:35:35, 56.81s/it] {'loss': 1.9439, 'learning_rate': 2e-05, 'epoch': 0.18} 18%|█▊ | 322/1817 [5:05:41<23:35:35, 56.81s/it] 18%|█▊ | 323/1817 [5:06:38<23:30:14, 56.64s/it] {'loss': 2.0133, 'learning_rate': 2e-05, 'epoch': 0.18} 18%|█▊ | 323/1817 [5:06:37<23:30:14, 56.64s/it] 18%|█▊ | 324/1817 [5:07:34<23:25:07, 56.47s/it] {'loss': 1.9292, 'learning_rate': 2e-05, 'epoch': 0.18} 18%|█▊ | 324/1817 [5:07:33<23:25:07, 56.47s/it] 18%|█▊ | 325/1817 [5:08:30<23:20:30, 56.32s/it] {'loss': 1.9743, 'learning_rate': 2e-05, 'epoch': 0.18} 18%|█▊ | 325/1817 [5:08:29<23:20:30, 56.32s/it] 18%|█▊ | 326/1817 [5:09:28<23:32:04, 56.82s/it] {'loss': 1.9638, 'learning_rate': 2e-05, 'epoch': 0.18} 18%|█▊ | 326/1817 [5:09:27<23:32:04, 56.82s/it] 18%|█▊ | 327/1817 [5:10:26<23:40:39, 57.21s/it] {'loss': 1.9778, 'learning_rate': 2e-05, 'epoch': 0.18} 18%|█▊ | 327/1817 [5:10:25<23:40:39, 57.21s/it] 18%|█▊ | 328/1817 [5:11:22<23:32:30, 56.92s/it] {'loss': 1.9561, 'learning_rate': 2e-05, 'epoch': 0.18} 18%|█▊ | 328/1817 [5:11:22<23:32:31, 56.92s/it] 18%|█▊ | 329/1817 [5:12:18<23:26:53, 56.73s/it] {'loss': 1.9262, 'learning_rate': 2e-05, 'epoch': 0.18} 18%|█▊ | 329/1817 [5:12:18<23:26:54, 56.73s/it] 18%|█▊ | 330/1817 [5:13:14<23:21:51, 56.56s/it] {'loss': 1.9109, 'learning_rate': 2e-05, 'epoch': 0.18} 18%|█▊ | 330/1817 [5:13:14<23:21:51, 56.56s/it] 18%|█▊ | 331/1817 [5:14:11<23:18:33, 56.47s/it] {'loss': 1.9878, 'learning_rate': 2e-05, 'epoch': 0.18} 18%|█▊ | 331/1817 [5:14:10<23:18:33, 56.47s/it] 18%|█▊ | 332/1817 [5:15:06<23:09:29, 56.14s/it] {'loss': 2.0057, 'learning_rate': 2e-05, 'epoch': 0.18} 18%|█▊ | 332/1817 [5:15:06<23:09:29, 56.14s/it] 18%|█▊ | 333/1817 [5:16:02<23:09:46, 56.19s/it] {'loss': 1.9342, 'learning_rate': 2e-05, 'epoch': 0.18} 18%|█▊ | 333/1817 [5:16:02<23:09:48, 56.19s/it] 18%|█▊ | 334/1817 [5:16:59<23:10:40, 56.26s/it] {'loss': 1.9707, 'learning_rate': 2e-05, 'epoch': 0.18} 18%|█▊ | 334/1817 [5:16:59<23:10:41, 56.27s/it] 18%|█▊ | 335/1817 [5:17:55<23:10:05, 56.28s/it] {'loss': 1.9654, 'learning_rate': 2e-05, 'epoch': 0.18} 18%|█▊ | 335/1817 [5:17:55<23:10:06, 56.28s/it] 18%|█▊ | 336/1817 [5:18:52<23:10:26, 56.33s/it] {'loss': 1.9366, 'learning_rate': 2e-05, 'epoch': 0.18} 18%|█▊ | 336/1817 [5:18:51<23:10:28, 56.33s/it] 19%|█▊ | 337/1817 [5:19:48<23:08:57, 56.31s/it] {'loss': 1.9641, 'learning_rate': 2e-05, 'epoch': 0.19} 19%|█▊ | 337/1817 [5:19:48<23:08:58, 56.31s/it] 19%|█▊ | 338/1817 [5:20:44<23:08:04, 56.31s/it] {'loss': 1.9693, 'learning_rate': 2e-05, 'epoch': 0.19} 19%|█▊ | 338/1817 [5:20:44<23:08:04, 56.31s/it] 19%|█▊ | 339/1817 [5:21:40<23:07:00, 56.31s/it] {'loss': 1.9442, 'learning_rate': 2e-05, 'epoch': 0.19} 19%|█▊ | 339/1817 [5:21:40<23:07:01, 56.31s/it] 19%|█▊ | 340/1817 [5:22:37<23:04:14, 56.23s/it] {'loss': 1.9382, 'learning_rate': 2e-05, 'epoch': 0.19} 19%|█▊ | 340/1817 [5:22:36<23:04:15, 56.23s/it] 19%|█▉ | 341/1817 [5:23:33<23:02:46, 56.21s/it] {'loss': 1.9858, 'learning_rate': 2e-05, 'epoch': 0.19} 19%|█▉ | 341/1817 [5:23:32<23:02:47, 56.21s/it] 19%|█▉ | 342/1817 [5:24:30<23:11:23, 56.60s/it] {'loss': 2.009, 'learning_rate': 2e-05, 'epoch': 0.19} 19%|█▉ | 342/1817 [5:24:30<23:11:23, 56.60s/it] 19%|█▉ | 343/1817 [5:25:28<23:20:29, 57.01s/it] {'loss': 1.953, 'learning_rate': 2e-05, 'epoch': 0.19} 19%|█▉ | 343/1817 [5:25:28<23:20:30, 57.01s/it] 19%|█▉ | 344/1817 [5:26:24<23:13:11, 56.75s/it] {'loss': 1.941, 'learning_rate': 2e-05, 'epoch': 0.19} 19%|█▉ | 344/1817 [5:26:24<23:13:13, 56.75s/it] 19%|█▉ | 345/1817 [5:27:22<23:17:22, 56.96s/it] {'loss': 1.9367, 'learning_rate': 2e-05, 'epoch': 0.19} 19%|█▉ | 345/1817 [5:27:21<23:17:21, 56.96s/it] 19%|█▉ | 346/1817 [5:28:20<23:23:25, 57.24s/it] {'loss': 1.9947, 'learning_rate': 2e-05, 'epoch': 0.19} 19%|█▉ | 346/1817 [5:28:19<23:23:26, 57.24s/it] 19%|█▉ | 347/1817 [5:29:17<23:27:33, 57.45s/it] {'loss': 1.9673, 'learning_rate': 2e-05, 'epoch': 0.19} 19%|█▉ | 347/1817 [5:29:17<23:27:33, 57.45s/it] 19%|█▉ | 348/1817 [5:30:12<23:04:28, 56.55s/it] {'loss': 1.9758, 'learning_rate': 2e-05, 'epoch': 0.19} 19%|█▉ | 348/1817 [5:30:12<23:04:29, 56.55s/it] 19%|█▉ | 349/1817 [5:31:10<23:11:14, 56.86s/it] {'loss': 1.9155, 'learning_rate': 2e-05, 'epoch': 0.19} 19%|█▉ | 349/1817 [5:31:09<23:11:14, 56.86s/it] 19%|█▉ | 350/1817 [5:32:07<23:15:30, 57.08s/it] {'loss': 1.9778, 'learning_rate': 2e-05, 'epoch': 0.19} 19%|█▉ | 350/1817 [5:32:07<23:15:30, 57.08s/it] 19%|█▉ | 351/1817 [5:33:05<23:22:46, 57.41s/it] {'loss': 1.9708, 'learning_rate': 2e-05, 'epoch': 0.19} 19%|█▉ | 351/1817 [5:33:05<23:22:47, 57.41s/it] 19%|█▉ | 352/1817 [5:34:02<23:13:01, 57.05s/it] {'loss': 1.9242, 'learning_rate': 2e-05, 'epoch': 0.19} 19%|█▉ | 352/1817 [5:34:01<23:13:02, 57.05s/it] 19%|█▉ | 353/1817 [5:34:58<23:06:17, 56.81s/it] {'loss': 1.9713, 'learning_rate': 2e-05, 'epoch': 0.19} 19%|█▉ | 353/1817 [5:34:58<23:06:17, 56.81s/it] 19%|█▉ | 354/1817 [5:35:55<23:06:50, 56.88s/it] {'loss': 1.9658, 'learning_rate': 2e-05, 'epoch': 0.19} 19%|█▉ | 354/1817 [5:35:55<23:06:50, 56.88s/it] 20%|█▉ | 355/1817 [5:36:53<23:14:31, 57.23s/it] {'loss': 1.9115, 'learning_rate': 2e-05, 'epoch': 0.2} 20%|█▉ | 355/1817 [5:36:53<23:14:31, 57.23s/it] 20%|█▉ | 356/1817 [5:37:49<23:06:02, 56.92s/it] {'loss': 1.9874, 'learning_rate': 2e-05, 'epoch': 0.2} 20%|█▉ | 356/1817 [5:37:49<23:06:03, 56.92s/it] 20%|█▉ | 357/1817 [5:38:45<23:01:00, 56.75s/it] {'loss': 1.9713, 'learning_rate': 2e-05, 'epoch': 0.2} 20%|█▉ | 357/1817 [5:38:45<23:01:00, 56.75s/it] 20%|█▉ | 358/1817 [5:39:42<22:56:35, 56.61s/it] {'loss': 2.0034, 'learning_rate': 2e-05, 'epoch': 0.2} 20%|█▉ | 358/1817 [5:39:41<22:56:35, 56.61s/it] 20%|█▉ | 359/1817 [5:40:38<22:52:29, 56.48s/it] {'loss': 1.9878, 'learning_rate': 2e-05, 'epoch': 0.2} 20%|█▉ | 359/1817 [5:40:38<22:52:30, 56.48s/it] 20%|█▉ | 360/1817 [5:41:34<22:50:13, 56.43s/it] {'loss': 1.9778, 'learning_rate': 2e-05, 'epoch': 0.2} 20%|█▉ | 360/1817 [5:41:34<22:50:13, 56.43s/it] 20%|█▉ | 361/1817 [5:42:30<22:47:54, 56.37s/it] {'loss': 1.955, 'learning_rate': 2e-05, 'epoch': 0.2} 20%|█▉ | 361/1817 [5:42:30<22:47:54, 56.37s/it] 20%|█▉ | 362/1817 [5:43:27<22:45:44, 56.32s/it] {'loss': 1.9734, 'learning_rate': 2e-05, 'epoch': 0.2} 20%|█▉ | 362/1817 [5:43:26<22:45:44, 56.32s/it] 20%|█▉ | 363/1817 [5:44:23<22:45:09, 56.33s/it] {'loss': 1.9623, 'learning_rate': 2e-05, 'epoch': 0.2} 20%|█▉ | 363/1817 [5:44:23<22:45:09, 56.33s/it] 20%|██ | 364/1817 [5:45:20<22:51:27, 56.63s/it] {'loss': 1.9698, 'learning_rate': 2e-05, 'epoch': 0.2} 20%|██ | 364/1817 [5:45:20<22:51:27, 56.63s/it] 20%|██ | 365/1817 [5:46:17<22:47:25, 56.50s/it] {'loss': 1.9563, 'learning_rate': 2e-05, 'epoch': 0.2} 20%|██ | 365/1817 [5:46:16<22:47:24, 56.50s/it] 20%|██ | 366/1817 [5:47:13<22:44:12, 56.41s/it] {'loss': 1.9644, 'learning_rate': 2e-05, 'epoch': 0.2} 20%|██ | 366/1817 [5:47:12<22:44:12, 56.41s/it] 20%|██ | 367/1817 [5:48:10<22:52:12, 56.78s/it] {'loss': 1.9751, 'learning_rate': 2e-05, 'epoch': 0.2} 20%|██ | 367/1817 [5:48:10<22:52:12, 56.78s/it] 20%|██ | 368/1817 [5:49:07<22:46:44, 56.59s/it] {'loss': 1.9616, 'learning_rate': 2e-05, 'epoch': 0.2} 20%|██ | 368/1817 [5:49:06<22:46:44, 56.59s/it] 20%|██ | 369/1817 [5:50:04<22:54:06, 56.94s/it] {'loss': 1.9692, 'learning_rate': 2e-05, 'epoch': 0.2} 20%|██ | 369/1817 [5:50:04<22:54:05, 56.94s/it] 20%|██ | 370/1817 [5:51:00<22:47:01, 56.68s/it] {'loss': 2.0184, 'learning_rate': 2e-05, 'epoch': 0.2} 20%|██ | 370/1817 [5:51:00<22:47:01, 56.68s/it] 20%|██ | 371/1817 [5:51:56<22:42:39, 56.54s/it] {'loss': 1.9445, 'learning_rate': 2e-05, 'epoch': 0.2} 20%|██ | 371/1817 [5:51:56<22:42:39, 56.54s/it] 20%|██ | 372/1817 [5:52:53<22:39:25, 56.45s/it] {'loss': 1.9966, 'learning_rate': 2e-05, 'epoch': 0.2} 20%|██ | 372/1817 [5:52:53<22:39:25, 56.45s/it] 21%|██ | 373/1817 [5:53:49<22:36:18, 56.36s/it] {'loss': 1.9646, 'learning_rate': 2e-05, 'epoch': 0.21} 21%|██ | 373/1817 [5:53:49<22:36:18, 56.36s/it] 21%|██ | 374/1817 [5:54:45<22:35:32, 56.36s/it] {'loss': 1.9759, 'learning_rate': 2e-05, 'epoch': 0.21} 21%|██ | 374/1817 [5:54:45<22:35:33, 56.36s/it] 21%|██ | 375/1817 [5:55:42<22:37:40, 56.49s/it] {'loss': 1.9829, 'learning_rate': 2e-05, 'epoch': 0.21} 21%|██ | 375/1817 [5:55:42<22:37:40, 56.49s/it] 21%|██ | 376/1817 [5:56:39<22:35:45, 56.45s/it] {'loss': 1.9743, 'learning_rate': 2e-05, 'epoch': 0.21} 21%|██ | 376/1817 [5:56:38<22:35:45, 56.45s/it] 21%|██ | 377/1817 [5:57:36<22:44:40, 56.86s/it] {'loss': 1.9554, 'learning_rate': 2e-05, 'epoch': 0.21} 21%|██ | 377/1817 [5:57:36<22:44:40, 56.86s/it] 21%|██ | 378/1817 [5:58:33<22:39:13, 56.67s/it] {'loss': 1.9363, 'learning_rate': 2e-05, 'epoch': 0.21} 21%|██ | 378/1817 [5:58:32<22:39:14, 56.67s/it] 21%|██ | 379/1817 [5:59:29<22:35:53, 56.57s/it] {'loss': 1.9383, 'learning_rate': 2e-05, 'epoch': 0.21} 21%|██ | 379/1817 [5:59:29<22:35:53, 56.57s/it] 21%|██ | 380/1817 [6:00:25<22:34:13, 56.54s/it] {'loss': 1.9642, 'learning_rate': 2e-05, 'epoch': 0.21} 21%|██ | 380/1817 [6:00:25<22:34:13, 56.54s/it] 21%|██ | 381/1817 [6:01:22<22:30:25, 56.42s/it] {'loss': 1.9746, 'learning_rate': 2e-05, 'epoch': 0.21} 21%|██ | 381/1817 [6:01:21<22:30:25, 56.42s/it] 21%|██ | 382/1817 [6:02:21<22:49:13, 57.25s/it] {'loss': 1.9679, 'learning_rate': 2e-05, 'epoch': 0.21} 21%|██ | 382/1817 [6:02:20<22:49:12, 57.25s/it] 21%|██ | 383/1817 [6:03:17<22:41:49, 56.98s/it] {'loss': 1.9786, 'learning_rate': 2e-05, 'epoch': 0.21} 21%|██ | 383/1817 [6:03:17<22:41:27, 56.96s/it] 21%|██ | 384/1817 [6:04:14<22:43:27, 57.09s/it] {'loss': 1.9723, 'learning_rate': 2e-05, 'epoch': 0.21} 21%|██ | 384/1817 [6:04:14<22:43:12, 57.08s/it] 21%|██ | 385/1817 [6:05:14<22:59:11, 57.79s/it] {'loss': 1.9604, 'learning_rate': 2e-05, 'epoch': 0.21} 21%|██ | 385/1817 [6:05:13<22:59:08, 57.79s/it] 21%|██ | 386/1817 [6:06:10<22:47:12, 57.33s/it] {'loss': 1.9542, 'learning_rate': 2e-05, 'epoch': 0.21} 21%|██ | 386/1817 [6:06:10<22:47:06, 57.32s/it] 21%|██▏ | 387/1817 [6:07:07<22:40:50, 57.10s/it] {'loss': 1.9603, 'learning_rate': 2e-05, 'epoch': 0.21} 21%|██▏ | 387/1817 [6:07:06<22:40:50, 57.10s/it] 21%|██▏ | 388/1817 [6:08:03<22:34:19, 56.86s/it] {'loss': 1.9662, 'learning_rate': 2e-05, 'epoch': 0.21} 21%|██▏ | 388/1817 [6:08:03<22:34:28, 56.87s/it] 21%|██▏ | 389/1817 [6:09:00<22:35:00, 56.93s/it] {'loss': 1.9868, 'learning_rate': 2e-05, 'epoch': 0.21} 21%|██▏ | 389/1817 [6:09:00<22:35:00, 56.93s/it] 21%|██▏ | 390/1817 [6:09:58<22:45:00, 57.39s/it] {'loss': 1.9645, 'learning_rate': 2e-05, 'epoch': 0.21} 21%|██▏ | 390/1817 [6:09:58<22:44:59, 57.39s/it] 22%|██▏ | 391/1817 [6:10:55<22:40:52, 57.26s/it] {'loss': 1.9735, 'learning_rate': 2e-05, 'epoch': 0.22} 22%|██▏ | 391/1817 [6:10:55<22:40:58, 57.26s/it] 22%|██▏ | 392/1817 [6:11:53<22:40:03, 57.27s/it] {'loss': 1.994, 'learning_rate': 2e-05, 'epoch': 0.22} 22%|██▏ | 392/1817 [6:11:52<22:40:02, 57.27s/it] 22%|██▏ | 393/1817 [6:12:50<22:35:37, 57.12s/it] {'loss': 1.9315, 'learning_rate': 2e-05, 'epoch': 0.22} 22%|██▏ | 393/1817 [6:12:49<22:35:40, 57.12s/it] 22%|██▏ | 394/1817 [6:13:46<22:27:59, 56.84s/it] {'loss': 1.9667, 'learning_rate': 2e-05, 'epoch': 0.22} 22%|██▏ | 394/1817 [6:13:45<22:28:02, 56.84s/it] 22%|██▏ | 395/1817 [6:14:42<22:22:46, 56.66s/it] {'loss': 1.9683, 'learning_rate': 2e-05, 'epoch': 0.22} 22%|██▏ | 395/1817 [6:14:42<22:22:46, 56.66s/it] 22%|██▏ | 396/1817 [6:15:38<22:16:30, 56.43s/it] {'loss': 1.9848, 'learning_rate': 2e-05, 'epoch': 0.22} 22%|██▏ | 396/1817 [6:15:38<22:16:32, 56.43s/it] 22%|██▏ | 397/1817 [6:16:34<22:13:30, 56.35s/it] {'loss': 1.944, 'learning_rate': 2e-05, 'epoch': 0.22} 22%|██▏ | 397/1817 [6:16:34<22:13:30, 56.35s/it] 22%|██▏ | 398/1817 [6:17:30<22:11:30, 56.30s/it] {'loss': 1.9382, 'learning_rate': 2e-05, 'epoch': 0.22} 22%|██▏ | 398/1817 [6:17:30<22:11:30, 56.30s/it] 22%|██▏ | 399/1817 [6:18:26<22:09:39, 56.26s/it] {'loss': 1.9684, 'learning_rate': 2e-05, 'epoch': 0.22} 22%|██▏ | 399/1817 [6:18:26<22:09:41, 56.26s/it] 22%|██▏ | 400/1817 [6:19:23<22:09:43, 56.30s/it] {'loss': 1.9466, 'learning_rate': 2e-05, 'epoch': 0.22} 22%|██▏ | 400/1817 [6:19:22<22:09:43, 56.30s/it] 22%|██▏ | 401/1817 [6:20:20<22:16:13, 56.62s/it] {'loss': 1.9557, 'learning_rate': 2e-05, 'epoch': 0.22} 22%|██▏ | 401/1817 [6:20:20<22:16:13, 56.62s/it] 22%|██▏ | 402/1817 [6:21:16<22:13:46, 56.56s/it] {'loss': 1.941, 'learning_rate': 2e-05, 'epoch': 0.22} 22%|██▏ | 402/1817 [6:21:16<22:13:46, 56.56s/it] 22%|██▏ | 403/1817 [6:22:11<21:58:27, 55.95s/it] {'loss': 1.963, 'learning_rate': 2e-05, 'epoch': 0.22} 22%|██▏ | 403/1817 [6:22:11<21:58:27, 55.95s/it] 22%|██▏ | 404/1817 [6:23:09<22:14:08, 56.65s/it] {'loss': 1.9181, 'learning_rate': 2e-05, 'epoch': 0.22} 22%|██▏ | 404/1817 [6:23:09<22:14:08, 56.65s/it] 22%|██▏ | 405/1817 [6:24:05<22:09:58, 56.51s/it] {'loss': 1.953, 'learning_rate': 2e-05, 'epoch': 0.22} 22%|██▏ | 405/1817 [6:24:05<22:09:58, 56.51s/it] 22%|██▏ | 406/1817 [6:25:04<22:22:12, 57.07s/it] {'loss': 1.9693, 'learning_rate': 2e-05, 'epoch': 0.22} 22%|██▏ | 406/1817 [6:25:04<22:22:13, 57.08s/it] 22%|██▏ | 407/1817 [6:26:02<22:30:26, 57.47s/it] {'loss': 1.9806, 'learning_rate': 2e-05, 'epoch': 0.22} 22%|██▏ | 407/1817 [6:26:02<22:30:27, 57.47s/it] 22%|██▏ | 408/1817 [6:26:58<22:20:24, 57.08s/it] {'loss': 2.0137, 'learning_rate': 2e-05, 'epoch': 0.22} 22%|██▏ | 408/1817 [6:26:58<22:20:24, 57.08s/it] 23%|██▎ | 409/1817 [6:27:58<22:34:08, 57.71s/it] {'loss': 1.9564, 'learning_rate': 2e-05, 'epoch': 0.23} 23%|██▎ | 409/1817 [6:27:57<22:34:08, 57.71s/it] 23%|██▎ | 410/1817 [6:28:57<22:45:24, 58.23s/it] {'loss': 1.9771, 'learning_rate': 2e-05, 'epoch': 0.23} 23%|██▎ | 410/1817 [6:28:57<22:45:25, 58.23s/it] 23%|██▎ | 411/1817 [6:29:55<22:41:52, 58.12s/it] {'loss': 1.9283, 'learning_rate': 2e-05, 'epoch': 0.23} 23%|██▎ | 411/1817 [6:29:55<22:41:52, 58.12s/it] 23%|██▎ | 412/1817 [6:30:51<22:27:54, 57.56s/it] {'loss': 1.9445, 'learning_rate': 2e-05, 'epoch': 0.23} 23%|██▎ | 412/1817 [6:30:51<22:27:54, 57.56s/it] 23%|██▎ | 413/1817 [6:31:46<22:07:02, 56.71s/it] {'loss': 1.938, 'learning_rate': 2e-05, 'epoch': 0.23} 23%|██▎ | 413/1817 [6:31:46<22:07:02, 56.71s/it] 23%|██▎ | 414/1817 [6:32:42<22:02:24, 56.55s/it] {'loss': 1.9564, 'learning_rate': 2e-05, 'epoch': 0.23} 23%|██▎ | 414/1817 [6:32:42<22:02:24, 56.55s/it] 23%|██▎ | 415/1817 [6:33:40<22:07:42, 56.82s/it] {'loss': 1.9561, 'learning_rate': 2e-05, 'epoch': 0.23} 23%|██▎ | 415/1817 [6:33:39<22:07:42, 56.82s/it] 23%|██▎ | 416/1817 [6:34:38<22:17:18, 57.27s/it] {'loss': 1.9512, 'learning_rate': 2e-05, 'epoch': 0.23} 23%|██▎ | 416/1817 [6:34:38<22:17:18, 57.27s/it] 23%|██▎ | 417/1817 [6:35:34<22:09:12, 56.97s/it] {'loss': 1.9453, 'learning_rate': 2e-05, 'epoch': 0.23} 23%|██▎ | 417/1817 [6:35:34<22:09:11, 56.97s/it] 23%|██▎ | 418/1817 [6:36:33<22:23:57, 57.64s/it] {'loss': 1.919, 'learning_rate': 2e-05, 'epoch': 0.23} 23%|██▎ | 418/1817 [6:36:33<22:23:57, 57.64s/it] 23%|██▎ | 419/1817 [6:37:31<22:23:04, 57.64s/it] {'loss': 1.9396, 'learning_rate': 2e-05, 'epoch': 0.23} 23%|██▎ | 419/1817 [6:37:31<22:23:04, 57.64s/it] 23%|██▎ | 420/1817 [6:38:27<22:12:44, 57.24s/it] {'loss': 1.9758, 'learning_rate': 2e-05, 'epoch': 0.23} 23%|██▎ | 420/1817 [6:38:27<22:12:44, 57.24s/it] 23%|██▎ | 421/1817 [6:39:24<22:05:50, 56.98s/it] {'loss': 1.9253, 'learning_rate': 2e-05, 'epoch': 0.23} 23%|██▎ | 421/1817 [6:39:23<22:05:50, 56.98s/it] 23%|██▎ | 422/1817 [6:40:20<22:00:28, 56.79s/it] {'loss': 1.9653, 'learning_rate': 2e-05, 'epoch': 0.23} 23%|██▎ | 422/1817 [6:40:20<22:00:28, 56.79s/it] 23%|██▎ | 423/1817 [6:41:16<21:56:35, 56.67s/it] {'loss': 1.9236, 'learning_rate': 2e-05, 'epoch': 0.23} 23%|██▎ | 423/1817 [6:41:16<21:56:35, 56.67s/it] 23%|██▎ | 424/1817 [6:42:14<22:03:48, 57.02s/it] {'loss': 1.969, 'learning_rate': 2e-05, 'epoch': 0.23} 23%|██▎ | 424/1817 [6:42:14<22:03:48, 57.02s/it] 23%|██▎ | 425/1817 [6:43:08<21:42:53, 56.16s/it] {'loss': 2.0114, 'learning_rate': 2e-05, 'epoch': 0.23} 23%|██▎ | 425/1817 [6:43:08<21:42:53, 56.16s/it] 23%|██▎ | 426/1817 [6:44:05<21:43:46, 56.24s/it] {'loss': 1.9834, 'learning_rate': 2e-05, 'epoch': 0.23} 23%|██▎ | 426/1817 [6:44:05<21:43:46, 56.24s/it] 24%|██▎ | 427/1817 [6:45:01<21:45:26, 56.35s/it] {'loss': 1.9647, 'learning_rate': 2e-05, 'epoch': 0.23} 24%|██▎ | 427/1817 [6:45:01<21:45:27, 56.35s/it] 24%|██▎ | 428/1817 [6:45:59<21:53:23, 56.73s/it] {'loss': 1.9799, 'learning_rate': 2e-05, 'epoch': 0.24} 24%|██▎ | 428/1817 [6:45:59<21:53:24, 56.73s/it] 24%|██▎ | 429/1817 [6:46:58<22:07:58, 57.41s/it] {'loss': 1.9874, 'learning_rate': 2e-05, 'epoch': 0.24} 24%|██▎ | 429/1817 [6:46:58<22:07:58, 57.41s/it] 24%|██▎ | 430/1817 [6:47:54<21:59:22, 57.07s/it] {'loss': 1.9325, 'learning_rate': 2e-05, 'epoch': 0.24} 24%|██▎ | 430/1817 [6:47:54<21:59:22, 57.07s/it] 24%|██▎ | 431/1817 [6:48:51<21:54:03, 56.89s/it] {'loss': 1.9778, 'learning_rate': 2e-05, 'epoch': 0.24} 24%|██▎ | 431/1817 [6:48:50<21:54:04, 56.89s/it] 24%|██▍ | 432/1817 [6:49:47<21:49:46, 56.74s/it] {'loss': 1.9622, 'learning_rate': 2e-05, 'epoch': 0.24} 24%|██▍ | 432/1817 [6:49:47<21:49:46, 56.74s/it] 24%|██▍ | 433/1817 [6:50:43<21:46:12, 56.63s/it] {'loss': 1.9686, 'learning_rate': 2e-05, 'epoch': 0.24} 24%|██▍ | 433/1817 [6:50:43<21:46:12, 56.63s/it] 24%|██▍ | 434/1817 [6:51:44<22:09:09, 57.66s/it] {'loss': 1.9542, 'learning_rate': 2e-05, 'epoch': 0.24} 24%|██▍ | 434/1817 [6:51:43<22:09:11, 57.67s/it] 24%|██▍ | 435/1817 [6:52:39<21:56:39, 57.16s/it] {'loss': 1.9687, 'learning_rate': 2e-05, 'epoch': 0.24} 24%|██▍ | 435/1817 [6:52:39<21:56:39, 57.16s/it] 24%|██▍ | 436/1817 [6:53:36<21:50:41, 56.95s/it] {'loss': 1.9473, 'learning_rate': 2e-05, 'epoch': 0.24} 24%|██▍ | 436/1817 [6:53:36<21:50:41, 56.95s/it] 24%|██▍ | 437/1817 [6:54:37<22:15:58, 58.09s/it] {'loss': 1.9657, 'learning_rate': 2e-05, 'epoch': 0.24} 24%|██▍ | 437/1817 [6:54:37<22:15:59, 58.09s/it] 24%|██▍ | 438/1817 [6:55:33<22:02:31, 57.54s/it] {'loss': 1.9616, 'learning_rate': 2e-05, 'epoch': 0.24} 24%|██▍ | 438/1817 [6:55:33<22:02:32, 57.54s/it] 24%|██▍ | 439/1817 [6:56:31<22:07:17, 57.79s/it] {'loss': 1.9536, 'learning_rate': 2e-05, 'epoch': 0.24} 24%|██▍ | 439/1817 [6:56:31<22:07:17, 57.79s/it] 24%|██▍ | 440/1817 [6:57:29<22:03:41, 57.68s/it] {'loss': 1.9555, 'learning_rate': 2e-05, 'epoch': 0.24} 24%|██▍ | 440/1817 [6:57:29<22:03:41, 57.68s/it] 24%|██▍ | 441/1817 [6:58:25<21:54:27, 57.32s/it] {'loss': 1.9475, 'learning_rate': 2e-05, 'epoch': 0.24} 24%|██▍ | 441/1817 [6:58:25<21:54:28, 57.32s/it] 24%|██▍ | 442/1817 [6:59:26<22:15:42, 58.29s/it] {'loss': 1.9395, 'learning_rate': 2e-05, 'epoch': 0.24} 24%|██▍ | 442/1817 [6:59:26<22:15:43, 58.29s/it] 24%|██▍ | 443/1817 [7:00:24<22:10:24, 58.10s/it] {'loss': 1.8895, 'learning_rate': 2e-05, 'epoch': 0.24} 24%|██▍ | 443/1817 [7:00:23<22:10:14, 58.09s/it] 24%|██▍ | 444/1817 [7:01:20<21:57:06, 57.56s/it] {'loss': 1.969, 'learning_rate': 2e-05, 'epoch': 0.24} 24%|██▍ | 444/1817 [7:01:20<21:57:06, 57.56s/it] 24%|██▍ | 445/1817 [7:02:17<21:52:13, 57.39s/it] {'loss': 1.9405, 'learning_rate': 2e-05, 'epoch': 0.24} 24%|██▍ | 445/1817 [7:02:16<21:51:48, 57.37s/it] 25%|██▍ | 446/1817 [7:03:13<21:43:51, 57.06s/it] {'loss': 1.9519, 'learning_rate': 2e-05, 'epoch': 0.25} 25%|██▍ | 446/1817 [7:03:13<21:43:51, 57.06s/it] 25%|██▍ | 447/1817 [7:04:09<21:37:54, 56.84s/it] {'loss': 1.9265, 'learning_rate': 2e-05, 'epoch': 0.25} 25%|██▍ | 447/1817 [7:04:09<21:37:54, 56.84s/it] 25%|██▍ | 448/1817 [7:05:05<21:31:01, 56.58s/it] {'loss': 1.9473, 'learning_rate': 2e-05, 'epoch': 0.25} 25%|██▍ | 448/1817 [7:05:05<21:30:51, 56.58s/it] 25%|██▍ | 449/1817 [7:06:02<21:29:35, 56.56s/it] {'loss': 1.9339, 'learning_rate': 2e-05, 'epoch': 0.25} 25%|██▍ | 449/1817 [7:06:02<21:29:36, 56.56s/it] 25%|██▍ | 450/1817 [7:06:57<21:21:17, 56.24s/it] {'loss': 1.997, 'learning_rate': 2e-05, 'epoch': 0.25} 25%|██▍ | 450/1817 [7:06:57<21:21:13, 56.24s/it] 25%|██▍ | 451/1817 [7:07:55<21:31:22, 56.72s/it] {'loss': 1.9444, 'learning_rate': 2e-05, 'epoch': 0.25} 25%|██▍ | 451/1817 [7:07:55<21:31:22, 56.72s/it] 25%|██▍ | 452/1817 [7:08:51<21:28:03, 56.62s/it] {'loss': 1.931, 'learning_rate': 2e-05, 'epoch': 0.25} 25%|██▍ | 452/1817 [7:08:51<21:28:03, 56.62s/it] 25%|██▍ | 453/1817 [7:09:48<21:25:57, 56.57s/it] {'loss': 1.9693, 'learning_rate': 2e-05, 'epoch': 0.25} 25%|██▍ | 453/1817 [7:09:48<21:25:58, 56.57s/it] 25%|██▍ | 454/1817 [7:10:46<21:34:48, 57.00s/it] {'loss': 1.94, 'learning_rate': 2e-05, 'epoch': 0.25} 25%|██▍ | 454/1817 [7:10:46<21:34:48, 57.00s/it] 25%|██▌ | 455/1817 [7:11:42<21:28:45, 56.77s/it] {'loss': 1.9445, 'learning_rate': 2e-05, 'epoch': 0.25} 25%|██▌ | 455/1817 [7:11:42<21:28:46, 56.77s/it] 25%|██▌ | 456/1817 [7:12:41<21:42:45, 57.43s/it] {'loss': 1.9496, 'learning_rate': 2e-05, 'epoch': 0.25} 25%|██▌ | 456/1817 [7:12:41<21:42:45, 57.43s/it] 25%|██▌ | 457/1817 [7:13:37<21:30:15, 56.92s/it] {'loss': 1.9931, 'learning_rate': 2e-05, 'epoch': 0.25} 25%|██▌ | 457/1817 [7:13:37<21:30:15, 56.92s/it] 25%|██▌ | 458/1817 [7:14:38<21:59:29, 58.26s/it] {'loss': 1.9826, 'learning_rate': 2e-05, 'epoch': 0.25} 25%|██▌ | 458/1817 [7:14:38<21:59:31, 58.26s/it] 25%|██▌ | 459/1817 [7:15:35<21:45:08, 57.66s/it] {'loss': 1.9667, 'learning_rate': 2e-05, 'epoch': 0.25} 25%|██▌ | 459/1817 [7:15:34<21:45:09, 57.67s/it] 25%|██▌ | 460/1817 [7:16:33<21:49:44, 57.91s/it] {'loss': 1.9401, 'learning_rate': 2e-05, 'epoch': 0.25} 25%|██▌ | 460/1817 [7:16:33<21:49:44, 57.91s/it] 25%|██▌ | 461/1817 [7:17:30<21:37:59, 57.43s/it] {'loss': 1.9396, 'learning_rate': 2e-05, 'epoch': 0.25} 25%|██▌ | 461/1817 [7:17:29<21:38:00, 57.43s/it] 25%|██▌ | 462/1817 [7:18:26<21:29:06, 57.08s/it] {'loss': 1.9186, 'learning_rate': 2e-05, 'epoch': 0.25} 25%|██▌ | 462/1817 [7:18:25<21:29:07, 57.08s/it] 25%|██▌ | 463/1817 [7:19:22<21:23:50, 56.89s/it] {'loss': 1.9747, 'learning_rate': 2e-05, 'epoch': 0.25} 25%|██▌ | 463/1817 [7:19:22<21:23:50, 56.89s/it] 26%|██▌ | 464/1817 [7:20:20<21:33:10, 57.35s/it] {'loss': 1.9567, 'learning_rate': 2e-05, 'epoch': 0.26} 26%|██▌ | 464/1817 [7:20:20<21:33:10, 57.35s/it] 26%|██▌ | 465/1817 [7:21:17<21:29:02, 57.21s/it] {'loss': 1.9276, 'learning_rate': 2e-05, 'epoch': 0.26} 26%|██▌ | 465/1817 [7:21:17<21:29:02, 57.21s/it] 26%|██▌ | 466/1817 [7:22:13<21:18:32, 56.78s/it] {'loss': 1.9639, 'learning_rate': 2e-05, 'epoch': 0.26} 26%|██▌ | 466/1817 [7:22:13<21:18:32, 56.78s/it] 26%|██▌ | 467/1817 [7:23:09<21:15:12, 56.68s/it] {'loss': 1.9595, 'learning_rate': 2e-05, 'epoch': 0.26} 26%|██▌ | 467/1817 [7:23:09<21:15:12, 56.68s/it] 26%|██▌ | 468/1817 [7:24:06<21:12:02, 56.58s/it] {'loss': 1.9912, 'learning_rate': 2e-05, 'epoch': 0.26} 26%|██▌ | 468/1817 [7:24:06<21:12:02, 56.58s/it] 26%|██▌ | 469/1817 [7:25:02<21:09:47, 56.52s/it] {'loss': 1.961, 'learning_rate': 2e-05, 'epoch': 0.26} 26%|██▌ | 469/1817 [7:25:02<21:09:47, 56.52s/it] 26%|██▌ | 470/1817 [7:25:59<21:08:46, 56.52s/it] {'loss': 1.9833, 'learning_rate': 2e-05, 'epoch': 0.26} 26%|██▌ | 470/1817 [7:25:59<21:08:47, 56.52s/it] 26%|██▌ | 471/1817 [7:26:57<21:16:31, 56.90s/it] {'loss': 1.9512, 'learning_rate': 2e-05, 'epoch': 0.26} 26%|██▌ | 471/1817 [7:26:56<21:16:31, 56.90s/it] 26%|██▌ | 472/1817 [7:27:53<21:11:48, 56.73s/it] {'loss': 1.9895, 'learning_rate': 2e-05, 'epoch': 0.26} 26%|██▌ | 472/1817 [7:27:53<21:11:49, 56.74s/it] 26%|██▌ | 473/1817 [7:28:50<21:16:25, 56.98s/it] {'loss': 1.9678, 'learning_rate': 2e-05, 'epoch': 0.26} 26%|██▌ | 473/1817 [7:28:50<21:16:25, 56.98s/it] 26%|██▌ | 474/1817 [7:29:48<21:18:57, 57.14s/it] {'loss': 1.9488, 'learning_rate': 2e-05, 'epoch': 0.26} 26%|██▌ | 474/1817 [7:29:48<21:18:57, 57.14s/it] 26%|██▌ | 475/1817 [7:30:48<21:34:22, 57.87s/it] {'loss': 1.9528, 'learning_rate': 2e-05, 'epoch': 0.26} 26%|██▌ | 475/1817 [7:30:47<21:34:22, 57.87s/it] 26%|██▌ | 476/1817 [7:31:45<21:29:02, 57.68s/it] {'loss': 1.9171, 'learning_rate': 2e-05, 'epoch': 0.26} 26%|██▌ | 476/1817 [7:31:45<21:29:02, 57.68s/it] 26%|██▋ | 477/1817 [7:32:43<21:30:55, 57.80s/it] {'loss': 1.9099, 'learning_rate': 2e-05, 'epoch': 0.26} 26%|██▋ | 477/1817 [7:32:43<21:30:55, 57.80s/it] 26%|██▋ | 478/1817 [7:33:40<21:21:47, 57.44s/it] {'loss': 1.9362, 'learning_rate': 2e-05, 'epoch': 0.26} 26%|██▋ | 478/1817 [7:33:39<21:21:47, 57.44s/it] 26%|██▋ | 479/1817 [7:34:37<21:22:31, 57.51s/it] {'loss': 1.9684, 'learning_rate': 2e-05, 'epoch': 0.26} 26%|██▋ | 479/1817 [7:34:37<21:22:31, 57.51s/it] 26%|██▋ | 480/1817 [7:35:34<21:14:24, 57.19s/it] {'loss': 1.9566, 'learning_rate': 2e-05, 'epoch': 0.26} 26%|██▋ | 480/1817 [7:35:33<21:14:24, 57.19s/it] 26%|██▋ | 481/1817 [7:36:28<20:51:30, 56.21s/it] {'loss': 2.0094, 'learning_rate': 2e-05, 'epoch': 0.26} 26%|██▋ | 481/1817 [7:36:27<20:51:31, 56.21s/it] 27%|██▋ | 482/1817 [7:37:22<20:40:57, 55.77s/it] {'loss': 1.9438, 'learning_rate': 2e-05, 'epoch': 0.27} 27%|██▋ | 482/1817 [7:37:22<20:40:57, 55.77s/it] 27%|██▋ | 483/1817 [7:38:20<20:55:04, 56.45s/it] {'loss': 1.9848, 'learning_rate': 2e-05, 'epoch': 0.27} 27%|██▋ | 483/1817 [7:38:20<20:55:04, 56.45s/it] 27%|██▋ | 484/1817 [7:39:17<20:53:00, 56.40s/it] {'loss': 1.9482, 'learning_rate': 2e-05, 'epoch': 0.27} 27%|██▋ | 484/1817 [7:39:16<20:53:00, 56.40s/it] 27%|██▋ | 485/1817 [7:40:13<20:52:21, 56.41s/it] {'loss': 1.9425, 'learning_rate': 2e-05, 'epoch': 0.27} 27%|██▋ | 485/1817 [7:40:13<20:52:22, 56.41s/it] 27%|██▋ | 486/1817 [7:41:09<20:50:55, 56.39s/it] {'loss': 1.9348, 'learning_rate': 2e-05, 'epoch': 0.27} 27%|██▋ | 486/1817 [7:41:09<20:50:55, 56.39s/it] 27%|██▋ | 487/1817 [7:42:04<20:40:18, 55.95s/it] {'loss': 1.9393, 'learning_rate': 2e-05, 'epoch': 0.27} 27%|██▋ | 487/1817 [7:42:04<20:40:18, 55.95s/it] 27%|██▋ | 488/1817 [7:43:00<20:41:48, 56.06s/it] {'loss': 1.9556, 'learning_rate': 2e-05, 'epoch': 0.27} 27%|██▋ | 488/1817 [7:43:01<20:41:48, 56.06s/it] 27%|██▋ | 489/1817 [7:43:56<20:37:50, 55.93s/it] {'loss': 1.9338, 'learning_rate': 2e-05, 'epoch': 0.27} 27%|██▋ | 489/1817 [7:43:56<20:37:49, 55.93s/it] 27%|██▋ | 490/1817 [7:44:53<20:38:45, 56.01s/it] {'loss': 1.9785, 'learning_rate': 2e-05, 'epoch': 0.27} 27%|██▋ | 490/1817 [7:44:52<20:38:46, 56.01s/it] 27%|██▋ | 491/1817 [7:45:49<20:42:28, 56.22s/it] {'loss': 1.9821, 'learning_rate': 2e-05, 'epoch': 0.27} 27%|██▋ | 491/1817 [7:45:49<20:42:28, 56.22s/it] 27%|██▋ | 492/1817 [7:46:46<20:43:44, 56.32s/it] {'loss': 1.9814, 'learning_rate': 2e-05, 'epoch': 0.27} 27%|██▋ | 492/1817 [7:46:46<20:43:45, 56.32s/it] 27%|██▋ | 493/1817 [7:47:44<20:55:35, 56.90s/it] {'loss': 1.9278, 'learning_rate': 2e-05, 'epoch': 0.27} 27%|██▋ | 493/1817 [7:47:44<20:55:36, 56.90s/it] 27%|██▋ | 494/1817 [7:48:41<20:51:42, 56.77s/it] {'loss': 1.9718, 'learning_rate': 2e-05, 'epoch': 0.27} 27%|██▋ | 494/1817 [7:48:40<20:51:42, 56.77s/it] 27%|██▋ | 495/1817 [7:49:37<20:49:33, 56.71s/it] {'loss': 1.9828, 'learning_rate': 2e-05, 'epoch': 0.27} 27%|██▋ | 495/1817 [7:49:37<20:49:35, 56.71s/it] 27%|██▋ | 496/1817 [7:50:35<20:58:30, 57.16s/it] {'loss': 1.9945, 'learning_rate': 2e-05, 'epoch': 0.27} 27%|██▋ | 496/1817 [7:50:35<20:58:30, 57.16s/it] 27%|██▋ | 497/1817 [7:51:33<21:01:24, 57.34s/it] {'loss': 1.9385, 'learning_rate': 2e-05, 'epoch': 0.27} 27%|██▋ | 497/1817 [7:51:33<21:01:24, 57.34s/it] 27%|██▋ | 498/1817 [7:52:30<20:54:48, 57.08s/it] {'loss': 1.9746, 'learning_rate': 2e-05, 'epoch': 0.27} 27%|██▋ | 498/1817 [7:52:29<20:54:49, 57.08s/it] 27%|██▋ | 499/1817 [7:53:26<20:50:26, 56.92s/it] {'loss': 1.9493, 'learning_rate': 2e-05, 'epoch': 0.27} 27%|██▋ | 499/1817 [7:53:26<20:50:27, 56.93s/it] 28%|██▊ | 500/1817 [7:54:23<20:46:35, 56.79s/it] {'loss': 1.9145, 'learning_rate': 2e-05, 'epoch': 0.28} 28%|██▊ | 500/1817 [7:54:22<20:46:36, 56.79s/it][INFO|trainer.py:2830] 2024-01-28 12:48:07,186 >> Saving model checkpoint to ./checkpoints_ct/ac/allm-ac-7b/checkpoint-500 [INFO|configuration_utils.py:457] 2024-01-28 12:48:07,191 >> Configuration saved in ./checkpoints_ct/ac/allm-ac-7b/checkpoint-500/config.json [INFO|configuration_utils.py:362] 2024-01-28 12:48:07,196 >> Configuration saved in ./checkpoints_ct/ac/allm-ac-7b/checkpoint-500/generation_config.json [INFO|modeling_utils.py:1759] 2024-01-28 12:48:07,224 >> Model weights saved in ./checkpoints_ct/ac/allm-ac-7b/checkpoint-500/pytorch_model.bin [INFO|tokenization_utils_base.py:2164] 2024-01-28 12:48:07,226 >> tokenizer config file saved in ./checkpoints_ct/ac/allm-ac-7b/checkpoint-500/tokenizer_config.json [INFO|tokenization_utils_base.py:2171] 2024-01-28 12:48:07,227 >> Special tokens file saved in ./checkpoints_ct/ac/allm-ac-7b/checkpoint-500/special_tokens_map.json [INFO|tokenization_utils_base.py:2221] 2024-01-28 12:48:07,228 >> added tokens file saved in ./checkpoints_ct/ac/allm-ac-7b/checkpoint-500/added_tokens.json [2024-01-28 12:48:14,847] [INFO] [logging.py:96:log_dist] [Rank 0] [Torch] Checkpoint global_step500 is about to be saved! [2024-01-28 12:48:14,848] [INFO] [engine.py:3492:save_16bit_model] Saving model weights to ./checkpoints_ct/ac/allm-ac-7b/checkpoint-500/pytorch_model.bin, tag: global_step500 [2024-01-28 12:48:14,848] [INFO] [torch_checkpoint_engine.py:21:save] [Torch] Saving ./checkpoints_ct/ac/allm-ac-7b/checkpoint-500/pytorch_model.bin... /apdcephfs/share_733425/vinnylywang/jianhuipang/llama2_sft/envs/lib/python3.8/site-packages/torch/nn/modules/module.py:1432: UserWarning: Positional args are being deprecated, use kwargs instead. Refer to https://pytorch.org/docs/master/generated/torch.nn.Module.html#torch.nn.Module.state_dict for details. warnings.warn( /apdcephfs/share_733425/vinnylywang/jianhuipang/llama2_sft/envs/lib/python3.8/site-packages/torch/nn/modules/module.py:1432: UserWarning: Positional args are being deprecated, use kwargs instead. Refer to https://pytorch.org/docs/master/generated/torch.nn.Module.html#torch.nn.Module.state_dict for details. warnings.warn( /apdcephfs/share_733425/vinnylywang/jianhuipang/llama2_sft/envs/lib/python3.8/site-packages/torch/nn/modules/module.py:1432: UserWarning: Positional args are being deprecated, use kwargs instead. Refer to https://pytorch.org/docs/master/generated/torch.nn.Module.html#torch.nn.Module.state_dict for details. warnings.warn( /apdcephfs/share_733425/vinnylywang/jianhuipang/llama2_sft/envs/lib/python3.8/site-packages/torch/nn/modules/module.py:1432: UserWarning: Positional args are being deprecated, use kwargs instead. Refer to https://pytorch.org/docs/master/generated/torch.nn.Module.html#torch.nn.Module.state_dict for details. warnings.warn( /apdcephfs/share_733425/vinnylywang/jianhuipang/llama2_sft/envs/lib/python3.8/site-packages/torch/nn/modules/module.py:1432: UserWarning: Positional args are being deprecated, use kwargs instead. Refer to https://pytorch.org/docs/master/generated/torch.nn.Module.html#torch.nn.Module.state_dict for details. warnings.warn( /apdcephfs/share_733425/vinnylywang/jianhuipang/llama2_sft/envs/lib/python3.8/site-packages/torch/nn/modules/module.py:1432: UserWarning: Positional args are being deprecated, use kwargs instead. Refer to https://pytorch.org/docs/master/generated/torch.nn.Module.html#torch.nn.Module.state_dict for details. warnings.warn( /apdcephfs/share_733425/vinnylywang/jianhuipang/llama2_sft/envs/lib/python3.8/site-packages/torch/nn/modules/module.py:1432: UserWarning: Positional args are being deprecated, use kwargs instead. Refer to https://pytorch.org/docs/master/generated/torch.nn.Module.html#torch.nn.Module.state_dict for details. warnings.warn( /apdcephfs/share_733425/vinnylywang/jianhuipang/llama2_sft/envs/lib/python3.8/site-packages/torch/nn/modules/module.py:1432: UserWarning: Positional args are being deprecated, use kwargs instead. Refer to https://pytorch.org/docs/master/generated/torch.nn.Module.html#torch.nn.Module.state_dict for details. warnings.warn( [2024-01-28 12:48:31,064] [INFO] [torch_checkpoint_engine.py:21:save] [Torch] Saving ./checkpoints_ct/ac/allm-ac-7b/checkpoint-500/global_step500/zero_pp_rank_8_mp_rank_00_model_states.pt... [2024-01-28 12:48:31,107] [INFO] [torch_checkpoint_engine.py:23:save] [Torch] Saved ./checkpoints_ct/ac/allm-ac-7b/checkpoint-500/global_step500/zero_pp_rank_8_mp_rank_00_model_states.pt. [2024-01-28 12:48:31,126] [INFO] [torch_checkpoint_engine.py:21:save] [Torch] Saving ./checkpoints_ct/ac/allm-ac-7b/checkpoint-500/global_step500/bf16_zero_pp_rank_8_mp_rank_00_optim_states.pt... .. [2024-01-28 12:48:38,584] [INFO] [torch_checkpoint_engine.py:23:save] [Torch] Saved ./checkpoints_ct/ac/allm-ac-7b/checkpoint-500/global_step500/bf16_zero_pp_rank_24_mp_rank_00_optim_states.pt. [2024-01-28 12:48:38,585] [INFO] [engine.py:3381:_save_zero_checkpoint] zero checkpoint saved ./checkpoints_ct/ac/allm-ac-7b/checkpoint-500/global_step500/bf16_zero_pp_rank_24_mp_rank_00_optim_states.pt [2024-01-28 12:48:38,598] [INFO] [torch_checkpoint_engine.py:33:commit] [Torch] Checkpoint global_step500 is ready now! 0/global_step500/bf16_zero_pp_rank_0_mp_rank_00_optim_states.pt... [2024-01-28 12:48:38,410] [INFO] [torch_checkpoint_engine.py:23:save] [Torch] Saved ./checkpoints_ct/ac/allm-ac-7b/checkpoint-500/global_step500/bf16_zero_pp_rank_16_mp_rank_00_optim_states.pt. [2024-01-28 12:48:38,411] [INFO] [engine.py:3381:_save_zero_checkpoint] zero checkpoint saved ./checkpoints_ct/ac/allm-ac-7b/checkpoint-500/global_step500/bf16_zero_pp_rank_16_mp_rank_00_optim_states.p[2024-01-28 12:48:38,658] [INFO] [torch_checkpoint_engine.py:33:commit] [Torch] Checkpoint global_step500 is ready now! ! 28%|██▊ | 501/1817 [7:55:51<24:13:18, 66.26s/it] {'loss': 1.9406, 'learning_rate': 2e-05, 'epoch': 0.28} 28%|██▊ | 501/1817 [7:55:51<24:13:18, 66.26s/it] 28%|██▊ | 502/1817 [7:56:47<23:06:19, 63.25s/it] {'loss': 1.9053, 'learning_rate': 2e-05, 'epoch': 0.28} 28%|██▊ | 502/1817 [7:56:47<23:06:19, 63.25s/it] 28%|██▊ | 503/1817 [7:57:42<22:11:01, 60.78s/it] {'loss': 1.9291, 'learning_rate': 2e-05, 'epoch': 0.28} 28%|██▊ | 503/1817 [7:57:42<22:11:03, 60.78s/it] 28%|██▊ | 504/1817 [7:58:40<21:53:13, 60.01s/it] {'loss': 1.9282, 'learning_rate': 2e-05, 'epoch': 0.28} 28%|██▊ | 504/1817 [7:58:40<21:53:13, 60.01s/it] 28%|██▊ | 505/1817 [7:59:37<21:32:33, 59.11s/it] {'loss': 1.9616, 'learning_rate': 2e-05, 'epoch': 0.28} 28%|██▊ | 505/1817 [7:59:37<21:32:34, 59.11s/it] 28%|██▊ | 506/1817 [8:00:38<21:40:52, 59.54s/it] {'loss': 1.9665, 'learning_rate': 2e-05, 'epoch': 0.28} 28%|██▊ | 506/1817 [8:00:38<21:40:41, 59.53s/it] 28%|██▊ | 507/1817 [8:01:34<21:18:29, 58.56s/it] {'loss': 1.9599, 'learning_rate': 2e-05, 'epoch': 0.28} 28%|██▊ | 507/1817 [8:01:34<21:18:29, 58.56s/it] 28%|██▊ | 508/1817 [8:02:30<21:01:50, 57.84s/it] {'loss': 1.946, 'learning_rate': 2e-05, 'epoch': 0.28} 28%|██▊ | 508/1817 [8:02:30<21:01:50, 57.84s/it] 28%|██▊ | 509/1817 [8:03:27<20:50:03, 57.34s/it] {'loss': 1.9742, 'learning_rate': 2e-05, 'epoch': 0.28} 28%|██▊ | 509/1817 [8:03:26<20:49:46, 57.33s/it] 28%|██▊ | 510/1817 [8:04:23<20:41:22, 56.99s/it] {'loss': 1.9168, 'learning_rate': 2e-05, 'epoch': 0.28} 28%|██▊ | 510/1817 [8:04:22<20:41:10, 56.98s/it] 28%|██▊ | 511/1817 [8:05:19<20:36:23, 56.80s/it] {'loss': 1.9276, 'learning_rate': 2e-05, 'epoch': 0.28} 28%|██▊ | 511/1817 [8:05:19<20:36:14, 56.80s/it] 28%|██▊ | 512/1817 [8:06:17<20:45:09, 57.25s/it] {'loss': 1.897, 'learning_rate': 2e-05, 'epoch': 0.28} 28%|██▊ | 512/1817 [8:06:17<20:45:09, 57.25s/it] 28%|██▊ | 513/1817 [8:07:14<20:37:06, 56.92s/it] {'loss': 1.955, 'learning_rate': 2e-05, 'epoch': 0.28} 28%|██▊ | 513/1817 [8:07:13<20:37:06, 56.92s/it] 28%|██▊ | 514/1817 [8:08:10<20:31:27, 56.71s/it] {'loss': 1.9089, 'learning_rate': 2e-05, 'epoch': 0.28} 28%|██▊ | 514/1817 [8:08:09<20:31:27, 56.71s/it] 28%|██▊ | 515/1817 [8:09:06<20:27:54, 56.59s/it] {'loss': 1.9492, 'learning_rate': 2e-05, 'epoch': 0.28} 28%|██▊ | 515/1817 [8:09:06<20:27:54, 56.59s/it] 28%|██▊ | 516/1817 [8:10:04<20:35:25, 56.98s/it] {'loss': 2.0255, 'learning_rate': 2e-05, 'epoch': 0.28} 28%|██▊ | 516/1817 [8:10:04<20:35:25, 56.98s/it] 28%|██▊ | 517/1817 [8:11:00<20:31:35, 56.84s/it] {'loss': 1.9539, 'learning_rate': 2e-05, 'epoch': 0.28} 28%|██▊ | 517/1817 [8:11:00<20:31:35, 56.84s/it] 29%|██▊ | 518/1817 [8:12:00<20:46:57, 57.60s/it] {'loss': 1.9558, 'learning_rate': 2e-05, 'epoch': 0.29} 29%|██▊ | 518/1817 [8:12:00<20:46:57, 57.60s/it] 29%|██▊ | 519/1817 [8:13:00<21:06:32, 58.55s/it] {'loss': 1.9631, 'learning_rate': 2e-05, 'epoch': 0.29} 29%|██▊ | 519/1817 [8:13:00<21:06:32, 58.55s/it] 29%|██▊ | 520/1817 [8:14:00<21:11:13, 58.81s/it] {'loss': 1.9395, 'learning_rate': 2e-05, 'epoch': 0.29} 29%|██▊ | 520/1817 [8:14:00<21:11:13, 58.81s/it] 29%|██▊ | 521/1817 [8:14:56<20:55:01, 58.10s/it] {'loss': 1.9901, 'learning_rate': 2e-05, 'epoch': 0.29} 29%|██▊ | 521/1817 [8:14:56<20:55:01, 58.10s/it] 29%|██▊ | 522/1817 [8:15:52<20:41:43, 57.53s/it] {'loss': 1.9257, 'learning_rate': 2e-05, 'epoch': 0.29} 29%|██▊ | 522/1817 [8:15:52<20:41:43, 57.53s/it] 29%|██▉ | 523/1817 [8:16:49<20:32:52, 57.17s/it] {'loss': 1.929, 'learning_rate': 2e-05, 'epoch': 0.29} 29%|██▉ | 523/1817 [8:16:49<20:32:54, 57.17s/it] 29%|██▉ | 524/1817 [8:17:45<20:26:38, 56.92s/it] {'loss': 1.9361, 'learning_rate': 2e-05, 'epoch': 0.29} 29%|██▉ | 524/1817 [8:17:45<20:26:40, 56.92s/it] 29%|██▉ | 525/1817 [8:18:43<20:32:41, 57.25s/it] {'loss': 1.9683, 'learning_rate': 2e-05, 'epoch': 0.29} 29%|██▉ | 525/1817 [8:18:43<20:32:41, 57.25s/it] 29%|██▉ | 526/1817 [8:19:41<20:33:46, 57.34s/it] {'loss': 1.9278, 'learning_rate': 2e-05, 'epoch': 0.29} 29%|██▉ | 526/1817 [8:19:41<20:33:47, 57.34s/it] 29%|██▉ | 527/1817 [8:20:38<20:33:36, 57.38s/it] {'loss': 1.9796, 'learning_rate': 2e-05, 'epoch': 0.29} 29%|██▉ | 527/1817 [8:20:38<20:33:36, 57.38s/it] 29%|██▉ | 528/1817 [8:21:37<20:38:52, 57.67s/it] {'loss': 1.9741, 'learning_rate': 2e-05, 'epoch': 0.29} 29%|██▉ | 528/1817 [8:21:36<20:38:54, 57.67s/it] 29%|██▉ | 529/1817 [8:22:33<20:28:54, 57.25s/it] {'loss': 1.9637, 'learning_rate': 2e-05, 'epoch': 0.29} 29%|██▉ | 529/1817 [8:22:33<20:28:54, 57.25s/it] 29%|██▉ | 530/1817 [8:23:29<20:21:24, 56.94s/it] {'loss': 1.8992, 'learning_rate': 2e-05, 'epoch': 0.29} 29%|██▉ | 530/1817 [8:23:29<20:21:24, 56.94s/it] 29%|██▉ | 531/1817 [8:24:25<20:15:28, 56.71s/it] {'loss': 1.9234, 'learning_rate': 2e-05, 'epoch': 0.29} 29%|██▉ | 531/1817 [8:24:25<20:15:28, 56.71s/it] 29%|██▉ | 532/1817 [8:25:21<20:12:06, 56.60s/it] {'loss': 1.981, 'learning_rate': 2e-05, 'epoch': 0.29} 29%|██▉ | 532/1817 [8:25:21<20:12:06, 56.60s/it] 29%|██▉ | 533/1817 [8:26:20<20:26:08, 57.30s/it] {'loss': 1.9594, 'learning_rate': 2e-05, 'epoch': 0.29} 29%|██▉ | 533/1817 [8:26:20<20:26:08, 57.30s/it] 29%|██▉ | 534/1817 [8:27:17<20:19:39, 57.04s/it] {'loss': 1.8986, 'learning_rate': 2e-05, 'epoch': 0.29} 29%|██▉ | 534/1817 [8:27:17<20:19:40, 57.04s/it] 29%|██▉ | 535/1817 [8:28:13<20:14:03, 56.82s/it] {'loss': 1.9401, 'learning_rate': 2e-05, 'epoch': 0.29} 29%|██▉ | 535/1817 [8:28:13<20:14:03, 56.82s/it] 29%|██▉ | 536/1817 [8:29:08<20:01:56, 56.30s/it] {'loss': 1.9542, 'learning_rate': 2e-05, 'epoch': 0.29} 29%|██▉ | 536/1817 [8:29:08<20:01:57, 56.30s/it] 30%|██▉ | 537/1817 [8:30:05<20:04:10, 56.45s/it] {'loss': 1.9474, 'learning_rate': 2e-05, 'epoch': 0.3} 30%|██▉ | 537/1817 [8:30:05<20:04:10, 56.45s/it] 30%|██▉ | 538/1817 [8:31:01<20:01:57, 56.39s/it] {'loss': 1.9037, 'learning_rate': 2e-05, 'epoch': 0.3} 30%|██▉ | 538/1817 [8:31:01<20:01:57, 56.39s/it] 30%|██▉ | 539/1817 [8:31:59<20:09:04, 56.76s/it] {'loss': 1.9766, 'learning_rate': 2e-05, 'epoch': 0.3} 30%|██▉ | 539/1817 [8:31:59<20:09:05, 56.77s/it] 30%|██▉ | 540/1817 [8:32:55<20:05:02, 56.62s/it] {'loss': 1.9672, 'learning_rate': 2e-05, 'epoch': 0.3} 30%|██▉ | 540/1817 [8:32:55<20:05:03, 56.62s/it] 30%|██▉ | 541/1817 [8:33:55<20:20:14, 57.38s/it] {'loss': 1.9495, 'learning_rate': 2e-05, 'epoch': 0.3} 30%|██▉ | 541/1817 [8:33:54<20:20:16, 57.38s/it] 30%|██▉ | 542/1817 [8:34:51<20:11:30, 57.01s/it] {'loss': 1.9207, 'learning_rate': 2e-05, 'epoch': 0.3} 30%|██▉ | 542/1817 [8:34:50<20:11:31, 57.01s/it] 30%|██▉ | 543/1817 [8:35:47<20:04:38, 56.73s/it] {'loss': 1.9447, 'learning_rate': 2e-05, 'epoch': 0.3} 30%|██▉ | 543/1817 [8:35:47<20:04:39, 56.73s/it] 30%|██▉ | 544/1817 [8:36:44<20:06:32, 56.87s/it] {'loss': 1.9187, 'learning_rate': 2e-05, 'epoch': 0.3} 30%|██▉ | 544/1817 [8:36:44<20:06:32, 56.87s/it] 30%|██▉ | 545/1817 [8:37:40<20:02:43, 56.73s/it] {'loss': 1.9871, 'learning_rate': 2e-05, 'epoch': 0.3} 30%|██▉ | 545/1817 [8:37:40<20:02:44, 56.73s/it] 30%|███ | 546/1817 [8:38:36<19:53:43, 56.35s/it] {'loss': 1.9543, 'learning_rate': 2e-05, 'epoch': 0.3} 30%|███ | 546/1817 [8:38:36<19:53:43, 56.35s/it] 30%|███ | 547/1817 [8:39:33<20:02:16, 56.80s/it] {'loss': 1.9182, 'learning_rate': 2e-05, 'epoch': 0.3} 30%|███ | 547/1817 [8:39:33<20:02:16, 56.80s/it] 30%|███ | 548/1817 [8:40:30<19:59:15, 56.70s/it] {'loss': 1.9572, 'learning_rate': 2e-05, 'epoch': 0.3} 30%|███ | 548/1817 [8:40:30<19:59:15, 56.70s/it] 30%|███ | 549/1817 [8:41:27<19:55:51, 56.59s/it] {'loss': 1.9552, 'learning_rate': 2e-05, 'epoch': 0.3} 30%|███ | 549/1817 [8:41:26<19:55:51, 56.59s/it] 30%|███ | 550/1817 [8:42:24<19:59:35, 56.81s/it] {'loss': 1.9437, 'learning_rate': 2e-05, 'epoch': 0.3} 30%|███ | 550/1817 [8:42:24<19:59:35, 56.81s/it] 30%|███ | 551/1817 [8:43:20<19:56:11, 56.69s/it] {'loss': 1.9695, 'learning_rate': 2e-05, 'epoch': 0.3} 30%|███ | 551/1817 [8:43:20<19:56:11, 56.69s/it] 30%|███ | 552/1817 [8:44:20<20:12:36, 57.52s/it] {'loss': 1.9372, 'learning_rate': 2e-05, 'epoch': 0.3} 30%|███ | 552/1817 [8:44:19<20:12:36, 57.52s/it] 30%|███ | 553/1817 [8:45:20<20:26:55, 58.24s/it] {'loss': 1.981, 'learning_rate': 2e-05, 'epoch': 0.3} 30%|███ | 553/1817 [8:45:19<20:26:56, 58.24s/it] 30%|███ | 554/1817 [8:46:16<20:12:59, 57.62s/it] {'loss': 1.9582, 'learning_rate': 2e-05, 'epoch': 0.3} 30%|███ | 554/1817 [8:46:16<20:13:00, 57.62s/it] 31%|███ | 555/1817 [8:47:12<20:02:19, 57.16s/it] {'loss': 1.9143, 'learning_rate': 2e-05, 'epoch': 0.31} 31%|███ | 555/1817 [8:47:12<20:02:21, 57.16s/it] 31%|███ | 556/1817 [8:48:08<19:55:39, 56.89s/it] {'loss': 1.9377, 'learning_rate': 2e-05, 'epoch': 0.31} 31%|███ | 556/1817 [8:48:08<19:55:39, 56.89s/it] 31%|███ | 557/1817 [8:49:04<19:50:11, 56.68s/it] {'loss': 1.9588, 'learning_rate': 2e-05, 'epoch': 0.31} 31%|███ | 557/1817 [8:49:04<19:50:11, 56.68s/it] 31%|███ | 558/1817 [8:50:03<20:04:33, 57.41s/it] {'loss': 1.9394, 'learning_rate': 2e-05, 'epoch': 0.31} 31%|███ | 558/1817 [8:50:03<20:04:34, 57.41s/it] 31%|███ | 559/1817 [8:51:01<20:07:23, 57.59s/it] {'loss': 1.916, 'learning_rate': 2e-05, 'epoch': 0.31} 31%|███ | 559/1817 [8:51:01<20:07:23, 57.59s/it] 31%|███ | 560/1817 [8:52:00<20:13:02, 57.90s/it] {'loss': 1.9406, 'learning_rate': 2e-05, 'epoch': 0.31} 31%|███ | 560/1817 [8:52:00<20:13:02, 57.90s/it] 31%|███ | 561/1817 [8:52:58<20:13:25, 57.97s/it] {'loss': 1.965, 'learning_rate': 2e-05, 'epoch': 0.31} 31%|███ | 561/1817 [8:52:58<20:13:27, 57.97s/it] 31%|███ | 562/1817 [8:53:56<20:14:39, 58.07s/it] {'loss': 1.9003, 'learning_rate': 2e-05, 'epoch': 0.31} 31%|███ | 562/1817 [8:53:56<20:14:39, 58.07s/it] 31%|███ | 563/1817 [8:54:53<20:01:12, 57.47s/it] {'loss': 1.9493, 'learning_rate': 2e-05, 'epoch': 0.31} 31%|███ | 563/1817 [8:54:52<20:01:12, 57.47s/it] 31%|███ | 564/1817 [8:55:49<19:52:35, 57.11s/it] {'loss': 1.8993, 'learning_rate': 2e-05, 'epoch': 0.31} 31%|███ | 564/1817 [8:55:49<19:52:35, 57.11s/it] 31%|███ | 565/1817 [8:56:48<20:02:41, 57.64s/it] {'loss': 1.922, 'learning_rate': 2e-05, 'epoch': 0.31} 31%|███ | 565/1817 [8:56:47<20:02:41, 57.64s/it] 31%|███ | 566/1817 [8:57:44<19:53:16, 57.23s/it] {'loss': 1.9709, 'learning_rate': 2e-05, 'epoch': 0.31} 31%|███ | 566/1817 [8:57:44<19:53:17, 57.23s/it] 31%|███ | 567/1817 [8:58:40<19:47:54, 57.02s/it] {'loss': 1.95, 'learning_rate': 2e-05, 'epoch': 0.31} 31%|███ | 567/1817 [8:58:40<19:47:54, 57.02s/it] 31%|███▏ | 568/1817 [8:59:38<19:51:52, 57.26s/it] {'loss': 1.9532, 'learning_rate': 2e-05, 'epoch': 0.31} 31%|███▏ | 568/1817 [8:59:38<19:51:52, 57.26s/it] 31%|███▏ | 569/1817 [9:00:35<19:44:00, 56.92s/it] {'loss': 1.944, 'learning_rate': 2e-05, 'epoch': 0.31} 31%|███▏ | 569/1817 [9:00:34<19:44:00, 56.92s/it] 31%|███▏ | 570/1817 [9:01:31<19:40:00, 56.78s/it] {'loss': 1.9785, 'learning_rate': 2e-05, 'epoch': 0.31} 31%|███▏ | 570/1817 [9:01:31<19:40:00, 56.78s/it] 31%|███▏ | 571/1817 [9:02:27<19:31:48, 56.43s/it] {'loss': 1.9504, 'learning_rate': 2e-05, 'epoch': 0.31} 31%|███▏ | 571/1817 [9:02:26<19:31:48, 56.43s/it] 31%|███▏ | 572/1817 [9:03:24<19:38:25, 56.79s/it] {'loss': 1.9401, 'learning_rate': 2e-05, 'epoch': 0.31} 31%|███▏ | 572/1817 [9:03:24<19:38:25, 56.79s/it] 32%|███▏ | 573/1817 [9:04:20<19:33:15, 56.59s/it] {'loss': 1.9944, 'learning_rate': 2e-05, 'epoch': 0.32} 32%|███▏ | 573/1817 [9:04:20<19:33:15, 56.59s/it] 32%|███▏ | 574/1817 [9:05:16<19:29:41, 56.46s/it] {'loss': 1.9676, 'learning_rate': 2e-05, 'epoch': 0.32} 32%|███▏ | 574/1817 [9:05:16<19:29:41, 56.46s/it] 32%|███▏ | 575/1817 [9:06:13<19:27:48, 56.42s/it] {'loss': 1.9794, 'learning_rate': 2e-05, 'epoch': 0.32} 32%|███▏ | 575/1817 [9:06:13<19:27:48, 56.42s/it] 32%|███▏ | 576/1817 [9:07:09<19:24:44, 56.31s/it] {'loss': 1.9401, 'learning_rate': 2e-05, 'epoch': 0.32} 32%|███▏ | 576/1817 [9:07:09<19:24:44, 56.31s/it] 32%|███▏ | 577/1817 [9:08:06<19:31:51, 56.70s/it] {'loss': 1.9358, 'learning_rate': 2e-05, 'epoch': 0.32} 32%|███▏ | 577/1817 [9:08:06<19:31:51, 56.70s/it] 32%|███▏ | 578/1817 [9:09:03<19:27:26, 56.53s/it] {'loss': 1.9803, 'learning_rate': 2e-05, 'epoch': 0.32} 32%|███▏ | 578/1817 [9:09:02<19:27:28, 56.54s/it] 32%|███▏ | 579/1817 [9:10:00<19:29:47, 56.69s/it] {'loss': 1.9326, 'learning_rate': 2e-05, 'epoch': 0.32} 32%|███▏ | 579/1817 [9:09:59<19:29:47, 56.69s/it] 32%|███▏ | 580/1817 [9:10:56<19:25:52, 56.55s/it] {'loss': 1.9531, 'learning_rate': 2e-05, 'epoch': 0.32} 32%|███▏ | 580/1817 [9:10:56<19:25:52, 56.55s/it] 32%|███▏ | 581/1817 [9:11:56<19:47:58, 57.67s/it] {'loss': 1.9209, 'learning_rate': 2e-05, 'epoch': 0.32} 32%|███▏ | 581/1817 [9:11:56<19:47:58, 57.67s/it] 32%|███▏ | 582/1817 [9:12:53<19:38:43, 57.27s/it] {'loss': 1.9282, 'learning_rate': 2e-05, 'epoch': 0.32} 32%|███▏ | 582/1817 [9:12:52<19:38:46, 57.27s/it] 32%|███▏ | 583/1817 [9:13:50<19:41:09, 57.43s/it] {'loss': 1.9299, 'learning_rate': 2e-05, 'epoch': 0.32} 32%|███▏ | 583/1817 [9:13:50<19:41:09, 57.43s/it] 32%|███▏ | 584/1817 [9:14:49<19:50:08, 57.91s/it] {'loss': 1.9577, 'learning_rate': 2e-05, 'epoch': 0.32} 32%|███▏ | 584/1817 [9:14:49<19:50:10, 57.92s/it] 32%|███▏ | 585/1817 [9:15:45<19:38:46, 57.41s/it] {'loss': 1.9127, 'learning_rate': 2e-05, 'epoch': 0.32} 32%|███▏ | 585/1817 [9:15:45<19:38:46, 57.41s/it] 32%|███▏ | 586/1817 [9:16:44<19:43:04, 57.66s/it] {'loss': 1.9415, 'learning_rate': 2e-05, 'epoch': 0.32} 32%|███▏ | 586/1817 [9:16:44<19:43:04, 57.66s/it] 32%|███▏ | 587/1817 [9:17:41<19:40:28, 57.58s/it] {'loss': 1.8958, 'learning_rate': 2e-05, 'epoch': 0.32} 32%|███▏ | 587/1817 [9:17:41<19:40:28, 57.58s/it] 32%|███▏ | 588/1817 [9:18:37<19:30:21, 57.14s/it] {'loss': 1.944, 'learning_rate': 2e-05, 'epoch': 0.32} 32%|███▏ | 588/1817 [9:18:37<19:30:21, 57.14s/it] 32%|███▏ | 589/1817 [9:19:33<19:22:33, 56.80s/it] {'loss': 1.9644, 'learning_rate': 2e-05, 'epoch': 0.32} 32%|███▏ | 589/1817 [9:19:33<19:22:34, 56.80s/it] 32%|███▏ | 590/1817 [9:20:31<19:29:06, 57.17s/it] {'loss': 1.9193, 'learning_rate': 2e-05, 'epoch': 0.32} 32%|███▏ | 590/1817 [9:20:31<19:29:07, 57.17s/it] 33%|███▎ | 591/1817 [9:21:27<19:21:16, 56.83s/it] {'loss': 1.9441, 'learning_rate': 2e-05, 'epoch': 0.33} 33%|███▎ | 591/1817 [9:21:27<19:21:16, 56.83s/it] 33%|███▎ | 592/1817 [9:22:23<19:15:27, 56.59s/it] {'loss': 1.9332, 'learning_rate': 2e-05, 'epoch': 0.33} 33%|███▎ | 592/1817 [9:22:23<19:15:27, 56.59s/it] 33%|███▎ | 593/1817 [9:23:21<19:19:44, 56.85s/it] {'loss': 1.923, 'learning_rate': 2e-05, 'epoch': 0.33} 33%|███▎ | 593/1817 [9:23:21<19:19:45, 56.85s/it] 33%|███▎ | 594/1817 [9:24:17<19:14:24, 56.63s/it] {'loss': 1.9204, 'learning_rate': 2e-05, 'epoch': 0.33} 33%|███▎ | 594/1817 [9:24:17<19:14:24, 56.64s/it] 33%|███▎ | 595/1817 [9:25:13<19:10:45, 56.50s/it] {'loss': 1.9372, 'learning_rate': 2e-05, 'epoch': 0.33} 33%|███▎ | 595/1817 [9:25:13<19:10:45, 56.50s/it] 33%|███▎ | 596/1817 [9:26:07<18:55:04, 55.78s/it] {'loss': 1.9648, 'learning_rate': 2e-05, 'epoch': 0.33} 33%|███▎ | 596/1817 [9:26:07<18:55:04, 55.78s/it] 33%|███▎ | 597/1817 [9:27:05<19:04:54, 56.31s/it] {'loss': 1.9093, 'learning_rate': 2e-05, 'epoch': 0.33} 33%|███▎ | 597/1817 [9:27:05<19:04:54, 56.31s/it] 33%|███▎ | 598/1817 [9:28:01<19:02:53, 56.25s/it] {'loss': 1.9304, 'learning_rate': 2e-05, 'epoch': 0.33} 33%|███▎ | 598/1817 [9:28:01<19:02:53, 56.25s/it] 33%|███▎ | 599/1817 [9:28:57<19:01:34, 56.23s/it] {'loss': 1.9618, 'learning_rate': 2e-05, 'epoch': 0.33} 33%|███▎ | 599/1817 [9:28:57<19:01:35, 56.24s/it] 33%|███▎ | 600/1817 [9:29:52<18:54:31, 55.93s/it] {'loss': 1.9417, 'learning_rate': 2e-05, 'epoch': 0.33} 33%|███▎ | 600/1817 [9:29:52<18:54:31, 55.93s/it] 33%|███▎ | 601/1817 [9:30:48<18:55:08, 56.01s/it] {'loss': 1.931, 'learning_rate': 2e-05, 'epoch': 0.33} 33%|███▎ | 601/1817 [9:30:48<18:55:08, 56.01s/it] 33%|███▎ | 602/1817 [9:31:47<19:08:34, 56.72s/it] {'loss': 1.9415, 'learning_rate': 2e-05, 'epoch': 0.33} 33%|███▎ | 602/1817 [9:31:47<19:08:35, 56.72s/it] 33%|███▎ | 603/1817 [9:32:43<19:04:24, 56.56s/it] {'loss': 1.9105, 'learning_rate': 2e-05, 'epoch': 0.33} 33%|███▎ | 603/1817 [9:32:43<19:04:25, 56.56s/it] 33%|███▎ | 604/1817 [9:33:38<18:56:44, 56.23s/it] {'loss': 1.9754, 'learning_rate': 2e-05, 'epoch': 0.33} 33%|███▎ | 604/1817 [9:33:38<18:56:44, 56.23s/it] 33%|███▎ | 605/1817 [9:34:34<18:54:47, 56.18s/it] {'loss': 1.9344, 'learning_rate': 2e-05, 'epoch': 0.33} 33%|███▎ | 605/1817 [9:34:34<18:54:47, 56.18s/it] 33%|███▎ | 606/1817 [9:35:33<19:06:52, 56.82s/it] {'loss': 1.9926, 'learning_rate': 2e-05, 'epoch': 0.33} 33%|███▎ | 606/1817 [9:35:33<19:06:52, 56.82s/it] 33%|███▎ | 607/1817 [9:36:33<19:22:23, 57.64s/it] {'loss': 1.9162, 'learning_rate': 2e-05, 'epoch': 0.33} 33%|███▎ | 607/1817 [9:36:32<19:22:24, 57.64s/it] 33%|███▎ | 608/1817 [9:37:29<19:12:55, 57.22s/it] {'loss': 1.9197, 'learning_rate': 2e-05, 'epoch': 0.33} 33%|███▎ | 608/1817 [9:37:29<19:12:56, 57.22s/it] 34%|███▎ | 609/1817 [9:38:25<19:05:29, 56.90s/it] {'loss': 1.9075, 'learning_rate': 2e-05, 'epoch': 0.34} 34%|███▎ | 609/1817 [9:38:25<19:05:29, 56.90s/it] 34%|███▎ | 610/1817 [9:39:22<19:09:12, 57.13s/it] {'loss': 1.9567, 'learning_rate': 2e-05, 'epoch': 0.34} 34%|███▎ | 610/1817 [9:39:22<19:09:12, 57.13s/it] 34%|███▎ | 611/1817 [9:40:18<19:02:13, 56.83s/it] {'loss': 1.9178, 'learning_rate': 2e-05, 'epoch': 0.34} 34%|███▎ | 611/1817 [9:40:18<19:02:13, 56.83s/it] 34%|███▎ | 612/1817 [9:41:15<18:57:46, 56.65s/it] {'loss': 1.924, 'learning_rate': 2e-05, 'epoch': 0.34} 34%|███▎ | 612/1817 [9:41:15<18:57:46, 56.65s/it] 34%|███▎ | 613/1817 [9:42:11<18:54:17, 56.53s/it] {'loss': 1.9683, 'learning_rate': 2e-05, 'epoch': 0.34} 34%|███▎ | 613/1817 [9:42:11<18:54:17, 56.53s/it] 34%|███▍ | 614/1817 [9:43:08<18:59:25, 56.83s/it] {'loss': 1.9442, 'learning_rate': 2e-05, 'epoch': 0.34} 34%|███▍ | 614/1817 [9:43:08<18:59:25, 56.83s/it] 34%|███▍ | 615/1817 [9:44:07<19:06:02, 57.21s/it] {'loss': 1.956, 'learning_rate': 2e-05, 'epoch': 0.34} 34%|███▍ | 615/1817 [9:44:07<19:06:02, 57.21s/it] 34%|███▍ | 616/1817 [9:45:03<18:59:10, 56.91s/it] {'loss': 1.9379, 'learning_rate': 2e-05, 'epoch': 0.34} 34%|███▍ | 616/1817 [9:45:03<18:59:12, 56.91s/it] 34%|███▍ | 617/1817 [9:46:01<19:02:32, 57.13s/it] {'loss': 1.9298, 'learning_rate': 2e-05, 'epoch': 0.34} 34%|███▍ | 617/1817 [9:46:00<19:02:32, 57.13s/it] 34%|███▍ | 618/1817 [9:46:57<18:56:31, 56.87s/it] {'loss': 1.9026, 'learning_rate': 2e-05, 'epoch': 0.34} 34%|███▍ | 618/1817 [9:46:57<18:56:31, 56.87s/it] 34%|███▍ | 619/1817 [9:47:53<18:51:53, 56.69s/it] {'loss': 1.9376, 'learning_rate': 2e-05, 'epoch': 0.34} 34%|███▍ | 619/1817 [9:47:53<18:51:53, 56.69s/it] 34%|███▍ | 620/1817 [9:48:51<18:57:07, 57.00s/it] {'loss': 1.9402, 'learning_rate': 2e-05, 'epoch': 0.34} 34%|███▍ | 620/1817 [9:48:51<18:57:07, 57.00s/it] 34%|███▍ | 621/1817 [9:49:50<19:07:28, 57.57s/it] {'loss': 1.8958, 'learning_rate': 2e-05, 'epoch': 0.34} 34%|███▍ | 621/1817 [9:49:50<19:07:29, 57.57s/it] 34%|███▍ | 622/1817 [9:50:49<19:15:21, 58.01s/it] {'loss': 1.9629, 'learning_rate': 2e-05, 'epoch': 0.34} 34%|███▍ | 622/1817 [9:50:49<19:15:22, 58.01s/it] 34%|███▍ | 623/1817 [9:51:49<19:27:42, 58.68s/it] {'loss': 1.881, 'learning_rate': 2e-05, 'epoch': 0.34} 34%|███▍ | 623/1817 [9:51:49<19:27:42, 58.68s/it] 34%|███▍ | 624/1817 [9:52:50<19:38:45, 59.28s/it] {'loss': 1.9392, 'learning_rate': 2e-05, 'epoch': 0.34} 34%|███▍ | 624/1817 [9:52:50<19:38:45, 59.28s/it] 34%|███▍ | 625/1817 [9:53:47<19:26:26, 58.71s/it] {'loss': 1.9175, 'learning_rate': 2e-05, 'epoch': 0.34} 34%|███▍ | 625/1817 [9:53:47<19:26:26, 58.71s/it] 34%|███▍ | 626/1817 [9:54:43<19:10:02, 57.94s/it] {'loss': 1.9691, 'learning_rate': 2e-05, 'epoch': 0.34} 34%|███▍ | 626/1817 [9:54:43<19:10:02, 57.94s/it] 35%|███▍ | 627/1817 [9:55:40<19:05:42, 57.77s/it] {'loss': 1.9418, 'learning_rate': 2e-05, 'epoch': 0.34} 35%|███▍ | 627/1817 [9:55:40<19:05:42, 57.77s/it] 35%|███▍ | 628/1817 [9:56:37<18:55:06, 57.28s/it] {'loss': 1.9584, 'learning_rate': 2e-05, 'epoch': 0.35} 35%|███▍ | 628/1817 [9:56:37<18:55:06, 57.28s/it] 35%|███▍ | 629/1817 [9:57:33<18:51:31, 57.15s/it] {'loss': 1.9335, 'learning_rate': 2e-05, 'epoch': 0.35} 35%|███▍ | 629/1817 [9:57:33<18:51:31, 57.15s/it] 35%|███▍ | 630/1817 [9:58:31<18:52:52, 57.26s/it] {'loss': 1.9732, 'learning_rate': 2e-05, 'epoch': 0.35} 35%|███▍ | 630/1817 [9:58:31<18:52:52, 57.26s/it] 35%|███▍ | 631/1817 [9:59:24<18:29:21, 56.12s/it] {'loss': 1.9384, 'learning_rate': 2e-05, 'epoch': 0.35} 35%|███▍ | 631/1817 [9:59:24<18:29:21, 56.12s/it] 35%|███▍ | 632/1817 [10:00:21<18:29:06, 56.16s/it] {'loss': 1.9194, 'learning_rate': 2e-05, 'epoch': 0.35} 35%|███▍ | 632/1817 [10:00:21<18:29:06, 56.16s/it] 35%|███▍ | 633/1817 [10:01:19<18:40:29, 56.78s/it] {'loss': 1.9632, 'learning_rate': 2e-05, 'epoch': 0.35} 35%|███▍ | 633/1817 [10:01:19<18:40:29, 56.78s/it] 35%|███▍ | 634/1817 [10:02:15<18:34:21, 56.52s/it] {'loss': 1.9055, 'learning_rate': 2e-05, 'epoch': 0.35} 35%|███▍ | 634/1817 [10:02:15<18:34:21, 56.52s/it] 35%|███▍ | 635/1817 [10:03:11<18:31:48, 56.44s/it] {'loss': 1.9291, 'learning_rate': 2e-05, 'epoch': 0.35} 35%|███▍ | 635/1817 [10:03:11<18:31:48, 56.44s/it] 35%|███▌ | 636/1817 [10:04:06<18:22:19, 56.00s/it] {'loss': 1.9379, 'learning_rate': 2e-05, 'epoch': 0.35} 35%|███▌ | 636/1817 [10:04:06<18:22:19, 56.00s/it] 35%|███▌ | 637/1817 [10:05:03<18:22:37, 56.07s/it] {'loss': 1.9464, 'learning_rate': 2e-05, 'epoch': 0.35} 35%|███▌ | 637/1817 [10:05:02<18:22:35, 56.06s/it] 35%|███▌ | 638/1817 [10:06:00<18:32:12, 56.60s/it] {'loss': 1.9341, 'learning_rate': 2e-05, 'epoch': 0.35} 35%|███▌ | 638/1817 [10:06:00<18:32:11, 56.60s/it] 35%|███▌ | 639/1817 [10:06:57<18:35:54, 56.84s/it] {'loss': 1.9892, 'learning_rate': 2e-05, 'epoch': 0.35} 35%|███▌ | 639/1817 [10:06:57<18:35:54, 56.84s/it] 35%|███▌ | 640/1817 [10:07:57<18:49:21, 57.57s/it] {'loss': 1.9599, 'learning_rate': 2e-05, 'epoch': 0.35} 35%|███▌ | 640/1817 [10:07:57<18:49:20, 57.57s/it] 35%|███▌ | 641/1817 [10:08:54<18:47:08, 57.51s/it] {'loss': 1.941, 'learning_rate': 2e-05, 'epoch': 0.35} 35%|███▌ | 641/1817 [10:08:54<18:47:09, 57.51s/it] 35%|███▌ | 642/1817 [10:09:53<18:54:42, 57.94s/it] {'loss': 1.9584, 'learning_rate': 2e-05, 'epoch': 0.35} 35%|███▌ | 642/1817 [10:09:53<18:54:42, 57.94s/it] 35%|███▌ | 643/1817 [10:10:49<18:39:25, 57.21s/it] {'loss': 1.9297, 'learning_rate': 2e-05, 'epoch': 0.35} 35%|███▌ | 643/1817 [10:10:49<18:39:25, 57.21s/it] 35%|███▌ | 644/1817 [10:11:47<18:43:18, 57.46s/it] {'loss': 1.92, 'learning_rate': 2e-05, 'epoch': 0.35} 35%|███▌ | 644/1817 [10:11:47<18:43:18, 57.46s/it] 35%|███▌ | 645/1817 [10:12:43<18:36:20, 57.15s/it] {'loss': 1.9706, 'learning_rate': 2e-05, 'epoch': 0.35} 35%|███▌ | 645/1817 [10:12:43<18:36:20, 57.15s/it] 36%|███▌ | 646/1817 [10:13:39<18:30:49, 56.92s/it] {'loss': 1.9571, 'learning_rate': 2e-05, 'epoch': 0.36} 36%|███▌ | 646/1817 [10:13:39<18:30:49, 56.92s/it] 36%|███▌ | 647/1817 [10:14:40<18:51:53, 58.05s/it] {'loss': 1.9443, 'learning_rate': 2e-05, 'epoch': 0.36} 36%|███▌ | 647/1817 [10:14:40<18:51:53, 58.05s/it] 36%|███▌ | 648/1817 [10:15:37<18:42:38, 57.62s/it] {'loss': 1.9559, 'learning_rate': 2e-05, 'epoch': 0.36} 36%|███▌ | 648/1817 [10:15:37<18:42:38, 57.62s/it] 36%|███▌ | 649/1817 [10:16:35<18:42:53, 57.68s/it] {'loss': 1.9282, 'learning_rate': 2e-05, 'epoch': 0.36} 36%|███▌ | 649/1817 [10:16:35<18:42:53, 57.68s/it] 36%|███▌ | 650/1817 [10:17:34<18:50:37, 58.13s/it] {'loss': 1.9509, 'learning_rate': 2e-05, 'epoch': 0.36} 36%|███▌ | 650/1817 [10:17:34<18:50:37, 58.13s/it] 36%|███▌ | 651/1817 [10:18:30<18:39:24, 57.60s/it] {'loss': 1.8683, 'learning_rate': 2e-05, 'epoch': 0.36} 36%|███▌ | 651/1817 [10:18:30<18:39:24, 57.60s/it] 36%|███▌ | 652/1817 [10:19:28<18:42:32, 57.81s/it] {'loss': 1.9268, 'learning_rate': 2e-05, 'epoch': 0.36} 36%|███▌ | 652/1817 [10:19:28<18:42:32, 57.81s/it] 36%|███▌ | 653/1817 [10:20:24<18:24:55, 56.95s/it] {'loss': 1.9242, 'learning_rate': 2e-05, 'epoch': 0.36} 36%|███▌ | 653/1817 [10:20:23<18:24:55, 56.95s/it] 36%|███▌ | 654/1817 [10:21:20<18:21:20, 56.82s/it] {'loss': 1.9395, 'learning_rate': 2e-05, 'epoch': 0.36} 36%|███▌ | 654/1817 [10:21:20<18:21:20, 56.82s/it] 36%|███▌ | 655/1817 [10:22:16<18:15:51, 56.58s/it] {'loss': 1.9114, 'learning_rate': 2e-05, 'epoch': 0.36} 36%|███▌ | 655/1817 [10:22:16<18:15:51, 56.58s/it] 36%|███▌ | 656/1817 [10:23:12<18:12:11, 56.44s/it] {'loss': 1.9505, 'learning_rate': 2e-05, 'epoch': 0.36} 36%|███▌ | 656/1817 [10:23:12<18:12:11, 56.44s/it] 36%|███▌ | 657/1817 [10:24:10<18:19:04, 56.85s/it] {'loss': 1.951, 'learning_rate': 2e-05, 'epoch': 0.36} 36%|███▌ | 657/1817 [10:24:10<18:19:04, 56.85s/it] 36%|███▌ | 658/1817 [10:25:10<18:33:41, 57.65s/it] {'loss': 1.9316, 'learning_rate': 2e-05, 'epoch': 0.36} 36%|███▌ | 658/1817 [10:25:09<18:33:41, 57.65s/it] 36%|███▋ | 659/1817 [10:26:06<18:24:56, 57.25s/it] {'loss': 1.9314, 'learning_rate': 2e-05, 'epoch': 0.36} 36%|███▋ | 659/1817 [10:26:06<18:24:56, 57.25s/it] 36%|███▋ | 660/1817 [10:27:02<18:18:27, 56.96s/it] {'loss': 1.9689, 'learning_rate': 2e-05, 'epoch': 0.36} 36%|███▋ | 660/1817 [10:27:02<18:18:27, 56.96s/it] 36%|███▋ | 661/1817 [10:28:00<18:22:15, 57.21s/it] {'loss': 1.966, 'learning_rate': 2e-05, 'epoch': 0.36} 36%|███▋ | 661/1817 [10:28:00<18:22:15, 57.21s/it] 36%|███▋ | 662/1817 [10:28:59<18:28:59, 57.61s/it] {'loss': 1.9217, 'learning_rate': 2e-05, 'epoch': 0.36} 36%|███▋ | 662/1817 [10:28:58<18:28:59, 57.61s/it] 36%|███▋ | 663/1817 [10:29:55<18:21:33, 57.27s/it] {'loss': 1.996, 'learning_rate': 2e-05, 'epoch': 0.36} 36%|███▋ | 663/1817 [10:29:55<18:21:33, 57.27s/it] 37%|███▋ | 664/1817 [10:30:51<18:14:36, 56.96s/it] {'loss': 1.9476, 'learning_rate': 2e-05, 'epoch': 0.37} 37%|███▋ | 664/1817 [10:30:51<18:14:36, 56.96s/it] 37%|███▋ | 665/1817 [10:31:49<18:17:11, 57.15s/it] {'loss': 1.9128, 'learning_rate': 2e-05, 'epoch': 0.37} 37%|███▋ | 665/1817 [10:31:49<18:17:11, 57.15s/it] 37%|███▋ | 666/1817 [10:32:45<18:11:29, 56.90s/it] {'loss': 1.9469, 'learning_rate': 2e-05, 'epoch': 0.37} 37%|███▋ | 666/1817 [10:32:45<18:11:29, 56.90s/it] 37%|███▋ | 667/1817 [10:33:43<18:14:19, 57.09s/it] {'loss': 1.9574, 'learning_rate': 2e-05, 'epoch': 0.37} 37%|███▋ | 667/1817 [10:33:42<18:14:19, 57.09s/it] 37%|███▋ | 668/1817 [10:34:40<18:17:29, 57.31s/it] {'loss': 1.9143, 'learning_rate': 2e-05, 'epoch': 0.37} 37%|███▋ | 668/1817 [10:34:40<18:17:29, 57.31s/it] 37%|███▋ | 669/1817 [10:35:37<18:11:01, 57.02s/it] {'loss': 1.9687, 'learning_rate': 2e-05, 'epoch': 0.37} 37%|███▋ | 669/1817 [10:35:37<18:11:01, 57.02s/it] 37%|███▋ | 670/1817 [10:36:34<18:12:33, 57.15s/it] {'loss': 1.9195, 'learning_rate': 2e-05, 'epoch': 0.37} 37%|███▋ | 670/1817 [10:36:34<18:12:33, 57.15s/it] 37%|███▋ | 671/1817 [10:37:32<18:13:21, 57.24s/it] {'loss': 1.9316, 'learning_rate': 2e-05, 'epoch': 0.37} 37%|███▋ | 671/1817 [10:37:32<18:13:21, 57.24s/it] 37%|███▋ | 672/1817 [10:38:28<18:07:16, 56.98s/it] {'loss': 1.9479, 'learning_rate': 2e-05, 'epoch': 0.37} 37%|███▋ | 672/1817 [10:38:28<18:07:16, 56.97s/it] 37%|███▋ | 673/1817 [10:39:25<18:05:02, 56.91s/it] {'loss': 1.9027, 'learning_rate': 2e-05, 'epoch': 0.37} 37%|███▋ | 673/1817 [10:39:25<18:05:02, 56.91s/it] 37%|███▋ | 674/1817 [10:40:21<17:59:52, 56.69s/it] {'loss': 1.9081, 'learning_rate': 2e-05, 'epoch': 0.37} 37%|███▋ | 674/1817 [10:40:21<17:59:52, 56.69s/it] 37%|███▋ | 675/1817 [10:41:17<17:56:30, 56.56s/it] {'loss': 1.9158, 'learning_rate': 2e-05, 'epoch': 0.37} 37%|███▋ | 675/1817 [10:41:17<17:56:30, 56.56s/it] 37%|███▋ | 676/1817 [10:42:13<17:53:00, 56.42s/it] {'loss': 1.9128, 'learning_rate': 2e-05, 'epoch': 0.37} 37%|███▋ | 676/1817 [10:42:13<17:53:00, 56.42s/it] 37%|███▋ | 677/1817 [10:43:10<17:55:33, 56.61s/it] {'loss': 1.9617, 'learning_rate': 2e-05, 'epoch': 0.37} 37%|███▋ | 677/1817 [10:43:10<17:55:33, 56.61s/it] 37%|███▋ | 678/1817 [10:44:09<18:03:11, 57.06s/it] {'loss': 1.9618, 'learning_rate': 2e-05, 'epoch': 0.37} 37%|███▋ | 678/1817 [10:44:08<18:03:11, 57.06s/it] 37%|███▋ | 679/1817 [10:45:06<18:03:48, 57.14s/it] {'loss': 1.9527, 'learning_rate': 2e-05, 'epoch': 0.37} 37%|███▋ | 679/1817 [10:45:06<18:03:48, 57.14s/it] 37%|███▋ | 680/1817 [10:46:04<18:07:54, 57.41s/it] {'loss': 1.9377, 'learning_rate': 2e-05, 'epoch': 0.37} 37%|███▋ | 680/1817 [10:46:04<18:07:54, 57.41s/it] 37%|███▋ | 681/1817 [10:47:02<18:08:04, 57.47s/it] {'loss': 1.9465, 'learning_rate': 2e-05, 'epoch': 0.37} 37%|███▋ | 681/1817 [10:47:01<18:08:04, 57.47s/it] 38%|███▊ | 682/1817 [10:47:59<18:07:30, 57.49s/it] {'loss': 1.9044, 'learning_rate': 2e-05, 'epoch': 0.38} 38%|███▊ | 682/1817 [10:47:59<18:07:30, 57.49s/it] 38%|███▊ | 683/1817 [10:48:57<18:09:23, 57.64s/it] {'loss': 1.9386, 'learning_rate': 2e-05, 'epoch': 0.38} 38%|███▊ | 683/1817 [10:48:57<18:09:23, 57.64s/it] 38%|███▊ | 684/1817 [10:49:53<18:01:03, 57.25s/it] {'loss': 1.9692, 'learning_rate': 2e-05, 'epoch': 0.38} 38%|███▊ | 684/1817 [10:49:53<18:01:03, 57.25s/it] 38%|███▊ | 685/1817 [10:50:50<17:55:43, 57.02s/it] {'loss': 1.9241, 'learning_rate': 2e-05, 'epoch': 0.38} 38%|███▊ | 685/1817 [10:50:50<17:55:43, 57.02s/it] 38%|███▊ | 686/1817 [10:51:46<17:49:34, 56.74s/it] {'loss': 1.9492, 'learning_rate': 2e-05, 'epoch': 0.38} 38%|███▊ | 686/1817 [10:51:46<17:49:34, 56.74s/it] 38%|███▊ | 687/1817 [10:52:42<17:45:32, 56.58s/it] {'loss': 1.9042, 'learning_rate': 2e-05, 'epoch': 0.38} 38%|███▊ | 687/1817 [10:52:42<17:45:32, 56.58s/it] 38%|███▊ | 688/1817 [10:53:38<17:42:16, 56.45s/it] {'loss': 1.9589, 'learning_rate': 2e-05, 'epoch': 0.38} 38%|███▊ | 688/1817 [10:53:38<17:42:16, 56.45s/it] 38%|███▊ | 689/1817 [10:54:35<17:39:43, 56.37s/it] {'loss': 1.9203, 'learning_rate': 2e-05, 'epoch': 0.38} 38%|███▊ | 689/1817 [10:54:34<17:39:43, 56.37s/it] 38%|███▊ | 690/1817 [10:55:31<17:39:08, 56.39s/it] {'loss': 1.9148, 'learning_rate': 2e-05, 'epoch': 0.38} 38%|███▊ | 690/1817 [10:55:31<17:39:08, 56.39s/it] 38%|███▊ | 691/1817 [10:56:29<17:46:57, 56.85s/it] {'loss': 1.9187, 'learning_rate': 2e-05, 'epoch': 0.38} 38%|███▊ | 691/1817 [10:56:29<17:46:57, 56.85s/it] 38%|███▊ | 692/1817 [10:57:25<17:42:27, 56.66s/it] {'loss': 1.9215, 'learning_rate': 2e-05, 'epoch': 0.38} 38%|███▊ | 692/1817 [10:57:25<17:42:27, 56.66s/it] 38%|███▊ | 693/1817 [10:58:21<17:39:04, 56.53s/it] {'loss': 1.9447, 'learning_rate': 2e-05, 'epoch': 0.38} 38%|███▊ | 693/1817 [10:58:21<17:39:04, 56.53s/it] 38%|███▊ | 694/1817 [10:59:18<17:36:04, 56.42s/it] {'loss': 1.9066, 'learning_rate': 2e-05, 'epoch': 0.38} 38%|███▊ | 694/1817 [10:59:17<17:36:04, 56.42s/it] 38%|███▊ | 695/1817 [11:00:14<17:33:54, 56.36s/it] {'loss': 1.9469, 'learning_rate': 2e-05, 'epoch': 0.38} 38%|███▊ | 695/1817 [11:00:13<17:33:54, 56.36s/it] 38%|███▊ | 696/1817 [11:01:07<17:17:58, 55.56s/it] {'loss': 1.9203, 'learning_rate': 2e-05, 'epoch': 0.38} 38%|███▊ | 696/1817 [11:01:07<17:17:58, 55.56s/it] 38%|███▊ | 697/1817 [11:02:04<17:20:59, 55.77s/it] {'loss': 1.9118, 'learning_rate': 2e-05, 'epoch': 0.38} 38%|███▊ | 697/1817 [11:02:03<17:21:00, 55.77s/it] 38%|███▊ | 698/1817 [11:03:00<17:23:09, 55.93s/it] {'loss': 1.9113, 'learning_rate': 2e-05, 'epoch': 0.38} 38%|███▊ | 698/1817 [11:03:00<17:23:06, 55.93s/it] 38%|███▊ | 699/1817 [11:03:56<17:24:38, 56.06s/it] {'loss': 1.9048, 'learning_rate': 2e-05, 'epoch': 0.38} 38%|███▊ | 699/1817 [11:03:56<17:24:36, 56.06s/it] 39%|███▊ | 700/1817 [11:04:54<17:33:00, 56.56s/it] {'loss': 1.9099, 'learning_rate': 2e-05, 'epoch': 0.39} 39%|███▊ | 700/1817 [11:04:54<17:32:59, 56.56s/it] 39%|███▊ | 701/1817 [11:05:52<17:37:35, 56.86s/it] {'loss': 1.9226, 'learning_rate': 2e-05, 'epoch': 0.39} 39%|███▊ | 701/1817 [11:05:51<17:37:34, 56.86s/it] 39%|███▊ | 702/1817 [11:06:48<17:33:32, 56.69s/it] {'loss': 1.9015, 'learning_rate': 2e-05, 'epoch': 0.39} 39%|███▊ | 702/1817 [11:06:48<17:33:31, 56.69s/it] 39%|███▊ | 703/1817 [11:07:44<17:30:35, 56.58s/it] {'loss': 1.9565, 'learning_rate': 2e-05, 'epoch': 0.39} 39%|███▊ | 703/1817 [11:07:44<17:30:34, 56.58s/it] 39%|███▊ | 704/1817 [11:08:43<17:43:50, 57.35s/it] {'loss': 1.9305, 'learning_rate': 2e-05, 'epoch': 0.39} 39%|███▊ | 704/1817 [11:08:43<17:43:50, 57.35s/it] 39%|███▉ | 705/1817 [11:09:40<17:36:49, 57.02s/it] {'loss': 1.9354, 'learning_rate': 2e-05, 'epoch': 0.39} 39%|███▉ | 705/1817 [11:09:39<17:36:49, 57.02s/it] 39%|███▉ | 706/1817 [11:10:36<17:30:57, 56.76s/it] {'loss': 1.9558, 'learning_rate': 2e-05, 'epoch': 0.39} 39%|███▉ | 706/1817 [11:10:36<17:30:57, 56.76s/it] 39%|███▉ | 707/1817 [11:11:35<17:41:25, 57.37s/it] {'loss': 1.9163, 'learning_rate': 2e-05, 'epoch': 0.39} 39%|███▉ | 707/1817 [11:11:34<17:41:25, 57.37s/it] 39%|███▉ | 708/1817 [11:12:30<17:31:35, 56.89s/it] {'loss': 1.9538, 'learning_rate': 2e-05, 'epoch': 0.39} 39%|███▉ | 708/1817 [11:12:30<17:31:35, 56.89s/it] 39%|███▉ | 709/1817 [11:13:29<17:38:32, 57.32s/it] {'loss': 1.9411, 'learning_rate': 2e-05, 'epoch': 0.39} 39%|███▉ | 709/1817 [11:13:28<17:38:32, 57.32s/it] 39%|███▉ | 710/1817 [11:14:27<17:40:34, 57.48s/it] {'loss': 1.9448, 'learning_rate': 2e-05, 'epoch': 0.39} 39%|███▉ | 710/1817 [11:14:26<17:40:34, 57.48s/it] 39%|███▉ | 711/1817 [11:15:23<17:32:02, 57.07s/it] {'loss': 1.9153, 'learning_rate': 2e-05, 'epoch': 0.39} 39%|███▉ | 711/1817 [11:15:22<17:32:02, 57.07s/it] 39%|███▉ | 712/1817 [11:16:22<17:42:59, 57.72s/it] {'loss': 1.9135, 'learning_rate': 2e-05, 'epoch': 0.39} 39%|███▉ | 712/1817 [11:16:22<17:42:59, 57.72s/it] 39%|███▉ | 713/1817 [11:17:18<17:34:16, 57.30s/it] {'loss': 1.914, 'learning_rate': 2e-05, 'epoch': 0.39} 39%|███▉ | 713/1817 [11:17:18<17:34:16, 57.30s/it] 39%|███▉ | 714/1817 [11:18:17<17:38:56, 57.60s/it] {'loss': 1.9263, 'learning_rate': 2e-05, 'epoch': 0.39} 39%|███▉ | 714/1817 [11:18:16<17:38:56, 57.60s/it] 39%|███▉ | 715/1817 [11:19:12<17:30:07, 57.18s/it] {'loss': 1.9462, 'learning_rate': 2e-05, 'epoch': 0.39} 39%|███▉ | 715/1817 [11:19:12<17:30:07, 57.18s/it] 39%|███▉ | 716/1817 [11:20:09<17:25:49, 56.99s/it] {'loss': 1.9481, 'learning_rate': 2e-05, 'epoch': 0.39} 39%|███▉ | 716/1817 [11:20:09<17:25:49, 56.99s/it] 39%|███▉ | 717/1817 [11:21:06<17:20:30, 56.76s/it] {'loss': 1.9155, 'learning_rate': 2e-05, 'epoch': 0.39} 39%|███▉ | 717/1817 [11:21:05<17:20:31, 56.76s/it] 40%|███▉ | 718/1817 [11:22:03<17:23:25, 56.97s/it] {'loss': 1.8248, 'learning_rate': 2e-05, 'epoch': 0.4} 40%|███▉ | 718/1817 [11:22:03<17:23:25, 56.97s/it] 40%|███▉ | 719/1817 [11:22:59<17:18:42, 56.76s/it] {'loss': 1.9709, 'learning_rate': 2e-05, 'epoch': 0.4} 40%|███▉ | 719/1817 [11:22:59<17:18:42, 56.76s/it] 40%|███▉ | 720/1817 [11:23:58<17:27:15, 57.28s/it] {'loss': 1.9812, 'learning_rate': 2e-05, 'epoch': 0.4} 40%|███▉ | 720/1817 [11:23:57<17:27:15, 57.28s/it] 40%|███▉ | 721/1817 [11:24:55<17:27:07, 57.32s/it] {'loss': 1.9349, 'learning_rate': 2e-05, 'epoch': 0.4} 40%|███▉ | 721/1817 [11:24:55<17:27:07, 57.32s/it] 40%|███▉ | 722/1817 [11:25:55<17:38:53, 58.02s/it] {'loss': 1.9288, 'learning_rate': 2e-05, 'epoch': 0.4} 40%|███▉ | 722/1817 [11:25:55<17:38:53, 58.02s/it] 40%|███▉ | 723/1817 [11:26:51<17:28:14, 57.49s/it] {'loss': 1.9478, 'learning_rate': 2e-05, 'epoch': 0.4} 40%|███▉ | 723/1817 [11:26:51<17:28:14, 57.49s/it] 40%|███▉ | 724/1817 [11:27:45<17:09:35, 56.52s/it] {'loss': 1.9532, 'learning_rate': 2e-05, 'epoch': 0.4} 40%|███▉ | 724/1817 [11:27:45<17:09:35, 56.52s/it] 40%|███▉ | 725/1817 [11:28:42<17:08:51, 56.53s/it] {'loss': 1.9314, 'learning_rate': 2e-05, 'epoch': 0.4} 40%|███▉ | 725/1817 [11:28:42<17:08:51, 56.53s/it] 40%|███▉ | 726/1817 [11:29:40<17:15:33, 56.95s/it] {'loss': 1.937, 'learning_rate': 2e-05, 'epoch': 0.4} 40%|███▉ | 726/1817 [11:29:40<17:15:33, 56.95s/it] 40%|████ | 727/1817 [11:30:36<17:11:43, 56.79s/it] {'loss': 1.8968, 'learning_rate': 2e-05, 'epoch': 0.4} 40%|████ | 727/1817 [11:30:36<17:11:43, 56.79s/it] 40%|████ | 728/1817 [11:31:32<17:06:54, 56.58s/it] {'loss': 1.9113, 'learning_rate': 2e-05, 'epoch': 0.4} 40%|████ | 728/1817 [11:31:32<17:06:54, 56.58s/it] 40%|████ | 729/1817 [11:32:29<17:04:25, 56.49s/it] {'loss': 1.8974, 'learning_rate': 2e-05, 'epoch': 0.4} 40%|████ | 729/1817 [11:32:28<17:04:25, 56.49s/it] 40%|████ | 730/1817 [11:33:25<17:00:49, 56.35s/it] {'loss': 1.9188, 'learning_rate': 2e-05, 'epoch': 0.4} 40%|████ | 730/1817 [11:33:24<17:00:49, 56.35s/it] 40%|████ | 731/1817 [11:34:21<17:00:41, 56.39s/it] {'loss': 1.9029, 'learning_rate': 2e-05, 'epoch': 0.4} 40%|████ | 731/1817 [11:34:21<17:00:41, 56.39s/it] 40%|████ | 732/1817 [11:35:17<17:00:27, 56.43s/it] {'loss': 1.9103, 'learning_rate': 2e-05, 'epoch': 0.4} 40%|████ | 732/1817 [11:35:17<17:00:27, 56.43s/it] 40%|████ | 733/1817 [11:36:14<16:58:41, 56.38s/it] {'loss': 1.9285, 'learning_rate': 2e-05, 'epoch': 0.4} 40%|████ | 733/1817 [11:36:14<16:58:41, 56.38s/it] 40%|████ | 734/1817 [11:37:10<16:57:42, 56.38s/it] {'loss': 1.9201, 'learning_rate': 2e-05, 'epoch': 0.4} 40%|████ | 734/1817 [11:37:10<16:57:42, 56.38s/it] 40%|████ | 735/1817 [11:38:08<17:06:12, 56.91s/it] {'loss': 1.9212, 'learning_rate': 2e-05, 'epoch': 0.4} 40%|████ | 735/1817 [11:38:08<17:06:12, 56.91s/it] 41%|████ | 736/1817 [11:39:04<16:55:40, 56.37s/it] {'loss': 1.8721, 'learning_rate': 2e-05, 'epoch': 0.4} 41%|████ | 736/1817 [11:39:03<16:55:40, 56.37s/it] 41%|████ | 737/1817 [11:40:02<17:05:54, 56.99s/it] {'loss': 1.9395, 'learning_rate': 2e-05, 'epoch': 0.41} 41%|████ | 737/1817 [11:40:02<17:05:54, 57.00s/it] 41%|████ | 738/1817 [11:40:58<17:01:31, 56.80s/it] {'loss': 1.929, 'learning_rate': 2e-05, 'epoch': 0.41} 41%|████ | 738/1817 [11:40:58<17:01:31, 56.80s/it] 41%|████ | 739/1817 [11:41:55<16:57:39, 56.64s/it] {'loss': 1.9176, 'learning_rate': 2e-05, 'epoch': 0.41} 41%|████ | 739/1817 [11:41:54<16:57:39, 56.64s/it] 41%|████ | 740/1817 [11:42:52<16:58:31, 56.74s/it] {'loss': 1.9373, 'learning_rate': 2e-05, 'epoch': 0.41} 41%|████ | 740/1817 [11:42:51<16:58:31, 56.74s/it] 41%|████ | 741/1817 [11:43:48<16:54:31, 56.57s/it] {'loss': 1.9771, 'learning_rate': 2e-05, 'epoch': 0.41} 41%|████ | 741/1817 [11:43:47<16:54:31, 56.57s/it] 41%|████ | 742/1817 [11:44:44<16:51:17, 56.44s/it] {'loss': 1.9231, 'learning_rate': 2e-05, 'epoch': 0.41} 41%|████ | 742/1817 [11:44:44<16:51:17, 56.44s/it] 41%|████ | 743/1817 [11:45:40<16:49:11, 56.38s/it] {'loss': 1.9104, 'learning_rate': 2e-05, 'epoch': 0.41} 41%|████ | 743/1817 [11:45:40<16:49:11, 56.38s/it] 41%|████ | 744/1817 [11:46:36<16:47:16, 56.33s/it] {'loss': 1.9098, 'learning_rate': 2e-05, 'epoch': 0.41} 41%|████ | 744/1817 [11:46:36<16:47:16, 56.33s/it] 41%|████ | 745/1817 [11:47:33<16:46:37, 56.34s/it] {'loss': 1.9019, 'learning_rate': 2e-05, 'epoch': 0.41} 41%|████ | 745/1817 [11:47:32<16:46:37, 56.34s/it] 41%|████ | 746/1817 [11:48:32<17:00:08, 57.15s/it] {'loss': 1.9571, 'learning_rate': 2e-05, 'epoch': 0.41} 41%|████ | 746/1817 [11:48:31<17:00:08, 57.15s/it] 41%|████ | 747/1817 [11:49:27<16:51:38, 56.73s/it] {'loss': 1.9712, 'learning_rate': 2e-05, 'epoch': 0.41} 41%|████ | 747/1817 [11:49:27<16:51:38, 56.73s/it] 41%|████ | 748/1817 [11:50:25<16:57:10, 57.09s/it] {'loss': 1.9384, 'learning_rate': 2e-05, 'epoch': 0.41} 41%|████ | 748/1817 [11:50:25<16:57:10, 57.09s/it] 41%|████ | 749/1817 [11:51:22<16:52:35, 56.89s/it] {'loss': 1.9746, 'learning_rate': 2e-05, 'epoch': 0.41} 41%|████ | 749/1817 [11:51:22<16:52:35, 56.89s/it] 41%|████▏ | 750/1817 [11:52:19<16:54:06, 57.03s/it] {'loss': 1.9427, 'learning_rate': 2e-05, 'epoch': 0.41} 41%|████▏ | 750/1817 [11:52:19<16:54:06, 57.03s/it] 41%|████▏ | 751/1817 [11:53:17<16:59:48, 57.40s/it] {'loss': 1.9232, 'learning_rate': 2e-05, 'epoch': 0.41} 41%|████▏ | 751/1817 [11:53:17<16:59:48, 57.40s/it] 41%|████▏ | 752/1817 [11:54:13<16:51:06, 56.96s/it] {'loss': 1.8874, 'learning_rate': 2e-05, 'epoch': 0.41} 41%|████▏ | 752/1817 [11:54:13<16:51:06, 56.96s/it] 41%|████▏ | 753/1817 [11:55:10<16:46:17, 56.75s/it] {'loss': 1.9126, 'learning_rate': 2e-05, 'epoch': 0.41} 41%|████▏ | 753/1817 [11:55:09<16:46:17, 56.75s/it] 41%|████▏ | 754/1817 [11:56:07<16:50:27, 57.03s/it] {'loss': 1.9477, 'learning_rate': 2e-05, 'epoch': 0.41} 41%|████▏ | 754/1817 [11:56:07<16:50:27, 57.03s/it] 42%|████▏ | 755/1817 [11:57:03<16:44:49, 56.77s/it] {'loss': 1.9715, 'learning_rate': 2e-05, 'epoch': 0.42} 42%|████▏ | 755/1817 [11:57:03<16:44:49, 56.77s/it] 42%|████▏ | 756/1817 [11:58:00<16:45:30, 56.86s/it] {'loss': 1.8883, 'learning_rate': 2e-05, 'epoch': 0.42} 42%|████▏ | 756/1817 [11:58:00<16:45:30, 56.86s/it] 42%|████▏ | 757/1817 [11:58:57<16:41:29, 56.69s/it] {'loss': 1.9519, 'learning_rate': 2e-05, 'epoch': 0.42} 42%|████▏ | 757/1817 [11:58:57<16:41:29, 56.69s/it] 42%|████▏ | 758/1817 [11:59:53<16:38:19, 56.56s/it] {'loss': 1.9531, 'learning_rate': 2e-05, 'epoch': 0.42} 42%|████▏ | 758/1817 [11:59:53<16:38:19, 56.56s/it] 42%|████▏ | 759/1817 [12:00:50<16:36:47, 56.53s/it] {'loss': 1.9487, 'learning_rate': 2e-05, 'epoch': 0.42} 42%|████▏ | 759/1817 [12:00:49<16:36:47, 56.53s/it] 42%|████▏ | 760/1817 [12:01:44<16:27:14, 56.04s/it] {'loss': 1.9371, 'learning_rate': 2e-05, 'epoch': 0.42} 42%|████▏ | 760/1817 [12:01:44<16:27:14, 56.04s/it] 42%|████▏ | 761/1817 [12:02:41<16:27:50, 56.13s/it] {'loss': 1.9089, 'learning_rate': 2e-05, 'epoch': 0.42} 42%|████▏ | 761/1817 [12:02:41<16:27:47, 56.12s/it] 42%|████▏ | 762/1817 [12:03:37<16:27:44, 56.18s/it] {'loss': 1.9288, 'learning_rate': 2e-05, 'epoch': 0.42} 42%|████▏ | 762/1817 [12:03:37<16:27:43, 56.17s/it] 42%|████▏ | 763/1817 [12:04:38<16:50:49, 57.54s/it] {'loss': 1.9195, 'learning_rate': 2e-05, 'epoch': 0.42} 42%|████▏ | 763/1817 [12:04:38<16:50:47, 57.54s/it] 42%|████▏ | 764/1817 [12:05:35<16:50:10, 57.56s/it] {'loss': 1.9181, 'learning_rate': 2e-05, 'epoch': 0.42} 42%|████▏ | 764/1817 [12:05:35<16:50:10, 57.56s/it] 42%|████▏ | 765/1817 [12:06:31<16:39:12, 56.99s/it] {'loss': 1.9141, 'learning_rate': 2e-05, 'epoch': 0.42} 42%|████▏ | 765/1817 [12:06:31<16:39:12, 56.99s/it] 42%|████▏ | 766/1817 [12:07:31<16:51:21, 57.74s/it] {'loss': 1.9207, 'learning_rate': 2e-05, 'epoch': 0.42} 42%|████▏ | 766/1817 [12:07:30<16:51:21, 57.74s/it] 42%|████▏ | 767/1817 [12:08:27<16:43:04, 57.32s/it] {'loss': 1.961, 'learning_rate': 2e-05, 'epoch': 0.42} 42%|████▏ | 767/1817 [12:08:27<16:43:05, 57.32s/it] 42%|████▏ | 768/1817 [12:09:24<16:43:13, 57.38s/it] {'loss': 1.9248, 'learning_rate': 2e-05, 'epoch': 0.42} 42%|████▏ | 768/1817 [12:09:24<16:43:13, 57.38s/it] 42%|████▏ | 769/1817 [12:10:24<16:52:35, 57.97s/it] {'loss': 1.9496, 'learning_rate': 2e-05, 'epoch': 0.42} 42%|████▏ | 769/1817 [12:10:24<16:52:36, 57.97s/it] 42%|████▏ | 770/1817 [12:11:20<16:42:57, 57.48s/it] {'loss': 1.9298, 'learning_rate': 2e-05, 'epoch': 0.42} 42%|████▏ | 770/1817 [12:11:20<16:42:57, 57.48s/it] 42%|████▏ | 771/1817 [12:12:16<16:34:45, 57.06s/it] {'loss': 1.9325, 'learning_rate': 2e-05, 'epoch': 0.42} 42%|████▏ | 771/1817 [12:12:16<16:34:45, 57.06s/it] 42%|████▏ | 772/1817 [12:13:14<16:36:14, 57.20s/it] {'loss': 1.9397, 'learning_rate': 2e-05, 'epoch': 0.42} 42%|████▏ | 772/1817 [12:13:13<16:36:14, 57.20s/it] 43%|████▎ | 773/1817 [12:14:10<16:29:35, 56.87s/it] {'loss': 1.8977, 'learning_rate': 2e-05, 'epoch': 0.43} 43%|████▎ | 773/1817 [12:14:10<16:29:36, 56.87s/it] 43%|████▎ | 774/1817 [12:15:06<16:26:10, 56.73s/it] {'loss': 1.9208, 'learning_rate': 2e-05, 'epoch': 0.43} 43%|████▎ | 774/1817 [12:15:06<16:26:10, 56.73s/it] 43%|████▎ | 775/1817 [12:16:05<16:36:28, 57.38s/it] {'loss': 1.8862, 'learning_rate': 2e-05, 'epoch': 0.43} 43%|████▎ | 775/1817 [12:16:05<16:36:28, 57.38s/it] 43%|████▎ | 776/1817 [12:17:02<16:34:40, 57.33s/it] {'loss': 1.9099, 'learning_rate': 2e-05, 'epoch': 0.43} 43%|████▎ | 776/1817 [12:17:02<16:34:41, 57.33s/it] 43%|████▎ | 777/1817 [12:18:00<16:36:15, 57.48s/it] {'loss': 1.8834, 'learning_rate': 2e-05, 'epoch': 0.43} 43%|████▎ | 777/1817 [12:18:00<16:36:15, 57.48s/it] 43%|████▎ | 778/1817 [12:18:59<16:40:05, 57.75s/it] {'loss': 1.9424, 'learning_rate': 2e-05, 'epoch': 0.43} 43%|████▎ | 778/1817 [12:18:58<16:40:05, 57.75s/it] 43%|████▎ | 779/1817 [12:19:56<16:35:03, 57.52s/it] {'loss': 1.9242, 'learning_rate': 2e-05, 'epoch': 0.43} 43%|████▎ | 779/1817 [12:19:55<16:35:03, 57.52s/it] 43%|████▎ | 780/1817 [12:20:55<16:41:54, 57.97s/it] {'loss': 1.8696, 'learning_rate': 2e-05, 'epoch': 0.43} 43%|████▎ | 780/1817 [12:20:54<16:41:54, 57.97s/it] 43%|████▎ | 781/1817 [12:21:51<16:32:18, 57.47s/it] {'loss': 1.9354, 'learning_rate': 2e-05, 'epoch': 0.43} 43%|████▎ | 781/1817 [12:21:51<16:32:18, 57.47s/it] 43%|████▎ | 782/1817 [12:22:47<16:25:03, 57.10s/it] {'loss': 1.9151, 'learning_rate': 2e-05, 'epoch': 0.43} 43%|████▎ | 782/1817 [12:22:47<16:25:03, 57.11s/it] 43%|████▎ | 783/1817 [12:23:43<16:19:23, 56.83s/it] {'loss': 1.8678, 'learning_rate': 2e-05, 'epoch': 0.43} 43%|████▎ | 783/1817 [12:23:43<16:19:23, 56.83s/it] 43%|████▎ | 784/1817 [12:24:38<16:10:52, 56.39s/it] {'loss': 1.9283, 'learning_rate': 2e-05, 'epoch': 0.43} 43%|████▎ | 784/1817 [12:24:38<16:10:52, 56.39s/it] 43%|████▎ | 785/1817 [12:25:35<16:09:13, 56.35s/it] {'loss': 1.9492, 'learning_rate': 2e-05, 'epoch': 0.43} 43%|████▎ | 785/1817 [12:25:35<16:09:13, 56.35s/it] 43%|████▎ | 786/1817 [12:26:33<16:15:41, 56.78s/it] {'loss': 1.9222, 'learning_rate': 2e-05, 'epoch': 0.43} 43%|████▎ | 786/1817 [12:26:32<16:15:41, 56.78s/it] 43%|████▎ | 787/1817 [12:27:34<16:36:25, 58.04s/it] {'loss': 1.9611, 'learning_rate': 2e-05, 'epoch': 0.43} 43%|████▎ | 787/1817 [12:27:33<16:36:25, 58.04s/it] 43%|████▎ | 788/1817 [12:28:30<16:26:45, 57.54s/it] {'loss': 1.9283, 'learning_rate': 2e-05, 'epoch': 0.43} 43%|████▎ | 788/1817 [12:28:30<16:26:45, 57.54s/it] 43%|████▎ | 789/1817 [12:29:26<16:19:53, 57.19s/it] {'loss': 1.9334, 'learning_rate': 2e-05, 'epoch': 0.43} 43%|████▎ | 789/1817 [12:29:26<16:19:53, 57.19s/it] 43%|████▎ | 790/1817 [12:30:25<16:24:41, 57.53s/it] {'loss': 1.8995, 'learning_rate': 2e-05, 'epoch': 0.43} 43%|████▎ | 790/1817 [12:30:24<16:24:41, 57.53s/it] 44%|████▎ | 791/1817 [12:31:22<16:24:30, 57.57s/it] {'loss': 1.9099, 'learning_rate': 2e-05, 'epoch': 0.44} 44%|████▎ | 791/1817 [12:31:22<16:24:30, 57.57s/it] 44%|████▎ | 792/1817 [12:32:19<16:18:11, 57.26s/it] {'loss': 1.9014, 'learning_rate': 2e-05, 'epoch': 0.44} 44%|████▎ | 792/1817 [12:32:19<16:18:11, 57.26s/it] 44%|████▎ | 793/1817 [12:33:15<16:13:11, 57.02s/it] {'loss': 1.9247, 'learning_rate': 2e-05, 'epoch': 0.44} 44%|████▎ | 793/1817 [12:33:15<16:13:11, 57.02s/it] 44%|████▎ | 794/1817 [12:34:12<16:08:24, 56.80s/it] {'loss': 1.9461, 'learning_rate': 2e-05, 'epoch': 0.44} 44%|████▎ | 794/1817 [12:34:11<16:08:24, 56.80s/it] 44%|████▍ | 795/1817 [12:35:05<15:53:03, 55.95s/it] {'loss': 1.878, 'learning_rate': 2e-05, 'epoch': 0.44} 44%|████▍ | 795/1817 [12:35:05<15:53:03, 55.95s/it] 44%|████▍ | 796/1817 [12:36:02<15:53:39, 56.04s/it] {'loss': 1.8854, 'learning_rate': 2e-05, 'epoch': 0.44} 44%|████▍ | 796/1817 [12:36:02<15:53:39, 56.04s/it] 44%|████▍ | 797/1817 [12:36:58<15:51:33, 55.97s/it] {'loss': 1.9536, 'learning_rate': 2e-05, 'epoch': 0.44} 44%|████▍ | 797/1817 [12:36:57<15:51:33, 55.97s/it] 44%|████▍ | 798/1817 [12:37:54<15:53:34, 56.15s/it] {'loss': 1.9157, 'learning_rate': 2e-05, 'epoch': 0.44} 44%|████▍ | 798/1817 [12:37:54<15:53:34, 56.15s/it] 44%|████▍ | 799/1817 [12:38:50<15:53:31, 56.20s/it] {'loss': 1.9347, 'learning_rate': 2e-05, 'epoch': 0.44} 44%|████▍ | 799/1817 [12:38:50<15:53:31, 56.20s/it] 44%|████▍ | 800/1817 [12:39:47<15:53:24, 56.25s/it] {'loss': 1.8734, 'learning_rate': 2e-05, 'epoch': 0.44} 44%|████▍ | 800/1817 [12:39:47<15:53:24, 56.25s/it] 44%|████▍ | 801/1817 [12:40:42<15:46:44, 55.91s/it] {'loss': 1.9432, 'learning_rate': 2e-05, 'epoch': 0.44} 44%|████▍ | 801/1817 [12:40:42<15:46:44, 55.91s/it] 44%|████▍ | 802/1817 [12:41:41<16:01:44, 56.85s/it] {'loss': 1.9293, 'learning_rate': 2e-05, 'epoch': 0.44} 44%|████▍ | 802/1817 [12:41:41<16:01:44, 56.85s/it] 44%|████▍ | 803/1817 [12:42:38<15:58:43, 56.73s/it] {'loss': 1.9611, 'learning_rate': 2e-05, 'epoch': 0.44} 44%|████▍ | 803/1817 [12:42:37<15:58:43, 56.73s/it] 44%|████▍ | 804/1817 [12:43:34<15:56:22, 56.65s/it] {'loss': 1.8827, 'learning_rate': 2e-05, 'epoch': 0.44} 44%|████▍ | 804/1817 [12:43:34<15:56:22, 56.65s/it] 44%|████▍ | 805/1817 [12:44:28<15:43:34, 55.94s/it] {'loss': 1.9318, 'learning_rate': 2e-05, 'epoch': 0.44} 44%|████▍ | 805/1817 [12:44:28<15:43:34, 55.94s/it] 44%|████▍ | 806/1817 [12:45:25<15:45:16, 56.10s/it] {'loss': 1.8961, 'learning_rate': 2e-05, 'epoch': 0.44} 44%|████▍ | 806/1817 [12:45:25<15:45:16, 56.10s/it] 44%|████▍ | 807/1817 [12:46:19<15:37:31, 55.69s/it] {'loss': 1.939, 'learning_rate': 2e-05, 'epoch': 0.44} 44%|████▍ | 807/1817 [12:46:19<15:37:31, 55.69s/it] 44%|████▍ | 808/1817 [12:47:17<15:46:03, 56.26s/it] {'loss': 1.9232, 'learning_rate': 2e-05, 'epoch': 0.44} 44%|████▍ | 808/1817 [12:47:17<15:46:03, 56.26s/it] 45%|████▍ | 809/1817 [12:48:15<15:52:09, 56.68s/it] {'loss': 1.8973, 'learning_rate': 2e-05, 'epoch': 0.45} 45%|████▍ | 809/1817 [12:48:15<15:52:09, 56.68s/it] 45%|████▍ | 810/1817 [12:49:11<15:50:06, 56.61s/it] {'loss': 1.9109, 'learning_rate': 2e-05, 'epoch': 0.45} 45%|████▍ | 810/1817 [12:49:11<15:50:06, 56.61s/it] 45%|████▍ | 811/1817 [12:50:09<15:56:37, 57.06s/it] {'loss': 1.8909, 'learning_rate': 2e-05, 'epoch': 0.45} 45%|████▍ | 811/1817 [12:50:09<15:56:37, 57.06s/it] 45%|████▍ | 812/1817 [12:51:04<15:46:20, 56.50s/it] {'loss': 1.9566, 'learning_rate': 2e-05, 'epoch': 0.45} 45%|████▍ | 812/1817 [12:51:04<15:46:20, 56.50s/it] 45%|████▍ | 813/1817 [12:52:04<16:00:49, 57.42s/it] {'loss': 1.892, 'learning_rate': 2e-05, 'epoch': 0.45} 45%|████▍ | 813/1817 [12:52:04<16:00:49, 57.42s/it] 45%|████▍ | 814/1817 [12:53:00<15:54:16, 57.09s/it] {'loss': 1.9457, 'learning_rate': 2e-05, 'epoch': 0.45} 45%|████▍ | 814/1817 [12:53:00<15:54:16, 57.09s/it] 45%|████▍ | 815/1817 [12:53:56<15:47:24, 56.73s/it] {'loss': 1.9396, 'learning_rate': 2e-05, 'epoch': 0.45} 45%|████▍ | 815/1817 [12:53:56<15:47:25, 56.73s/it] 45%|████▍ | 816/1817 [12:54:53<15:44:58, 56.64s/it] {'loss': 1.8983, 'learning_rate': 2e-05, 'epoch': 0.45} 45%|████▍ | 816/1817 [12:54:52<15:44:58, 56.64s/it] 45%|████▍ | 817/1817 [12:55:49<15:42:04, 56.52s/it] {'loss': 1.9539, 'learning_rate': 2e-05, 'epoch': 0.45} 45%|████▍ | 817/1817 [12:55:49<15:42:04, 56.52s/it] 45%|████▌ | 818/1817 [12:56:47<15:50:35, 57.09s/it] {'loss': 1.9262, 'learning_rate': 2e-05, 'epoch': 0.45} 45%|████▌ | 818/1817 [12:56:47<15:50:35, 57.09s/it] 45%|████▌ | 819/1817 [12:57:44<15:45:56, 56.87s/it] {'loss': 1.9045, 'learning_rate': 2e-05, 'epoch': 0.45} 45%|████▌ | 819/1817 [12:57:43<15:45:56, 56.87s/it] 45%|████▌ | 820/1817 [12:58:39<15:38:20, 56.47s/it] {'loss': 1.8846, 'learning_rate': 2e-05, 'epoch': 0.45} 45%|████▌ | 820/1817 [12:58:39<15:38:20, 56.47s/it] 45%|████▌ | 821/1817 [12:59:38<15:47:11, 57.06s/it] {'loss': 1.908, 'learning_rate': 2e-05, 'epoch': 0.45} 45%|████▌ | 821/1817 [12:59:37<15:47:11, 57.06s/it] 45%|████▌ | 822/1817 [13:00:34<15:42:38, 56.84s/it] {'loss': 1.9242, 'learning_rate': 2e-05, 'epoch': 0.45} 45%|████▌ | 822/1817 [13:00:34<15:42:38, 56.84s/it] 45%|████▌ | 823/1817 [13:01:30<15:39:21, 56.70s/it] {'loss': 1.8977, 'learning_rate': 2e-05, 'epoch': 0.45} 45%|████▌ | 823/1817 [13:01:30<15:39:07, 56.69s/it] 45%|████▌ | 824/1817 [13:02:27<15:36:56, 56.61s/it] {'loss': 1.9147, 'learning_rate': 2e-05, 'epoch': 0.45} 45%|████▌ | 824/1817 [13:02:27<15:36:54, 56.61s/it] 45%|████▌ | 825/1817 [13:03:23<15:34:40, 56.53s/it] {'loss': 1.9249, 'learning_rate': 2e-05, 'epoch': 0.45} 45%|████▌ | 825/1817 [13:03:23<15:34:40, 56.53s/it] 45%|████▌ | 826/1817 [13:04:21<15:40:22, 56.93s/it] {'loss': 1.9105, 'learning_rate': 2e-05, 'epoch': 0.45} 45%|████▌ | 826/1817 [13:04:21<15:40:22, 56.93s/it] 46%|████▌ | 827/1817 [13:05:19<15:45:40, 57.31s/it] {'loss': 1.9395, 'learning_rate': 2e-05, 'epoch': 0.46} 46%|████▌ | 827/1817 [13:05:19<15:45:33, 57.31s/it] 46%|████▌ | 828/1817 [13:06:17<15:45:13, 57.34s/it] {'loss': 1.9407, 'learning_rate': 2e-05, 'epoch': 0.46} 46%|████▌ | 828/1817 [13:06:16<15:45:09, 57.34s/it] 46%|████▌ | 829/1817 [13:07:13<15:39:35, 57.06s/it] {'loss': 1.8988, 'learning_rate': 2e-05, 'epoch': 0.46} 46%|████▌ | 829/1817 [13:07:13<15:39:34, 57.06s/it] 46%|████▌ | 830/1817 [13:08:09<15:35:33, 56.87s/it] {'loss': 1.9223, 'learning_rate': 2e-05, 'epoch': 0.46} 46%|████▌ | 830/1817 [13:08:09<15:35:33, 56.87s/it] 46%|████▌ | 831/1817 [13:09:06<15:31:59, 56.71s/it] {'loss': 1.9118, 'learning_rate': 2e-05, 'epoch': 0.46} 46%|████▌ | 831/1817 [13:09:06<15:32:01, 56.72s/it] 46%|████▌ | 832/1817 [13:10:02<15:28:56, 56.59s/it] {'loss': 1.8943, 'learning_rate': 2e-05, 'epoch': 0.46} 46%|████▌ | 832/1817 [13:10:02<15:29:01, 56.59s/it] 46%|████▌ | 833/1817 [13:10:59<15:27:18, 56.54s/it] {'loss': 1.8802, 'learning_rate': 2e-05, 'epoch': 0.46} 46%|████▌ | 833/1817 [13:10:58<15:27:18, 56.54s/it] 46%|████▌ | 834/1817 [13:11:56<15:33:42, 56.99s/it] {'loss': 1.9332, 'learning_rate': 2e-05, 'epoch': 0.46} 46%|████▌ | 834/1817 [13:11:56<15:33:42, 56.99s/it] 46%|████▌ | 835/1817 [13:12:54<15:33:51, 57.06s/it] {'loss': 1.9576, 'learning_rate': 2e-05, 'epoch': 0.46} 46%|████▌ | 835/1817 [13:12:54<15:33:51, 57.06s/it] 46%|████▌ | 836/1817 [13:13:50<15:29:16, 56.84s/it] {'loss': 1.8994, 'learning_rate': 2e-05, 'epoch': 0.46} 46%|████▌ | 836/1817 [13:13:50<15:29:16, 56.84s/it] 46%|████▌ | 837/1817 [13:14:48<15:36:57, 57.36s/it] {'loss': 1.9055, 'learning_rate': 2e-05, 'epoch': 0.46} 46%|████▌ | 837/1817 [13:14:48<15:36:57, 57.36s/it] 46%|████▌ | 838/1817 [13:15:45<15:30:55, 57.05s/it] {'loss': 1.9484, 'learning_rate': 2e-05, 'epoch': 0.46} 46%|████▌ | 838/1817 [13:15:45<15:30:55, 57.05s/it] 46%|████▌ | 839/1817 [13:16:43<15:37:44, 57.53s/it] {'loss': 1.926, 'learning_rate': 2e-05, 'epoch': 0.46} 46%|████▌ | 839/1817 [13:16:43<15:37:44, 57.53s/it] 46%|████▌ | 840/1817 [13:17:40<15:30:57, 57.17s/it] {'loss': 1.9634, 'learning_rate': 2e-05, 'epoch': 0.46} 46%|████▌ | 840/1817 [13:17:40<15:30:57, 57.17s/it] 46%|████▋ | 841/1817 [13:18:36<15:25:03, 56.87s/it] {'loss': 1.9369, 'learning_rate': 2e-05, 'epoch': 0.46} 46%|████▋ | 841/1817 [13:18:36<15:25:03, 56.87s/it] 46%|████▋ | 842/1817 [13:19:32<15:20:41, 56.66s/it] {'loss': 1.9182, 'learning_rate': 2e-05, 'epoch': 0.46} 46%|████▋ | 842/1817 [13:19:32<15:20:41, 56.66s/it] 46%|████▋ | 843/1817 [13:20:30<15:25:17, 57.00s/it] {'loss': 1.9032, 'learning_rate': 2e-05, 'epoch': 0.46} 46%|████▋ | 843/1817 [13:20:30<15:25:17, 57.00s/it] 46%|████▋ | 844/1817 [13:21:31<15:40:37, 58.00s/it] {'loss': 1.9138, 'learning_rate': 2e-05, 'epoch': 0.46} 46%|████▋ | 844/1817 [13:21:30<15:40:37, 58.00s/it] 47%|████▋ | 845/1817 [13:22:29<15:41:31, 58.12s/it] {'loss': 1.9492, 'learning_rate': 2e-05, 'epoch': 0.46} 47%|████▋ | 845/1817 [13:22:29<15:41:31, 58.12s/it] 47%|████▋ | 846/1817 [13:23:25<15:31:16, 57.55s/it] {'loss': 1.9677, 'learning_rate': 2e-05, 'epoch': 0.47} 47%|████▋ | 846/1817 [13:23:25<15:31:16, 57.55s/it] 47%|████▋ | 847/1817 [13:24:21<15:24:06, 57.16s/it] {'loss': 1.9397, 'learning_rate': 2e-05, 'epoch': 0.47} 47%|████▋ | 847/1817 [13:24:21<15:24:06, 57.16s/it] 47%|████▋ | 848/1817 [13:25:17<15:19:06, 56.91s/it] {'loss': 1.9698, 'learning_rate': 2e-05, 'epoch': 0.47} 47%|████▋ | 848/1817 [13:25:17<15:19:06, 56.91s/it] 47%|████▋ | 849/1817 [13:26:14<15:15:22, 56.74s/it] {'loss': 1.9113, 'learning_rate': 2e-05, 'epoch': 0.47} 47%|████▋ | 849/1817 [13:26:14<15:15:22, 56.74s/it] 47%|████▋ | 850/1817 [13:27:11<15:17:42, 56.94s/it] {'loss': 1.8996, 'learning_rate': 2e-05, 'epoch': 0.47} 47%|████▋ | 850/1817 [13:27:11<15:17:42, 56.94s/it] 47%|████▋ | 851/1817 [13:28:08<15:13:21, 56.73s/it] {'loss': 1.894, 'learning_rate': 2e-05, 'epoch': 0.47} 47%|████▋ | 851/1817 [13:28:07<15:13:21, 56.73s/it] 47%|████▋ | 852/1817 [13:29:04<15:10:33, 56.62s/it] {'loss': 1.9414, 'learning_rate': 2e-05, 'epoch': 0.47} 47%|████▋ | 852/1817 [13:29:04<15:10:33, 56.62s/it] 47%|████▋ | 853/1817 [13:30:00<15:08:22, 56.54s/it] {'loss': 1.9265, 'learning_rate': 2e-05, 'epoch': 0.47} 47%|████▋ | 853/1817 [13:30:00<15:08:22, 56.54s/it] 47%|████▋ | 854/1817 [13:30:56<15:06:41, 56.49s/it] {'loss': 1.9031, 'learning_rate': 2e-05, 'epoch': 0.47} 47%|████▋ | 854/1817 [13:30:56<15:06:41, 56.49s/it] 47%|████▋ | 855/1817 [13:31:54<15:10:27, 56.79s/it] {'loss': 1.884, 'learning_rate': 2e-05, 'epoch': 0.47} 47%|████▋ | 855/1817 [13:31:54<15:10:27, 56.79s/it] 47%|████▋ | 856/1817 [13:32:51<15:08:20, 56.71s/it] {'loss': 1.9243, 'learning_rate': 2e-05, 'epoch': 0.47} 47%|████▋ | 856/1817 [13:32:51<15:08:20, 56.71s/it] 47%|████▋ | 857/1817 [13:33:47<15:05:31, 56.60s/it] {'loss': 1.9195, 'learning_rate': 2e-05, 'epoch': 0.47} 47%|████▋ | 857/1817 [13:33:47<15:05:31, 56.60s/it] 47%|████▋ | 858/1817 [13:34:46<15:17:29, 57.40s/it] {'loss': 1.9178, 'learning_rate': 2e-05, 'epoch': 0.47} 47%|████▋ | 858/1817 [13:34:46<15:17:29, 57.40s/it] 47%|████▋ | 859/1817 [13:35:44<15:18:44, 57.54s/it] {'loss': 1.8785, 'learning_rate': 2e-05, 'epoch': 0.47} 47%|████▋ | 859/1817 [13:35:44<15:18:44, 57.54s/it] 47%|████▋ | 860/1817 [13:36:40<15:12:06, 57.19s/it] {'loss': 1.943, 'learning_rate': 2e-05, 'epoch': 0.47} 47%|████▋ | 860/1817 [13:36:40<15:12:06, 57.19s/it] 47%|████▋ | 861/1817 [13:37:37<15:07:27, 56.95s/it] {'loss': 1.8934, 'learning_rate': 2e-05, 'epoch': 0.47} 47%|████▋ | 861/1817 [13:37:37<15:07:27, 56.95s/it] 47%|████▋ | 862/1817 [13:38:33<15:03:29, 56.76s/it] {'loss': 1.9214, 'learning_rate': 2e-05, 'epoch': 0.47} 47%|████▋ | 862/1817 [13:38:33<15:03:29, 56.76s/it] 47%|████▋ | 863/1817 [13:39:31<15:07:27, 57.07s/it] {'loss': 1.8987, 'learning_rate': 2e-05, 'epoch': 0.47} 47%|████▋ | 863/1817 [13:39:31<15:07:27, 57.07s/it] 48%|████▊ | 864/1817 [13:40:27<15:02:55, 56.85s/it] {'loss': 1.8909, 'learning_rate': 2e-05, 'epoch': 0.48} 48%|████▊ | 864/1817 [13:40:27<15:02:55, 56.85s/it] 48%|████▊ | 865/1817 [13:41:24<14:59:24, 56.68s/it] {'loss': 1.906, 'learning_rate': 2e-05, 'epoch': 0.48} 48%|████▊ | 865/1817 [13:41:23<14:59:23, 56.68s/it] 48%|████▊ | 866/1817 [13:42:20<14:57:53, 56.65s/it] {'loss': 1.9409, 'learning_rate': 2e-05, 'epoch': 0.48} 48%|████▊ | 866/1817 [13:42:20<14:57:53, 56.65s/it] 48%|████▊ | 867/1817 [13:43:18<15:01:08, 56.91s/it] {'loss': 1.9422, 'learning_rate': 2e-05, 'epoch': 0.48} 48%|████▊ | 867/1817 [13:43:18<15:01:08, 56.91s/it] 48%|████▊ | 868/1817 [13:44:15<15:04:02, 57.16s/it] {'loss': 1.9205, 'learning_rate': 2e-05, 'epoch': 0.48} 48%|████▊ | 868/1817 [13:44:15<15:04:02, 57.16s/it] 48%|████▊ | 869/1817 [13:45:11<14:54:16, 56.60s/it] {'loss': 1.942, 'learning_rate': 2e-05, 'epoch': 0.48} 48%|████▊ | 869/1817 [13:45:11<14:54:16, 56.60s/it] 48%|████▊ | 870/1817 [13:46:11<15:09:19, 57.61s/it] {'loss': 1.9132, 'learning_rate': 2e-05, 'epoch': 0.48} 48%|████▊ | 870/1817 [13:46:11<15:09:19, 57.61s/it] 48%|████▊ | 871/1817 [13:47:07<15:02:16, 57.23s/it] {'loss': 1.9142, 'learning_rate': 2e-05, 'epoch': 0.48} 48%|████▊ | 871/1817 [13:47:07<15:02:16, 57.23s/it] 48%|████▊ | 872/1817 [13:48:03<14:56:03, 56.89s/it] {'loss': 1.874, 'learning_rate': 2e-05, 'epoch': 0.48} 48%|████▊ | 872/1817 [13:48:03<14:56:03, 56.89s/it] 48%|████▊ | 873/1817 [13:49:01<14:57:53, 57.07s/it] {'loss': 1.8581, 'learning_rate': 2e-05, 'epoch': 0.48} 48%|████▊ | 873/1817 [13:49:01<14:57:53, 57.07s/it] 48%|████▊ | 874/1817 [13:49:57<14:56:05, 57.02s/it] {'loss': 1.9459, 'learning_rate': 2e-05, 'epoch': 0.48} 48%|████▊ | 874/1817 [13:49:57<14:56:05, 57.02s/it] 48%|████▊ | 875/1817 [13:50:54<14:51:44, 56.80s/it] {'loss': 1.9106, 'learning_rate': 2e-05, 'epoch': 0.48} 48%|████▊ | 875/1817 [13:50:54<14:51:44, 56.80s/it] 48%|████▊ | 876/1817 [13:51:53<15:04:08, 57.65s/it] {'loss': 1.9017, 'learning_rate': 2e-05, 'epoch': 0.48} 48%|████▊ | 876/1817 [13:51:53<15:04:08, 57.65s/it] 48%|████▊ | 877/1817 [13:52:49<14:51:50, 56.93s/it] {'loss': 1.9543, 'learning_rate': 2e-05, 'epoch': 0.48} 48%|████▊ | 877/1817 [13:52:49<14:51:50, 56.93s/it] 48%|████▊ | 878/1817 [13:53:45<14:47:53, 56.73s/it] {'loss': 1.9308, 'learning_rate': 2e-05, 'epoch': 0.48} 48%|████▊ | 878/1817 [13:53:45<14:47:53, 56.73s/it] 48%|████▊ | 879/1817 [13:54:41<14:44:40, 56.59s/it] {'loss': 1.9294, 'learning_rate': 2e-05, 'epoch': 0.48} 48%|████▊ | 879/1817 [13:54:41<14:44:40, 56.59s/it] 48%|████▊ | 880/1817 [13:55:38<14:42:40, 56.52s/it] {'loss': 1.9189, 'learning_rate': 2e-05, 'epoch': 0.48} 48%|████▊ | 880/1817 [13:55:37<14:42:40, 56.52s/it] 48%|████▊ | 881/1817 [13:56:34<14:39:40, 56.39s/it] {'loss': 1.9344, 'learning_rate': 2e-05, 'epoch': 0.48} 48%|████▊ | 881/1817 [13:56:34<14:39:40, 56.39s/it] 49%|████▊ | 882/1817 [13:57:32<14:45:12, 56.80s/it] {'loss': 1.9113, 'learning_rate': 2e-05, 'epoch': 0.49} 49%|████▊ | 882/1817 [13:57:31<14:45:11, 56.80s/it] 49%|████▊ | 883/1817 [13:58:30<14:50:39, 57.22s/it] {'loss': 1.9116, 'learning_rate': 2e-05, 'epoch': 0.49} 49%|████▊ | 883/1817 [13:58:30<14:50:39, 57.22s/it] 49%|████▊ | 884/1817 [13:59:26<14:45:28, 56.94s/it] {'loss': 1.9143, 'learning_rate': 2e-05, 'epoch': 0.49} 49%|████▊ | 884/1817 [13:59:26<14:45:28, 56.94s/it] 49%|████▊ | 885/1817 [14:00:21<14:35:18, 56.35s/it] {'loss': 1.8939, 'learning_rate': 2e-05, 'epoch': 0.49} 49%|████▊ | 885/1817 [14:00:21<14:35:18, 56.35s/it] 49%|████▉ | 886/1817 [14:01:20<14:45:28, 57.07s/it] {'loss': 1.9441, 'learning_rate': 2e-05, 'epoch': 0.49} 49%|████▉ | 886/1817 [14:01:20<14:45:28, 57.07s/it] 49%|████▉ | 887/1817 [14:02:17<14:47:08, 57.24s/it] {'loss': 1.8901, 'learning_rate': 2e-05, 'epoch': 0.49} 49%|████▉ | 887/1817 [14:02:17<14:47:08, 57.24s/it] 49%|████▉ | 888/1817 [14:03:14<14:41:16, 56.92s/it] {'loss': 1.9086, 'learning_rate': 2e-05, 'epoch': 0.49} 49%|████▉ | 888/1817 [14:03:13<14:41:13, 56.91s/it] 49%|████▉ | 889/1817 [14:04:10<14:39:23, 56.86s/it] {'loss': 1.9043, 'learning_rate': 2e-05, 'epoch': 0.49} 49%|████▉ | 889/1817 [14:04:10<14:39:23, 56.86s/it] 49%|████▉ | 890/1817 [14:05:10<14:50:48, 57.66s/it] {'loss': 1.8989, 'learning_rate': 2e-05, 'epoch': 0.49} 49%|████▉ | 890/1817 [14:05:10<14:50:46, 57.66s/it] 49%|████▉ | 891/1817 [14:06:08<14:52:35, 57.83s/it] {'loss': 1.9161, 'learning_rate': 2e-05, 'epoch': 0.49} 49%|████▉ | 891/1817 [14:06:08<14:52:34, 57.83s/it] 49%|████▉ | 892/1817 [14:07:06<14:52:07, 57.87s/it] {'loss': 1.9201, 'learning_rate': 2e-05, 'epoch': 0.49} 49%|████▉ | 892/1817 [14:07:06<14:52:07, 57.87s/it] 49%|████▉ | 893/1817 [14:08:07<15:05:20, 58.79s/it] {'loss': 1.9138, 'learning_rate': 2e-05, 'epoch': 0.49} 49%|████▉ | 893/1817 [14:08:07<15:05:19, 58.79s/it] 49%|████▉ | 894/1817 [14:09:05<15:00:01, 58.51s/it] {'loss': 1.9045, 'learning_rate': 2e-05, 'epoch': 0.49} 49%|████▉ | 894/1817 [14:09:05<15:00:01, 58.51s/it] 49%|████▉ | 895/1817 [14:10:02<14:56:17, 58.33s/it] {'loss': 1.908, 'learning_rate': 2e-05, 'epoch': 0.49} 49%|████▉ | 895/1817 [14:10:02<14:56:17, 58.33s/it] 49%|████▉ | 896/1817 [14:10:59<14:46:45, 57.77s/it] {'loss': 1.9442, 'learning_rate': 2e-05, 'epoch': 0.49} 49%|████▉ | 896/1817 [14:10:59<14:46:45, 57.77s/it] 49%|████▉ | 897/1817 [14:11:55<14:38:18, 57.28s/it] {'loss': 1.9296, 'learning_rate': 2e-05, 'epoch': 0.49} 49%|████▉ | 897/1817 [14:11:55<14:38:18, 57.28s/it] 49%|████▉ | 898/1817 [14:12:53<14:37:31, 57.29s/it] {'loss': 1.8856, 'learning_rate': 2e-05, 'epoch': 0.49} 49%|████▉ | 898/1817 [14:12:52<14:37:31, 57.29s/it] 49%|████▉ | 899/1817 [14:13:50<14:37:36, 57.36s/it] {'loss': 1.8817, 'learning_rate': 2e-05, 'epoch': 0.49} 49%|████▉ | 899/1817 [14:13:50<14:37:36, 57.36s/it] 50%|████▉ | 900/1817 [14:14:46<14:31:31, 57.02s/it] {'loss': 1.899, 'learning_rate': 2e-05, 'epoch': 0.5} 50%|████▉ | 900/1817 [14:14:46<14:31:31, 57.02s/it] 50%|████▉ | 901/1817 [14:15:42<14:26:33, 56.76s/it] {'loss': 1.9403, 'learning_rate': 2e-05, 'epoch': 0.5} 50%|████▉ | 901/1817 [14:15:42<14:26:33, 56.76s/it] 50%|████▉ | 902/1817 [14:16:39<14:24:09, 56.67s/it] {'loss': 1.8742, 'learning_rate': 2e-05, 'epoch': 0.5} 50%|████▉ | 902/1817 [14:16:39<14:24:09, 56.67s/it] 50%|████▉ | 903/1817 [14:17:36<14:23:37, 56.69s/it] {'loss': 1.9014, 'learning_rate': 2e-05, 'epoch': 0.5} 50%|████▉ | 903/1817 [14:17:35<14:23:37, 56.69s/it] 50%|████▉ | 904/1817 [14:18:34<14:30:57, 57.24s/it] {'loss': 1.8938, 'learning_rate': 2e-05, 'epoch': 0.5} 50%|████▉ | 904/1817 [14:18:34<14:30:57, 57.24s/it] 50%|████▉ | 905/1817 [14:19:31<14:28:16, 57.12s/it] {'loss': 1.9287, 'learning_rate': 2e-05, 'epoch': 0.5} 50%|████▉ | 905/1817 [14:19:31<14:28:16, 57.12s/it] 50%|████▉ | 906/1817 [14:20:28<14:27:31, 57.14s/it] {'loss': 1.9334, 'learning_rate': 2e-05, 'epoch': 0.5} 50%|████▉ | 906/1817 [14:20:28<14:27:31, 57.14s/it] 50%|████▉ | 907/1817 [14:21:25<14:27:07, 57.17s/it] {'loss': 1.904, 'learning_rate': 2e-05, 'epoch': 0.5} 50%|████▉ | 907/1817 [14:21:25<14:27:07, 57.17s/it] 50%|████▉ | 908/1817 [14:22:24<14:30:57, 57.49s/it] {'loss': 1.9111, 'learning_rate': 2e-05, 'epoch': 0.5} 50%|████▉ | 908/1817 [14:22:23<14:30:57, 57.49s/it] 50%|█████ | 909/1817 [14:23:22<14:33:30, 57.72s/it] {'loss': 1.8692, 'learning_rate': 2e-05, 'epoch': 0.5} 50%|█████ | 909/1817 [14:23:22<14:33:30, 57.72s/it] 50%|█████ | 910/1817 [14:24:18<14:26:45, 57.34s/it] {'loss': 1.8926, 'learning_rate': 2e-05, 'epoch': 0.5} 50%|█████ | 910/1817 [14:24:18<14:26:45, 57.34s/it] 50%|█████ | 911/1817 [14:25:15<14:25:02, 57.29s/it] {'loss': 1.9059, 'learning_rate': 2e-05, 'epoch': 0.5} 50%|█████ | 911/1817 [14:25:15<14:25:02, 57.29s/it] 50%|█████ | 912/1817 [14:26:14<14:29:08, 57.62s/it] {'loss': 1.9203, 'learning_rate': 2e-05, 'epoch': 0.5} 50%|█████ | 912/1817 [14:26:14<14:29:08, 57.62s/it] 50%|█████ | 913/1817 [14:27:10<14:21:53, 57.21s/it] {'loss': 1.9124, 'learning_rate': 2e-05, 'epoch': 0.5} 50%|█████ | 913/1817 [14:27:10<14:21:53, 57.21s/it] 50%|█████ | 914/1817 [14:28:06<14:16:47, 56.93s/it] {'loss': 1.9205, 'learning_rate': 2e-05, 'epoch': 0.5} 50%|█████ | 914/1817 [14:28:06<14:16:47, 56.93s/it] 50%|█████ | 915/1817 [14:29:03<14:13:13, 56.76s/it] {'loss': 1.8994, 'learning_rate': 2e-05, 'epoch': 0.5} 50%|█████ | 915/1817 [14:29:03<14:13:13, 56.76s/it] 50%|█████ | 916/1817 [14:29:59<14:11:26, 56.70s/it] {'loss': 1.9112, 'learning_rate': 2e-05, 'epoch': 0.5} 50%|█████ | 916/1817 [14:29:59<14:11:26, 56.70s/it] 50%|█████ | 917/1817 [14:30:56<14:09:17, 56.62s/it] {'loss': 1.8931, 'learning_rate': 2e-05, 'epoch': 0.5} 50%|█████ | 917/1817 [14:30:56<14:09:17, 56.62s/it] 51%|█████ | 918/1817 [14:31:55<14:21:03, 57.47s/it] {'loss': 1.916, 'learning_rate': 2e-05, 'epoch': 0.51} 51%|█████ | 918/1817 [14:31:55<14:21:03, 57.47s/it] 51%|█████ | 919/1817 [14:32:51<14:15:17, 57.15s/it] {'loss': 1.9192, 'learning_rate': 2e-05, 'epoch': 0.51} 51%|█████ | 919/1817 [14:32:51<14:15:17, 57.15s/it] 51%|█████ | 920/1817 [14:33:48<14:11:33, 56.96s/it] {'loss': 1.9365, 'learning_rate': 2e-05, 'epoch': 0.51} 51%|█████ | 920/1817 [14:33:48<14:11:33, 56.96s/it] 51%|█████ | 921/1817 [14:34:46<14:16:04, 57.33s/it] {'loss': 1.9319, 'learning_rate': 2e-05, 'epoch': 0.51} 51%|█████ | 921/1817 [14:34:46<14:16:04, 57.33s/it] 51%|█████ | 922/1817 [14:35:43<14:11:26, 57.08s/it] {'loss': 1.9138, 'learning_rate': 2e-05, 'epoch': 0.51} 51%|█████ | 922/1817 [14:35:43<14:11:26, 57.08s/it] 51%|█████ | 923/1817 [14:36:39<14:07:36, 56.89s/it] {'loss': 1.9092, 'learning_rate': 2e-05, 'epoch': 0.51} 51%|█████ | 923/1817 [14:36:39<14:07:36, 56.89s/it] 51%|█████ | 924/1817 [14:37:35<14:04:16, 56.73s/it] {'loss': 1.8876, 'learning_rate': 2e-05, 'epoch': 0.51} 51%|█████ | 924/1817 [14:37:35<14:04:16, 56.73s/it] 51%|█████ | 925/1817 [14:38:32<14:01:28, 56.60s/it] {'loss': 1.9461, 'learning_rate': 2e-05, 'epoch': 0.51} 51%|█████ | 925/1817 [14:38:32<14:01:28, 56.60s/it] 51%|█████ | 926/1817 [14:39:29<14:03:27, 56.80s/it] {'loss': 1.876, 'learning_rate': 2e-05, 'epoch': 0.51} 51%|█████ | 926/1817 [14:39:29<14:03:27, 56.80s/it] 51%|█████ | 927/1817 [14:40:25<14:00:18, 56.65s/it] {'loss': 1.9158, 'learning_rate': 2e-05, 'epoch': 0.51} 51%|█████ | 927/1817 [14:40:25<14:00:18, 56.65s/it] 51%|█████ | 928/1817 [14:41:22<13:57:46, 56.54s/it] {'loss': 1.8801, 'learning_rate': 2e-05, 'epoch': 0.51} 51%|█████ | 928/1817 [14:41:22<13:57:46, 56.54s/it] 51%|█████ | 929/1817 [14:42:17<13:52:40, 56.26s/it] {'loss': 1.9228, 'learning_rate': 2e-05, 'epoch': 0.51} 51%|█████ | 929/1817 [14:42:17<13:52:40, 56.26s/it] 51%|█████ | 930/1817 [14:43:14<13:52:01, 56.28s/it] {'loss': 1.9325, 'learning_rate': 2e-05, 'epoch': 0.51} 51%|█████ | 930/1817 [14:43:14<13:52:01, 56.28s/it] 51%|█████ | 931/1817 [14:44:10<13:51:07, 56.28s/it] {'loss': 1.8941, 'learning_rate': 2e-05, 'epoch': 0.51} 51%|█████ | 931/1817 [14:44:10<13:51:07, 56.28s/it] 51%|█████▏ | 932/1817 [14:45:08<13:55:58, 56.68s/it] {'loss': 1.9429, 'learning_rate': 2e-05, 'epoch': 0.51} 51%|█████▏ | 932/1817 [14:45:07<13:55:58, 56.68s/it] 51%|█████▏ | 933/1817 [14:46:04<13:53:35, 56.58s/it] {'loss': 1.9287, 'learning_rate': 2e-05, 'epoch': 0.51} 51%|█████▏ | 933/1817 [14:46:04<13:53:35, 56.58s/it] 51%|█████▏ | 934/1817 [14:47:02<14:00:36, 57.12s/it] {'loss': 1.9281, 'learning_rate': 2e-05, 'epoch': 0.51} 51%|█████▏ | 934/1817 [14:47:02<14:00:36, 57.12s/it] 51%|█████▏ | 935/1817 [14:47:59<13:56:05, 56.88s/it] {'loss': 1.9539, 'learning_rate': 2e-05, 'epoch': 0.51} 51%|█████▏ | 935/1817 [14:47:59<13:56:05, 56.88s/it] 52%|█████▏ | 936/1817 [14:48:56<13:55:17, 56.89s/it] {'loss': 1.9059, 'learning_rate': 2e-05, 'epoch': 0.51} 52%|█████▏ | 936/1817 [14:48:55<13:55:17, 56.89s/it] 52%|█████▏ | 937/1817 [14:49:53<13:57:30, 57.10s/it] {'loss': 1.8899, 'learning_rate': 2e-05, 'epoch': 0.52} 52%|█████▏ | 937/1817 [14:49:53<13:57:30, 57.10s/it] 52%|█████▏ | 938/1817 [14:50:53<14:07:49, 57.87s/it] {'loss': 1.9194, 'learning_rate': 2e-05, 'epoch': 0.52} 52%|█████▏ | 938/1817 [14:50:53<14:07:49, 57.87s/it] 52%|█████▏ | 939/1817 [14:51:49<13:59:32, 57.37s/it] {'loss': 1.9574, 'learning_rate': 2e-05, 'epoch': 0.52} 52%|█████▏ | 939/1817 [14:51:49<13:59:32, 57.37s/it] 52%|█████▏ | 940/1817 [14:52:45<13:53:15, 57.01s/it] {'loss': 1.9161, 'learning_rate': 2e-05, 'epoch': 0.52} 52%|█████▏ | 940/1817 [14:52:45<13:53:15, 57.01s/it] 52%|█████▏ | 941/1817 [14:53:42<13:50:29, 56.88s/it] {'loss': 1.899, 'learning_rate': 2e-05, 'epoch': 0.52} 52%|█████▏ | 941/1817 [14:53:42<13:50:29, 56.88s/it] 52%|█████▏ | 942/1817 [14:54:38<13:48:10, 56.79s/it] {'loss': 1.9232, 'learning_rate': 2e-05, 'epoch': 0.52} 52%|█████▏ | 942/1817 [14:54:38<13:48:10, 56.79s/it] 52%|█████▏ | 943/1817 [14:55:37<13:53:50, 57.24s/it] {'loss': 1.9342, 'learning_rate': 2e-05, 'epoch': 0.52} 52%|█████▏ | 943/1817 [14:55:37<13:53:50, 57.24s/it] 52%|█████▏ | 944/1817 [14:56:33<13:49:45, 57.03s/it] {'loss': 1.9085, 'learning_rate': 2e-05, 'epoch': 0.52} 52%|█████▏ | 944/1817 [14:56:33<13:49:45, 57.03s/it] 52%|█████▏ | 945/1817 [14:57:30<13:46:14, 56.85s/it] {'loss': 1.8948, 'learning_rate': 2e-05, 'epoch': 0.52} 52%|█████▏ | 945/1817 [14:57:29<13:46:14, 56.85s/it] 52%|█████▏ | 946/1817 [14:58:26<13:43:07, 56.70s/it] {'loss': 1.9388, 'learning_rate': 2e-05, 'epoch': 0.52} 52%|█████▏ | 946/1817 [14:58:26<13:43:07, 56.70s/it] 52%|█████▏ | 947/1817 [14:59:23<13:41:11, 56.63s/it] {'loss': 1.8636, 'learning_rate': 2e-05, 'epoch': 0.52} 52%|█████▏ | 947/1817 [14:59:22<13:41:11, 56.63s/it] 52%|█████▏ | 948/1817 [15:00:19<13:40:14, 56.63s/it] {'loss': 1.8847, 'learning_rate': 2e-05, 'epoch': 0.52} 52%|█████▏ | 948/1817 [15:00:19<13:40:14, 56.63s/it] 52%|█████▏ | 949/1817 [15:01:16<13:37:52, 56.54s/it] {'loss': 1.8789, 'learning_rate': 2e-05, 'epoch': 0.52} 52%|█████▏ | 949/1817 [15:01:15<13:37:52, 56.54s/it] 52%|█████▏ | 950/1817 [15:02:12<13:37:17, 56.56s/it] {'loss': 1.8895, 'learning_rate': 2e-05, 'epoch': 0.52} 52%|█████▏ | 950/1817 [15:02:12<13:37:17, 56.56s/it] 52%|█████▏ | 951/1817 [15:03:13<13:53:23, 57.74s/it] {'loss': 1.9401, 'learning_rate': 2e-05, 'epoch': 0.52} 52%|█████▏ | 951/1817 [15:03:12<13:53:22, 57.74s/it] 52%|█████▏ | 952/1817 [15:04:11<13:54:48, 57.91s/it] {'loss': 1.9211, 'learning_rate': 2e-05, 'epoch': 0.52} 52%|█████▏ | 952/1817 [15:04:11<13:54:47, 57.90s/it] 52%|█████▏ | 953/1817 [15:05:07<13:47:37, 57.47s/it] {'loss': 1.9017, 'learning_rate': 2e-05, 'epoch': 0.52} 52%|█████▏ | 953/1817 [15:05:07<13:47:37, 57.47s/it] 53%|█████▎ | 954/1817 [15:06:07<13:55:13, 58.07s/it] {'loss': 1.9055, 'learning_rate': 2e-05, 'epoch': 0.52} 53%|█████▎ | 954/1817 [15:06:07<13:55:13, 58.07s/it] 53%|█████▎ | 955/1817 [15:07:03<13:47:09, 57.58s/it] {'loss': 1.8922, 'learning_rate': 2e-05, 'epoch': 0.53} 53%|█████▎ | 955/1817 [15:07:03<13:47:09, 57.57s/it] 53%|█████▎ | 956/1817 [15:08:00<13:40:56, 57.21s/it] {'loss': 1.9156, 'learning_rate': 2e-05, 'epoch': 0.53} 53%|█████▎ | 956/1817 [15:07:59<13:40:55, 57.21s/it] 53%|█████▎ | 957/1817 [15:08:55<13:33:58, 56.79s/it] {'loss': 1.919, 'learning_rate': 2e-05, 'epoch': 0.53} 53%|█████▎ | 957/1817 [15:08:55<13:33:58, 56.79s/it] 53%|█████▎ | 958/1817 [15:09:52<13:31:16, 56.67s/it] {'loss': 1.9061, 'learning_rate': 2e-05, 'epoch': 0.53} 53%|█████▎ | 958/1817 [15:09:52<13:31:16, 56.67s/it] 53%|█████▎ | 959/1817 [15:10:50<13:34:48, 56.98s/it] {'loss': 1.9378, 'learning_rate': 2e-05, 'epoch': 0.53} 53%|█████▎ | 959/1817 [15:10:49<13:34:49, 56.98s/it] 53%|█████▎ | 960/1817 [15:11:46<13:30:45, 56.76s/it] {'loss': 1.8971, 'learning_rate': 2e-05, 'epoch': 0.53} 53%|█████▎ | 960/1817 [15:11:46<13:30:45, 56.76s/it] 53%|█████▎ | 961/1817 [15:12:46<13:47:43, 58.02s/it] {'loss': 1.9223, 'learning_rate': 2e-05, 'epoch': 0.53} 53%|█████▎ | 961/1817 [15:12:46<13:47:43, 58.02s/it] 53%|█████▎ | 962/1817 [15:13:44<13:44:00, 57.83s/it] {'loss': 1.9374, 'learning_rate': 2e-05, 'epoch': 0.53} 53%|█████▎ | 962/1817 [15:13:44<13:44:00, 57.83s/it] 53%|█████▎ | 963/1817 [15:14:41<13:37:42, 57.45s/it] {'loss': 1.8618, 'learning_rate': 2e-05, 'epoch': 0.53} 53%|█████▎ | 963/1817 [15:14:40<13:37:42, 57.45s/it] 53%|█████▎ | 964/1817 [15:15:36<13:29:52, 56.97s/it] {'loss': 1.9305, 'learning_rate': 2e-05, 'epoch': 0.53} 53%|█████▎ | 964/1817 [15:15:36<13:29:52, 56.97s/it] 53%|█████▎ | 965/1817 [15:16:33<13:25:27, 56.72s/it] {'loss': 1.9182, 'learning_rate': 2e-05, 'epoch': 0.53} 53%|█████▎ | 965/1817 [15:16:32<13:25:27, 56.72s/it] 53%|█████▎ | 966/1817 [15:17:30<13:28:19, 56.99s/it] {'loss': 1.8577, 'learning_rate': 2e-05, 'epoch': 0.53} 53%|█████▎ | 966/1817 [15:17:30<13:28:19, 56.99s/it] 53%|█████▎ | 967/1817 [15:18:25<13:18:29, 56.36s/it] {'loss': 1.8976, 'learning_rate': 2e-05, 'epoch': 0.53} 53%|█████▎ | 967/1817 [15:18:25<13:18:29, 56.36s/it] 53%|█████▎ | 968/1817 [15:19:23<13:24:29, 56.85s/it] {'loss': 1.8718, 'learning_rate': 2e-05, 'epoch': 0.53} 53%|█████▎ | 968/1817 [15:19:23<13:24:29, 56.85s/it] 53%|█████▎ | 969/1817 [15:20:22<13:31:08, 57.39s/it] {'loss': 1.9491, 'learning_rate': 2e-05, 'epoch': 0.53} 53%|█████▎ | 969/1817 [15:20:22<13:31:08, 57.39s/it] 53%|█████▎ | 970/1817 [15:21:20<13:32:21, 57.55s/it] {'loss': 1.9182, 'learning_rate': 2e-05, 'epoch': 0.53} 53%|█████▎ | 970/1817 [15:21:19<13:32:21, 57.55s/it] 53%|█████▎ | 971/1817 [15:22:16<13:25:46, 57.15s/it] {'loss': 1.8961, 'learning_rate': 2e-05, 'epoch': 0.53} 53%|█████▎ | 971/1817 [15:22:16<13:25:46, 57.15s/it] 53%|█████▎ | 972/1817 [15:23:14<13:27:04, 57.31s/it] {'loss': 1.9482, 'learning_rate': 2e-05, 'epoch': 0.53} 53%|█████▎ | 972/1817 [15:23:13<13:27:04, 57.31s/it] 54%|█████▎ | 973/1817 [15:24:08<13:15:01, 56.52s/it] {'loss': 1.9219, 'learning_rate': 2e-05, 'epoch': 0.54} 54%|█████▎ | 973/1817 [15:24:08<13:15:01, 56.52s/it] 54%|█████▎ | 974/1817 [15:25:05<13:13:00, 56.44s/it] {'loss': 1.8907, 'learning_rate': 2e-05, 'epoch': 0.54} 54%|█████▎ | 974/1817 [15:25:04<13:13:00, 56.44s/it] 54%|█████▎ | 975/1817 [15:25:59<13:03:48, 55.85s/it] {'loss': 1.8809, 'learning_rate': 2e-05, 'epoch': 0.54} 54%|█████▎ | 975/1817 [15:25:59<13:03:48, 55.85s/it] 54%|█████▎ | 976/1817 [15:26:55<13:04:46, 55.99s/it] {'loss': 1.9281, 'learning_rate': 2e-05, 'epoch': 0.54} 54%|█████▎ | 976/1817 [15:26:55<13:04:46, 55.99s/it] 54%|█████▍ | 977/1817 [15:27:53<13:09:40, 56.41s/it] {'loss': 1.9282, 'learning_rate': 2e-05, 'epoch': 0.54} 54%|█████▍ | 977/1817 [15:27:52<13:09:40, 56.41s/it] 54%|█████▍ | 978/1817 [15:28:48<13:05:28, 56.17s/it] {'loss': 1.9025, 'learning_rate': 2e-05, 'epoch': 0.54} 54%|█████▍ | 978/1817 [15:28:48<13:05:28, 56.17s/it] 54%|█████▍ | 979/1817 [15:29:44<13:03:55, 56.13s/it] {'loss': 1.9362, 'learning_rate': 2e-05, 'epoch': 0.54} 54%|█████▍ | 979/1817 [15:29:44<13:03:55, 56.13s/it] 54%|█████▍ | 980/1817 [15:30:41<13:04:05, 56.21s/it] {'loss': 1.9312, 'learning_rate': 2e-05, 'epoch': 0.54} 54%|█████▍ | 980/1817 [15:30:41<13:04:05, 56.21s/it] 54%|█████▍ | 981/1817 [15:31:37<13:03:27, 56.23s/it] {'loss': 1.8967, 'learning_rate': 2e-05, 'epoch': 0.54} 54%|█████▍ | 981/1817 [15:31:37<13:03:28, 56.23s/it] 54%|█████▍ | 982/1817 [15:32:34<13:04:00, 56.34s/it] {'loss': 1.8916, 'learning_rate': 2e-05, 'epoch': 0.54} 54%|█████▍ | 982/1817 [15:32:33<13:04:00, 56.34s/it] 54%|█████▍ | 983/1817 [15:33:30<13:03:18, 56.35s/it] {'loss': 1.9163, 'learning_rate': 2e-05, 'epoch': 0.54} 54%|█████▍ | 983/1817 [15:33:30<13:03:17, 56.35s/it] 54%|█████▍ | 984/1817 [15:34:26<13:01:37, 56.30s/it] {'loss': 1.9278, 'learning_rate': 2e-05, 'epoch': 0.54} 54%|█████▍ | 984/1817 [15:34:26<13:01:37, 56.30s/it] 54%|█████▍ | 985/1817 [15:35:23<13:00:59, 56.32s/it] {'loss': 1.8928, 'learning_rate': 2e-05, 'epoch': 0.54} 54%|█████▍ | 985/1817 [15:35:22<13:00:59, 56.32s/it] 54%|█████▍ | 986/1817 [15:36:23<13:19:44, 57.74s/it] {'loss': 1.8991, 'learning_rate': 2e-05, 'epoch': 0.54} 54%|█████▍ | 986/1817 [15:36:23<13:19:44, 57.74s/it] 54%|█████▍ | 987/1817 [15:37:20<13:12:49, 57.31s/it] {'loss': 1.8875, 'learning_rate': 2e-05, 'epoch': 0.54} 54%|█████▍ | 987/1817 [15:37:20<13:12:49, 57.31s/it] 54%|█████▍ | 988/1817 [15:38:16<13:08:15, 57.05s/it] {'loss': 1.9007, 'learning_rate': 2e-05, 'epoch': 0.54} 54%|█████▍ | 988/1817 [15:38:16<13:08:15, 57.05s/it] 54%|█████▍ | 989/1817 [15:39:13<13:04:54, 56.88s/it] {'loss': 1.8657, 'learning_rate': 2e-05, 'epoch': 0.54} 54%|█████▍ | 989/1817 [15:39:13<13:04:54, 56.88s/it] 54%|█████▍ | 990/1817 [15:40:10<13:06:36, 57.07s/it] {'loss': 1.8924, 'learning_rate': 2e-05, 'epoch': 0.54} 54%|█████▍ | 990/1817 [15:40:10<13:06:36, 57.07s/it] 55%|█████▍ | 991/1817 [15:41:07<13:03:08, 56.89s/it] {'loss': 1.9056, 'learning_rate': 2e-05, 'epoch': 0.55} 55%|█████▍ | 991/1817 [15:41:07<13:03:08, 56.89s/it] 55%|█████▍ | 992/1817 [15:42:04<13:04:40, 57.07s/it] {'loss': 1.955, 'learning_rate': 2e-05, 'epoch': 0.55} 55%|█████▍ | 992/1817 [15:42:04<13:04:41, 57.07s/it] 55%|█████▍ | 993/1817 [15:43:00<12:59:41, 56.77s/it] {'loss': 1.9316, 'learning_rate': 2e-05, 'epoch': 0.55} 55%|█████▍ | 993/1817 [15:43:00<12:59:41, 56.77s/it] 55%|█████▍ | 994/1817 [15:43:57<12:57:54, 56.71s/it] {'loss': 1.8922, 'learning_rate': 2e-05, 'epoch': 0.55} 55%|█████▍ | 994/1817 [15:43:57<12:57:54, 56.71s/it] 55%|█████▍ | 995/1817 [15:44:53<12:54:58, 56.57s/it] {'loss': 1.9, 'learning_rate': 2e-05, 'epoch': 0.55} 55%|█████▍ | 995/1817 [15:44:53<12:54:58, 56.57s/it] 55%|█████▍ | 996/1817 [15:45:49<12:53:16, 56.51s/it] {'loss': 1.905, 'learning_rate': 2e-05, 'epoch': 0.55} 55%|█████▍ | 996/1817 [15:45:49<12:53:16, 56.51s/it] 55%|█████▍ | 997/1817 [15:46:46<12:51:01, 56.42s/it] {'loss': 1.905, 'learning_rate': 2e-05, 'epoch': 0.55} 55%|█████▍ | 997/1817 [15:46:46<12:51:01, 56.42s/it] 55%|█████▍ | 998/1817 [15:47:43<12:53:10, 56.64s/it] {'loss': 1.8933, 'learning_rate': 2e-05, 'epoch': 0.55} 55%|█████▍ | 998/1817 [15:47:43<12:53:10, 56.64s/it] 55%|█████▍ | 999/1817 [15:48:39<12:50:30, 56.52s/it] {'loss': 1.9002, 'learning_rate': 2e-05, 'epoch': 0.55} 55%|█████▍ | 999/1817 [15:48:39<12:50:30, 56.52s/it] 55%|█████▌ | 1000/1817 [15:49:37<12:53:01, 56.77s/it] {'loss': 1.9801, 'learning_rate': 2e-05, 'epoch': 0.55} 55%|█████▌ | 1000/1817 [15:49:37<12:53:01, 56.77s/it]5, CurrSamplesPerSec=8.931138187718732, MemAllocated=0.09GB, MaxMemAllocated=23.01GB 55%|█████▌ | 1000/1817 [15:49:36<12:53:02, 56.77s/it] {'loss': 1.9801, 'learning_rate': 2e-05, 'epoch': 0.55} 55%|█████▌ | 1000/1817 [15:49:36<12:53:02, 56.77s/it][INFO|trainer.py:2830] 2024-01-28 20:43:21,146 >> Saving model checkpoint to ./checkpoints_ct/ac/allm-ac-7b/checkpoint-1000 [INFO|configuration_utils.py:457] 2024-01-28 20:43:21,151 >> Configuration saved in ./checkpoints_ct/ac/allm-ac-7b/checkpoint-1000/config.json [INFO|configuration_utils.py:362] 2024-01-28 20:43:21,155 >> Configuration saved in ./checkpoints_ct/ac/allm-ac-7b/checkpoint-1000/generation_config.json [INFO|modeling_utils.py:1759] 2024-01-28 20:43:21,185 >> Model weights saved in ./checkpoints_ct/ac/allm-ac-7b/checkpoint-1000/pytorch_model.bin [INFO|tokenization_utils_base.py:2164] 2024-01-28 20:43:21,187 >> tokenizer config file saved in ./checkpoints_ct/ac/allm-ac-7b/checkpoint-1000/tokenizer_config.json [INFO|tokenization_utils_base.py:2171] 2024-01-28 20:43:21,187 >> Special tokens file saved in ./checkpoints_ct/ac/allm-ac-7b/checkpoint-1000/special_tokens_map.json [INFO|tokenization_utils_base.py:2221] 2024-01-28 20:43:21,188 >> added tokens file saved in ./checkpoints_ct/ac/allm-ac-7b/checkpoint-1000/added_tokens.json [2024-01-28 20:43:28,919] [INFO] [logging.py:96:log_dist] [Rank 0] [Torch] Checkpoint global_step1000 is about to be saved! [2024-01-28 20:43:28,920] [INFO] [engine.py:3492:save_16bit_model] Saving model weights to ./checkpoints_ct/ac/allm-ac-7b/checkpoint-1000/pytorch_model.bin, tag: global_step1000 [2024-01-28 20:43:28,920] [INFO] [torch_checkpoint_engine.py:21:save] [Torch] Saving ./checkpoints_ct/ac/allm-ac-7b/checkpoint-1000/pytorch_model.bin... [2024-01-28 20:43:43,325] [INFO] [torch_checkpoint_engine.py:21:save] [Torch] Saving ./checkpoints_ct/ac/allm-ac-7b/checkpoint-1000/global_step1000/zero_pp_rank_16_mp_rank_00_model_states.pt...[[2024-01-28 20:43:43,364] [INFO] [torch_checkpoint_engine.py:23:save] [Torch] Saved ./checkpoints_ct/ac/allm-ac-7b/checkpoint-1000/global_step1000/zero_pp_rank_16_mp_rank_00_model_states.pt[2[2024-01-28 20:43:43,388] [INFO] [torch_checkpoint_engine.py:21:save] [Torch] Saving ./checkpoints_ct/ac/allm-ac-7b/checkpoint-1000/global_step1000/bf16_zero_pp_rank_16_mp_rank_00_optim_states.pt... ates.pt [2024-01-28 20:43:43,331] [INFO] [torch_checkpoint_engine.py:21:save] [Torch] Saving ./checkpoints_ct/ac/allm-ac-7b/checkpoint-1000/global_step1000/zero_pp_rank_0_mp_rank_00_model_states.pt... [2024-01-28 20:43:43,355] [INFO] [torch_checkpoint_engine.py:23:save] [Torch] Saved ./checkpoints_ct/ac/allm-ac-7b/checkpoint-1000/global_step1000/zero_pp_rank_0_mp_rank_00_model_states.pt. [2024-01-28 20:43:43,396] [INFO] [torch_checkpoint_engine.py:21:save] [Torch] Saving ./checkpoints_ct/ac/allm-ac-7b/checkpoint-1000/global_step1000/bf16_zero_pp_rank_0_mp_rank_00_optim_states.pt... [2024-01-28 20:43:50,712] [INFO] [torch_checkpoint_engine.py:23:save] [Torch] Saved ./checkpoints_ct/ac/allm-ac-7b/checkpoint-1000/global_step1000/bf16_zero_pp_rank_8_mp_rank_00_optim_states.pt. [2024-01-28 20:43:50,712] [INFO] [engine.py:3381:_save_zero_checkpoint] zero checkpoint saved ./checkpoints_ct/ac/allm-ac-7b/checkpoint-1000/global_step1000/bf16_zero_pp_rank_8_mp_rank_00_optim_states.pt [2024-01-28 20:43:50,839] [INFO] [torch_checkpoint_engine.py:33:commit] [Torch] Checkpoint global_step1000 is ready now! [INFO|trainer.py:2908] 2024-01-28 20:43:51,276 >> Deleting older checkpoint [checkpoints_ct/ac/allm-ac-7b/checkpoint-500] due to args.save_total_limit 55%|█████▌ | 1001/1817 [15:51:03<14:54:26, 65.77s/it] {'loss': 1.919, 'learning_rate': 2e-05, 'epoch': 0.55} 55%|█████▌ | 1001/1817 [15:51:03<14:54:26, 65.77s/it] 55%|█████▌ | 1002/1817 [15:51:59<14:14:14, 62.89s/it] {'loss': 1.8916, 'learning_rate': 2e-05, 'epoch': 0.55} 55%|█████▌ | 1002/1817 [15:51:59<14:14:14, 62.89s/it] 55%|█████▌ | 1003/1817 [15:52:56<13:46:27, 60.92s/it] {'loss': 1.9445, 'learning_rate': 2e-05, 'epoch': 0.55} 55%|█████▌ | 1003/1817 [15:52:56<13:46:27, 60.92s/it] 55%|█████▌ | 1004/1817 [15:53:52<13:26:20, 59.51s/it] {'loss': 1.9319, 'learning_rate': 2e-05, 'epoch': 0.55} 55%|█████▌ | 1004/1817 [15:53:52<13:26:20, 59.51s/it] 55%|█████▌ | 1005/1817 [15:54:49<13:16:56, 58.89s/it] {'loss': 1.9203, 'learning_rate': 2e-05, 'epoch': 0.55} 55%|█████▌ | 1005/1817 [15:54:49<13:16:57, 58.89s/it] 55%|█████▌ | 1006/1817 [15:55:45<13:05:13, 58.09s/it] {'loss': 1.8989, 'learning_rate': 2e-05, 'epoch': 0.55} 55%|█████▌ | 1006/1817 [15:55:45<13:05:13, 58.09s/it] 55%|█████▌ | 1007/1817 [15:56:42<12:57:26, 57.59s/it] {'loss': 1.9186, 'learning_rate': 2e-05, 'epoch': 0.55} 55%|█████▌ | 1007/1817 [15:56:42<12:57:26, 57.59s/it] 55%|█████▌ | 1008/1817 [15:57:40<12:57:30, 57.66s/it] {'loss': 1.9238, 'learning_rate': 2e-05, 'epoch': 0.55} 55%|█████▌ | 1008/1817 [15:57:40<12:57:30, 57.66s/it] 56%|█████▌ | 1009/1817 [15:58:37<12:55:42, 57.60s/it] {'loss': 1.9008, 'learning_rate': 2e-05, 'epoch': 0.56} 56%|█████▌ | 1009/1817 [15:58:37<12:55:43, 57.60s/it] 56%|█████▌ | 1010/1817 [15:59:35<12:55:22, 57.65s/it] {'loss': 1.915, 'learning_rate': 2e-05, 'epoch': 0.56} 56%|█████▌ | 1010/1817 [15:59:35<12:55:22, 57.65s/it] 56%|█████▌ | 1011/1817 [16:00:31<12:49:11, 57.26s/it] {'loss': 1.9132, 'learning_rate': 2e-05, 'epoch': 0.56} 56%|█████▌ | 1011/1817 [16:00:31<12:49:11, 57.26s/it] 56%|█████▌ | 1012/1817 [16:01:29<12:49:37, 57.36s/it] {'loss': 1.9589, 'learning_rate': 2e-05, 'epoch': 0.56} 56%|█████▌ | 1012/1817 [16:01:29<12:49:26, 57.35s/it] 56%|█████▌ | 1013/1817 [16:02:26<12:45:00, 57.09s/it] {'loss': 1.8784, 'learning_rate': 2e-05, 'epoch': 0.56} 56%|█████▌ | 1013/1817 [16:02:25<12:44:46, 57.07s/it] 56%|█████▌ | 1014/1817 [16:03:23<12:45:13, 57.18s/it] {'loss': 1.9455, 'learning_rate': 2e-05, 'epoch': 0.56} 56%|█████▌ | 1014/1817 [16:03:23<12:45:01, 57.16s/it] 56%|█████▌ | 1015/1817 [16:04:19<12:41:01, 56.93s/it] {'loss': 1.9233, 'learning_rate': 2e-05, 'epoch': 0.56} 56%|█████▌ | 1015/1817 [16:04:19<12:40:53, 56.92s/it] 56%|█████▌ | 1016/1817 [16:05:15<12:36:58, 56.70s/it] {'loss': 1.8869, 'learning_rate': 2e-05, 'epoch': 0.56} 56%|█████▌ | 1016/1817 [16:05:15<12:36:57, 56.70s/it] 56%|█████▌ | 1017/1817 [16:06:13<12:40:29, 57.04s/it] {'loss': 1.9061, 'learning_rate': 2e-05, 'epoch': 0.56} 56%|█████▌ | 1017/1817 [16:06:13<12:40:29, 57.04s/it] 56%|█████▌ | 1018/1817 [16:07:09<12:37:23, 56.88s/it] {'loss': 1.9545, 'learning_rate': 2e-05, 'epoch': 0.56} 56%|█████▌ | 1018/1817 [16:07:09<12:37:23, 56.88s/it] 56%|█████▌ | 1019/1817 [16:08:08<12:42:42, 57.35s/it] {'loss': 1.9252, 'learning_rate': 2e-05, 'epoch': 0.56} 56%|█████▌ | 1019/1817 [16:08:08<12:42:42, 57.35s/it] 56%|█████▌ | 1020/1817 [16:09:06<12:45:00, 57.59s/it] {'loss': 1.8777, 'learning_rate': 2e-05, 'epoch': 0.56} 56%|█████▌ | 1020/1817 [16:09:06<12:45:05, 57.60s/it] 56%|█████▌ | 1021/1817 [16:10:02<12:37:54, 57.13s/it] {'loss': 1.9425, 'learning_rate': 2e-05, 'epoch': 0.56} 56%|█████▌ | 1021/1817 [16:10:02<12:37:58, 57.13s/it] 56%|█████▌ | 1022/1817 [16:10:59<12:37:03, 57.14s/it] {'loss': 1.8991, 'learning_rate': 2e-05, 'epoch': 0.56} 56%|█████▌ | 1022/1817 [16:10:59<12:37:03, 57.14s/it] 56%|█████▋ | 1023/1817 [16:11:57<12:38:06, 57.29s/it] {'loss': 1.9091, 'learning_rate': 2e-05, 'epoch': 0.56} 56%|█████▋ | 1023/1817 [16:11:57<12:38:06, 57.29s/it] 56%|█████▋ | 1024/1817 [16:12:53<12:32:43, 56.95s/it] {'loss': 1.9436, 'learning_rate': 2e-05, 'epoch': 0.56} 56%|█████▋ | 1024/1817 [16:12:53<12:32:45, 56.96s/it] 56%|█████▋ | 1025/1817 [16:13:51<12:32:36, 57.02s/it] {'loss': 1.9265, 'learning_rate': 2e-05, 'epoch': 0.56} 56%|█████▋ | 1025/1817 [16:13:50<12:32:37, 57.02s/it] 56%|█████▋ | 1026/1817 [16:14:47<12:28:50, 56.80s/it] {'loss': 1.9693, 'learning_rate': 2e-05, 'epoch': 0.56} 56%|█████▋ | 1026/1817 [16:14:47<12:28:51, 56.80s/it] 57%|█████▋ | 1027/1817 [16:15:43<12:25:58, 56.66s/it] {'loss': 1.9241, 'learning_rate': 2e-05, 'epoch': 0.57} 57%|█████▋ | 1027/1817 [16:15:43<12:25:58, 56.66s/it] 57%|█████▋ | 1028/1817 [16:16:39<12:23:15, 56.52s/it] {'loss': 1.9111, 'learning_rate': 2e-05, 'epoch': 0.57} 57%|█████▋ | 1028/1817 [16:16:39<12:23:14, 56.52s/it] 57%|█████▋ | 1029/1817 [16:17:37<12:28:44, 57.01s/it] {'loss': 1.8884, 'learning_rate': 2e-05, 'epoch': 0.57} 57%|█████▋ | 1029/1817 [16:17:37<12:28:44, 57.01s/it] 57%|█████▋ | 1030/1817 [16:18:34<12:25:19, 56.82s/it] {'loss': 1.8767, 'learning_rate': 2e-05, 'epoch': 0.57} 57%|█████▋ | 1030/1817 [16:18:34<12:25:19, 56.82s/it] 57%|█████▋ | 1031/1817 [16:19:32<12:30:56, 57.32s/it] {'loss': 1.8921, 'learning_rate': 2e-05, 'epoch': 0.57} 57%|█████▋ | 1031/1817 [16:19:32<12:30:56, 57.32s/it] 57%|█████▋ | 1032/1817 [16:20:29<12:27:08, 57.11s/it] {'loss': 1.8988, 'learning_rate': 2e-05, 'epoch': 0.57} 57%|█████▋ | 1032/1817 [16:20:29<12:27:08, 57.11s/it] 57%|█████▋ | 1033/1817 [16:21:26<12:25:45, 57.07s/it] {'loss': 1.896, 'learning_rate': 2e-05, 'epoch': 0.57} 57%|█████▋ | 1033/1817 [16:21:26<12:25:45, 57.07s/it] 57%|█████▋ | 1034/1817 [16:22:26<12:36:08, 57.94s/it] {'loss': 1.8838, 'learning_rate': 2e-05, 'epoch': 0.57} 57%|█████▋ | 1034/1817 [16:22:26<12:36:09, 57.94s/it] 57%|█████▋ | 1035/1817 [16:23:23<12:32:05, 57.70s/it] {'loss': 1.8892, 'learning_rate': 2e-05, 'epoch': 0.57} 57%|█████▋ | 1035/1817 [16:23:23<12:32:05, 57.70s/it] 57%|█████▋ | 1036/1817 [16:24:19<12:26:39, 57.36s/it] {'loss': 1.8835, 'learning_rate': 2e-05, 'epoch': 0.57} 57%|█████▋ | 1036/1817 [16:24:19<12:26:39, 57.36s/it] 57%|█████▋ | 1037/1817 [16:25:16<12:21:38, 57.05s/it] {'loss': 1.899, 'learning_rate': 2e-05, 'epoch': 0.57} 57%|█████▋ | 1037/1817 [16:25:16<12:21:38, 57.05s/it] 57%|█████▋ | 1038/1817 [16:26:14<12:24:59, 57.38s/it] {'loss': 1.9312, 'learning_rate': 2e-05, 'epoch': 0.57} 57%|█████▋ | 1038/1817 [16:26:14<12:24:59, 57.38s/it] 57%|█████▋ | 1039/1817 [16:27:11<12:23:55, 57.37s/it] {'loss': 1.9411, 'learning_rate': 2e-05, 'epoch': 0.57} 57%|█████▋ | 1039/1817 [16:27:11<12:23:55, 57.37s/it] 57%|█████▋ | 1040/1817 [16:28:09<12:25:53, 57.60s/it] {'loss': 1.9082, 'learning_rate': 2e-05, 'epoch': 0.57} 57%|█████▋ | 1040/1817 [16:28:09<12:25:53, 57.60s/it] 57%|█████▋ | 1041/1817 [16:29:05<12:18:45, 57.12s/it] {'loss': 1.886, 'learning_rate': 2e-05, 'epoch': 0.57} 57%|█████▋ | 1041/1817 [16:29:05<12:18:45, 57.12s/it] 57%|█████▋ | 1042/1817 [16:30:02<12:14:30, 56.86s/it] {'loss': 1.9338, 'learning_rate': 2e-05, 'epoch': 0.57} 57%|█████▋ | 1042/1817 [16:30:02<12:14:31, 56.87s/it] 57%|█████▋ | 1043/1817 [16:30:58<12:10:53, 56.66s/it] {'loss': 1.9365, 'learning_rate': 2e-05, 'epoch': 0.57} 57%|█████▋ | 1043/1817 [16:30:58<12:10:53, 56.66s/it] 57%|█████▋ | 1044/1817 [16:31:55<12:12:56, 56.89s/it] {'loss': 1.9842, 'learning_rate': 2e-05, 'epoch': 0.57} 57%|█████▋ | 1044/1817 [16:31:55<12:12:56, 56.89s/it] 58%|█████▊ | 1045/1817 [16:32:52<12:09:27, 56.69s/it] {'loss': 1.9097, 'learning_rate': 2e-05, 'epoch': 0.57} 58%|█████▊ | 1045/1817 [16:32:51<12:09:27, 56.69s/it] 58%|█████▊ | 1046/1817 [16:33:48<12:07:10, 56.59s/it] {'loss': 1.8748, 'learning_rate': 2e-05, 'epoch': 0.58} 58%|█████▊ | 1046/1817 [16:33:48<12:07:10, 56.59s/it] 58%|█████▊ | 1047/1817 [16:34:42<11:57:23, 55.90s/it] {'loss': 1.9338, 'learning_rate': 2e-05, 'epoch': 0.58} 58%|█████▊ | 1047/1817 [16:34:42<11:57:24, 55.90s/it] 58%|█████▊ | 1048/1817 [16:35:42<12:08:42, 56.86s/it] {'loss': 1.8928, 'learning_rate': 2e-05, 'epoch': 0.58} 58%|█████▊ | 1048/1817 [16:35:41<12:08:43, 56.86s/it] 58%|█████▊ | 1049/1817 [16:36:37<12:05:28, 56.68s/it] {'loss': 1.9193, 'learning_rate': 2e-05, 'epoch': 0.58} 58%|█████▊ | 1049/1817 [16:36:37<12:05:28, 56.68s/it] 58%|█████▊ | 1050/1817 [16:37:34<12:03:06, 56.57s/it] {'loss': 1.8707, 'learning_rate': 2e-05, 'epoch': 0.58} 58%|█████▊ | 1050/1817 [16:37:34<12:03:06, 56.57s/it] 58%|█████▊ | 1051/1817 [16:38:32<12:05:43, 56.85s/it] {'loss': 1.9104, 'learning_rate': 2e-05, 'epoch': 0.58} 58%|█████▊ | 1051/1817 [16:38:31<12:05:43, 56.85s/it] 58%|█████▊ | 1052/1817 [16:39:30<12:10:31, 57.30s/it] {'loss': 1.9184, 'learning_rate': 2e-05, 'epoch': 0.58} 58%|█████▊ | 1052/1817 [16:39:30<12:10:31, 57.30s/it] 58%|█████▊ | 1053/1817 [16:40:28<12:11:06, 57.42s/it] {'loss': 1.8941, 'learning_rate': 2e-05, 'epoch': 0.58} 58%|█████▊ | 1053/1817 [16:40:27<12:11:06, 57.42s/it] 58%|█████▊ | 1054/1817 [16:41:26<12:14:19, 57.74s/it] {'loss': 1.8891, 'learning_rate': 2e-05, 'epoch': 0.58} 58%|█████▊ | 1054/1817 [16:41:26<12:14:18, 57.74s/it] 58%|█████▊ | 1055/1817 [16:42:22<12:07:45, 57.30s/it] {'loss': 1.9193, 'learning_rate': 2e-05, 'epoch': 0.58} 58%|█████▊ | 1055/1817 [16:42:22<12:07:45, 57.30s/it] 58%|█████▊ | 1056/1817 [16:43:19<12:05:36, 57.21s/it] {'loss': 1.9257, 'learning_rate': 2e-05, 'epoch': 0.58} 58%|█████▊ | 1056/1817 [16:43:19<12:05:36, 57.21s/it] 58%|█████▊ | 1057/1817 [16:44:19<12:13:45, 57.93s/it] {'loss': 1.8834, 'learning_rate': 2e-05, 'epoch': 0.58} 58%|█████▊ | 1057/1817 [16:44:19<12:13:45, 57.93s/it] 58%|█████▊ | 1058/1817 [16:45:15<12:06:25, 57.43s/it] {'loss': 1.9046, 'learning_rate': 2e-05, 'epoch': 0.58} 58%|█████▊ | 1058/1817 [16:45:15<12:06:26, 57.43s/it] 58%|█████▊ | 1059/1817 [16:46:13<12:05:28, 57.43s/it] {'loss': 1.9194, 'learning_rate': 2e-05, 'epoch': 0.58} 58%|█████▊ | 1059/1817 [16:46:12<12:05:28, 57.43s/it] 58%|█████▊ | 1060/1817 [16:47:09<11:59:16, 57.01s/it] {'loss': 1.8812, 'learning_rate': 2e-05, 'epoch': 0.58} 58%|█████▊ | 1060/1817 [16:47:08<11:59:16, 57.01s/it] 58%|█████▊ | 1061/1817 [16:48:05<11:55:06, 56.75s/it] {'loss': 1.8702, 'learning_rate': 2e-05, 'epoch': 0.58} 58%|█████▊ | 1061/1817 [16:48:05<11:55:06, 56.75s/it] 58%|█████▊ | 1062/1817 [16:49:01<11:51:11, 56.52s/it] {'loss': 1.9278, 'learning_rate': 2e-05, 'epoch': 0.58} 58%|█████▊ | 1062/1817 [16:49:01<11:51:11, 56.52s/it] 59%|█████▊ | 1063/1817 [16:49:54<11:37:28, 55.50s/it] {'loss': 1.9469, 'learning_rate': 2e-05, 'epoch': 0.58} 59%|█████▊ | 1063/1817 [16:49:54<11:37:28, 55.50s/it] 59%|█████▊ | 1064/1817 [16:50:49<11:32:57, 55.22s/it] {'loss': 1.9109, 'learning_rate': 2e-05, 'epoch': 0.59} 59%|█████▊ | 1064/1817 [16:50:48<11:32:57, 55.22s/it] 59%|█████▊ | 1065/1817 [16:51:45<11:35:41, 55.51s/it] {'loss': 1.9061, 'learning_rate': 2e-05, 'epoch': 0.59} 59%|█████▊ | 1065/1817 [16:51:44<11:35:41, 55.51s/it] 59%|█████▊ | 1066/1817 [16:52:42<11:41:37, 56.06s/it] {'loss': 1.907, 'learning_rate': 2e-05, 'epoch': 0.59} 59%|█████▊ | 1066/1817 [16:52:42<11:41:37, 56.06s/it] 59%|█████▊ | 1067/1817 [16:53:39<11:43:31, 56.28s/it] {'loss': 1.9263, 'learning_rate': 2e-05, 'epoch': 0.59} 59%|█████▊ | 1067/1817 [16:53:39<11:43:31, 56.28s/it] 59%|█████▉ | 1068/1817 [16:54:37<11:49:55, 56.87s/it] {'loss': 1.9163, 'learning_rate': 2e-05, 'epoch': 0.59} 59%|█████▉ | 1068/1817 [16:54:37<11:49:56, 56.87s/it] 59%|█████▉ | 1069/1817 [16:55:33<11:46:45, 56.69s/it] {'loss': 1.923, 'learning_rate': 2e-05, 'epoch': 0.59} 59%|█████▉ | 1069/1817 [16:55:33<11:46:45, 56.69s/it] 59%|█████▉ | 1070/1817 [16:56:27<11:35:05, 55.83s/it] {'loss': 1.9259, 'learning_rate': 2e-05, 'epoch': 0.59} 59%|█████▉ | 1070/1817 [16:56:27<11:35:05, 55.83s/it] 59%|█████▉ | 1071/1817 [16:57:23<11:35:05, 55.91s/it] {'loss': 1.9271, 'learning_rate': 2e-05, 'epoch': 0.59} 59%|█████▉ | 1071/1817 [16:57:23<11:35:05, 55.91s/it] 59%|█████▉ | 1072/1817 [16:58:20<11:35:28, 56.01s/it] {'loss': 1.8975, 'learning_rate': 2e-05, 'epoch': 0.59} 59%|█████▉ | 1072/1817 [16:58:19<11:35:28, 56.01s/it] 59%|█████▉ | 1073/1817 [16:59:19<11:46:02, 56.94s/it] {'loss': 1.8953, 'learning_rate': 2e-05, 'epoch': 0.59} 59%|█████▉ | 1073/1817 [16:59:18<11:46:02, 56.94s/it] 59%|█████▉ | 1074/1817 [17:00:16<11:46:41, 57.07s/it] {'loss': 1.9556, 'learning_rate': 2e-05, 'epoch': 0.59} 59%|█████▉ | 1074/1817 [17:00:16<11:46:41, 57.07s/it] 59%|█████▉ | 1075/1817 [17:01:11<11:38:00, 56.44s/it] {'loss': 1.9131, 'learning_rate': 2e-05, 'epoch': 0.59} 59%|█████▉ | 1075/1817 [17:01:11<11:38:00, 56.44s/it] 59%|█████▉ | 1076/1817 [17:02:07<11:36:14, 56.38s/it] {'loss': 1.9243, 'learning_rate': 2e-05, 'epoch': 0.59} 59%|█████▉ | 1076/1817 [17:02:07<11:36:14, 56.38s/it] 59%|█████▉ | 1077/1817 [17:03:05<11:40:20, 56.78s/it] {'loss': 1.9213, 'learning_rate': 2e-05, 'epoch': 0.59} 59%|█████▉ | 1077/1817 [17:03:05<11:40:20, 56.79s/it] 59%|█████▉ | 1078/1817 [17:04:02<11:41:32, 56.96s/it] {'loss': 1.9346, 'learning_rate': 2e-05, 'epoch': 0.59} 59%|█████▉ | 1078/1817 [17:04:02<11:41:32, 56.96s/it] 59%|█████▉ | 1079/1817 [17:04:58<11:37:22, 56.70s/it] {'loss': 1.8825, 'learning_rate': 2e-05, 'epoch': 0.59} 59%|█████▉ | 1079/1817 [17:04:58<11:37:17, 56.69s/it] 59%|█████▉ | 1080/1817 [17:05:57<11:43:30, 57.27s/it] {'loss': 1.9115, 'learning_rate': 2e-05, 'epoch': 0.59} 59%|█████▉ | 1080/1817 [17:05:57<11:43:30, 57.27s/it] 59%|█████▉ | 1081/1817 [17:06:56<11:47:51, 57.71s/it] {'loss': 1.9528, 'learning_rate': 2e-05, 'epoch': 0.59} 59%|█████▉ | 1081/1817 [17:06:55<11:47:49, 57.70s/it] 60%|█████▉ | 1082/1817 [17:07:52<11:42:06, 57.31s/it] {'loss': 1.8954, 'learning_rate': 2e-05, 'epoch': 0.6} 60%|█████▉ | 1082/1817 [17:07:52<11:42:06, 57.31s/it] 60%|█████▉ | 1083/1817 [17:08:50<11:42:32, 57.43s/it] {'loss': 1.8903, 'learning_rate': 2e-05, 'epoch': 0.6} 60%|█████▉ | 1083/1817 [17:08:50<11:42:32, 57.43s/it] 60%|█████▉ | 1084/1817 [17:09:48<11:43:27, 57.58s/it] {'loss': 1.9214, 'learning_rate': 2e-05, 'epoch': 0.6} 60%|█████▉ | 1084/1817 [17:09:47<11:43:27, 57.58s/it] 60%|█████▉ | 1085/1817 [17:10:45<11:42:08, 57.55s/it] {'loss': 1.9049, 'learning_rate': 2e-05, 'epoch': 0.6} 60%|█████▉ | 1085/1817 [17:10:45<11:42:08, 57.55s/it] 60%|█████▉ | 1086/1817 [17:11:44<11:44:02, 57.79s/it] {'loss': 1.9336, 'learning_rate': 2e-05, 'epoch': 0.6} 60%|█████▉ | 1086/1817 [17:11:43<11:44:02, 57.79s/it] 60%|█████▉ | 1087/1817 [17:12:40<11:37:07, 57.30s/it] {'loss': 1.8831, 'learning_rate': 2e-05, 'epoch': 0.6} 60%|█████▉ | 1087/1817 [17:12:39<11:37:07, 57.30s/it] 60%|█████▉ | 1088/1817 [17:13:36<11:32:31, 57.00s/it] {'loss': 1.9231, 'learning_rate': 2e-05, 'epoch': 0.6} 60%|█████▉ | 1088/1817 [17:13:36<11:32:31, 57.00s/it] 60%|█████▉ | 1089/1817 [17:14:33<11:32:52, 57.11s/it] {'loss': 1.91, 'learning_rate': 2e-05, 'epoch': 0.6} 60%|█████▉ | 1089/1817 [17:14:33<11:32:52, 57.11s/it] 60%|█████▉ | 1090/1817 [17:15:30<11:29:04, 56.87s/it] {'loss': 1.9233, 'learning_rate': 2e-05, 'epoch': 0.6} 60%|█████▉ | 1090/1817 [17:15:29<11:29:04, 56.87s/it] 60%|██████ | 1091/1817 [17:16:26<11:25:46, 56.68s/it] {'loss': 1.8909, 'learning_rate': 2e-05, 'epoch': 0.6} 60%|██████ | 1091/1817 [17:16:26<11:25:46, 56.68s/it] 60%|██████ | 1092/1817 [17:17:24<11:28:20, 56.97s/it] {'loss': 1.9338, 'learning_rate': 2e-05, 'epoch': 0.6} 60%|██████ | 1092/1817 [17:17:23<11:28:20, 56.97s/it] 60%|██████ | 1093/1817 [17:18:22<11:31:00, 57.27s/it] {'loss': 1.8822, 'learning_rate': 2e-05, 'epoch': 0.6} 60%|██████ | 1093/1817 [17:18:21<11:31:01, 57.27s/it] 60%|██████ | 1094/1817 [17:19:17<11:26:14, 56.95s/it] {'loss': 1.8902, 'learning_rate': 2e-05, 'epoch': 0.6} 60%|██████ | 1094/1817 [17:19:17<11:26:14, 56.95s/it] 60%|██████ | 1095/1817 [17:20:14<11:21:45, 56.66s/it] {'loss': 1.9269, 'learning_rate': 2e-05, 'epoch': 0.6} 60%|██████ | 1095/1817 [17:20:13<11:21:46, 56.66s/it] 60%|██████ | 1096/1817 [17:21:12<11:25:01, 57.01s/it] {'loss': 1.9037, 'learning_rate': 2e-05, 'epoch': 0.6} 60%|██████ | 1096/1817 [17:21:11<11:25:01, 57.01s/it] 60%|██████ | 1097/1817 [17:22:07<11:20:53, 56.74s/it] {'loss': 1.8851, 'learning_rate': 2e-05, 'epoch': 0.6} 60%|██████ | 1097/1817 [17:22:07<11:20:53, 56.74s/it] 60%|██████ | 1098/1817 [17:23:03<11:14:31, 56.29s/it] {'loss': 1.8774, 'learning_rate': 2e-05, 'epoch': 0.6} 60%|██████ | 1098/1817 [17:23:03<11:14:31, 56.29s/it] 60%|██████ | 1099/1817 [17:23:59<11:13:41, 56.30s/it] {'loss': 1.8959, 'learning_rate': 2e-05, 'epoch': 0.6} 60%|██████ | 1099/1817 [17:23:59<11:13:41, 56.30s/it] 61%|██████ | 1100/1817 [17:24:55<11:12:30, 56.28s/it] {'loss': 1.9002, 'learning_rate': 2e-05, 'epoch': 0.61} 61%|██████ | 1100/1817 [17:24:55<11:12:30, 56.28s/it] 61%|██████ | 1101/1817 [17:25:52<11:11:39, 56.28s/it] {'loss': 1.9466, 'learning_rate': 2e-05, 'epoch': 0.61} 61%|██████ | 1101/1817 [17:25:51<11:11:39, 56.28s/it] 61%|██████ | 1102/1817 [17:26:49<11:15:50, 56.71s/it] {'loss': 1.9353, 'learning_rate': 2e-05, 'epoch': 0.61} 61%|██████ | 1102/1817 [17:26:49<11:15:50, 56.71s/it] 61%|██████ | 1103/1817 [17:27:46<11:12:29, 56.51s/it] {'loss': 1.9319, 'learning_rate': 2e-05, 'epoch': 0.61} 61%|██████ | 1103/1817 [17:27:45<11:12:29, 56.51s/it] 61%|██████ | 1104/1817 [17:28:44<11:21:20, 57.34s/it] {'loss': 1.9197, 'learning_rate': 2e-05, 'epoch': 0.61} 61%|██████ | 1104/1817 [17:28:44<11:21:20, 57.34s/it] 61%|██████ | 1105/1817 [17:29:41<11:16:12, 56.98s/it] {'loss': 1.9158, 'learning_rate': 2e-05, 'epoch': 0.61} 61%|██████ | 1105/1817 [17:29:41<11:16:12, 56.98s/it] 61%|██████ | 1106/1817 [17:30:37<11:12:30, 56.75s/it] {'loss': 1.9674, 'learning_rate': 2e-05, 'epoch': 0.61} 61%|██████ | 1106/1817 [17:30:37<11:12:30, 56.75s/it] 61%|██████ | 1107/1817 [17:31:33<11:10:04, 56.63s/it] {'loss': 1.8882, 'learning_rate': 2e-05, 'epoch': 0.61} 61%|██████ | 1107/1817 [17:31:33<11:10:04, 56.63s/it] 61%|██████ | 1108/1817 [17:32:30<11:08:37, 56.58s/it] {'loss': 1.8946, 'learning_rate': 2e-05, 'epoch': 0.61} 61%|██████ | 1108/1817 [17:32:30<11:08:37, 56.58s/it] 61%|██████ | 1109/1817 [17:33:26<11:06:13, 56.46s/it] {'loss': 1.8641, 'learning_rate': 2e-05, 'epoch': 0.61} 61%|██████ | 1109/1817 [17:33:26<11:06:13, 56.46s/it] 61%|██████ | 1110/1817 [17:34:22<11:04:45, 56.42s/it] {'loss': 1.884, 'learning_rate': 2e-05, 'epoch': 0.61} 61%|██████ | 1110/1817 [17:34:22<11:04:45, 56.42s/it] 61%|██████ | 1111/1817 [17:35:19<11:03:45, 56.41s/it] {'loss': 1.9447, 'learning_rate': 2e-05, 'epoch': 0.61} 61%|██████ | 1111/1817 [17:35:19<11:03:45, 56.41s/it] 61%|██████ | 1112/1817 [17:36:16<11:07:58, 56.85s/it] {'loss': 1.8804, 'learning_rate': 2e-05, 'epoch': 0.61} 61%|██████ | 1112/1817 [17:36:16<11:07:58, 56.85s/it] 61%|██████▏ | 1113/1817 [17:37:12<11:02:15, 56.44s/it] {'loss': 1.8948, 'learning_rate': 2e-05, 'epoch': 0.61} 61%|██████▏ | 1113/1817 [17:37:12<11:02:15, 56.44s/it] 61%|██████▏ | 1114/1817 [17:38:08<11:01:14, 56.44s/it] {'loss': 1.9182, 'learning_rate': 2e-05, 'epoch': 0.61} 61%|██████▏ | 1114/1817 [17:38:08<11:01:14, 56.44s/it] 61%|██████▏ | 1115/1817 [17:39:05<10:59:51, 56.40s/it] {'loss': 1.889, 'learning_rate': 2e-05, 'epoch': 0.61} 61%|██████▏ | 1115/1817 [17:39:05<10:59:52, 56.40s/it] 61%|██████▏ | 1116/1817 [17:40:01<10:58:27, 56.36s/it] {'loss': 1.9411, 'learning_rate': 2e-05, 'epoch': 0.61} 61%|██████▏ | 1116/1817 [17:40:01<10:58:27, 56.36s/it] 61%|██████▏ | 1117/1817 [17:41:01<11:09:26, 57.38s/it] {'loss': 1.9351, 'learning_rate': 2e-05, 'epoch': 0.61} 61%|██████▏ | 1117/1817 [17:41:01<11:09:26, 57.38s/it] 62%|██████▏ | 1118/1817 [17:41:57<11:04:52, 57.07s/it] {'loss': 1.9243, 'learning_rate': 2e-05, 'epoch': 0.62} 62%|██████▏ | 1118/1817 [17:41:57<11:04:53, 57.07s/it] 62%|██████▏ | 1119/1817 [17:42:54<11:02:16, 56.93s/it] {'loss': 1.8489, 'learning_rate': 2e-05, 'epoch': 0.62} 62%|██████▏ | 1119/1817 [17:42:54<11:02:16, 56.93s/it] 62%|██████▏ | 1120/1817 [17:43:50<10:59:55, 56.81s/it] {'loss': 1.8613, 'learning_rate': 2e-05, 'epoch': 0.62} 62%|██████▏ | 1120/1817 [17:43:50<10:59:55, 56.81s/it] 62%|██████▏ | 1121/1817 [17:44:48<11:03:02, 57.16s/it] {'loss': 1.8945, 'learning_rate': 2e-05, 'epoch': 0.62} 62%|██████▏ | 1121/1817 [17:44:48<11:03:02, 57.16s/it] 62%|██████▏ | 1122/1817 [17:45:44<10:59:03, 56.90s/it] {'loss': 1.9423, 'learning_rate': 2e-05, 'epoch': 0.62} 62%|██████▏ | 1122/1817 [17:45:44<10:59:03, 56.90s/it] 62%|██████▏ | 1123/1817 [17:46:41<10:55:28, 56.67s/it] {'loss': 1.8926, 'learning_rate': 2e-05, 'epoch': 0.62} 62%|██████▏ | 1123/1817 [17:46:41<10:55:28, 56.67s/it] 62%|██████▏ | 1124/1817 [17:47:36<10:48:44, 56.17s/it] {'loss': 1.9124, 'learning_rate': 2e-05, 'epoch': 0.62} 62%|██████▏ | 1124/1817 [17:47:36<10:48:45, 56.17s/it] 62%|██████▏ | 1125/1817 [17:48:34<10:55:05, 56.80s/it] {'loss': 1.9278, 'learning_rate': 2e-05, 'epoch': 0.62} 62%|██████▏ | 1125/1817 [17:48:34<10:55:05, 56.80s/it] 62%|██████▏ | 1126/1817 [17:49:30<10:51:53, 56.60s/it] {'loss': 1.9291, 'learning_rate': 2e-05, 'epoch': 0.62} 62%|██████▏ | 1126/1817 [17:49:30<10:51:53, 56.60s/it] 62%|██████▏ | 1127/1817 [17:50:28<10:57:01, 57.13s/it] {'loss': 1.9015, 'learning_rate': 2e-05, 'epoch': 0.62} 62%|██████▏ | 1127/1817 [17:50:28<10:57:01, 57.13s/it] 62%|██████▏ | 1128/1817 [17:51:26<10:59:28, 57.43s/it] {'loss': 1.8933, 'learning_rate': 2e-05, 'epoch': 0.62} 62%|██████▏ | 1128/1817 [17:51:26<10:59:28, 57.43s/it] 62%|██████▏ | 1129/1817 [17:52:24<10:59:23, 57.50s/it] {'loss': 1.9189, 'learning_rate': 2e-05, 'epoch': 0.62} 62%|██████▏ | 1129/1817 [17:52:24<10:59:23, 57.50s/it] 62%|██████▏ | 1130/1817 [17:53:22<10:58:02, 57.47s/it] {'loss': 1.9304, 'learning_rate': 2e-05, 'epoch': 0.62} 62%|██████▏ | 1130/1817 [17:53:22<10:58:02, 57.47s/it] 62%|██████▏ | 1131/1817 [17:54:20<11:00:08, 57.74s/it] {'loss': 1.9234, 'learning_rate': 2e-05, 'epoch': 0.62} 62%|██████▏ | 1131/1817 [17:54:20<11:00:08, 57.74s/it] 62%|██████▏ | 1132/1817 [17:55:18<11:00:42, 57.87s/it] {'loss': 1.8982, 'learning_rate': 2e-05, 'epoch': 0.62} 62%|██████▏ | 1132/1817 [17:55:18<11:00:42, 57.87s/it] 62%|██████▏ | 1133/1817 [17:56:17<11:04:13, 58.27s/it] {'loss': 1.9047, 'learning_rate': 2e-05, 'epoch': 0.62} 62%|██████▏ | 1133/1817 [17:56:17<11:04:13, 58.27s/it] 62%|██████▏ | 1134/1817 [17:57:13<10:55:46, 57.61s/it] {'loss': 1.9054, 'learning_rate': 2e-05, 'epoch': 0.62} 62%|██████▏ | 1134/1817 [17:57:13<10:55:46, 57.61s/it] 62%|██████▏ | 1135/1817 [17:58:13<11:02:14, 58.26s/it] {'loss': 1.9219, 'learning_rate': 2e-05, 'epoch': 0.62} 62%|██████▏ | 1135/1817 [17:58:13<11:02:14, 58.26s/it] 63%|██████▎ | 1136/1817 [17:59:11<10:58:28, 58.02s/it] {'loss': 1.8941, 'learning_rate': 2e-05, 'epoch': 0.63} 63%|██████▎ | 1136/1817 [17:59:11<10:58:28, 58.02s/it] 63%|██████▎ | 1137/1817 [18:00:07<10:51:14, 57.46s/it] {'loss': 1.9393, 'learning_rate': 2e-05, 'epoch': 0.63} 63%|██████▎ | 1137/1817 [18:00:07<10:51:14, 57.46s/it] 63%|██████▎ | 1138/1817 [18:01:04<10:49:47, 57.42s/it] {'loss': 1.8799, 'learning_rate': 2e-05, 'epoch': 0.63} 63%|██████▎ | 1138/1817 [18:01:04<10:49:47, 57.42s/it] 63%|██████▎ | 1139/1817 [18:02:02<10:49:48, 57.50s/it] {'loss': 1.9107, 'learning_rate': 2e-05, 'epoch': 0.63} 63%|██████▎ | 1139/1817 [18:02:02<10:49:36, 57.49s/it] 63%|██████▎ | 1140/1817 [18:02:58<10:43:43, 57.05s/it] {'loss': 1.8967, 'learning_rate': 2e-05, 'epoch': 0.63} 63%|██████▎ | 1140/1817 [18:02:58<10:43:43, 57.05s/it] 63%|██████▎ | 1141/1817 [18:03:54<10:40:24, 56.84s/it] {'loss': 1.8722, 'learning_rate': 2e-05, 'epoch': 0.63} 63%|██████▎ | 1141/1817 [18:03:54<10:40:24, 56.84s/it] 63%|██████▎ | 1142/1817 [18:04:50<10:37:34, 56.67s/it] {'loss': 1.9187, 'learning_rate': 2e-05, 'epoch': 0.63} 63%|██████▎ | 1142/1817 [18:04:50<10:37:34, 56.67s/it] 63%|██████▎ | 1143/1817 [18:05:47<10:36:10, 56.63s/it] {'loss': 1.9394, 'learning_rate': 2e-05, 'epoch': 0.63} 63%|██████▎ | 1143/1817 [18:05:47<10:36:10, 56.63s/it] 63%|██████▎ | 1144/1817 [18:06:45<10:40:21, 57.09s/it] {'loss': 1.9015, 'learning_rate': 2e-05, 'epoch': 0.63} 63%|██████▎ | 1144/1817 [18:06:45<10:40:21, 57.09s/it] 63%|██████▎ | 1145/1817 [18:07:41<10:37:04, 56.88s/it] {'loss': 1.8712, 'learning_rate': 2e-05, 'epoch': 0.63} 63%|██████▎ | 1145/1817 [18:07:41<10:37:04, 56.88s/it] 63%|██████▎ | 1146/1817 [18:08:41<10:45:09, 57.69s/it] {'loss': 1.9337, 'learning_rate': 2e-05, 'epoch': 0.63} 63%|██████▎ | 1146/1817 [18:08:41<10:45:09, 57.69s/it] 63%|██████▎ | 1147/1817 [18:09:37<10:40:02, 57.32s/it] {'loss': 1.8811, 'learning_rate': 2e-05, 'epoch': 0.63} 63%|██████▎ | 1147/1817 [18:09:37<10:40:02, 57.32s/it] 63%|██████▎ | 1148/1817 [18:10:34<10:37:18, 57.16s/it] {'loss': 1.9125, 'learning_rate': 2e-05, 'epoch': 0.63} 63%|██████▎ | 1148/1817 [18:10:34<10:37:18, 57.16s/it] 63%|██████▎ | 1149/1817 [18:11:31<10:33:20, 56.89s/it] {'loss': 1.9038, 'learning_rate': 2e-05, 'epoch': 0.63} 63%|██████▎ | 1149/1817 [18:11:31<10:33:20, 56.89s/it] 63%|██████▎ | 1150/1817 [18:12:27<10:30:35, 56.73s/it] {'loss': 1.8901, 'learning_rate': 2e-05, 'epoch': 0.63} 63%|██████▎ | 1150/1817 [18:12:27<10:30:35, 56.73s/it] 63%|██████▎ | 1151/1817 [18:13:27<10:40:54, 57.74s/it] {'loss': 1.9274, 'learning_rate': 2e-05, 'epoch': 0.63} 63%|██████▎ | 1151/1817 [18:13:27<10:40:54, 57.74s/it] 63%|██████▎ | 1152/1817 [18:14:24<10:37:52, 57.55s/it] {'loss': 1.8974, 'learning_rate': 2e-05, 'epoch': 0.63} 63%|██████▎ | 1152/1817 [18:14:24<10:37:52, 57.55s/it] 63%|██████▎ | 1153/1817 [18:15:20<10:32:52, 57.19s/it] {'loss': 1.8927, 'learning_rate': 2e-05, 'epoch': 0.63} 63%|██████▎ | 1153/1817 [18:15:20<10:32:52, 57.19s/it] 64%|██████▎ | 1154/1817 [18:16:21<10:41:44, 58.08s/it] {'loss': 1.8723, 'learning_rate': 2e-05, 'epoch': 0.63} 64%|██████▎ | 1154/1817 [18:16:21<10:41:44, 58.08s/it] 64%|██████▎ | 1155/1817 [18:17:18<10:39:29, 57.96s/it] {'loss': 1.9132, 'learning_rate': 2e-05, 'epoch': 0.64} 64%|██████▎ | 1155/1817 [18:17:18<10:39:29, 57.96s/it] 64%|██████▎ | 1156/1817 [18:18:15<10:33:30, 57.50s/it] {'loss': 1.8757, 'learning_rate': 2e-05, 'epoch': 0.64} 64%|██████▎ | 1156/1817 [18:18:15<10:33:30, 57.50s/it] 64%|██████▎ | 1157/1817 [18:19:13<10:33:56, 57.63s/it] {'loss': 1.9209, 'learning_rate': 2e-05, 'epoch': 0.64} 64%|██████▎ | 1157/1817 [18:19:13<10:33:56, 57.63s/it] 64%|██████▎ | 1158/1817 [18:20:13<10:42:30, 58.50s/it] {'loss': 1.9229, 'learning_rate': 2e-05, 'epoch': 0.64} 64%|██████▎ | 1158/1817 [18:20:13<10:42:30, 58.50s/it] 64%|██████▍ | 1159/1817 [18:21:10<10:35:56, 57.99s/it] {'loss': 1.8917, 'learning_rate': 2e-05, 'epoch': 0.64} 64%|██████▍ | 1159/1817 [18:21:10<10:35:56, 57.99s/it] 64%|██████▍ | 1160/1817 [18:22:07<10:33:12, 57.83s/it] {'loss': 1.8911, 'learning_rate': 2e-05, 'epoch': 0.64} 64%|██████▍ | 1160/1817 [18:22:07<10:33:12, 57.83s/it] 64%|██████▍ | 1161/1817 [18:23:05<10:30:17, 57.65s/it] {'loss': 1.882, 'learning_rate': 2e-05, 'epoch': 0.64} 64%|██████▍ | 1161/1817 [18:23:05<10:30:17, 57.65s/it] 64%|██████▍ | 1162/1817 [18:24:02<10:28:12, 57.55s/it] {'loss': 1.8609, 'learning_rate': 2e-05, 'epoch': 0.64} 64%|██████▍ | 1162/1817 [18:24:02<10:28:12, 57.55s/it] 64%|██████▍ | 1163/1817 [18:24:59<10:24:10, 57.26s/it] {'loss': 1.9107, 'learning_rate': 2e-05, 'epoch': 0.64} 64%|██████▍ | 1163/1817 [18:24:59<10:24:10, 57.26s/it] 64%|██████▍ | 1164/1817 [18:25:56<10:23:07, 57.25s/it] {'loss': 1.8687, 'learning_rate': 2e-05, 'epoch': 0.64} 64%|██████▍ | 1164/1817 [18:25:56<10:23:07, 57.25s/it] 64%|██████▍ | 1165/1817 [18:26:51<10:15:19, 56.63s/it] {'loss': 1.9189, 'learning_rate': 2e-05, 'epoch': 0.64} 64%|██████▍ | 1165/1817 [18:26:51<10:15:19, 56.63s/it] 64%|██████▍ | 1166/1817 [18:27:49<10:18:51, 57.04s/it] {'loss': 1.8737, 'learning_rate': 2e-05, 'epoch': 0.64} 64%|██████▍ | 1166/1817 [18:27:49<10:18:51, 57.04s/it] 64%|██████▍ | 1167/1817 [18:28:45<10:15:09, 56.78s/it] {'loss': 1.9371, 'learning_rate': 2e-05, 'epoch': 0.64} 64%|██████▍ | 1167/1817 [18:28:45<10:15:09, 56.78s/it] 64%|██████▍ | 1168/1817 [18:29:43<10:17:03, 57.05s/it] {'loss': 1.9365, 'learning_rate': 2e-05, 'epoch': 0.64} 64%|██████▍ | 1168/1817 [18:29:43<10:17:03, 57.05s/it] 64%|██████▍ | 1169/1817 [18:30:44<10:28:47, 58.22s/it] {'loss': 1.9018, 'learning_rate': 2e-05, 'epoch': 0.64} 64%|██████▍ | 1169/1817 [18:30:44<10:28:47, 58.22s/it] 64%|██████▍ | 1170/1817 [18:31:38<10:15:57, 57.12s/it] {'loss': 1.9144, 'learning_rate': 2e-05, 'epoch': 0.64} 64%|██████▍ | 1170/1817 [18:31:38<10:15:57, 57.12s/it] 64%|██████▍ | 1171/1817 [18:32:37<10:17:32, 57.36s/it] {'loss': 1.9048, 'learning_rate': 2e-05, 'epoch': 0.64} 64%|██████▍ | 1171/1817 [18:32:36<10:17:32, 57.36s/it] 65%|██████▍ | 1172/1817 [18:33:33<10:14:47, 57.19s/it] {'loss': 1.915, 'learning_rate': 2e-05, 'epoch': 0.64} 65%|██████▍ | 1172/1817 [18:33:33<10:14:47, 57.19s/it] 65%|██████▍ | 1173/1817 [18:34:30<10:11:32, 56.98s/it] {'loss': 1.8994, 'learning_rate': 2e-05, 'epoch': 0.65} 65%|██████▍ | 1173/1817 [18:34:29<10:11:32, 56.98s/it] 65%|██████▍ | 1174/1817 [18:35:28<10:14:54, 57.38s/it] {'loss': 1.9424, 'learning_rate': 2e-05, 'epoch': 0.65} 65%|██████▍ | 1174/1817 [18:35:28<10:14:55, 57.38s/it] 65%|██████▍ | 1175/1817 [18:36:24<10:10:57, 57.10s/it] {'loss': 1.9452, 'learning_rate': 2e-05, 'epoch': 0.65} 65%|██████▍ | 1175/1817 [18:36:24<10:10:57, 57.10s/it] 65%|██████▍ | 1176/1817 [18:37:21<10:08:23, 56.95s/it] {'loss': 1.9099, 'learning_rate': 2e-05, 'epoch': 0.65} 65%|██████▍ | 1176/1817 [18:37:21<10:08:24, 56.95s/it] 65%|██████▍ | 1177/1817 [18:38:20<10:14:47, 57.64s/it] {'loss': 1.9334, 'learning_rate': 2e-05, 'epoch': 0.65} 65%|██████▍ | 1177/1817 [18:38:20<10:14:47, 57.64s/it] 65%|██████▍ | 1178/1817 [18:39:21<10:21:57, 58.40s/it] {'loss': 1.8772, 'learning_rate': 2e-05, 'epoch': 0.65} 65%|██████▍ | 1178/1817 [18:39:20<10:21:57, 58.40s/it] 65%|██████▍ | 1179/1817 [18:40:16<10:13:58, 57.74s/it] {'loss': 1.9177, 'learning_rate': 2e-05, 'epoch': 0.65} 65%|██████▍ | 1179/1817 [18:40:16<10:13:58, 57.74s/it] 65%|██████▍ | 1180/1817 [18:41:15<10:15:27, 57.97s/it] {'loss': 1.9185, 'learning_rate': 2e-05, 'epoch': 0.65} 65%|██████▍ | 1180/1817 [18:41:15<10:15:27, 57.97s/it] 65%|██████▍ | 1181/1817 [18:42:13<10:13:57, 57.92s/it] {'loss': 1.8946, 'learning_rate': 2e-05, 'epoch': 0.65} 65%|██████▍ | 1181/1817 [18:42:13<10:13:57, 57.92s/it] 65%|██████▌ | 1182/1817 [18:43:11<10:13:13, 57.94s/it] {'loss': 1.8935, 'learning_rate': 2e-05, 'epoch': 0.65} 65%|██████▌ | 1182/1817 [18:43:11<10:13:13, 57.94s/it] 65%|██████▌ | 1183/1817 [18:44:07<10:07:44, 57.51s/it] {'loss': 1.8897, 'learning_rate': 2e-05, 'epoch': 0.65} 65%|██████▌ | 1183/1817 [18:44:07<10:07:44, 57.51s/it] 65%|██████▌ | 1184/1817 [18:45:06<10:09:54, 57.81s/it] {'loss': 1.9064, 'learning_rate': 2e-05, 'epoch': 0.65} 65%|██████▌ | 1184/1817 [18:45:06<10:09:54, 57.81s/it] 65%|██████▌ | 1185/1817 [18:46:04<10:11:08, 58.02s/it] {'loss': 1.9426, 'learning_rate': 2e-05, 'epoch': 0.65} 65%|██████▌ | 1185/1817 [18:46:04<10:11:08, 58.02s/it] 65%|██████▌ | 1186/1817 [18:47:01<10:04:49, 57.51s/it] {'loss': 1.8851, 'learning_rate': 2e-05, 'epoch': 0.65} 65%|██████▌ | 1186/1817 [18:47:01<10:04:49, 57.51s/it] 65%|██████▌ | 1187/1817 [18:47:57<10:00:32, 57.19s/it] {'loss': 1.9157, 'learning_rate': 2e-05, 'epoch': 0.65} 65%|██████▌ | 1187/1817 [18:47:57<10:00:32, 57.20s/it] 65%|██████▌ | 1188/1817 [18:48:56<10:03:42, 57.59s/it] {'loss': 1.9034, 'learning_rate': 2e-05, 'epoch': 0.65} 65%|██████▌ | 1188/1817 [18:48:56<10:03:42, 57.59s/it] 65%|██████▌ | 1189/1817 [18:49:54<10:02:52, 57.60s/it] {'loss': 1.9637, 'learning_rate': 2e-05, 'epoch': 0.65} 65%|██████▌ | 1189/1817 [18:49:53<10:02:52, 57.60s/it] 65%|██████▌ | 1190/1817 [18:50:49<9:57:40, 57.19s/it] {'loss': 1.8885, 'learning_rate': 2e-05, 'epoch': 0.65} 65%|██████▌ | 1190/1817 [18:50:49<9:57:40, 57.19s/it] 66%|██████▌ | 1191/1817 [18:51:46<9:54:12, 56.95s/it] {'loss': 1.8849, 'learning_rate': 2e-05, 'epoch': 0.66} 66%|██████▌ | 1191/1817 [18:51:46<9:54:13, 56.95s/it] 66%|██████▌ | 1192/1817 [18:52:43<9:51:38, 56.80s/it] {'loss': 1.9063, 'learning_rate': 2e-05, 'epoch': 0.66} 66%|██████▌ | 1192/1817 [18:52:42<9:51:38, 56.80s/it] 66%|██████▌ | 1193/1817 [18:53:39<9:49:37, 56.69s/it] {'loss': 1.9163, 'learning_rate': 2e-05, 'epoch': 0.66} 66%|██████▌ | 1193/1817 [18:53:39<9:49:37, 56.69s/it] 66%|██████▌ | 1194/1817 [18:54:35<9:48:14, 56.65s/it] {'loss': 1.8898, 'learning_rate': 2e-05, 'epoch': 0.66} 66%|██████▌ | 1194/1817 [18:54:35<9:48:14, 56.65s/it] 66%|██████▌ | 1195/1817 [18:55:34<9:53:05, 57.21s/it] {'loss': 1.8913, 'learning_rate': 2e-05, 'epoch': 0.66} 66%|██████▌ | 1195/1817 [18:55:34<9:53:05, 57.21s/it] 66%|██████▌ | 1196/1817 [18:56:33<9:56:03, 57.59s/it] {'loss': 1.9445, 'learning_rate': 2e-05, 'epoch': 0.66} 66%|██████▌ | 1196/1817 [18:56:32<9:56:03, 57.59s/it] 66%|██████▌ | 1197/1817 [18:57:29<9:51:21, 57.23s/it] {'loss': 1.8993, 'learning_rate': 2e-05, 'epoch': 0.66} 66%|██████▌ | 1197/1817 [18:57:29<9:51:21, 57.23s/it] 66%|██████▌ | 1198/1817 [18:58:25<9:48:32, 57.05s/it] {'loss': 1.9001, 'learning_rate': 2e-05, 'epoch': 0.66} 66%|██████▌ | 1198/1817 [18:58:25<9:48:32, 57.05s/it] 66%|██████▌ | 1199/1817 [18:59:24<9:51:51, 57.46s/it] {'loss': 1.9337, 'learning_rate': 2e-05, 'epoch': 0.66} 66%|██████▌ | 1199/1817 [18:59:24<9:51:51, 57.46s/it] 66%|██████▌ | 1200/1817 [19:00:23<9:55:48, 57.94s/it] {'loss': 1.8982, 'learning_rate': 2e-05, 'epoch': 0.66} 66%|██████▌ | 1200/1817 [19:00:23<9:55:48, 57.94s/it] 66%|██████▌ | 1201/1817 [19:01:20<9:53:52, 57.85s/it] {'loss': 1.913, 'learning_rate': 2e-05, 'epoch': 0.66} 66%|██████▌ | 1201/1817 [19:01:20<9:53:52, 57.85s/it] 66%|██████▌ | 1202/1817 [19:02:17<9:48:29, 57.41s/it] {'loss': 1.9481, 'learning_rate': 2e-05, 'epoch': 0.66} 66%|██████▌ | 1202/1817 [19:02:17<9:48:29, 57.41s/it] 66%|██████▌ | 1203/1817 [19:03:17<9:55:47, 58.22s/it] {'loss': 1.9061, 'learning_rate': 2e-05, 'epoch': 0.66} 66%|██████▌ | 1203/1817 [19:03:17<9:55:38, 58.21s/it] 66%|██████▋ | 1204/1817 [19:04:17<10:01:45, 58.90s/it] {'loss': 1.8548, 'learning_rate': 2e-05, 'epoch': 0.66} 66%|██████▋ | 1204/1817 [19:04:17<10:01:45, 58.90s/it] 66%|██████▋ | 1205/1817 [19:05:14<9:53:06, 58.15s/it] {'loss': 1.8532, 'learning_rate': 2e-05, 'epoch': 0.66} 66%|██████▋ | 1205/1817 [19:05:14<9:53:06, 58.15s/it] 66%|██████▋ | 1206/1817 [19:06:10<9:46:18, 57.57s/it] {'loss': 1.8542, 'learning_rate': 2e-05, 'epoch': 0.66} 66%|██████▋ | 1206/1817 [19:06:10<9:46:18, 57.57s/it] 66%|██████▋ | 1207/1817 [19:07:07<9:42:02, 57.25s/it] {'loss': 1.8894, 'learning_rate': 2e-05, 'epoch': 0.66} 66%|██████▋ | 1207/1817 [19:07:07<9:42:00, 57.25s/it] 66%|██████▋ | 1208/1817 [19:08:03<9:38:14, 56.97s/it] {'loss': 1.9079, 'learning_rate': 2e-05, 'epoch': 0.66} 66%|██████▋ | 1208/1817 [19:08:03<9:38:17, 56.98s/it] 67%|██████▋ | 1209/1817 [19:09:00<9:36:18, 56.87s/it] {'loss': 1.8983, 'learning_rate': 2e-05, 'epoch': 0.67} 67%|██████▋ | 1209/1817 [19:09:00<9:36:22, 56.88s/it] 67%|██████▋ | 1210/1817 [19:09:56<9:34:33, 56.79s/it] {'loss': 1.8813, 'learning_rate': 2e-05, 'epoch': 0.67} 67%|██████▋ | 1210/1817 [19:09:56<9:34:33, 56.79s/it] 67%|██████▋ | 1211/1817 [19:10:52<9:32:23, 56.67s/it] {'loss': 1.9262, 'learning_rate': 2e-05, 'epoch': 0.67} 67%|██████▋ | 1211/1817 [19:10:52<9:32:23, 56.67s/it] 67%|██████▋ | 1212/1817 [19:11:49<9:30:32, 56.58s/it] {'loss': 1.9189, 'learning_rate': 2e-05, 'epoch': 0.67} 67%|██████▋ | 1212/1817 [19:11:49<9:30:32, 56.58s/it] 67%|██████▋ | 1213/1817 [19:12:47<9:35:01, 57.12s/it] {'loss': 1.8874, 'learning_rate': 2e-05, 'epoch': 0.67} 67%|██████▋ | 1213/1817 [19:12:47<9:35:01, 57.12s/it] 67%|██████▋ | 1214/1817 [19:13:44<9:32:16, 56.94s/it] {'loss': 1.8825, 'learning_rate': 2e-05, 'epoch': 0.67} 67%|██████▋ | 1214/1817 [19:13:44<9:32:17, 56.94s/it] 67%|██████▋ | 1215/1817 [19:14:40<9:30:09, 56.83s/it] {'loss': 1.8766, 'learning_rate': 2e-05, 'epoch': 0.67} 67%|██████▋ | 1215/1817 [19:14:40<9:30:09, 56.83s/it] 67%|██████▋ | 1216/1817 [19:15:39<9:34:42, 57.37s/it] {'loss': 1.8759, 'learning_rate': 2e-05, 'epoch': 0.67} 67%|██████▋ | 1216/1817 [19:15:39<9:34:42, 57.38s/it] 67%|██████▋ | 1217/1817 [19:16:36<9:30:24, 57.04s/it] {'loss': 1.9008, 'learning_rate': 2e-05, 'epoch': 0.67} 67%|██████▋ | 1217/1817 [19:16:35<9:30:25, 57.04s/it] 67%|██████▋ | 1218/1817 [19:17:32<9:27:29, 56.84s/it] {'loss': 1.9018, 'learning_rate': 2e-05, 'epoch': 0.67} 67%|██████▋ | 1218/1817 [19:17:32<9:27:29, 56.84s/it] 67%|██████▋ | 1219/1817 [19:18:29<9:27:06, 56.90s/it] {'loss': 1.9183, 'learning_rate': 2e-05, 'epoch': 0.67} 67%|██████▋ | 1219/1817 [19:18:29<9:27:07, 56.90s/it] 67%|██████▋ | 1220/1817 [19:19:27<9:28:28, 57.13s/it] {'loss': 1.8996, 'learning_rate': 2e-05, 'epoch': 0.67} 67%|██████▋ | 1220/1817 [19:19:26<9:28:28, 57.13s/it] 67%|██████▋ | 1221/1817 [19:20:23<9:25:01, 56.88s/it] {'loss': 1.9553, 'learning_rate': 2e-05, 'epoch': 0.67} 67%|██████▋ | 1221/1817 [19:20:23<9:25:01, 56.88s/it] 67%|██████▋ | 1222/1817 [19:21:21<9:28:33, 57.33s/it] {'loss': 1.9137, 'learning_rate': 2e-05, 'epoch': 0.67} 67%|██████▋ | 1222/1817 [19:21:21<9:28:33, 57.33s/it] 67%|██████▋ | 1223/1817 [19:22:17<9:24:11, 56.99s/it] {'loss': 1.8822, 'learning_rate': 2e-05, 'epoch': 0.67} 67%|██████▋ | 1223/1817 [19:22:17<9:24:12, 56.99s/it] 67%|██████▋ | 1224/1817 [19:23:15<9:24:51, 57.15s/it] {'loss': 1.8634, 'learning_rate': 2e-05, 'epoch': 0.67} 67%|██████▋ | 1224/1817 [19:23:15<9:24:51, 57.15s/it] 67%|██████▋ | 1225/1817 [19:24:13<9:25:54, 57.36s/it] {'loss': 1.9188, 'learning_rate': 2e-05, 'epoch': 0.67} 67%|██████▋ | 1225/1817 [19:24:13<9:25:54, 57.36s/it] 67%|██████▋ | 1226/1817 [19:25:09<9:22:09, 57.07s/it] {'loss': 1.8507, 'learning_rate': 2e-05, 'epoch': 0.67} 67%|██████▋ | 1226/1817 [19:25:09<9:22:09, 57.07s/it] 68%|██████▊ | 1227/1817 [19:26:07<9:24:14, 57.38s/it] {'loss': 1.901, 'learning_rate': 2e-05, 'epoch': 0.68} 68%|██████▊ | 1227/1817 [19:26:07<9:24:14, 57.38s/it] 68%|██████▊ | 1228/1817 [19:27:03<9:20:21, 57.08s/it] {'loss': 1.8795, 'learning_rate': 2e-05, 'epoch': 0.68} 68%|██████▊ | 1228/1817 [19:27:03<9:20:21, 57.08s/it] 68%|██████▊ | 1229/1817 [19:28:03<9:27:07, 57.87s/it] {'loss': 1.8791, 'learning_rate': 2e-05, 'epoch': 0.68} 68%|██████▊ | 1229/1817 [19:28:03<9:27:08, 57.87s/it] 68%|██████▊ | 1230/1817 [19:29:00<9:24:14, 57.67s/it] {'loss': 1.8667, 'learning_rate': 2e-05, 'epoch': 0.68} 68%|██████▊ | 1230/1817 [19:29:00<9:24:14, 57.67s/it] 68%|██████▊ | 1231/1817 [19:29:54<9:12:49, 56.60s/it] {'loss': 1.9093, 'learning_rate': 2e-05, 'epoch': 0.68} 68%|██████▊ | 1231/1817 [19:29:54<9:12:49, 56.60s/it] 68%|██████▊ | 1232/1817 [19:30:51<9:11:23, 56.55s/it] {'loss': 1.8761, 'learning_rate': 2e-05, 'epoch': 0.68} 68%|██████▊ | 1232/1817 [19:30:51<9:11:23, 56.55s/it] 68%|██████▊ | 1233/1817 [19:31:47<9:10:02, 56.51s/it] {'loss': 1.9208, 'learning_rate': 2e-05, 'epoch': 0.68} 68%|██████▊ | 1233/1817 [19:31:47<9:10:02, 56.51s/it] 68%|██████▊ | 1234/1817 [19:32:44<9:08:27, 56.44s/it] {'loss': 1.9, 'learning_rate': 2e-05, 'epoch': 0.68} 68%|██████▊ | 1234/1817 [19:32:44<9:08:27, 56.44s/it] 68%|██████▊ | 1235/1817 [19:33:40<9:07:00, 56.39s/it] {'loss': 1.8785, 'learning_rate': 2e-05, 'epoch': 0.68} 68%|██████▊ | 1235/1817 [19:33:40<9:06:59, 56.39s/it] 68%|██████▊ | 1236/1817 [19:34:38<9:11:15, 56.93s/it] {'loss': 1.9002, 'learning_rate': 2e-05, 'epoch': 0.68} 68%|██████▊ | 1236/1817 [19:34:38<9:11:15, 56.93s/it] 68%|██████▊ | 1237/1817 [19:35:34<9:07:40, 56.66s/it] {'loss': 1.9159, 'learning_rate': 2e-05, 'epoch': 0.68} 68%|██████▊ | 1237/1817 [19:35:34<9:07:40, 56.66s/it] 68%|██████▊ | 1238/1817 [19:36:31<9:05:29, 56.53s/it] {'loss': 1.9149, 'learning_rate': 2e-05, 'epoch': 0.68} 68%|██████▊ | 1238/1817 [19:36:30<9:05:30, 56.53s/it] 68%|██████▊ | 1239/1817 [19:37:27<9:04:20, 56.51s/it] {'loss': 1.9188, 'learning_rate': 2e-05, 'epoch': 0.68} 68%|██████▊ | 1239/1817 [19:37:27<9:04:20, 56.51s/it] 68%|██████▊ | 1240/1817 [19:38:23<9:02:46, 56.44s/it] {'loss': 1.9585, 'learning_rate': 2e-05, 'epoch': 0.68} 68%|██████▊ | 1240/1817 [19:38:23<9:02:46, 56.44s/it] 68%|██████▊ | 1241/1817 [19:39:19<9:01:26, 56.40s/it] {'loss': 1.8825, 'learning_rate': 2e-05, 'epoch': 0.68} 68%|██████▊ | 1241/1817 [19:39:19<9:01:26, 56.40s/it] 68%|██████▊ | 1242/1817 [19:40:13<8:53:15, 55.64s/it] {'loss': 1.8788, 'learning_rate': 2e-05, 'epoch': 0.68} 68%|██████▊ | 1242/1817 [19:40:13<8:53:15, 55.64s/it] 68%|██████▊ | 1243/1817 [19:41:10<8:54:11, 55.84s/it] {'loss': 1.8452, 'learning_rate': 2e-05, 'epoch': 0.68} 68%|██████▊ | 1243/1817 [19:41:10<8:54:11, 55.84s/it] 68%|██████▊ | 1244/1817 [19:42:08<8:59:59, 56.54s/it] {'loss': 1.9086, 'learning_rate': 2e-05, 'epoch': 0.68} 68%|██████▊ | 1244/1817 [19:42:08<8:59:59, 56.54s/it] 69%|██████▊ | 1245/1817 [19:43:04<8:58:33, 56.49s/it] {'loss': 1.9362, 'learning_rate': 2e-05, 'epoch': 0.69} 69%|██████▊ | 1245/1817 [19:43:04<8:58:33, 56.49s/it] 69%|██████▊ | 1246/1817 [19:44:00<8:57:05, 56.44s/it] {'loss': 1.9217, 'learning_rate': 2e-05, 'epoch': 0.69} 69%|██████▊ | 1246/1817 [19:44:00<8:57:05, 56.44s/it] 69%|██████▊ | 1247/1817 [19:44:57<8:55:42, 56.39s/it] {'loss': 1.9214, 'learning_rate': 2e-05, 'epoch': 0.69} 69%|██████▊ | 1247/1817 [19:44:57<8:55:42, 56.39s/it] 69%|██████▊ | 1248/1817 [19:45:52<8:50:49, 55.98s/it] {'loss': 1.8996, 'learning_rate': 2e-05, 'epoch': 0.69} 69%|██████▊ | 1248/1817 [19:45:52<8:50:49, 55.98s/it] 69%|██████▊ | 1249/1817 [19:46:51<8:58:29, 56.88s/it] {'loss': 1.903, 'learning_rate': 2e-05, 'epoch': 0.69} 69%|██████▊ | 1249/1817 [19:46:51<8:58:29, 56.88s/it] 69%|██████▉ | 1250/1817 [19:47:47<8:55:29, 56.67s/it] {'loss': 1.8974, 'learning_rate': 2e-05, 'epoch': 0.69} 69%|██████▉ | 1250/1817 [19:47:47<8:55:29, 56.67s/it] 69%|██████▉ | 1251/1817 [19:48:43<8:53:26, 56.55s/it] {'loss': 1.8726, 'learning_rate': 2e-05, 'epoch': 0.69} 69%|██████▉ | 1251/1817 [19:48:43<8:53:26, 56.55s/it] 69%|██████▉ | 1252/1817 [19:49:39<8:51:38, 56.46s/it] {'loss': 1.9292, 'learning_rate': 2e-05, 'epoch': 0.69} 69%|██████▉ | 1252/1817 [19:49:39<8:51:38, 56.46s/it] 69%|██████▉ | 1253/1817 [19:50:36<8:49:49, 56.36s/it] {'loss': 1.9134, 'learning_rate': 2e-05, 'epoch': 0.69} 69%|██████▉ | 1253/1817 [19:50:36<8:49:49, 56.36s/it] 69%|██████▉ | 1254/1817 [19:51:32<8:48:23, 56.31s/it] {'loss': 1.9125, 'learning_rate': 2e-05, 'epoch': 0.69} 69%|██████▉ | 1254/1817 [19:51:32<8:48:23, 56.31s/it] 69%|██████▉ | 1255/1817 [19:52:28<8:47:09, 56.28s/it] {'loss': 1.8965, 'learning_rate': 2e-05, 'epoch': 0.69} 69%|██████▉ | 1255/1817 [19:52:28<8:47:09, 56.28s/it] 69%|██████▉ | 1256/1817 [19:53:24<8:46:11, 56.28s/it] {'loss': 1.8767, 'learning_rate': 2e-05, 'epoch': 0.69} 69%|██████▉ | 1256/1817 [19:53:24<8:46:11, 56.28s/it] 69%|██████▉ | 1257/1817 [19:54:23<8:50:56, 56.89s/it] {'loss': 1.9059, 'learning_rate': 2e-05, 'epoch': 0.69} 69%|██████▉ | 1257/1817 [19:54:23<8:50:56, 56.89s/it] 69%|██████▉ | 1258/1817 [19:55:19<8:47:49, 56.65s/it] {'loss': 1.905, 'learning_rate': 2e-05, 'epoch': 0.69} 69%|██████▉ | 1258/1817 [19:55:19<8:47:49, 56.65s/it] 69%|██████▉ | 1259/1817 [19:56:15<8:45:25, 56.50s/it] {'loss': 1.8837, 'learning_rate': 2e-05, 'epoch': 0.69} 69%|██████▉ | 1259/1817 [19:56:15<8:45:25, 56.50s/it] 69%|██████▉ | 1260/1817 [19:57:11<8:43:07, 56.35s/it] {'loss': 1.9185, 'learning_rate': 2e-05, 'epoch': 0.69} 69%|██████▉ | 1260/1817 [19:57:11<8:43:07, 56.35s/it] 69%|██████▉ | 1261/1817 [19:58:07<8:41:42, 56.30s/it] {'loss': 1.9106, 'learning_rate': 2e-05, 'epoch': 0.69} 69%|██████▉ | 1261/1817 [19:58:07<8:41:42, 56.30s/it] 69%|██████▉ | 1262/1817 [19:59:04<8:43:08, 56.56s/it] {'loss': 1.8948, 'learning_rate': 2e-05, 'epoch': 0.69} 69%|██████▉ | 1262/1817 [19:59:04<8:43:08, 56.56s/it] 70%|██████▉ | 1263/1817 [20:00:00<8:40:55, 56.42s/it] {'loss': 1.9043, 'learning_rate': 2e-05, 'epoch': 0.69} 70%|██████▉ | 1263/1817 [20:00:00<8:40:55, 56.42s/it] 70%|██████▉ | 1264/1817 [20:00:56<8:39:21, 56.35s/it] {'loss': 1.9174, 'learning_rate': 2e-05, 'epoch': 0.7} 70%|██████▉ | 1264/1817 [20:00:56<8:39:21, 56.35s/it] 70%|██████▉ | 1265/1817 [20:01:53<8:37:59, 56.30s/it] {'loss': 1.8772, 'learning_rate': 2e-05, 'epoch': 0.7} 70%|██████▉ | 1265/1817 [20:01:53<8:37:59, 56.30s/it] 70%|██████▉ | 1266/1817 [20:02:49<8:36:54, 56.29s/it] {'loss': 1.8987, 'learning_rate': 2e-05, 'epoch': 0.7} 70%|██████▉ | 1266/1817 [20:02:49<8:36:54, 56.29s/it] 70%|██████▉ | 1267/1817 [20:03:45<8:37:00, 56.40s/it] {'loss': 1.8467, 'learning_rate': 2e-05, 'epoch': 0.7} 70%|██████▉ | 1267/1817 [20:03:45<8:37:00, 56.40s/it] 70%|██████▉ | 1268/1817 [20:04:43<8:40:25, 56.88s/it] {'loss': 1.9181, 'learning_rate': 2e-05, 'epoch': 0.7} 70%|██████▉ | 1268/1817 [20:04:43<8:40:25, 56.88s/it] 70%|██████▉ | 1269/1817 [20:05:42<8:42:58, 57.26s/it] {'loss': 1.8721, 'learning_rate': 2e-05, 'epoch': 0.7} 70%|██████▉ | 1269/1817 [20:05:42<8:42:55, 57.25s/it] 70%|██████▉ | 1270/1817 [20:06:39<8:42:50, 57.35s/it] {'loss': 1.9198, 'learning_rate': 2e-05, 'epoch': 0.7} 70%|██████▉ | 1270/1817 [20:06:39<8:42:50, 57.35s/it] 70%|██████▉ | 1271/1817 [20:07:36<8:38:46, 57.01s/it] {'loss': 1.8547, 'learning_rate': 2e-05, 'epoch': 0.7} 70%|██████▉ | 1271/1817 [20:07:35<8:38:48, 57.01s/it] 70%|███████ | 1272/1817 [20:08:34<8:42:31, 57.53s/it] {'loss': 1.8761, 'learning_rate': 2e-05, 'epoch': 0.7} 70%|███████ | 1272/1817 [20:08:34<8:42:31, 57.53s/it] 70%|███████ | 1273/1817 [20:09:30<8:38:04, 57.14s/it] {'loss': 1.8831, 'learning_rate': 2e-05, 'epoch': 0.7} 70%|███████ | 1273/1817 [20:09:30<8:38:04, 57.14s/it] 70%|███████ | 1274/1817 [20:10:31<8:44:42, 57.98s/it] {'loss': 1.8684, 'learning_rate': 2e-05, 'epoch': 0.7} 70%|███████ | 1274/1817 [20:10:30<8:44:42, 57.98s/it] 70%|███████ | 1275/1817 [20:11:27<8:39:14, 57.48s/it] {'loss': 1.8555, 'learning_rate': 2e-05, 'epoch': 0.7} 70%|███████ | 1275/1817 [20:11:27<8:39:14, 57.48s/it] 70%|███████ | 1276/1817 [20:12:25<8:41:07, 57.80s/it] {'loss': 1.9193, 'learning_rate': 2e-05, 'epoch': 0.7} 70%|███████ | 1276/1817 [20:12:25<8:41:07, 57.80s/it] 70%|███████ | 1277/1817 [20:13:22<8:36:00, 57.33s/it] {'loss': 1.8861, 'learning_rate': 2e-05, 'epoch': 0.7} 70%|███████ | 1277/1817 [20:13:21<8:36:01, 57.34s/it] 70%|███████ | 1278/1817 [20:14:19<8:35:51, 57.42s/it] {'loss': 1.8941, 'learning_rate': 2e-05, 'epoch': 0.7} 70%|███████ | 1278/1817 [20:14:19<8:35:51, 57.42s/it] {'loss': 1.8941, 'learning_rate': 2e-05, 'epoch': 0.7} 70%|███████ | 1278/1817 [20:14:19<8:35:51, 57.42s/it] 70%|███████ | 1279/1817 [20:15:15<8:31:57, 57.10s/it] {'loss': 1.9104, 'learning_rate': 2e-05, 'epoch': 0.7} 70%|███████ | 1279/1817 [20:15:15<8:31:57, 57.10s/it] 70%|███████ | 1280/1817 [20:16:13<8:32:37, 57.28s/it] {'loss': 1.8709, 'learning_rate': 2e-05, 'epoch': 0.7} 70%|███████ | 1280/1817 [20:16:13<8:32:37, 57.28s/it] 71%|███████ | 1281/1817 [20:17:10<8:31:52, 57.30s/it] {'loss': 1.883, 'learning_rate': 2e-05, 'epoch': 0.7} 71%|███████ | 1281/1817 [20:17:10<8:31:52, 57.30s/it] 71%|███████ | 1282/1817 [20:18:07<8:28:41, 57.05s/it] {'loss': 1.8965, 'learning_rate': 2e-05, 'epoch': 0.71} 71%|███████ | 1282/1817 [20:18:07<8:28:41, 57.05s/it] 71%|███████ | 1283/1817 [20:19:03<8:25:27, 56.79s/it] {'loss': 1.9503, 'learning_rate': 2e-05, 'epoch': 0.71} 71%|███████ | 1283/1817 [20:19:03<8:25:27, 56.79s/it] 71%|███████ | 1284/1817 [20:19:58<8:18:29, 56.11s/it] {'loss': 1.8778, 'learning_rate': 2e-05, 'epoch': 0.71} 71%|███████ | 1284/1817 [20:19:58<8:18:29, 56.11s/it] 71%|███████ | 1285/1817 [20:20:55<8:20:07, 56.41s/it] {'loss': 1.8832, 'learning_rate': 2e-05, 'epoch': 0.71} 71%|███████ | 1285/1817 [20:20:55<8:20:08, 56.41s/it] 71%|███████ | 1286/1817 [20:21:51<8:19:06, 56.40s/it] {'loss': 1.9182, 'learning_rate': 2e-05, 'epoch': 0.71} 71%|███████ | 1286/1817 [20:21:51<8:19:06, 56.40s/it] 71%|███████ | 1287/1817 [20:22:47<8:18:04, 56.38s/it] {'loss': 1.9097, 'learning_rate': 2e-05, 'epoch': 0.71} 71%|███████ | 1287/1817 [20:22:47<8:18:04, 56.38s/it] 71%|███████ | 1288/1817 [20:23:44<8:18:02, 56.49s/it] {'loss': 1.8945, 'learning_rate': 2e-05, 'epoch': 0.71} 71%|███████ | 1288/1817 [20:23:44<8:18:02, 56.49s/it] 71%|███████ | 1289/1817 [20:24:41<8:18:55, 56.70s/it] {'loss': 1.8857, 'learning_rate': 2e-05, 'epoch': 0.71} 71%|███████ | 1289/1817 [20:24:41<8:18:55, 56.70s/it] 71%|███████ | 1290/1817 [20:25:38<8:18:58, 56.81s/it] {'loss': 1.8966, 'learning_rate': 2e-05, 'epoch': 0.71} 71%|███████ | 1290/1817 [20:25:38<8:18:58, 56.81s/it] 71%|███████ | 1291/1817 [20:26:35<8:17:53, 56.79s/it] {'loss': 1.9274, 'learning_rate': 2e-05, 'epoch': 0.71} 71%|███████ | 1291/1817 [20:26:35<8:17:53, 56.79s/it] 71%|███████ | 1292/1817 [20:27:34<8:21:41, 57.34s/it] {'loss': 1.9077, 'learning_rate': 2e-05, 'epoch': 0.71} 71%|███████ | 1292/1817 [20:27:34<8:21:41, 57.34s/it] 71%|███████ | 1293/1817 [20:28:31<8:19:45, 57.22s/it] {'loss': 1.9427, 'learning_rate': 2e-05, 'epoch': 0.71} 71%|███████ | 1293/1817 [20:28:31<8:19:45, 57.22s/it] 71%|███████ | 1294/1817 [20:29:28<8:19:51, 57.35s/it] {'loss': 1.8842, 'learning_rate': 2e-05, 'epoch': 0.71} 71%|███████ | 1294/1817 [20:29:28<8:19:51, 57.35s/it] 71%|███████▏ | 1295/1817 [20:30:24<8:12:30, 56.61s/it] {'loss': 1.9503, 'learning_rate': 2e-05, 'epoch': 0.71} 71%|███████▏ | 1295/1817 [20:30:23<8:12:30, 56.61s/it] 71%|███████▏ | 1296/1817 [20:31:20<8:10:45, 56.52s/it] {'loss': 1.9334, 'learning_rate': 2e-05, 'epoch': 0.71} 71%|███████▏ | 1296/1817 [20:31:20<8:10:45, 56.52s/it] 71%|███████▏ | 1297/1817 [20:32:16<8:08:30, 56.37s/it] {'loss': 1.9219, 'learning_rate': 2e-05, 'epoch': 0.71} 71%|███████▏ | 1297/1817 [20:32:16<8:08:30, 56.37s/it] 71%|███████▏ | 1298/1817 [20:33:13<8:10:16, 56.68s/it] {'loss': 1.8774, 'learning_rate': 2e-05, 'epoch': 0.71} 71%|███████▏ | 1298/1817 [20:33:13<8:10:16, 56.68s/it] 71%|███████▏ | 1299/1817 [20:34:09<8:07:57, 56.52s/it] {'loss': 1.8872, 'learning_rate': 2e-05, 'epoch': 0.71} 71%|███████▏ | 1299/1817 [20:34:09<8:07:57, 56.52s/it] 72%|███████▏ | 1300/1817 [20:35:07<8:08:35, 56.70s/it] {'loss': 1.9216, 'learning_rate': 2e-05, 'epoch': 0.72} 72%|███████▏ | 1300/1817 [20:35:06<8:08:36, 56.70s/it] 72%|███████▏ | 1301/1817 [20:36:03<8:06:42, 56.59s/it] {'loss': 1.9285, 'learning_rate': 2e-05, 'epoch': 0.72} 72%|███████▏ | 1301/1817 [20:36:03<8:06:42, 56.59s/it] 72%|███████▏ | 1302/1817 [20:36:59<8:03:39, 56.35s/it] {'loss': 1.9311, 'learning_rate': 2e-05, 'epoch': 0.72} 72%|███████▏ | 1302/1817 [20:36:58<8:03:39, 56.35s/it] 72%|███████▏ | 1303/1817 [20:37:56<8:06:17, 56.76s/it] {'loss': 1.904, 'learning_rate': 2e-05, 'epoch': 0.72} 72%|███████▏ | 1303/1817 [20:37:56<8:06:17, 56.77s/it] 72%|███████▏ | 1304/1817 [20:38:53<8:05:18, 56.76s/it] {'loss': 1.8506, 'learning_rate': 2e-05, 'epoch': 0.72} 72%|███████▏ | 1304/1817 [20:38:53<8:05:19, 56.76s/it] 72%|███████▏ | 1305/1817 [20:39:49<8:03:07, 56.62s/it] {'loss': 1.8827, 'learning_rate': 2e-05, 'epoch': 0.72} 72%|███████▏ | 1305/1817 [20:39:49<8:03:07, 56.62s/it] 72%|███████▏ | 1306/1817 [20:40:45<8:00:19, 56.40s/it] {'loss': 1.9129, 'learning_rate': 2e-05, 'epoch': 0.72} 72%|███████▏ | 1306/1817 [20:40:45<8:00:19, 56.40s/it] 72%|███████▏ | 1307/1817 [20:41:41<7:58:46, 56.33s/it] {'loss': 1.9052, 'learning_rate': 2e-05, 'epoch': 0.72} 72%|███████▏ | 1307/1817 [20:41:41<7:58:46, 56.33s/it] 72%|███████▏ | 1308/1817 [20:42:38<7:57:28, 56.28s/it] {'loss': 1.8868, 'learning_rate': 2e-05, 'epoch': 0.72} 72%|███████▏ | 1308/1817 [20:42:37<7:57:28, 56.28s/it] 72%|███████▏ | 1309/1817 [20:43:32<7:52:14, 55.78s/it] {'loss': 1.8654, 'learning_rate': 2e-05, 'epoch': 0.72} 72%|███████▏ | 1309/1817 [20:43:32<7:52:14, 55.78s/it] 72%|███████▏ | 1310/1817 [20:44:30<7:55:24, 56.26s/it] {'loss': 1.8991, 'learning_rate': 2e-05, 'epoch': 0.72} 72%|███████▏ | 1310/1817 [20:44:29<7:55:24, 56.26s/it] 72%|███████▏ | 1311/1817 [20:45:29<8:02:26, 57.21s/it] {'loss': 1.8897, 'learning_rate': 2e-05, 'epoch': 0.72} 72%|███████▏ | 1311/1817 [20:45:29<8:02:26, 57.21s/it] 72%|███████▏ | 1312/1817 [20:46:26<8:02:32, 57.33s/it] {'loss': 1.905, 'learning_rate': 2e-05, 'epoch': 0.72} 72%|███████▏ | 1312/1817 [20:46:26<8:02:32, 57.33s/it] 72%|███████▏ | 1313/1817 [20:47:25<8:03:33, 57.57s/it] {'loss': 1.9046, 'learning_rate': 2e-05, 'epoch': 0.72} 72%|███████▏ | 1313/1817 [20:47:25<8:03:33, 57.57s/it] 72%|███████▏ | 1314/1817 [20:48:22<8:03:22, 57.66s/it] {'loss': 1.882, 'learning_rate': 2e-05, 'epoch': 0.72} 72%|███████▏ | 1314/1817 [20:48:22<8:03:22, 57.66s/it] 72%|███████▏ | 1315/1817 [20:49:19<7:58:58, 57.25s/it] {'loss': 1.9263, 'learning_rate': 2e-05, 'epoch': 0.72} 72%|███████▏ | 1315/1817 [20:49:19<7:58:58, 57.25s/it] 72%|███████▏ | 1316/1817 [20:50:17<8:01:16, 57.64s/it] {'loss': 1.8968, 'learning_rate': 2e-05, 'epoch': 0.72} 72%|███████▏ | 1316/1817 [20:50:17<8:01:16, 57.64s/it] 72%|███████▏ | 1317/1817 [20:51:13<7:56:48, 57.22s/it] {'loss': 1.9244, 'learning_rate': 2e-05, 'epoch': 0.72} 72%|███████▏ | 1317/1817 [20:51:13<7:56:48, 57.22s/it] 73%|███████▎ | 1318/1817 [20:52:10<7:53:24, 56.92s/it] {'loss': 1.8802, 'learning_rate': 2e-05, 'epoch': 0.73} 73%|███████▎ | 1318/1817 [20:52:10<7:53:24, 56.92s/it] 73%|███████▎ | 1319/1817 [20:53:07<7:52:20, 56.91s/it] {'loss': 1.9586, 'learning_rate': 2e-05, 'epoch': 0.73} 73%|███████▎ | 1319/1817 [20:53:07<7:52:20, 56.91s/it] 73%|███████▎ | 1320/1817 [20:54:03<7:48:20, 56.54s/it] {'loss': 1.9088, 'learning_rate': 2e-05, 'epoch': 0.73} 73%|███████▎ | 1320/1817 [20:54:02<7:48:20, 56.54s/it] 73%|███████▎ | 1321/1817 [20:54:59<7:47:12, 56.52s/it] {'loss': 1.9058, 'learning_rate': 2e-05, 'epoch': 0.73} 73%|███████▎ | 1321/1817 [20:54:59<7:47:12, 56.52s/it] 73%|███████▎ | 1322/1817 [20:55:56<7:46:10, 56.51s/it] {'loss': 1.9122, 'learning_rate': 2e-05, 'epoch': 0.73} 73%|███████▎ | 1322/1817 [20:55:55<7:46:10, 56.51s/it] 73%|███████▎ | 1323/1817 [20:56:53<7:48:43, 56.93s/it] {'loss': 1.8862, 'learning_rate': 2e-05, 'epoch': 0.73} 73%|███████▎ | 1323/1817 [20:56:53<7:48:43, 56.93s/it] 73%|███████▎ | 1324/1817 [20:57:48<7:42:59, 56.35s/it] {'loss': 1.9093, 'learning_rate': 2e-05, 'epoch': 0.73} 73%|███████▎ | 1324/1817 [20:57:48<7:42:59, 56.35s/it] 73%|███████▎ | 1325/1817 [20:58:46<7:45:54, 56.82s/it] {'loss': 1.8912, 'learning_rate': 2e-05, 'epoch': 0.73} 73%|███████▎ | 1325/1817 [20:58:46<7:45:54, 56.82s/it] 73%|███████▎ | 1326/1817 [20:59:43<7:43:50, 56.68s/it] {'loss': 1.9135, 'learning_rate': 2e-05, 'epoch': 0.73} 73%|███████▎ | 1326/1817 [20:59:42<7:43:49, 56.68s/it] 73%|███████▎ | 1327/1817 [21:00:39<7:42:02, 56.58s/it] {'loss': 1.8993, 'learning_rate': 2e-05, 'epoch': 0.73} 73%|███████▎ | 1327/1817 [21:00:39<7:42:02, 56.58s/it] 73%|███████▎ | 1328/1817 [21:01:35<7:40:10, 56.46s/it] {'loss': 1.9208, 'learning_rate': 2e-05, 'epoch': 0.73} 73%|███████▎ | 1328/1817 [21:01:35<7:40:10, 56.46s/it] 73%|███████▎ | 1329/1817 [21:02:31<7:38:15, 56.34s/it] {'loss': 1.865, 'learning_rate': 2e-05, 'epoch': 0.73} 73%|███████▎ | 1329/1817 [21:02:31<7:38:15, 56.34s/it] 73%|███████▎ | 1330/1817 [21:03:27<7:37:20, 56.35s/it] {'loss': 1.8595, 'learning_rate': 2e-05, 'epoch': 0.73} 73%|███████▎ | 1330/1817 [21:03:27<7:37:20, 56.35s/it] 73%|███████▎ | 1331/1817 [21:04:24<7:36:10, 56.32s/it] {'loss': 1.9249, 'learning_rate': 2e-05, 'epoch': 0.73} 73%|███████▎ | 1331/1817 [21:04:24<7:36:11, 56.32s/it] 73%|███████▎ | 1332/1817 [21:05:20<7:35:16, 56.32s/it] {'loss': 1.8997, 'learning_rate': 2e-05, 'epoch': 0.73} 73%|███████▎ | 1332/1817 [21:05:20<7:35:16, 56.32s/it] 73%|███████▎ | 1333/1817 [21:06:16<7:34:13, 56.31s/it] {'loss': 1.9182, 'learning_rate': 2e-05, 'epoch': 0.73} 73%|███████▎ | 1333/1817 [21:06:16<7:34:13, 56.31s/it] 73%|███████▎ | 1334/1817 [21:07:13<7:33:27, 56.33s/it] {'loss': 1.8708, 'learning_rate': 2e-05, 'epoch': 0.73} 73%|███████▎ | 1334/1817 [21:07:13<7:33:27, 56.33s/it] 73%|███████▎ | 1335/1817 [21:08:09<7:32:50, 56.37s/it] {'loss': 1.9019, 'learning_rate': 2e-05, 'epoch': 0.73} 73%|███████▎ | 1335/1817 [21:08:09<7:32:50, 56.37s/it] 74%|███████▎ | 1336/1817 [21:09:05<7:30:09, 56.15s/it] {'loss': 1.9158, 'learning_rate': 2e-05, 'epoch': 0.74} 74%|███████▎ | 1336/1817 [21:09:05<7:30:09, 56.15s/it] 74%|███████▎ | 1337/1817 [21:10:04<7:37:55, 57.24s/it] {'loss': 1.879, 'learning_rate': 2e-05, 'epoch': 0.74} 74%|███████▎ | 1337/1817 [21:10:04<7:37:55, 57.24s/it] 74%|███████▎ | 1338/1817 [21:11:03<7:39:36, 57.57s/it] {'loss': 1.8836, 'learning_rate': 2e-05, 'epoch': 0.74} 74%|███████▎ | 1338/1817 [21:11:03<7:39:36, 57.57s/it] 74%|███████▎ | 1339/1817 [21:12:00<7:38:30, 57.55s/it] {'loss': 1.9166, 'learning_rate': 2e-05, 'epoch': 0.74} 74%|███████▎ | 1339/1817 [21:12:00<7:38:30, 57.55s/it] 74%|███████▎ | 1340/1817 [21:12:58<7:37:43, 57.58s/it] {'loss': 1.8766, 'learning_rate': 2e-05, 'epoch': 0.74} 74%|███████▎ | 1340/1817 [21:12:58<7:37:43, 57.58s/it] 74%|███████▍ | 1341/1817 [21:13:53<7:30:44, 56.82s/it] {'loss': 1.9038, 'learning_rate': 2e-05, 'epoch': 0.74} 74%|███████▍ | 1341/1817 [21:13:53<7:30:44, 56.82s/it] 74%|███████▍ | 1342/1817 [21:14:50<7:28:41, 56.68s/it] {'loss': 1.9149, 'learning_rate': 2e-05, 'epoch': 0.74} 74%|███████▍ | 1342/1817 [21:14:49<7:28:41, 56.68s/it] 74%|███████▍ | 1343/1817 [21:15:46<7:27:18, 56.62s/it] {'loss': 1.8926, 'learning_rate': 2e-05, 'epoch': 0.74} 74%|███████▍ | 1343/1817 [21:15:46<7:27:18, 56.62s/it] 74%|███████▍ | 1344/1817 [21:16:42<7:25:54, 56.56s/it] {'loss': 1.9152, 'learning_rate': 2e-05, 'epoch': 0.74} 74%|███████▍ | 1344/1817 [21:16:42<7:25:54, 56.56s/it] 74%|███████▍ | 1345/1817 [21:17:41<7:28:18, 56.99s/it] {'loss': 1.8942, 'learning_rate': 2e-05, 'epoch': 0.74} 74%|███████▍ | 1345/1817 [21:17:40<7:28:18, 56.99s/it] 74%|███████▍ | 1346/1817 [21:18:37<7:25:44, 56.78s/it] {'loss': 1.8868, 'learning_rate': 2e-05, 'epoch': 0.74} 74%|███████▍ | 1346/1817 [21:18:37<7:25:44, 56.78s/it] 74%|███████▍ | 1347/1817 [21:19:34<7:26:37, 57.02s/it] {'loss': 1.9353, 'learning_rate': 2e-05, 'epoch': 0.74} 74%|███████▍ | 1347/1817 [21:19:34<7:26:37, 57.02s/it] 74%|███████▍ | 1348/1817 [21:20:30<7:24:06, 56.82s/it] {'loss': 1.9406, 'learning_rate': 2e-05, 'epoch': 0.74} 74%|███████▍ | 1348/1817 [21:20:30<7:24:06, 56.82s/it] 74%|███████▍ | 1349/1817 [21:21:30<7:29:48, 57.67s/it] {'loss': 1.8705, 'learning_rate': 2e-05, 'epoch': 0.74} 74%|███████▍ | 1349/1817 [21:21:30<7:29:48, 57.67s/it] 74%|███████▍ | 1350/1817 [21:22:26<7:25:44, 57.27s/it] {'loss': 1.8618, 'learning_rate': 2e-05, 'epoch': 0.74} 74%|███████▍ | 1350/1817 [21:22:26<7:25:44, 57.27s/it] 74%|███████▍ | 1351/1817 [21:23:24<7:25:10, 57.32s/it] {'loss': 1.9215, 'learning_rate': 2e-05, 'epoch': 0.74} 74%|███████▍ | 1351/1817 [21:23:24<7:25:10, 57.32s/it] 74%|███████▍ | 1352/1817 [21:24:21<7:24:19, 57.33s/it] {'loss': 1.9082, 'learning_rate': 2e-05, 'epoch': 0.74} 74%|███████▍ | 1352/1817 [21:24:21<7:24:19, 57.33s/it] 74%|███████▍ | 1353/1817 [21:25:18<7:23:00, 57.29s/it] {'loss': 1.9353, 'learning_rate': 2e-05, 'epoch': 0.74} 74%|███████▍ | 1353/1817 [21:25:18<7:23:00, 57.29s/it] 75%|███████▍ | 1354/1817 [21:26:16<7:22:47, 57.38s/it] {'loss': 1.8963, 'learning_rate': 2e-05, 'epoch': 0.74} 75%|███████▍ | 1354/1817 [21:26:16<7:22:47, 57.38s/it] 75%|███████▍ | 1355/1817 [21:27:17<7:29:40, 58.40s/it] {'loss': 1.8758, 'learning_rate': 2e-05, 'epoch': 0.75} 75%|███████▍ | 1355/1817 [21:27:17<7:29:40, 58.40s/it] 75%|███████▍ | 1356/1817 [21:28:13<7:23:33, 57.73s/it] {'loss': 1.9006, 'learning_rate': 2e-05, 'epoch': 0.75} 75%|███████▍ | 1356/1817 [21:28:13<7:23:33, 57.73s/it] 75%|███████▍ | 1357/1817 [21:29:10<7:19:39, 57.35s/it] {'loss': 1.9116, 'learning_rate': 2e-05, 'epoch': 0.75} 75%|███████▍ | 1357/1817 [21:29:09<7:19:39, 57.35s/it] 75%|███████▍ | 1358/1817 [21:30:06<7:16:13, 57.02s/it] {'loss': 1.8721, 'learning_rate': 2e-05, 'epoch': 0.75} 75%|███████▍ | 1358/1817 [21:30:06<7:16:13, 57.02s/it] 75%|███████▍ | 1359/1817 [21:31:05<7:19:46, 57.61s/it] {'loss': 1.907, 'learning_rate': 2e-05, 'epoch': 0.75} 75%|███████▍ | 1359/1817 [21:31:05<7:19:46, 57.61s/it] 75%|███████▍ | 1360/1817 [21:32:00<7:14:36, 57.06s/it] {'loss': 1.8584, 'learning_rate': 2e-05, 'epoch': 0.75} 75%|███████▍ | 1360/1817 [21:32:00<7:14:36, 57.06s/it] 75%|███████▍ | 1361/1817 [21:32:57<7:12:00, 56.84s/it] {'loss': 1.9223, 'learning_rate': 2e-05, 'epoch': 0.75} 75%|███████▍ | 1361/1817 [21:32:57<7:11:59, 56.84s/it] 75%|███████▍ | 1362/1817 [21:33:53<7:09:16, 56.61s/it] {'loss': 1.886, 'learning_rate': 2e-05, 'epoch': 0.75} 75%|███████▍ | 1362/1817 [21:33:53<7:09:16, 56.61s/it] 75%|███████▌ | 1363/1817 [21:34:52<7:14:26, 57.42s/it] {'loss': 1.9078, 'learning_rate': 2e-05, 'epoch': 0.75} 75%|███████▌ | 1363/1817 [21:34:52<7:14:26, 57.42s/it] 75%|███████▌ | 1364/1817 [21:35:52<7:18:35, 58.09s/it] {'loss': 1.872, 'learning_rate': 2e-05, 'epoch': 0.75} 75%|███████▌ | 1364/1817 [21:35:52<7:18:36, 58.09s/it] 75%|███████▌ | 1365/1817 [21:36:48<7:13:27, 57.54s/it] {'loss': 1.8938, 'learning_rate': 2e-05, 'epoch': 0.75} 75%|███████▌ | 1365/1817 [21:36:48<7:13:27, 57.54s/it] 75%|███████▌ | 1366/1817 [21:37:47<7:13:55, 57.73s/it] {'loss': 1.8863, 'learning_rate': 2e-05, 'epoch': 0.75} 75%|███████▌ | 1366/1817 [21:37:46<7:13:55, 57.73s/it] 75%|███████▌ | 1367/1817 [21:38:43<7:09:26, 57.26s/it] {'loss': 1.8954, 'learning_rate': 2e-05, 'epoch': 0.75} 75%|███████▌ | 1367/1817 [21:38:42<7:09:26, 57.26s/it] 75%|███████▌ | 1368/1817 [21:39:39<7:06:23, 56.98s/it] {'loss': 1.9133, 'learning_rate': 2e-05, 'epoch': 0.75} 75%|███████▌ | 1368/1817 [21:39:39<7:06:23, 56.98s/it] 75%|███████▌ | 1369/1817 [21:40:35<7:03:49, 56.76s/it] {'loss': 1.9016, 'learning_rate': 2e-05, 'epoch': 0.75} 75%|███████▌ | 1369/1817 [21:40:35<7:03:49, 56.76s/it] 75%|███████▌ | 1370/1817 [21:41:31<7:00:53, 56.50s/it] {'loss': 1.879, 'learning_rate': 2e-05, 'epoch': 0.75} 75%|███████▌ | 1370/1817 [21:41:31<7:00:53, 56.50s/it] 75%|███████▌ | 1371/1817 [21:42:32<7:09:12, 57.74s/it] {'loss': 1.9054, 'learning_rate': 2e-05, 'epoch': 0.75} 75%|███████▌ | 1371/1817 [21:42:32<7:09:12, 57.74s/it] 76%|███████▌ | 1372/1817 [21:43:28<7:04:45, 57.27s/it] {'loss': 1.8951, 'learning_rate': 2e-05, 'epoch': 0.75} 76%|███████▌ | 1372/1817 [21:43:28<7:04:46, 57.27s/it] 76%|███████▌ | 1373/1817 [21:44:24<7:01:32, 56.96s/it] {'loss': 1.9275, 'learning_rate': 2e-05, 'epoch': 0.76} 76%|███████▌ | 1373/1817 [21:44:24<7:01:32, 56.96s/it] 76%|███████▌ | 1374/1817 [21:45:21<6:59:04, 56.76s/it] {'loss': 1.9048, 'learning_rate': 2e-05, 'epoch': 0.76} 76%|███████▌ | 1374/1817 [21:45:20<6:59:04, 56.76s/it] 76%|███████▌ | 1375/1817 [21:46:16<6:56:50, 56.58s/it] {'loss': 1.8833, 'learning_rate': 2e-05, 'epoch': 0.76} 76%|███████▌ | 1375/1817 [21:46:16<6:56:50, 56.58s/it] 76%|███████▌ | 1376/1817 [21:47:14<6:57:25, 56.79s/it] {'loss': 1.9127, 'learning_rate': 2e-05, 'epoch': 0.76} 76%|███████▌ | 1376/1817 [21:47:14<6:57:25, 56.79s/it] 76%|███████▌ | 1377/1817 [21:48:14<7:03:46, 57.79s/it] {'loss': 1.9315, 'learning_rate': 2e-05, 'epoch': 0.76} 76%|███████▌ | 1377/1817 [21:48:14<7:03:46, 57.79s/it] 76%|███████▌ | 1378/1817 [21:49:10<6:59:20, 57.31s/it] {'loss': 1.8951, 'learning_rate': 2e-05, 'epoch': 0.76} 76%|███████▌ | 1378/1817 [21:49:10<6:59:20, 57.31s/it] 76%|███████▌ | 1379/1817 [21:50:06<6:55:51, 56.97s/it] {'loss': 1.85, 'learning_rate': 2e-05, 'epoch': 0.76} 76%|███████▌ | 1379/1817 [21:50:06<6:55:51, 56.97s/it] 76%|███████▌ | 1380/1817 [21:51:03<6:53:11, 56.73s/it] {'loss': 1.8717, 'learning_rate': 2e-05, 'epoch': 0.76} 76%|███████▌ | 1380/1817 [21:51:02<6:53:11, 56.73s/it] 76%|███████▌ | 1381/1817 [21:52:00<6:54:15, 57.01s/it] {'loss': 1.8846, 'learning_rate': 2e-05, 'epoch': 0.76} 76%|███████▌ | 1381/1817 [21:52:00<6:54:16, 57.01s/it] 76%|███████▌ | 1382/1817 [21:53:00<6:59:24, 57.85s/it] {'loss': 1.8727, 'learning_rate': 2e-05, 'epoch': 0.76} 76%|███████▌ | 1382/1817 [21:53:00<6:59:24, 57.85s/it] 76%|███████▌ | 1383/1817 [21:53:59<7:01:49, 58.32s/it] {'loss': 1.9191, 'learning_rate': 2e-05, 'epoch': 0.76} 76%|███████▌ | 1383/1817 [21:53:59<7:01:49, 58.32s/it] 76%|███████▌ | 1384/1817 [21:54:58<7:01:56, 58.47s/it] {'loss': 1.9208, 'learning_rate': 2e-05, 'epoch': 0.76} 76%|███████▌ | 1384/1817 [21:54:58<7:01:56, 58.47s/it] 76%|███████▌ | 1385/1817 [21:55:57<7:01:32, 58.55s/it] {'loss': 1.9207, 'learning_rate': 2e-05, 'epoch': 0.76} 76%|███████▌ | 1385/1817 [21:55:57<7:01:32, 58.55s/it] 76%|███████▋ | 1386/1817 [21:56:52<6:54:19, 57.68s/it] {'loss': 1.9035, 'learning_rate': 2e-05, 'epoch': 0.76} 76%|███████▋ | 1386/1817 [21:56:52<6:54:19, 57.68s/it] 76%|███████▋ | 1387/1817 [21:57:52<6:57:40, 58.28s/it] {'loss': 1.8865, 'learning_rate': 2e-05, 'epoch': 0.76} 76%|███████▋ | 1387/1817 [21:57:52<6:57:40, 58.28s/it] 76%|███████▋ | 1388/1817 [21:58:50<6:55:07, 58.06s/it] {'loss': 1.8523, 'learning_rate': 2e-05, 'epoch': 0.76} 76%|███████▋ | 1388/1817 [21:58:50<6:55:07, 58.06s/it] 76%|███████▋ | 1389/1817 [21:59:46<6:50:36, 57.56s/it] {'loss': 1.9142, 'learning_rate': 2e-05, 'epoch': 0.76} 76%|███████▋ | 1389/1817 [21:59:46<6:50:36, 57.56s/it] 76%|███████▋ | 1390/1817 [22:00:41<6:42:42, 56.59s/it] {'loss': 1.9335, 'learning_rate': 2e-05, 'epoch': 0.76} 76%|███████▋ | 1390/1817 [22:00:40<6:42:38, 56.58s/it] 77%|███████▋ | 1391/1817 [22:01:37<6:41:11, 56.51s/it] {'loss': 1.868, 'learning_rate': 2e-05, 'epoch': 0.77} 77%|███████▋ | 1391/1817 [22:01:37<6:41:05, 56.49s/it] 77%|███████▋ | 1392/1817 [22:02:33<6:39:05, 56.34s/it] {'loss': 1.8913, 'learning_rate': 2e-05, 'epoch': 0.77} 77%|███████▋ | 1392/1817 [22:02:33<6:39:05, 56.34s/it] 77%|███████▋ | 1393/1817 [22:03:34<6:48:40, 57.83s/it] {'loss': 1.8888, 'learning_rate': 2e-05, 'epoch': 0.77} 77%|███████▋ | 1393/1817 [22:03:34<6:48:40, 57.83s/it] 77%|███████▋ | 1394/1817 [22:04:30<6:44:30, 57.38s/it] {'loss': 1.9144, 'learning_rate': 2e-05, 'epoch': 0.77} 77%|███████▋ | 1394/1817 [22:04:30<6:44:30, 57.38s/it] 77%|███████▋ | 1395/1817 [22:05:27<6:41:43, 57.12s/it] {'loss': 1.8761, 'learning_rate': 2e-05, 'epoch': 0.77} 77%|███████▋ | 1395/1817 [22:05:27<6:41:43, 57.12s/it] 77%|███████▋ | 1396/1817 [22:06:25<6:42:27, 57.36s/it] {'loss': 1.8716, 'learning_rate': 2e-05, 'epoch': 0.77} 77%|███████▋ | 1396/1817 [22:06:25<6:42:27, 57.36s/it] 77%|███████▋ | 1397/1817 [22:07:22<6:41:39, 57.38s/it] {'loss': 1.9306, 'learning_rate': 2e-05, 'epoch': 0.77} 77%|███████▋ | 1397/1817 [22:07:22<6:41:39, 57.38s/it] 77%|███████▋ | 1398/1817 [22:08:21<6:44:30, 57.93s/it] {'loss': 1.8826, 'learning_rate': 2e-05, 'epoch': 0.77} 77%|███████▋ | 1398/1817 [22:08:21<6:44:30, 57.93s/it] 77%|███████▋ | 1399/1817 [22:09:19<6:42:54, 57.83s/it] {'loss': 1.8673, 'learning_rate': 2e-05, 'epoch': 0.77} 77%|███████▋ | 1399/1817 [22:09:19<6:42:54, 57.83s/it] 77%|███████▋ | 1400/1817 [22:10:15<6:38:29, 57.34s/it] {'loss': 1.88, 'learning_rate': 2e-05, 'epoch': 0.77} 77%|███████▋ | 1400/1817 [22:10:15<6:38:29, 57.34s/it] 77%|███████▋ | 1401/1817 [22:11:12<6:35:11, 57.00s/it] {'loss': 1.8771, 'learning_rate': 2e-05, 'epoch': 0.77} 77%|███████▋ | 1401/1817 [22:11:11<6:35:11, 57.00s/it] 77%|███████▋ | 1402/1817 [22:12:10<6:36:55, 57.39s/it] {'loss': 1.8883, 'learning_rate': 2e-05, 'epoch': 0.77} 77%|███████▋ | 1402/1817 [22:12:10<6:36:55, 57.39s/it] 77%|███████▋ | 1403/1817 [22:13:06<6:33:34, 57.04s/it] {'loss': 1.9101, 'learning_rate': 2e-05, 'epoch': 0.77} 77%|███████▋ | 1403/1817 [22:13:06<6:33:34, 57.04s/it] 77%|███████▋ | 1404/1817 [22:14:02<6:31:16, 56.84s/it] {'loss': 1.9055, 'learning_rate': 2e-05, 'epoch': 0.77} 77%|███████▋ | 1404/1817 [22:14:02<6:31:16, 56.84s/it] 77%|███████▋ | 1405/1817 [22:15:00<6:32:22, 57.14s/it] {'loss': 1.9311, 'learning_rate': 2e-05, 'epoch': 0.77} 77%|███████▋ | 1405/1817 [22:15:00<6:32:22, 57.14s/it] 77%|███████▋ | 1406/1817 [22:15:56<6:29:21, 56.84s/it] {'loss': 1.8643, 'learning_rate': 2e-05, 'epoch': 0.77} 77%|███████▋ | 1406/1817 [22:15:56<6:29:21, 56.84s/it] 77%|███████▋ | 1407/1817 [22:16:55<6:31:16, 57.26s/it] {'loss': 1.8705, 'learning_rate': 2e-05, 'epoch': 0.77} 77%|███████▋ | 1407/1817 [22:16:54<6:31:17, 57.26s/it] 77%|███████▋ | 1408/1817 [22:17:54<6:34:13, 57.83s/it] {'loss': 1.8632, 'learning_rate': 2e-05, 'epoch': 0.77} 77%|███████▋ | 1408/1817 [22:17:54<6:34:13, 57.83s/it] 78%|███████▊ | 1409/1817 [22:18:50<6:29:33, 57.29s/it] {'loss': 1.8776, 'learning_rate': 2e-05, 'epoch': 0.78} 78%|███████▊ | 1409/1817 [22:18:50<6:29:33, 57.29s/it] 78%|███████▊ | 1410/1817 [22:19:46<6:26:56, 57.04s/it] {'loss': 1.8959, 'learning_rate': 2e-05, 'epoch': 0.78} 78%|███████▊ | 1410/1817 [22:19:46<6:26:56, 57.04s/it] 78%|███████▊ | 1411/1817 [22:20:42<6:24:21, 56.80s/it] {'loss': 1.8766, 'learning_rate': 2e-05, 'epoch': 0.78} 78%|███████▊ | 1411/1817 [22:20:42<6:24:21, 56.80s/it] 78%|███████▊ | 1412/1817 [22:21:40<6:25:08, 57.06s/it] {'loss': 1.8854, 'learning_rate': 2e-05, 'epoch': 0.78} 78%|███████▊ | 1412/1817 [22:21:40<6:25:08, 57.06s/it] 78%|███████▊ | 1413/1817 [22:22:36<6:22:08, 56.75s/it] {'loss': 1.9272, 'learning_rate': 2e-05, 'epoch': 0.78} 78%|███████▊ | 1413/1817 [22:22:36<6:22:08, 56.75s/it] 78%|███████▊ | 1414/1817 [22:23:31<6:17:30, 56.20s/it] {'loss': 1.9102, 'learning_rate': 2e-05, 'epoch': 0.78} 78%|███████▊ | 1414/1817 [22:23:31<6:17:30, 56.20s/it] 78%|███████▊ | 1415/1817 [22:24:27<6:16:58, 56.26s/it] {'loss': 1.881, 'learning_rate': 2e-05, 'epoch': 0.78} 78%|███████▊ | 1415/1817 [22:24:27<6:16:58, 56.26s/it] 78%|███████▊ | 1416/1817 [22:25:24<6:16:25, 56.32s/it] {'loss': 1.8741, 'learning_rate': 2e-05, 'epoch': 0.78} 78%|███████▊ | 1416/1817 [22:25:24<6:16:25, 56.32s/it] 78%|███████▊ | 1417/1817 [22:26:24<6:24:02, 57.61s/it] {'loss': 1.8903, 'learning_rate': 2e-05, 'epoch': 0.78} 78%|███████▊ | 1417/1817 [22:26:24<6:24:02, 57.61s/it] 78%|███████▊ | 1418/1817 [22:27:24<6:26:09, 58.07s/it] {'loss': 1.8926, 'learning_rate': 2e-05, 'epoch': 0.78} 78%|███████▊ | 1418/1817 [22:27:24<6:26:09, 58.07s/it] 78%|███████▊ | 1419/1817 [22:28:20<6:22:24, 57.65s/it] {'loss': 1.8934, 'learning_rate': 2e-05, 'epoch': 0.78} 78%|███████▊ | 1419/1817 [22:28:20<6:22:24, 57.65s/it] 78%|███████▊ | 1420/1817 [22:29:17<6:19:28, 57.35s/it] {'loss': 1.8667, 'learning_rate': 2e-05, 'epoch': 0.78} 78%|███████▊ | 1420/1817 [22:29:17<6:19:28, 57.35s/it] 78%|███████▊ | 1421/1817 [22:30:14<6:16:42, 57.08s/it] {'loss': 1.8953, 'learning_rate': 2e-05, 'epoch': 0.78} 78%|███████▊ | 1421/1817 [22:30:13<6:16:42, 57.08s/it] 78%|███████▊ | 1422/1817 [22:31:10<6:15:10, 56.99s/it] {'loss': 1.9079, 'learning_rate': 2e-05, 'epoch': 0.78} 78%|███████▊ | 1422/1817 [22:31:10<6:15:10, 56.99s/it] 78%|███████▊ | 1423/1817 [22:32:08<6:16:13, 57.29s/it] {'loss': 1.8942, 'learning_rate': 2e-05, 'epoch': 0.78} 78%|███████▊ | 1423/1817 [22:32:08<6:16:13, 57.29s/it] 78%|███████▊ | 1424/1817 [22:33:06<6:16:52, 57.54s/it] {'loss': 1.8904, 'learning_rate': 2e-05, 'epoch': 0.78} 78%|███████▊ | 1424/1817 [22:33:06<6:16:51, 57.54s/it] 78%|███████▊ | 1425/1817 [22:34:03<6:13:23, 57.15s/it] {'loss': 1.929, 'learning_rate': 2e-05, 'epoch': 0.78} 78%|███████▊ | 1425/1817 [22:34:02<6:13:23, 57.15s/it] 78%|███████▊ | 1426/1817 [22:35:00<6:12:59, 57.24s/it] {'loss': 1.8849, 'learning_rate': 2e-05, 'epoch': 0.78} 78%|███████▊ | 1426/1817 [22:35:00<6:12:59, 57.24s/it] 79%|███████▊ | 1427/1817 [22:35:59<6:15:28, 57.77s/it] {'loss': 1.8974, 'learning_rate': 2e-05, 'epoch': 0.79} 79%|███████▊ | 1427/1817 [22:35:59<6:15:28, 57.77s/it] 79%|███████▊ | 1428/1817 [22:36:59<6:17:35, 58.24s/it] {'loss': 1.8912, 'learning_rate': 2e-05, 'epoch': 0.79} 79%|███████▊ | 1428/1817 [22:36:58<6:17:35, 58.24s/it] 79%|███████▊ | 1429/1817 [22:37:56<6:15:29, 58.06s/it] {'loss': 1.9139, 'learning_rate': 2e-05, 'epoch': 0.79} 79%|███████▊ | 1429/1817 [22:37:56<6:15:29, 58.06s/it] 79%|███████▊ | 1430/1817 [22:38:52<6:10:59, 57.52s/it] {'loss': 1.8909, 'learning_rate': 2e-05, 'epoch': 0.79} 79%|███████▊ | 1430/1817 [22:38:52<6:10:59, 57.52s/it] 79%|███████▉ | 1431/1817 [22:39:49<6:07:43, 57.16s/it] {'loss': 1.8696, 'learning_rate': 2e-05, 'epoch': 0.79} 79%|███████▉ | 1431/1817 [22:39:48<6:07:43, 57.16s/it] 79%|███████▉ | 1432/1817 [22:40:44<6:02:23, 56.48s/it] {'loss': 1.8824, 'learning_rate': 2e-05, 'epoch': 0.79} 79%|███████▉ | 1432/1817 [22:40:43<6:02:22, 56.48s/it] 79%|███████▉ | 1433/1817 [22:41:43<6:06:26, 57.26s/it] {'loss': 1.8946, 'learning_rate': 2e-05, 'epoch': 0.79} 79%|███████▉ | 1433/1817 [22:41:42<6:06:26, 57.26s/it] 79%|███████▉ | 1434/1817 [22:42:39<6:03:23, 56.93s/it] {'loss': 1.8849, 'learning_rate': 2e-05, 'epoch': 0.79} 79%|███████▉ | 1434/1817 [22:42:39<6:03:23, 56.93s/it] 79%|███████▉ | 1435/1817 [22:43:35<6:01:01, 56.70s/it] {'loss': 1.9193, 'learning_rate': 2e-05, 'epoch': 0.79} 79%|███████▉ | 1435/1817 [22:43:35<6:01:01, 56.71s/it] 79%|███████▉ | 1436/1817 [22:44:31<5:59:11, 56.56s/it] {'loss': 1.8715, 'learning_rate': 2e-05, 'epoch': 0.79} 79%|███████▉ | 1436/1817 [22:44:31<5:59:11, 56.57s/it] 79%|███████▉ | 1437/1817 [22:45:29<6:00:10, 56.87s/it] {'loss': 1.913, 'learning_rate': 2e-05, 'epoch': 0.79} 79%|███████▉ | 1437/1817 [22:45:29<6:00:10, 56.87s/it] 79%|███████▉ | 1438/1817 [22:46:27<6:01:56, 57.30s/it] {'loss': 1.8752, 'learning_rate': 2e-05, 'epoch': 0.79} 79%|███████▉ | 1438/1817 [22:46:27<6:01:56, 57.30s/it] 79%|███████▉ | 1439/1817 [22:47:26<6:03:52, 57.76s/it] {'loss': 1.8792, 'learning_rate': 2e-05, 'epoch': 0.79} 79%|███████▉ | 1439/1817 [22:47:26<6:03:52, 57.76s/it] 79%|███████▉ | 1440/1817 [22:48:22<6:00:05, 57.31s/it] {'loss': 1.8891, 'learning_rate': 2e-05, 'epoch': 0.79} 79%|███████▉ | 1440/1817 [22:48:22<6:00:05, 57.31s/it] 79%|███████▉ | 1441/1817 [22:49:19<5:57:33, 57.06s/it] {'loss': 1.8956, 'learning_rate': 2e-05, 'epoch': 0.79} 79%|███████▉ | 1441/1817 [22:49:18<5:57:33, 57.06s/it] 79%|███████▉ | 1442/1817 [22:50:15<5:54:52, 56.78s/it] {'loss': 1.9269, 'learning_rate': 2e-05, 'epoch': 0.79} 79%|███████▉ | 1442/1817 [22:50:15<5:54:52, 56.78s/it] 79%|███████▉ | 1443/1817 [22:51:13<5:56:05, 57.13s/it] {'loss': 1.905, 'learning_rate': 2e-05, 'epoch': 0.79} 79%|███████▉ | 1443/1817 [22:51:13<5:56:05, 57.13s/it] 79%|███████▉ | 1444/1817 [22:52:11<5:57:01, 57.43s/it] {'loss': 1.9071, 'learning_rate': 2e-05, 'epoch': 0.79} 79%|███████▉ | 1444/1817 [22:52:11<5:57:02, 57.43s/it] 80%|███████▉ | 1445/1817 [22:53:07<5:54:20, 57.15s/it] {'loss': 1.9106, 'learning_rate': 2e-05, 'epoch': 0.8} 80%|███████▉ | 1445/1817 [22:53:07<5:54:20, 57.15s/it] 80%|███████▉ | 1446/1817 [22:54:04<5:51:55, 56.91s/it] {'loss': 1.912, 'learning_rate': 2e-05, 'epoch': 0.8} 80%|███████▉ | 1446/1817 [22:54:04<5:51:55, 56.91s/it] 80%|███████▉ | 1447/1817 [22:55:00<5:49:53, 56.74s/it] {'loss': 1.853, 'learning_rate': 2e-05, 'epoch': 0.8} 80%|███████▉ | 1447/1817 [22:55:00<5:49:53, 56.74s/it] 80%|███████▉ | 1448/1817 [22:55:56<5:48:13, 56.62s/it] {'loss': 1.9006, 'learning_rate': 2e-05, 'epoch': 0.8} 80%|███████▉ | 1448/1817 [22:55:56<5:48:13, 56.62s/it] 80%|███████▉ | 1449/1817 [22:56:54<5:49:02, 56.91s/it] {'loss': 1.9215, 'learning_rate': 2e-05, 'epoch': 0.8} 80%|███████▉ | 1449/1817 [22:56:54<5:49:02, 56.91s/it] 80%|███████▉ | 1450/1817 [22:57:50<5:46:56, 56.72s/it] {'loss': 1.9246, 'learning_rate': 2e-05, 'epoch': 0.8} 80%|███████▉ | 1450/1817 [22:57:50<5:46:56, 56.72s/it] 80%|███████▉ | 1451/1817 [22:58:48<5:48:00, 57.05s/it] {'loss': 1.9249, 'learning_rate': 2e-05, 'epoch': 0.8} 80%|███████▉ | 1451/1817 [22:58:48<5:48:00, 57.05s/it] 80%|███████▉ | 1452/1817 [22:59:44<5:45:31, 56.80s/it] {'loss': 1.9334, 'learning_rate': 2e-05, 'epoch': 0.8} 80%|███████▉ | 1452/1817 [22:59:44<5:45:31, 56.80s/it] 80%|███████▉ | 1453/1817 [23:00:41<5:43:25, 56.61s/it] {'loss': 1.9227, 'learning_rate': 2e-05, 'epoch': 0.8} 80%|███████▉ | 1453/1817 [23:00:40<5:43:22, 56.60s/it] 80%|████████ | 1454/1817 [23:01:38<5:44:53, 57.01s/it] {'loss': 1.8924, 'learning_rate': 2e-05, 'epoch': 0.8} 80%|████████ | 1454/1817 [23:01:38<5:44:53, 57.01s/it] 80%|████████ | 1455/1817 [23:02:35<5:43:01, 56.85s/it] {'loss': 1.8992, 'learning_rate': 2e-05, 'epoch': 0.8} 80%|████████ | 1455/1817 [23:02:35<5:42:54, 56.84s/it] 80%|████████ | 1456/1817 [23:03:33<5:43:19, 57.06s/it] {'loss': 1.877, 'learning_rate': 2e-05, 'epoch': 0.8} 80%|████████ | 1456/1817 [23:03:32<5:43:15, 57.05s/it] 80%|████████ | 1457/1817 [23:04:29<5:41:20, 56.89s/it] {'loss': 1.87, 'learning_rate': 2e-05, 'epoch': 0.8} 80%|████████ | 1457/1817 [23:04:29<5:41:21, 56.89s/it] 80%|████████ | 1458/1817 [23:05:25<5:39:31, 56.75s/it] {'loss': 1.9116, 'learning_rate': 2e-05, 'epoch': 0.8} 80%|████████ | 1458/1817 [23:05:25<5:39:32, 56.75s/it] 80%|████████ | 1459/1817 [23:06:22<5:38:06, 56.67s/it] {'loss': 1.9212, 'learning_rate': 2e-05, 'epoch': 0.8} 80%|████████ | 1459/1817 [23:06:22<5:38:06, 56.67s/it] 80%|████████ | 1460/1817 [23:07:20<5:40:08, 57.17s/it] {'loss': 1.8851, 'learning_rate': 2e-05, 'epoch': 0.8} 80%|████████ | 1460/1817 [23:07:20<5:40:08, 57.17s/it] 80%|████████ | 1461/1817 [23:08:20<5:43:35, 57.91s/it] {'loss': 1.8988, 'learning_rate': 2e-05, 'epoch': 0.8} 80%|████████ | 1461/1817 [23:08:20<5:43:34, 57.91s/it] 80%|████████ | 1462/1817 [23:09:16<5:39:49, 57.43s/it] {'loss': 1.9017, 'learning_rate': 2e-05, 'epoch': 0.8} 80%|████████ | 1462/1817 [23:09:16<5:39:49, 57.43s/it] 81%|████████ | 1463/1817 [23:10:11<5:33:49, 56.58s/it] {'loss': 1.8875, 'learning_rate': 2e-05, 'epoch': 0.8} 81%|████████ | 1463/1817 [23:10:10<5:33:49, 56.58s/it] 81%|████████ | 1464/1817 [23:11:09<5:35:15, 56.99s/it] {'loss': 1.9285, 'learning_rate': 2e-05, 'epoch': 0.81} 81%|████████ | 1464/1817 [23:11:08<5:35:15, 56.99s/it] 81%|████████ | 1465/1817 [23:12:05<5:33:02, 56.77s/it] {'loss': 1.8981, 'learning_rate': 2e-05, 'epoch': 0.81} 81%|████████ | 1465/1817 [23:12:05<5:33:02, 56.77s/it] 81%|████████ | 1466/1817 [23:13:03<5:34:10, 57.12s/it] {'loss': 1.9121, 'learning_rate': 2e-05, 'epoch': 0.81} 81%|████████ | 1466/1817 [23:13:03<5:34:10, 57.12s/it] 81%|████████ | 1467/1817 [23:13:59<5:31:44, 56.87s/it] {'loss': 1.8891, 'learning_rate': 2e-05, 'epoch': 0.81} 81%|████████ | 1467/1817 [23:13:59<5:31:44, 56.87s/it] 81%|████████ | 1468/1817 [23:14:56<5:30:06, 56.75s/it] {'loss': 1.9316, 'learning_rate': 2e-05, 'epoch': 0.81} 81%|████████ | 1468/1817 [23:14:55<5:30:06, 56.75s/it] 81%|████████ | 1469/1817 [23:15:52<5:28:37, 56.66s/it] {'loss': 1.9156, 'learning_rate': 2e-05, 'epoch': 0.81} 81%|████████ | 1469/1817 [23:15:52<5:28:37, 56.66s/it] 81%|████████ | 1470/1817 [23:16:48<5:27:02, 56.55s/it] {'loss': 1.9049, 'learning_rate': 2e-05, 'epoch': 0.81} 81%|████████ | 1470/1817 [23:16:48<5:27:02, 56.55s/it] 81%|████████ | 1471/1817 [23:17:44<5:25:17, 56.41s/it] {'loss': 1.9154, 'learning_rate': 2e-05, 'epoch': 0.81} 81%|████████ | 1471/1817 [23:17:44<5:25:17, 56.41s/it] 81%|████████ | 1472/1817 [23:18:41<5:24:15, 56.39s/it] {'loss': 1.9135, 'learning_rate': 2e-05, 'epoch': 0.81} 81%|████████ | 1472/1817 [23:18:41<5:24:15, 56.39s/it] 81%|████████ | 1473/1817 [23:19:37<5:23:15, 56.38s/it] {'loss': 1.9103, 'learning_rate': 2e-05, 'epoch': 0.81} 81%|████████ | 1473/1817 [23:19:37<5:23:15, 56.38s/it] 81%|████████ | 1474/1817 [23:20:34<5:23:23, 56.57s/it] {'loss': 1.8747, 'learning_rate': 2e-05, 'epoch': 0.81} 81%|████████ | 1474/1817 [23:20:34<5:23:23, 56.57s/it] 81%|████████ | 1475/1817 [23:21:30<5:21:48, 56.46s/it] {'loss': 1.9247, 'learning_rate': 2e-05, 'epoch': 0.81} 81%|████████ | 1475/1817 [23:21:30<5:21:48, 56.46s/it] 81%|████████ | 1476/1817 [23:22:30<5:26:13, 57.40s/it] {'loss': 1.8948, 'learning_rate': 2e-05, 'epoch': 0.81} 81%|████████ | 1476/1817 [23:22:30<5:26:13, 57.40s/it] 81%|████████▏ | 1477/1817 [23:23:28<5:25:41, 57.47s/it] {'loss': 1.8958, 'learning_rate': 2e-05, 'epoch': 0.81} 81%|████████▏ | 1477/1817 [23:23:27<5:25:41, 57.47s/it] 81%|████████▏ | 1478/1817 [23:24:25<5:24:45, 57.48s/it] {'loss': 1.855, 'learning_rate': 2e-05, 'epoch': 0.81} 81%|████████▏ | 1478/1817 [23:24:25<5:24:45, 57.48s/it] 81%|████████▏ | 1479/1817 [23:25:21<5:21:51, 57.13s/it] {'loss': 1.9358, 'learning_rate': 2e-05, 'epoch': 0.81} 81%|████████▏ | 1479/1817 [23:25:21<5:21:51, 57.13s/it] 81%|████████▏ | 1480/1817 [23:26:18<5:19:32, 56.89s/it] {'loss': 1.9413, 'learning_rate': 2e-05, 'epoch': 0.81} 81%|████████▏ | 1480/1817 [23:26:17<5:19:32, 56.89s/it] 82%|████████▏ | 1481/1817 [23:27:14<5:17:25, 56.68s/it] {'loss': 1.8968, 'learning_rate': 2e-05, 'epoch': 0.81} 82%|████████▏ | 1481/1817 [23:27:14<5:17:25, 56.68s/it] 82%|████████▏ | 1482/1817 [23:28:11<5:16:26, 56.68s/it] {'loss': 1.9389, 'learning_rate': 2e-05, 'epoch': 0.82} 82%|████████▏ | 1482/1817 [23:28:10<5:16:26, 56.68s/it] 82%|████████▏ | 1483/1817 [23:29:08<5:15:56, 56.75s/it] {'loss': 1.8877, 'learning_rate': 2e-05, 'epoch': 0.82} 82%|████████▏ | 1483/1817 [23:29:07<5:15:56, 56.75s/it] 82%|████████▏ | 1484/1817 [23:30:05<5:15:57, 56.93s/it] {'loss': 1.9037, 'learning_rate': 2e-05, 'epoch': 0.82} 82%|████████▏ | 1484/1817 [23:30:05<5:15:57, 56.93s/it] 82%|████████▏ | 1485/1817 [23:31:01<5:14:06, 56.77s/it] {'loss': 1.9354, 'learning_rate': 2e-05, 'epoch': 0.82} 82%|████████▏ | 1485/1817 [23:31:01<5:14:06, 56.77s/it] 82%|████████▏ | 1486/1817 [23:31:59<5:13:53, 56.90s/it] {'loss': 1.9027, 'learning_rate': 2e-05, 'epoch': 0.82} 82%|████████▏ | 1486/1817 [23:31:58<5:13:53, 56.90s/it] 82%|████████▏ | 1487/1817 [23:32:56<5:13:16, 56.96s/it] {'loss': 1.9015, 'learning_rate': 2e-05, 'epoch': 0.82} 82%|████████▏ | 1487/1817 [23:32:55<5:13:16, 56.96s/it] 82%|████████▏ | 1488/1817 [23:33:51<5:10:02, 56.54s/it] {'loss': 1.8841, 'learning_rate': 2e-05, 'epoch': 0.82} 82%|████████▏ | 1488/1817 [23:33:51<5:10:02, 56.54s/it] 82%|████████▏ | 1489/1817 [23:34:49<5:11:49, 57.04s/it] {'loss': 1.9094, 'learning_rate': 2e-05, 'epoch': 0.82} 82%|████████▏ | 1489/1817 [23:34:49<5:11:49, 57.04s/it] 82%|████████▏ | 1490/1817 [23:35:46<5:09:22, 56.77s/it] {'loss': 1.9027, 'learning_rate': 2e-05, 'epoch': 0.82} 82%|████████▏ | 1490/1817 [23:35:45<5:09:22, 56.77s/it] 82%|████████▏ | 1491/1817 [23:36:42<5:07:31, 56.60s/it] {'loss': 1.865, 'learning_rate': 2e-05, 'epoch': 0.82} 82%|████████▏ | 1491/1817 [23:36:41<5:07:31, 56.60s/it] 82%|████████▏ | 1492/1817 [23:37:40<5:08:35, 56.97s/it] {'loss': 1.8647, 'learning_rate': 2e-05, 'epoch': 0.82} 82%|████████▏ | 1492/1817 [23:37:39<5:08:35, 56.97s/it] 82%|████████▏ | 1493/1817 [23:38:37<5:07:53, 57.02s/it] {'loss': 1.8958, 'learning_rate': 2e-05, 'epoch': 0.82} 82%|████████▏ | 1493/1817 [23:38:36<5:07:53, 57.02s/it] 82%|████████▏ | 1494/1817 [23:39:33<5:05:43, 56.79s/it] {'loss': 1.903, 'learning_rate': 2e-05, 'epoch': 0.82} 82%|████████▏ | 1494/1817 [23:39:33<5:05:43, 56.79s/it] 82%|████████▏ | 1495/1817 [23:40:30<5:04:29, 56.74s/it] {'loss': 1.8753, 'learning_rate': 2e-05, 'epoch': 0.82} 82%|████████▏ | 1495/1817 [23:40:29<5:04:29, 56.74s/it] 82%|████████▏ | 1496/1817 [23:41:26<5:02:50, 56.61s/it] {'loss': 1.9074, 'learning_rate': 2e-05, 'epoch': 0.82} 82%|████████▏ | 1496/1817 [23:41:26<5:02:50, 56.61s/it] 82%|████████▏ | 1497/1817 [23:42:22<5:01:37, 56.55s/it] {'loss': 1.9044, 'learning_rate': 2e-05, 'epoch': 0.82} 82%|████████▏ | 1497/1817 [23:42:22<5:01:37, 56.55s/it] 82%|████████▏ | 1498/1817 [23:43:17<4:58:09, 56.08s/it] {'loss': 1.8523, 'learning_rate': 2e-05, 'epoch': 0.82} 82%|████████▏ | 1498/1817 [23:43:17<4:58:09, 56.08s/it] 82%|████████▏ | 1499/1817 [23:44:14<4:57:43, 56.17s/it] {'loss': 1.9226, 'learning_rate': 2e-05, 'epoch': 0.82} 82%|████████▏ | 1499/1817 [23:44:13<4:57:43, 56.17s/it] 83%|████████▎ | 1500/1817 [23:45:11<4:59:16, 56.65s/it] {'loss': 1.9049, 'learning_rate': 2e-05, 'epoch': 0.83} 83%|████████▎ | 1500/1817 [23:45:11<4:59:16, 56.65s/it][INFO|trainer.py:2830] 2024-01-29 04:38:55,939 >> Saving model checkpoint to ./checkpoints_ct/ac/allm-ac-7b/checkpoint-1500 [INFO|configuration_utils.py:457] 2024-01-29 04:38:55,984 >> Configuration saved in ./checkpoints_ct/ac/allm-ac-7b/checkpoint-1500/config.json [INFO|configuration_utils.py:362] 2024-01-29 04:38:55,988 >> Configuration saved in ./checkpoints_ct/ac/allm-ac-7b/checkpoint-1500/generation_config.json [INFO|modeling_utils.py:1759] 2024-01-29 04:38:56,017 >> Model weights saved in ./checkpoints_ct/ac/allm-ac-7b/checkpoint-1500/pytorch_model.bin [INFO|tokenization_utils_base.py:2164] 2024-01-29 04:38:56,019 >> tokenizer config file saved in ./checkpoints_ct/ac/allm-ac-7b/checkpoint-1500/tokenizer_config.json [INFO|tokenization_utils_base.py:2171] 2024-01-29 04:38:56,020 >> Special tokens file saved in ./checkpoints_ct/ac/allm-ac-7b/checkpoint-1500/special_tokens_map.json [INFO|tokenization_utils_base.py:2221] 2024-01-29 04:38:56,021 >> added tokens file saved in ./checkpoints_ct/ac/allm-ac-7b/checkpoint-1500/added_tokens.json [2024-01-29 04:39:03,863] [INFO] [logging.py:96:log_dist] [Rank 0] [Torch] Checkpoint global_step1500 is about to be saved! [2024-01-29 04:39:03,865] [INFO] [engine.py:3492:save_16bit_model] Saving model weights to ./checkpoints_ct/ac/allm-ac-7b/checkpoint-1500/pytorch_model.bin, tag: global_step1500 [2024-01-29 04:39:03,865] [INFO] [torch_checkpoint_engine.py:21:save] [Torch] Saving ./checkpoints_ct/ac/allm-ac-7b/checkpoint-1500/pytorch_model.bin... [2024-01-29 04:39:20,223] [INFO] [torch_checkpoint_engine.py:21:save] [Torch] Saving ./checkpoints_ct/ac/allm-ac-7b/checkpoint-1500/global_step1500/zero_pp_rank_8_mp_rank_00_model_states.pt... [2024-01-29 04:39:20,265] [INFO] [torch_checkpoint_engine.py:23:save] [Torch] Saved ./checkpoints_ct/ac/allm-ac-7b/checkpoint-1500/global_step1500/zero_pp_rank_8_mp_rank_00_model_states.pt. [2[2024-01-29 04:39:20,281] [INFO] [torch_checkpoint_engine.py:21:save] [Torch] Saving ./checkpoints_ct/ac/allm-ac-7b/checkpoint-1500/global_step1500/bf16_zero_pp_rank_16_mp_rank_00_optim_states.pt... [2024-01-29 04:39:27,668] [INFO] [torch_checkpoint_engine.py:23:save] [Torch] Saved ./checkpoints_ct/ac/allm-ac-7b/checkpoint-1500/global_step1500/bf16_zero_pp_rank_24_mp_rank_00_optim_states.pt. [2024-01-29 04:39:27,669] [INFO] [engine.py:3381:_save_zero_checkpoint] zero checkpoint saved ./checkpoints_ct/ac/allm-ac-7b/checkpoint-1500/global_step1500/bf16_zero_pp_rank_24_mp_rank_00_optim_states.pt [2024-01-29 04:39:27,761] [INFO] [torch_checkpoint_engine.py:33:commit] [Torch] Checkpoint global_step1500 is ready now! /global_step1500/bf16_zero_pp_rank_0_mp_rank_00_optim_states.pt... [2024-01-29 04:39:27,649] [INFO] [torch_checkpoint_engine.py:23:save] [Torch] Saved ./checkpoints_ct/ac/allm-ac-7b/checkpoint-1500/global_step1500/bf16_zero_pp_rank_8_mp_rank_00_optim_states.pt. [2024-01-29 04:39:27,667] [INFO] [engine.py:3381:_save_zero_checkpoint] zero checkpoint saved ./checkpoints_ct/ac/allm-ac-7b/checkpoint-1500/global_step1500/bf16_zero_pp_rank_0_mp_rank_00_optim_states.pt [2[2024-01-29 04:39:27,806] [INFO] [torch_checkpoint_engine.py:33:commit] [Torch] Checkpoint global_step1500 is ready now[INFO|trainer.py:2908] 2024-01-29 04:39:27,871 >> Deleting older checkpoint [checkpoints_ct/ac/allm-ac-7b/checkpoint-1000] due to args.save_total_limit 83%|████████▎ | 1501/1817 [23:46:40<5:49:09, 66.30s/it] {'loss': 1.9021, 'learning_rate': 2e-05, 'epoch': 0.83} 83%|████████▎ | 1501/1817 [23:46:40<5:49:09, 66.30s/it] 83%|████████▎ | 1502/1817 [23:47:38<5:34:51, 63.78s/it] {'loss': 1.8843, 'learning_rate': 2e-05, 'epoch': 0.83} 83%|████████▎ | 1502/1817 [23:47:38<5:34:51, 63.78s/it] 83%|████████▎ | 1503/1817 [23:48:32<5:18:54, 60.94s/it] {'loss': 1.9, 'learning_rate': 2e-05, 'epoch': 0.83} 83%|████████▎ | 1503/1817 [23:48:32<5:18:54, 60.94s/it] 83%|████████▎ | 1504/1817 [23:49:29<5:11:14, 59.66s/it] {'loss': 1.8842, 'learning_rate': 2e-05, 'epoch': 0.83} 83%|████████▎ | 1504/1817 [23:49:29<5:11:14, 59.66s/it] 83%|████████▎ | 1505/1817 [23:50:26<5:05:17, 58.71s/it] {'loss': 1.8443, 'learning_rate': 2e-05, 'epoch': 0.83} 83%|████████▎ | 1505/1817 [23:50:25<5:05:17, 58.71s/it] 83%|████████▎ | 1506/1817 [23:51:22<5:00:55, 58.06s/it] {'loss': 1.89, 'learning_rate': 2e-05, 'epoch': 0.83} 83%|████████▎ | 1506/1817 [23:51:22<5:00:55, 58.06s/it] 83%|████████▎ | 1507/1817 [23:52:18<4:57:43, 57.62s/it] {'loss': 1.8959, 'learning_rate': 2e-05, 'epoch': 0.83} 83%|████████▎ | 1507/1817 [23:52:18<4:57:43, 57.62s/it] 83%|████████▎ | 1508/1817 [23:53:16<4:56:37, 57.60s/it] {'loss': 1.9538, 'learning_rate': 2e-05, 'epoch': 0.83} 83%|████████▎ | 1508/1817 [23:53:16<4:56:37, 57.60s/it] 83%|████████▎ | 1509/1817 [23:54:14<4:55:44, 57.61s/it] {'loss': 1.883, 'learning_rate': 2e-05, 'epoch': 0.83} 83%|████████▎ | 1509/1817 [23:54:14<4:55:44, 57.61s/it] 83%|████████▎ | 1510/1817 [23:55:12<4:54:56, 57.64s/it] {'loss': 1.8624, 'learning_rate': 2e-05, 'epoch': 0.83} 83%|████████▎ | 1510/1817 [23:55:11<4:54:56, 57.64s/it] 83%|████████▎ | 1511/1817 [23:56:08<4:52:01, 57.26s/it] {'loss': 1.8882, 'learning_rate': 2e-05, 'epoch': 0.83} 83%|████████▎ | 1511/1817 [23:56:08<4:52:01, 57.26s/it] 83%|████████▎ | 1512/1817 [23:57:05<4:49:54, 57.03s/it] {'loss': 1.9289, 'learning_rate': 2e-05, 'epoch': 0.83} 83%|████████▎ | 1512/1817 [23:57:04<4:49:54, 57.03s/it] 83%|████████▎ | 1513/1817 [23:58:01<4:48:09, 56.87s/it] {'loss': 1.9491, 'learning_rate': 2e-05, 'epoch': 0.83} 83%|████████▎ | 1513/1817 [23:58:01<4:48:09, 56.87s/it] 83%|████████▎ | 1514/1817 [23:58:57<4:46:11, 56.67s/it] {'loss': 1.9563, 'learning_rate': 2e-05, 'epoch': 0.83} 83%|████████▎ | 1514/1817 [23:58:57<4:46:11, 56.67s/it] 83%|████████▎ | 1515/1817 [23:59:54<4:45:01, 56.63s/it] {'loss': 1.8533, 'learning_rate': 2e-05, 'epoch': 0.83} 83%|████████▎ | 1515/1817 [23:59:53<4:45:01, 56.63s/it] 83%|████████▎ | 1516/1817 [24:00:49<4:41:30, 56.12s/it] {'loss': 1.8741, 'learning_rate': 2e-05, 'epoch': 0.83} 83%|████████▎ | 1516/1817 [24:00:48<4:41:31, 56.12s/it] 83%|████████▎ | 1517/1817 [24:01:48<4:45:50, 57.17s/it] {'loss': 1.9011, 'learning_rate': 2e-05, 'epoch': 0.83} 83%|████████▎ | 1517/1817 [24:01:48<4:45:50, 57.17s/it] 84%|████████▎ | 1518/1817 [24:02:45<4:44:02, 57.00s/it] {'loss': 1.8857, 'learning_rate': 2e-05, 'epoch': 0.84} 84%|████████▎ | 1518/1817 [24:02:45<4:43:56, 56.98s/it] 84%|████████▎ | 1519/1817 [24:03:41<4:42:08, 56.81s/it] {'loss': 1.9173, 'learning_rate': 2e-05, 'epoch': 0.84} 84%|████████▎ | 1519/1817 [24:03:41<4:42:08, 56.81s/it] 84%|████████▎ | 1520/1817 [24:04:41<4:45:15, 57.63s/it] {'loss': 1.9364, 'learning_rate': 2e-05, 'epoch': 0.84} 84%|████████▎ | 1520/1817 [24:04:40<4:45:15, 57.63s/it] 84%|████████▎ | 1521/1817 [24:05:37<4:42:28, 57.26s/it] {'loss': 1.9074, 'learning_rate': 2e-05, 'epoch': 0.84} 84%|████████▎ | 1521/1817 [24:05:37<4:42:28, 57.26s/it] 84%|████████▍ | 1522/1817 [24:06:35<4:42:26, 57.44s/it] {'loss': 1.9035, 'learning_rate': 2e-05, 'epoch': 0.84} 84%|████████▍ | 1522/1817 [24:06:35<4:42:24, 57.44s/it] 84%|████████▍ | 1523/1817 [24:07:32<4:40:07, 57.17s/it] {'loss': 1.8968, 'learning_rate': 2e-05, 'epoch': 0.84} 84%|████████▍ | 1523/1817 [24:07:31<4:40:07, 57.17s/it] 84%|████████▍ | 1524/1817 [24:08:28<4:38:35, 57.05s/it] {'loss': 1.9074, 'learning_rate': 2e-05, 'epoch': 0.84} 84%|████████▍ | 1524/1817 [24:08:28<4:38:35, 57.05s/it] 84%|████████▍ | 1525/1817 [24:09:25<4:36:45, 56.87s/it] {'loss': 1.9239, 'learning_rate': 2e-05, 'epoch': 0.84} 84%|████████▍ | 1525/1817 [24:09:25<4:36:45, 56.87s/it] 84%|████████▍ | 1526/1817 [24:10:23<4:37:42, 57.26s/it] {'loss': 1.8526, 'learning_rate': 2e-05, 'epoch': 0.84} 84%|████████▍ | 1526/1817 [24:10:23<4:37:42, 57.26s/it] 84%|████████▍ | 1527/1817 [24:11:18<4:33:38, 56.62s/it] {'loss': 1.8636, 'learning_rate': 2e-05, 'epoch': 0.84} 84%|████████▍ | 1527/1817 [24:11:18<4:33:38, 56.62s/it] 84%|████████▍ | 1528/1817 [24:12:14<4:32:16, 56.53s/it] {'loss': 1.8515, 'learning_rate': 2e-05, 'epoch': 0.84} 84%|████████▍ | 1528/1817 [24:12:14<4:32:16, 56.53s/it] 84%|████████▍ | 1529/1817 [24:13:11<4:31:09, 56.49s/it] {'loss': 1.9215, 'learning_rate': 2e-05, 'epoch': 0.84} 84%|████████▍ | 1529/1817 [24:13:11<4:31:09, 56.49s/it] 84%|████████▍ | 1530/1817 [24:14:06<4:28:44, 56.18s/it] {'loss': 1.9211, 'learning_rate': 2e-05, 'epoch': 0.84} 84%|████████▍ | 1530/1817 [24:14:06<4:28:44, 56.18s/it] 84%|████████▍ | 1531/1817 [24:15:02<4:26:38, 55.94s/it] {'loss': 1.871, 'learning_rate': 2e-05, 'epoch': 0.84} 84%|████████▍ | 1531/1817 [24:15:01<4:26:38, 55.94s/it] 84%|████████▍ | 1532/1817 [24:15:58<4:26:21, 56.08s/it] {'loss': 1.8938, 'learning_rate': 2e-05, 'epoch': 0.84} 84%|████████▍ | 1532/1817 [24:15:58<4:26:21, 56.08s/it] 84%|████████▍ | 1533/1817 [24:16:56<4:28:46, 56.78s/it] {'loss': 1.9038, 'learning_rate': 2e-05, 'epoch': 0.84} 84%|████████▍ | 1533/1817 [24:16:56<4:28:46, 56.78s/it] 84%|████████▍ | 1534/1817 [24:17:54<4:28:59, 57.03s/it] {'loss': 1.898, 'learning_rate': 2e-05, 'epoch': 0.84} 84%|████████▍ | 1534/1817 [24:17:54<4:28:59, 57.03s/it] 84%|████████▍ | 1535/1817 [24:18:50<4:26:43, 56.75s/it] {'loss': 1.8682, 'learning_rate': 2e-05, 'epoch': 0.84} 84%|████████▍ | 1535/1817 [24:18:50<4:26:43, 56.75s/it] 85%|████████▍ | 1536/1817 [24:19:50<4:29:57, 57.64s/it] {'loss': 1.9057, 'learning_rate': 2e-05, 'epoch': 0.85} 85%|████████▍ | 1536/1817 [24:19:50<4:29:57, 57.64s/it] 85%|████████▍ | 1537/1817 [24:20:46<4:27:03, 57.23s/it] {'loss': 1.8873, 'learning_rate': 2e-05, 'epoch': 0.85} 85%|████████▍ | 1537/1817 [24:20:46<4:27:03, 57.23s/it] 85%|████████▍ | 1538/1817 [24:21:42<4:24:51, 56.96s/it] {'loss': 1.8641, 'learning_rate': 2e-05, 'epoch': 0.85} 85%|████████▍ | 1538/1817 [24:21:42<4:24:51, 56.96s/it] 85%|████████▍ | 1539/1817 [24:22:39<4:23:07, 56.79s/it] {'loss': 1.8486, 'learning_rate': 2e-05, 'epoch': 0.85} 85%|████████▍ | 1539/1817 [24:22:39<4:23:07, 56.79s/it] 85%|████████▍ | 1540/1817 [24:23:37<4:23:59, 57.18s/it] {'loss': 1.8946, 'learning_rate': 2e-05, 'epoch': 0.85} 85%|████████▍ | 1540/1817 [24:23:37<4:23:59, 57.18s/it] 85%|████████▍ | 1541/1817 [24:24:34<4:23:45, 57.34s/it] {'loss': 1.9038, 'learning_rate': 2e-05, 'epoch': 0.85} 85%|████████▍ | 1541/1817 [24:24:34<4:23:45, 57.34s/it] 85%|████████▍ | 1542/1817 [24:25:34<4:25:42, 57.97s/it] {'loss': 1.8578, 'learning_rate': 2e-05, 'epoch': 0.85} 85%|████████▍ | 1542/1817 [24:25:34<4:25:42, 57.97s/it] 85%|████████▍ | 1543/1817 [24:26:31<4:22:53, 57.57s/it] {'loss': 1.8949, 'learning_rate': 2e-05, 'epoch': 0.85} 85%|████████▍ | 1543/1817 [24:26:30<4:22:53, 57.57s/it] 85%|████████▍ | 1544/1817 [24:27:29<4:23:11, 57.84s/it] {'loss': 1.8629, 'learning_rate': 2e-05, 'epoch': 0.85} 85%|████████▍ | 1544/1817 [24:27:29<4:23:11, 57.84s/it] 85%|████████▌ | 1545/1817 [24:28:26<4:20:48, 57.53s/it] {'loss': 1.9213, 'learning_rate': 2e-05, 'epoch': 0.85} 85%|████████▌ | 1545/1817 [24:28:26<4:20:48, 57.53s/it] 85%|████████▌ | 1546/1817 [24:29:23<4:19:29, 57.45s/it] {'loss': 1.8836, 'learning_rate': 2e-05, 'epoch': 0.85} 85%|████████▌ | 1546/1817 [24:29:23<4:19:29, 57.45s/it] 85%|████████▌ | 1547/1817 [24:30:20<4:17:49, 57.29s/it] {'loss': 1.8813, 'learning_rate': 2e-05, 'epoch': 0.85} 85%|████████▌ | 1547/1817 [24:30:20<4:17:49, 57.29s/it] 85%|████████▌ | 1548/1817 [24:31:17<4:16:05, 57.12s/it] {'loss': 1.8763, 'learning_rate': 2e-05, 'epoch': 0.85} 85%|████████▌ | 1548/1817 [24:31:17<4:16:05, 57.12s/it] 85%|████████▌ | 1549/1817 [24:32:14<4:16:00, 57.32s/it] {'loss': 1.8301, 'learning_rate': 2e-05, 'epoch': 0.85} 85%|████████▌ | 1549/1817 [24:32:14<4:16:00, 57.32s/it] 85%|████████▌ | 1550/1817 [24:33:10<4:12:24, 56.72s/it] {'loss': 1.8669, 'learning_rate': 2e-05, 'epoch': 0.85} 85%|████████▌ | 1550/1817 [24:33:10<4:12:24, 56.72s/it] 85%|████████▌ | 1551/1817 [24:34:08<4:13:44, 57.23s/it] {'loss': 1.8751, 'learning_rate': 2e-05, 'epoch': 0.85} 85%|████████▌ | 1551/1817 [24:34:08<4:13:44, 57.23s/it] 85%|████████▌ | 1552/1817 [24:35:07<4:14:50, 57.70s/it] {'loss': 1.8926, 'learning_rate': 2e-05, 'epoch': 0.85} 85%|████████▌ | 1552/1817 [24:35:07<4:14:50, 57.70s/it] 85%|████████▌ | 1553/1817 [24:36:07<4:16:45, 58.35s/it] {'loss': 1.8385, 'learning_rate': 2e-05, 'epoch': 0.85} 85%|████████▌ | 1553/1817 [24:36:07<4:16:45, 58.35s/it] 86%|████████▌ | 1554/1817 [24:37:04<4:13:30, 57.83s/it] {'loss': 1.8806, 'learning_rate': 2e-05, 'epoch': 0.86} 86%|████████▌ | 1554/1817 [24:37:04<4:13:30, 57.83s/it] 86%|████████▌ | 1555/1817 [24:38:00<4:10:34, 57.38s/it] {'loss': 1.8812, 'learning_rate': 2e-05, 'epoch': 0.86} 86%|████████▌ | 1555/1817 [24:38:00<4:10:34, 57.38s/it] 86%|████████▌ | 1556/1817 [24:38:56<4:08:15, 57.07s/it] {'loss': 1.8811, 'learning_rate': 2e-05, 'epoch': 0.86} 86%|████████▌ | 1556/1817 [24:38:56<4:08:15, 57.07s/it] 86%|████████▌ | 1557/1817 [24:39:53<4:06:19, 56.85s/it] {'loss': 1.9084, 'learning_rate': 2e-05, 'epoch': 0.86} 86%|████████▌ | 1557/1817 [24:39:53<4:06:19, 56.85s/it] 86%|████████▌ | 1558/1817 [24:40:49<4:04:45, 56.70s/it] {'loss': 1.8717, 'learning_rate': 2e-05, 'epoch': 0.86} 86%|████████▌ | 1558/1817 [24:40:49<4:04:45, 56.70s/it] 86%|████████▌ | 1559/1817 [24:41:46<4:04:22, 56.83s/it] {'loss': 1.9214, 'learning_rate': 2e-05, 'epoch': 0.86} 86%|████████▌ | 1559/1817 [24:41:46<4:04:22, 56.83s/it] 86%|████████▌ | 1560/1817 [24:42:42<4:02:40, 56.65s/it] {'loss': 1.8881, 'learning_rate': 2e-05, 'epoch': 0.86} 86%|████████▌ | 1560/1817 [24:42:42<4:02:40, 56.65s/it] 86%|████████▌ | 1561/1817 [24:43:38<4:01:05, 56.51s/it] {'loss': 1.8806, 'learning_rate': 2e-05, 'epoch': 0.86} 86%|████████▌ | 1561/1817 [24:43:38<4:01:05, 56.51s/it] 86%|████████▌ | 1562/1817 [24:44:36<4:01:13, 56.76s/it] {'loss': 1.8591, 'learning_rate': 2e-05, 'epoch': 0.86} 86%|████████▌ | 1562/1817 [24:44:36<4:01:13, 56.76s/it] 86%|████████▌ | 1563/1817 [24:45:32<3:59:27, 56.56s/it] {'loss': 1.9151, 'learning_rate': 2e-05, 'epoch': 0.86} 86%|████████▌ | 1563/1817 [24:45:32<3:59:27, 56.56s/it] 86%|████████▌ | 1564/1817 [24:46:28<3:58:01, 56.45s/it] {'loss': 1.9365, 'learning_rate': 2e-05, 'epoch': 0.86} 86%|████████▌ | 1564/1817 [24:46:28<3:58:01, 56.45s/it] 86%|████████▌ | 1565/1817 [24:47:24<3:56:27, 56.30s/it] {'loss': 1.8693, 'learning_rate': 2e-05, 'epoch': 0.86} 86%|████████▌ | 1565/1817 [24:47:24<3:56:27, 56.30s/it] 86%|████████▌ | 1566/1817 [24:48:20<3:55:19, 56.25s/it] {'loss': 1.8923, 'learning_rate': 2e-05, 'epoch': 0.86} 86%|████████▌ | 1566/1817 [24:48:20<3:55:19, 56.25s/it] 86%|████████▌ | 1567/1817 [24:49:17<3:55:33, 56.53s/it] {'loss': 1.9143, 'learning_rate': 2e-05, 'epoch': 0.86} 86%|████████▌ | 1567/1817 [24:49:17<3:55:33, 56.53s/it] 86%|████████▋ | 1568/1817 [24:50:14<3:54:15, 56.45s/it] {'loss': 1.8634, 'learning_rate': 2e-05, 'epoch': 0.86} 86%|████████▋ | 1568/1817 [24:50:14<3:54:15, 56.45s/it] 86%|████████▋ | 1569/1817 [24:51:13<3:56:38, 57.25s/it] {'loss': 1.8879, 'learning_rate': 2e-05, 'epoch': 0.86} 86%|████████▋ | 1569/1817 [24:51:13<3:56:38, 57.25s/it] 86%|████████▋ | 1570/1817 [24:52:09<3:54:31, 56.97s/it] {'loss': 1.8413, 'learning_rate': 2e-05, 'epoch': 0.86} 86%|████████▋ | 1570/1817 [24:52:09<3:54:31, 56.97s/it] 86%|████████▋ | 1571/1817 [24:53:04<3:51:37, 56.49s/it] {'loss': 1.8882, 'learning_rate': 2e-05, 'epoch': 0.86} 86%|████████▋ | 1571/1817 [24:53:04<3:51:37, 56.49s/it] 87%|████████▋ | 1572/1817 [24:54:03<3:52:49, 57.02s/it] {'loss': 1.911, 'learning_rate': 2e-05, 'epoch': 0.86} 87%|████████▋ | 1572/1817 [24:54:03<3:52:49, 57.02s/it] 87%|████████▋ | 1573/1817 [24:54:59<3:50:51, 56.77s/it] {'loss': 1.9141, 'learning_rate': 2e-05, 'epoch': 0.87} 87%|████████▋ | 1573/1817 [24:54:59<3:50:51, 56.77s/it] 87%|████████▋ | 1574/1817 [24:55:55<3:49:18, 56.62s/it] {'loss': 1.9427, 'learning_rate': 2e-05, 'epoch': 0.87} 87%|████████▋ | 1574/1817 [24:55:55<3:49:18, 56.62s/it] 87%|████████▋ | 1575/1817 [24:56:53<3:49:32, 56.91s/it] {'loss': 1.9226, 'learning_rate': 2e-05, 'epoch': 0.87} 87%|████████▋ | 1575/1817 [24:56:53<3:49:32, 56.91s/it] 87%|████████▋ | 1576/1817 [24:57:50<3:49:18, 57.09s/it] {'loss': 1.8514, 'learning_rate': 2e-05, 'epoch': 0.87} 87%|████████▋ | 1576/1817 [24:57:50<3:49:18, 57.09s/it] 87%|████████▋ | 1577/1817 [24:58:46<3:46:39, 56.67s/it] {'loss': 1.9211, 'learning_rate': 2e-05, 'epoch': 0.87} 87%|████████▋ | 1577/1817 [24:58:46<3:46:39, 56.67s/it] 87%|████████▋ | 1578/1817 [24:59:42<3:45:20, 56.57s/it] {'loss': 1.8984, 'learning_rate': 2e-05, 'epoch': 0.87} 87%|████████▋ | 1578/1817 [24:59:42<3:45:20, 56.57s/it] 87%|████████▋ | 1579/1817 [25:00:41<3:46:32, 57.11s/it] {'loss': 1.898, 'learning_rate': 2e-05, 'epoch': 0.87} 87%|████████▋ | 1579/1817 [25:00:41<3:46:32, 57.11s/it] 87%|████████▋ | 1580/1817 [25:01:39<3:46:17, 57.29s/it] {'loss': 1.9323, 'learning_rate': 2e-05, 'epoch': 0.87} 87%|████████▋ | 1580/1817 [25:01:38<3:46:13, 57.27s/it] 87%|████████▋ | 1581/1817 [25:02:35<3:44:25, 57.06s/it] {'loss': 1.8921, 'learning_rate': 2e-05, 'epoch': 0.87} 87%|████████▋ | 1581/1817 [25:02:35<3:44:25, 57.06s/it] 87%|████████▋ | 1582/1817 [25:03:31<3:42:12, 56.74s/it] {'loss': 1.8566, 'learning_rate': 2e-05, 'epoch': 0.87} 87%|████████▋ | 1582/1817 [25:03:31<3:42:12, 56.74s/it] 87%|████████▋ | 1583/1817 [25:04:27<3:40:12, 56.46s/it] {'loss': 1.9061, 'learning_rate': 2e-05, 'epoch': 0.87} 87%|████████▋ | 1583/1817 [25:04:27<3:40:12, 56.46s/it] 87%|████████▋ | 1584/1817 [25:05:23<3:38:44, 56.33s/it] {'loss': 1.9272, 'learning_rate': 2e-05, 'epoch': 0.87} 87%|████████▋ | 1584/1817 [25:05:23<3:38:44, 56.33s/it] 87%|████████▋ | 1585/1817 [25:06:23<3:42:31, 57.55s/it] {'loss': 1.9473, 'learning_rate': 2e-05, 'epoch': 0.87} 87%|████████▋ | 1585/1817 [25:06:23<3:42:31, 57.55s/it] 87%|████████▋ | 1586/1817 [25:07:20<3:41:25, 57.51s/it] {'loss': 1.8821, 'learning_rate': 2e-05, 'epoch': 0.87} 87%|████████▋ | 1586/1817 [25:07:20<3:41:25, 57.51s/it] 87%|████████▋ | 1587/1817 [25:08:17<3:38:54, 57.11s/it] {'loss': 1.8989, 'learning_rate': 2e-05, 'epoch': 0.87} 87%|████████▋ | 1587/1817 [25:08:17<3:38:54, 57.11s/it] 87%|████████▋ | 1588/1817 [25:09:13<3:36:54, 56.83s/it] {'loss': 1.8652, 'learning_rate': 2e-05, 'epoch': 0.87} 87%|████████▋ | 1588/1817 [25:09:13<3:36:54, 56.83s/it] 87%|████████▋ | 1589/1817 [25:10:09<3:35:19, 56.67s/it] {'loss': 1.9159, 'learning_rate': 2e-05, 'epoch': 0.87} 87%|████████▋ | 1589/1817 [25:10:09<3:35:19, 56.67s/it] 88%|████████▊ | 1590/1817 [25:11:05<3:33:53, 56.54s/it] {'loss': 1.8878, 'learning_rate': 2e-05, 'epoch': 0.87} 88%|████████▊ | 1590/1817 [25:11:05<3:33:53, 56.54s/it] 88%|████████▊ | 1591/1817 [25:12:02<3:32:50, 56.50s/it] {'loss': 1.8428, 'learning_rate': 2e-05, 'epoch': 0.88} 88%|████████▊ | 1591/1817 [25:12:02<3:32:50, 56.50s/it] 88%|████████▊ | 1592/1817 [25:12:59<3:32:38, 56.70s/it] {'loss': 1.8961, 'learning_rate': 2e-05, 'epoch': 0.88} 88%|████████▊ | 1592/1817 [25:12:59<3:32:38, 56.70s/it] 88%|████████▊ | 1593/1817 [25:13:57<3:33:06, 57.08s/it] {'loss': 1.932, 'learning_rate': 2e-05, 'epoch': 0.88} 88%|████████▊ | 1593/1817 [25:13:57<3:33:06, 57.08s/it] 88%|████████▊ | 1594/1817 [25:14:53<3:31:19, 56.86s/it] {'loss': 1.8619, 'learning_rate': 2e-05, 'epoch': 0.88} 88%|████████▊ | 1594/1817 [25:14:53<3:31:19, 56.86s/it] 88%|████████▊ | 1595/1817 [25:15:49<3:29:43, 56.68s/it] {'loss': 1.8687, 'learning_rate': 2e-05, 'epoch': 0.88} 88%|████████▊ | 1595/1817 [25:15:49<3:29:43, 56.68s/it] 88%|████████▊ | 1596/1817 [25:16:46<3:28:23, 56.58s/it] {'loss': 1.8983, 'learning_rate': 2e-05, 'epoch': 0.88} 88%|████████▊ | 1596/1817 [25:16:46<3:28:23, 56.58s/it] 88%|████████▊ | 1597/1817 [25:17:43<3:27:19, 56.54s/it] {'loss': 1.9208, 'learning_rate': 2e-05, 'epoch': 0.88} 88%|████████▊ | 1597/1817 [25:17:42<3:27:19, 56.54s/it] 88%|████████▊ | 1598/1817 [25:18:41<3:28:24, 57.10s/it] {'loss': 1.8852, 'learning_rate': 2e-05, 'epoch': 0.88} 88%|████████▊ | 1598/1817 [25:18:41<3:28:24, 57.10s/it] 88%|████████▊ | 1599/1817 [25:19:37<3:26:42, 56.89s/it] {'loss': 1.8891, 'learning_rate': 2e-05, 'epoch': 0.88} 88%|████████▊ | 1599/1817 [25:19:37<3:26:42, 56.89s/it] 88%|████████▊ | 1600/1817 [25:20:33<3:25:12, 56.74s/it] {'loss': 1.8677, 'learning_rate': 2e-05, 'epoch': 0.88} 88%|████████▊ | 1600/1817 [25:20:33<3:25:12, 56.74s/it] 88%|████████▊ | 1601/1817 [25:21:32<3:25:50, 57.18s/it] {'loss': 1.9033, 'learning_rate': 2e-05, 'epoch': 0.88} 88%|████████▊ | 1601/1817 [25:21:32<3:25:50, 57.18s/it] 88%|████████▊ | 1602/1817 [25:22:28<3:24:02, 56.94s/it] {'loss': 1.9031, 'learning_rate': 2e-05, 'epoch': 0.88} 88%|████████▊ | 1602/1817 [25:22:28<3:24:02, 56.94s/it] 88%|████████▊ | 1603/1817 [25:23:22<3:20:18, 56.16s/it] {'loss': 1.86, 'learning_rate': 2e-05, 'epoch': 0.88} 88%|████████▊ | 1603/1817 [25:23:22<3:20:18, 56.16s/it] 88%|████████▊ | 1604/1817 [25:24:20<3:20:56, 56.60s/it] {'loss': 1.8923, 'learning_rate': 2e-05, 'epoch': 0.88} 88%|████████▊ | 1604/1817 [25:24:20<3:20:56, 56.60s/it] 88%|████████▊ | 1605/1817 [25:25:18<3:21:54, 57.15s/it] {'loss': 1.905, 'learning_rate': 2e-05, 'epoch': 0.88} 88%|████████▊ | 1605/1817 [25:25:18<3:21:54, 57.15s/it] 88%|████████▊ | 1606/1817 [25:26:15<3:20:05, 56.90s/it] {'loss': 1.8822, 'learning_rate': 2e-05, 'epoch': 0.88} 88%|████████▊ | 1606/1817 [25:26:15<3:20:05, 56.90s/it] 88%|████████▊ | 1607/1817 [25:27:12<3:19:45, 57.08s/it] {'loss': 1.894, 'learning_rate': 2e-05, 'epoch': 0.88} 88%|████████▊ | 1607/1817 [25:27:12<3:19:45, 57.08s/it] 88%|████████▊ | 1608/1817 [25:28:11<3:20:25, 57.54s/it] {'loss': 1.8999, 'learning_rate': 2e-05, 'epoch': 0.88} 88%|████████▊ | 1608/1817 [25:28:11<3:20:25, 57.54s/it] 89%|████████▊ | 1609/1817 [25:29:09<3:19:43, 57.61s/it] {'loss': 1.8723, 'learning_rate': 2e-05, 'epoch': 0.89} 89%|████████▊ | 1609/1817 [25:29:09<3:19:43, 57.61s/it] 89%|████████▊ | 1610/1817 [25:30:05<3:17:18, 57.19s/it] {'loss': 1.8515, 'learning_rate': 2e-05, 'epoch': 0.89} 89%|████████▊ | 1610/1817 [25:30:05<3:17:18, 57.19s/it] 89%|████████▊ | 1611/1817 [25:31:01<3:15:15, 56.87s/it] {'loss': 1.8721, 'learning_rate': 2e-05, 'epoch': 0.89} 89%|████████▊ | 1611/1817 [25:31:01<3:15:15, 56.87s/it] 89%|████████▊ | 1612/1817 [25:31:58<3:13:45, 56.71s/it] {'loss': 1.8893, 'learning_rate': 2e-05, 'epoch': 0.89} 89%|████████▊ | 1612/1817 [25:31:57<3:13:45, 56.71s/it] 89%|████████▉ | 1613/1817 [25:32:54<3:12:28, 56.61s/it] {'loss': 1.9022, 'learning_rate': 2e-05, 'epoch': 0.89} 89%|████████▉ | 1613/1817 [25:32:54<3:12:28, 56.61s/it] 89%|████████▉ | 1614/1817 [25:33:50<3:11:19, 56.55s/it] {'loss': 1.8725, 'learning_rate': 2e-05, 'epoch': 0.89} 89%|████████▉ | 1614/1817 [25:33:50<3:11:19, 56.55s/it] 89%|████████▉ | 1615/1817 [25:34:50<3:13:18, 57.42s/it] {'loss': 1.9169, 'learning_rate': 2e-05, 'epoch': 0.89} 89%|████████▉ | 1615/1817 [25:34:50<3:13:18, 57.42s/it] 89%|████████▉ | 1616/1817 [25:35:47<3:12:27, 57.45s/it] {'loss': 1.8711, 'learning_rate': 2e-05, 'epoch': 0.89} 89%|████████▉ | 1616/1817 [25:35:47<3:12:27, 57.45s/it] 89%|████████▉ | 1617/1817 [25:36:49<3:15:52, 58.76s/it] {'loss': 1.92, 'learning_rate': 2e-05, 'epoch': 0.89} 89%|████████▉ | 1617/1817 [25:36:49<3:15:52, 58.76s/it] 89%|████████▉ | 1618/1817 [25:37:44<3:11:21, 57.69s/it] {'loss': 1.881, 'learning_rate': 2e-05, 'epoch': 0.89} 89%|████████▉ | 1618/1817 [25:37:44<3:11:21, 57.69s/it] 89%|████████▉ | 1619/1817 [25:38:42<3:10:19, 57.68s/it] {'loss': 1.9106, 'learning_rate': 2e-05, 'epoch': 0.89} 89%|████████▉ | 1619/1817 [25:38:42<3:10:19, 57.68s/it] 89%|████████▉ | 1620/1817 [25:39:40<3:09:49, 57.81s/it] {'loss': 1.9373, 'learning_rate': 2e-05, 'epoch': 0.89} 89%|████████▉ | 1620/1817 [25:39:40<3:09:49, 57.81s/it] 89%|████████▉ | 1621/1817 [25:40:36<3:07:29, 57.39s/it] {'loss': 1.8995, 'learning_rate': 2e-05, 'epoch': 0.89} 89%|████████▉ | 1621/1817 [25:40:36<3:07:29, 57.39s/it] 89%|████████▉ | 1622/1817 [25:41:32<3:04:08, 56.66s/it] {'loss': 1.952, 'learning_rate': 2e-05, 'epoch': 0.89} 89%|████████▉ | 1622/1817 [25:41:31<3:04:09, 56.66s/it] 89%|████████▉ | 1623/1817 [25:42:29<3:04:23, 57.03s/it] {'loss': 1.9199, 'learning_rate': 2e-05, 'epoch': 0.89} 89%|████████▉ | 1623/1817 [25:42:29<3:04:23, 57.03s/it] 89%|████████▉ | 1624/1817 [25:43:27<3:03:50, 57.15s/it] {'loss': 1.8753, 'learning_rate': 2e-05, 'epoch': 0.89} 89%|████████▉ | 1624/1817 [25:43:27<3:03:50, 57.15s/it] 89%|████████▉ | 1625/1817 [25:44:25<3:03:23, 57.31s/it] {'loss': 1.8666, 'learning_rate': 2e-05, 'epoch': 0.89} 89%|████████▉ | 1625/1817 [25:44:24<3:03:23, 57.31s/it] 89%|████████▉ | 1626/1817 [25:45:21<3:01:37, 57.05s/it] {'loss': 1.8983, 'learning_rate': 2e-05, 'epoch': 0.89} 89%|████████▉ | 1626/1817 [25:45:21<3:01:37, 57.05s/it] 90%|████████▉ | 1627/1817 [25:46:17<2:59:59, 56.84s/it] {'loss': 1.907, 'learning_rate': 2e-05, 'epoch': 0.9} 90%|████████▉ | 1627/1817 [25:46:17<2:59:59, 56.84s/it] 90%|████████▉ | 1628/1817 [25:47:15<3:00:09, 57.19s/it] {'loss': 1.8645, 'learning_rate': 2e-05, 'epoch': 0.9} 90%|████████▉ | 1628/1817 [25:47:15<3:00:09, 57.19s/it] 90%|████████▉ | 1629/1817 [25:48:12<2:58:15, 56.89s/it] {'loss': 1.9197, 'learning_rate': 2e-05, 'epoch': 0.9} 90%|████████▉ | 1629/1817 [25:48:11<2:58:15, 56.89s/it] 90%|████████▉ | 1630/1817 [25:49:06<2:55:16, 56.24s/it] {'loss': 1.9041, 'learning_rate': 2e-05, 'epoch': 0.9} 90%|████████▉ | 1630/1817 [25:49:06<2:55:16, 56.24s/it] 90%|████████▉ | 1631/1817 [25:50:03<2:54:36, 56.32s/it] {'loss': 1.9385, 'learning_rate': 2e-05, 'epoch': 0.9} 90%|████████▉ | 1631/1817 [25:50:02<2:54:36, 56.32s/it] 90%|████████▉ | 1632/1817 [25:51:04<2:57:57, 57.72s/it] {'loss': 1.8779, 'learning_rate': 2e-05, 'epoch': 0.9} 90%|████████▉ | 1632/1817 [25:51:03<2:57:57, 57.72s/it] 90%|████████▉ | 1633/1817 [25:52:00<2:55:39, 57.28s/it] {'loss': 1.8198, 'learning_rate': 2e-05, 'epoch': 0.9} 90%|████████▉ | 1633/1817 [25:52:00<2:55:39, 57.28s/it] 90%|████████▉ | 1634/1817 [25:52:56<2:53:35, 56.91s/it] {'loss': 1.9242, 'learning_rate': 2e-05, 'epoch': 0.9} 90%|████████▉ | 1634/1817 [25:52:56<2:53:35, 56.91s/it] 90%|████████▉ | 1635/1817 [25:53:52<2:52:20, 56.81s/it] {'loss': 1.8788, 'learning_rate': 2e-05, 'epoch': 0.9} 90%|████████▉ | 1635/1817 [25:53:52<2:52:20, 56.81s/it] 90%|█████████ | 1636/1817 [25:54:51<2:52:30, 57.18s/it] {'loss': 1.8705, 'learning_rate': 2e-05, 'epoch': 0.9} 90%|█████████ | 1636/1817 [25:54:50<2:52:30, 57.18s/it] 90%|█████████ | 1637/1817 [25:55:48<2:52:06, 57.37s/it] {'loss': 1.8994, 'learning_rate': 2e-05, 'epoch': 0.9} 90%|█████████ | 1637/1817 [25:55:48<2:52:06, 57.37s/it] 90%|█████████ | 1638/1817 [25:56:46<2:51:22, 57.45s/it] {'loss': 1.9191, 'learning_rate': 2e-05, 'epoch': 0.9} 90%|█████████ | 1638/1817 [25:56:46<2:51:22, 57.45s/it] 90%|█████████ | 1639/1817 [25:57:42<2:49:15, 57.05s/it] {'loss': 1.8976, 'learning_rate': 2e-05, 'epoch': 0.9} 90%|█████████ | 1639/1817 [25:57:42<2:49:15, 57.05s/it] 90%|█████████ | 1640/1817 [25:58:40<2:48:49, 57.23s/it] {'loss': 1.9057, 'learning_rate': 2e-05, 'epoch': 0.9} 90%|█████████ | 1640/1817 [25:58:40<2:48:49, 57.23s/it] 90%|█████████ | 1641/1817 [25:59:36<2:47:11, 57.00s/it] {'loss': 1.9407, 'learning_rate': 2e-05, 'epoch': 0.9} 90%|█████████ | 1641/1817 [25:59:36<2:47:11, 57.00s/it] 90%|█████████ | 1642/1817 [26:00:32<2:45:32, 56.76s/it] {'loss': 1.9261, 'learning_rate': 2e-05, 'epoch': 0.9} 90%|█████████ | 1642/1817 [26:00:32<2:45:32, 56.76s/it] 90%|█████████ | 1643/1817 [26:01:29<2:44:15, 56.64s/it] {'loss': 1.8657, 'learning_rate': 2e-05, 'epoch': 0.9} 90%|█████████ | 1643/1817 [26:01:29<2:44:15, 56.64s/it] 90%|█████████ | 1644/1817 [26:02:28<2:45:48, 57.51s/it] {'loss': 1.9017, 'learning_rate': 2e-05, 'epoch': 0.9} 90%|█████████ | 1644/1817 [26:02:28<2:45:48, 57.51s/it] 91%|█████████ | 1645/1817 [26:03:26<2:45:26, 57.71s/it] {'loss': 1.8774, 'learning_rate': 2e-05, 'epoch': 0.91} 91%|█████████ | 1645/1817 [26:03:26<2:45:26, 57.71s/it] 91%|█████████ | 1646/1817 [26:04:24<2:44:02, 57.56s/it] {'loss': 1.8384, 'learning_rate': 2e-05, 'epoch': 0.91} 91%|█████████ | 1646/1817 [26:04:24<2:44:02, 57.56s/it] 91%|█████████ | 1647/1817 [26:05:22<2:43:45, 57.79s/it] {'loss': 1.9428, 'learning_rate': 2e-05, 'epoch': 0.91} 91%|█████████ | 1647/1817 [26:05:22<2:43:45, 57.79s/it] 91%|█████████ | 1648/1817 [26:06:18<2:41:42, 57.41s/it] {'loss': 1.8803, 'learning_rate': 2e-05, 'epoch': 0.91} 91%|█████████ | 1648/1817 [26:06:18<2:41:42, 57.41s/it] 91%|█████████ | 1649/1817 [26:07:15<2:40:07, 57.18s/it] {'loss': 1.9124, 'learning_rate': 2e-05, 'epoch': 0.91} 91%|█████████ | 1649/1817 [26:07:15<2:40:07, 57.18s/it] 91%|█████████ | 1650/1817 [26:08:14<2:40:48, 57.77s/it] {'loss': 1.9116, 'learning_rate': 2e-05, 'epoch': 0.91} 91%|█████████ | 1650/1817 [26:08:14<2:40:48, 57.77s/it] 91%|█████████ | 1651/1817 [26:09:10<2:38:34, 57.32s/it] {'loss': 1.8543, 'learning_rate': 2e-05, 'epoch': 0.91} 91%|█████████ | 1651/1817 [26:09:10<2:38:34, 57.32s/it] 91%|█████████ | 1652/1817 [26:10:05<2:35:03, 56.38s/it] {'loss': 1.8939, 'learning_rate': 2e-05, 'epoch': 0.91} 91%|█████████ | 1652/1817 [26:10:05<2:35:03, 56.38s/it] 91%|█████████ | 1653/1817 [26:11:03<2:35:48, 57.00s/it] {'loss': 1.8657, 'learning_rate': 2e-05, 'epoch': 0.91} 91%|█████████ | 1653/1817 [26:11:03<2:35:48, 57.00s/it] 91%|█████████ | 1654/1817 [26:11:59<2:34:17, 56.79s/it] {'loss': 1.9307, 'learning_rate': 2e-05, 'epoch': 0.91} 91%|█████████ | 1654/1817 [26:11:59<2:34:17, 56.79s/it] 91%|█████████ | 1655/1817 [26:12:56<2:33:09, 56.73s/it] {'loss': 1.8874, 'learning_rate': 2e-05, 'epoch': 0.91} 91%|█████████ | 1655/1817 [26:12:56<2:33:09, 56.73s/it] 91%|█████████ | 1656/1817 [26:13:55<2:33:28, 57.19s/it] {'loss': 1.8849, 'learning_rate': 2e-05, 'epoch': 0.91} 91%|█████████ | 1656/1817 [26:13:54<2:33:28, 57.19s/it] 91%|█████████ | 1657/1817 [26:14:52<2:32:41, 57.26s/it] {'loss': 1.9207, 'learning_rate': 2e-05, 'epoch': 0.91} 91%|█████████ | 1657/1817 [26:14:52<2:32:41, 57.26s/it] 91%|█████████ | 1658/1817 [26:15:48<2:30:57, 56.96s/it] {'loss': 1.8798, 'learning_rate': 2e-05, 'epoch': 0.91} 91%|█████████ | 1658/1817 [26:15:48<2:30:57, 56.96s/it] 91%|█████████▏| 1659/1817 [26:16:45<2:29:34, 56.80s/it] {'loss': 1.9098, 'learning_rate': 2e-05, 'epoch': 0.91} 91%|█████████▏| 1659/1817 [26:16:44<2:29:34, 56.80s/it] 91%|█████████▏| 1660/1817 [26:17:41<2:28:18, 56.68s/it] {'loss': 1.8698, 'learning_rate': 2e-05, 'epoch': 0.91} 91%|█████████▏| 1660/1817 [26:17:41<2:28:18, 56.68s/it] 91%|█████████▏| 1661/1817 [26:18:39<2:28:00, 56.93s/it] {'loss': 1.8945, 'learning_rate': 2e-05, 'epoch': 0.91} 91%|█████████▏| 1661/1817 [26:18:38<2:28:00, 56.93s/it] 91%|█████████▏| 1662/1817 [26:19:35<2:26:43, 56.79s/it] {'loss': 1.8739, 'learning_rate': 2e-05, 'epoch': 0.91} 91%|█████████▏| 1662/1817 [26:19:35<2:26:43, 56.79s/it] 92%|█████████▏| 1663/1817 [26:20:31<2:25:02, 56.51s/it] {'loss': 1.8861, 'learning_rate': 2e-05, 'epoch': 0.91} 92%|█████████▏| 1663/1817 [26:20:31<2:25:02, 56.51s/it] 92%|█████████▏| 1664/1817 [26:21:29<2:24:59, 56.86s/it] {'loss': 1.8698, 'learning_rate': 2e-05, 'epoch': 0.92} 92%|█████████▏| 1664/1817 [26:21:28<2:24:59, 56.86s/it] 92%|█████████▏| 1665/1817 [26:22:22<2:21:34, 55.89s/it] {'loss': 1.8796, 'learning_rate': 2e-05, 'epoch': 0.92} 92%|█████████▏| 1665/1817 [26:22:22<2:21:34, 55.89s/it] 92%|█████████▏| 1666/1817 [26:23:22<2:23:44, 57.11s/it] {'loss': 1.8598, 'learning_rate': 2e-05, 'epoch': 0.92} 92%|█████████▏| 1666/1817 [26:23:22<2:23:44, 57.11s/it] 92%|█████████▏| 1667/1817 [26:24:20<2:23:15, 57.31s/it] {'loss': 1.8671, 'learning_rate': 2e-05, 'epoch': 0.92} 92%|█████████▏| 1667/1817 [26:24:20<2:23:15, 57.31s/it] 92%|█████████▏| 1668/1817 [26:25:16<2:21:36, 57.02s/it] {'loss': 1.9368, 'learning_rate': 2e-05, 'epoch': 0.92} 92%|█████████▏| 1668/1817 [26:25:16<2:21:36, 57.02s/it] 92%|█████████▏| 1669/1817 [26:26:10<2:18:20, 56.09s/it] {'loss': 1.9004, 'learning_rate': 2e-05, 'epoch': 0.92} 92%|█████████▏| 1669/1817 [26:26:10<2:18:20, 56.09s/it] 92%|█████████▏| 1670/1817 [26:27:08<2:18:38, 56.59s/it] {'loss': 1.8819, 'learning_rate': 2e-05, 'epoch': 0.92} 92%|█████████▏| 1670/1817 [26:27:08<2:18:38, 56.59s/it] 92%|█████████▏| 1671/1817 [26:28:06<2:19:12, 57.21s/it] {'loss': 1.8986, 'learning_rate': 2e-05, 'epoch': 0.92} 92%|█████████▏| 1671/1817 [26:28:06<2:19:12, 57.21s/it] 92%|█████████▏| 1672/1817 [26:29:07<2:20:57, 58.33s/it] {'loss': 1.9275, 'learning_rate': 2e-05, 'epoch': 0.92} 92%|█████████▏| 1672/1817 [26:29:07<2:20:57, 58.33s/it] 92%|█████████▏| 1673/1817 [26:30:06<2:20:12, 58.42s/it] {'loss': 1.8776, 'learning_rate': 2e-05, 'epoch': 0.92} 92%|█████████▏| 1673/1817 [26:30:06<2:20:12, 58.42s/it] 92%|█████████▏| 1674/1817 [26:31:02<2:17:46, 57.81s/it] {'loss': 1.9297, 'learning_rate': 2e-05, 'epoch': 0.92} 92%|█████████▏| 1674/1817 [26:31:02<2:17:46, 57.81s/it] 92%|█████████▏| 1675/1817 [26:31:59<2:16:12, 57.55s/it] {'loss': 1.8877, 'learning_rate': 2e-05, 'epoch': 0.92} 92%|█████████▏| 1675/1817 [26:31:59<2:16:12, 57.55s/it] 92%|█████████▏| 1676/1817 [26:32:56<2:14:37, 57.28s/it] {'loss': 1.9236, 'learning_rate': 2e-05, 'epoch': 0.92} 92%|█████████▏| 1676/1817 [26:32:56<2:14:37, 57.28s/it] 92%|█████████▏| 1677/1817 [26:33:54<2:13:53, 57.38s/it] {'loss': 1.8892, 'learning_rate': 2e-05, 'epoch': 0.92} 92%|█████████▏| 1677/1817 [26:33:53<2:13:53, 57.38s/it] 92%|█████████▏| 1678/1817 [26:34:48<2:11:07, 56.60s/it] {'loss': 1.8899, 'learning_rate': 2e-05, 'epoch': 0.92} 92%|█████████▏| 1678/1817 [26:34:48<2:11:07, 56.60s/it] 92%|█████████▏| 1679/1817 [26:35:45<2:10:11, 56.61s/it] {'loss': 1.9453, 'learning_rate': 2e-05, 'epoch': 0.92} 92%|█████████▏| 1679/1817 [26:35:45<2:10:11, 56.61s/it] 92%|█████████▏| 1680/1817 [26:36:43<2:10:05, 56.97s/it] {'loss': 1.9368, 'learning_rate': 2e-05, 'epoch': 0.92} 92%|█████████▏| 1680/1817 [26:36:43<2:10:05, 56.97s/it] 93%|█████████▎| 1681/1817 [26:37:38<2:08:20, 56.62s/it] {'loss': 1.9243, 'learning_rate': 2e-05, 'epoch': 0.92} 93%|█████████▎| 1681/1817 [26:37:39<2:08:20, 56.62s/it] 93%|█████████▎| 1682/1817 [26:38:35<2:07:17, 56.58s/it] {'loss': 1.9135, 'learning_rate': 2e-05, 'epoch': 0.93} 93%|█████████▎| 1682/1817 [26:38:35<2:07:17, 56.58s/it] 93%|█████████▎| 1683/1817 [26:39:32<2:06:25, 56.61s/it] {'loss': 1.8927, 'learning_rate': 2e-05, 'epoch': 0.93} 93%|█████████▎| 1683/1817 [26:39:32<2:06:25, 56.61s/it] 93%|█████████▎| 1684/1817 [26:40:28<2:05:26, 56.59s/it] {'loss': 1.8601, 'learning_rate': 2e-05, 'epoch': 0.93} 93%|█████████▎| 1684/1817 [26:40:28<2:05:26, 56.59s/it] 93%|█████████▎| 1685/1817 [26:41:26<2:04:59, 56.82s/it] {'loss': 1.9063, 'learning_rate': 2e-05, 'epoch': 0.93} 93%|█████████▎| 1685/1817 [26:41:26<2:04:59, 56.82s/it] 93%|█████████▎| 1686/1817 [26:42:23<2:04:09, 56.87s/it] {'loss': 1.9136, 'learning_rate': 2e-05, 'epoch': 0.93} 93%|█████████▎| 1686/1817 [26:42:23<2:04:09, 56.87s/it] 93%|█████████▎| 1687/1817 [26:43:19<2:03:10, 56.85s/it] {'loss': 1.8806, 'learning_rate': 2e-05, 'epoch': 0.93} 93%|█████████▎| 1687/1817 [26:43:19<2:03:10, 56.85s/it] 93%|█████████▎| 1688/1817 [26:44:15<2:01:21, 56.44s/it] {'loss': 1.9051, 'learning_rate': 2e-05, 'epoch': 0.93} 93%|█████████▎| 1688/1817 [26:44:15<2:01:21, 56.44s/it] 93%|█████████▎| 1689/1817 [26:45:15<2:03:02, 57.68s/it] {'loss': 1.9087, 'learning_rate': 2e-05, 'epoch': 0.93} 93%|█████████▎| 1689/1817 [26:45:15<2:03:02, 57.68s/it] 93%|█████████▎| 1690/1817 [26:46:12<2:01:21, 57.33s/it] {'loss': 1.9347, 'learning_rate': 2e-05, 'epoch': 0.93} 93%|█████████▎| 1690/1817 [26:46:12<2:01:21, 57.33s/it] 93%|█████████▎| 1691/1817 [26:47:10<2:00:55, 57.59s/it] {'loss': 1.8694, 'learning_rate': 2e-05, 'epoch': 0.93} 93%|█████████▎| 1691/1817 [26:47:10<2:00:55, 57.59s/it] 93%|█████████▎| 1692/1817 [26:48:07<1:59:36, 57.41s/it] {'loss': 1.8785, 'learning_rate': 2e-05, 'epoch': 0.93} 93%|█████████▎| 1692/1817 [26:48:07<1:59:36, 57.41s/it] 93%|█████████▎| 1693/1817 [26:49:05<1:58:43, 57.45s/it] {'loss': 1.9222, 'learning_rate': 2e-05, 'epoch': 0.93} 93%|█████████▎| 1693/1817 [26:49:05<1:58:43, 57.45s/it] 93%|█████████▎| 1694/1817 [26:50:05<1:59:20, 58.22s/it] {'loss': 1.8843, 'learning_rate': 2e-05, 'epoch': 0.93} 93%|█████████▎| 1694/1817 [26:50:05<1:59:20, 58.22s/it] 93%|█████████▎| 1695/1817 [26:51:03<1:58:11, 58.12s/it] {'loss': 1.8581, 'learning_rate': 2e-05, 'epoch': 0.93} 93%|█████████▎| 1695/1817 [26:51:03<1:58:11, 58.12s/it] 93%|█████████▎| 1696/1817 [26:51:59<1:56:13, 57.63s/it] {'loss': 1.931, 'learning_rate': 2e-05, 'epoch': 0.93} 93%|█████████▎| 1696/1817 [26:51:59<1:56:13, 57.63s/it] 93%|█████████▎| 1697/1817 [26:52:55<1:54:33, 57.28s/it] {'loss': 1.8851, 'learning_rate': 2e-05, 'epoch': 0.93} 93%|█████████▎| 1697/1817 [26:52:55<1:54:33, 57.28s/it] 93%|█████████▎| 1698/1817 [26:53:53<1:53:45, 57.36s/it] {'loss': 1.9192, 'learning_rate': 2e-05, 'epoch': 0.93} 93%|█████████▎| 1698/1817 [26:53:53<1:53:45, 57.36s/it] 94%|█████████▎| 1699/1817 [26:54:49<1:52:09, 57.03s/it] {'loss': 1.8996, 'learning_rate': 2e-05, 'epoch': 0.93} 94%|█████████▎| 1699/1817 [26:54:49<1:52:09, 57.03s/it] 94%|█████████▎| 1700/1817 [26:55:48<1:51:55, 57.40s/it] {'loss': 1.9191, 'learning_rate': 2e-05, 'epoch': 0.94} 94%|█████████▎| 1700/1817 [26:55:48<1:51:55, 57.40s/it] 94%|█████████▎| 1701/1817 [26:56:47<1:52:04, 57.97s/it] {'loss': 1.9111, 'learning_rate': 2e-05, 'epoch': 0.94} 94%|█████████▎| 1701/1817 [26:56:47<1:52:04, 57.97s/it] 94%|█████████▎| 1702/1817 [26:57:44<1:50:14, 57.51s/it] {'loss': 1.9191, 'learning_rate': 2e-05, 'epoch': 0.94} 94%|█████████▎| 1702/1817 [26:57:43<1:50:14, 57.51s/it] 94%|█████████▎| 1703/1817 [26:58:40<1:48:38, 57.18s/it] {'loss': 1.9232, 'learning_rate': 2e-05, 'epoch': 0.94} 94%|█████████▎| 1703/1817 [26:58:40<1:48:38, 57.18s/it] 94%|█████████▍| 1704/1817 [26:59:38<1:47:58, 57.33s/it] {'loss': 1.907, 'learning_rate': 2e-05, 'epoch': 0.94} 94%|█████████▍| 1704/1817 [26:59:37<1:47:58, 57.33s/it] 94%|█████████▍| 1705/1817 [27:00:34<1:46:28, 57.04s/it] {'loss': 1.8619, 'learning_rate': 2e-05, 'epoch': 0.94} 94%|█████████▍| 1705/1817 [27:00:34<1:46:28, 57.04s/it] 94%|█████████▍| 1706/1817 [27:01:32<1:45:47, 57.18s/it] {'loss': 1.9235, 'learning_rate': 2e-05, 'epoch': 0.94} 94%|█████████▍| 1706/1817 [27:01:31<1:45:47, 57.19s/it] 94%|█████████▍| 1707/1817 [27:02:28<1:44:30, 57.01s/it] {'loss': 1.9095, 'learning_rate': 2e-05, 'epoch': 0.94} 94%|█████████▍| 1707/1817 [27:02:28<1:44:30, 57.01s/it] 94%|█████████▍| 1708/1817 [27:03:24<1:43:10, 56.80s/it] {'loss': 1.8749, 'learning_rate': 2e-05, 'epoch': 0.94} 94%|█████████▍| 1708/1817 [27:03:24<1:43:10, 56.80s/it] 94%|█████████▍| 1709/1817 [27:04:20<1:41:35, 56.44s/it] {'loss': 1.9114, 'learning_rate': 2e-05, 'epoch': 0.94} 94%|█████████▍| 1709/1817 [27:04:20<1:41:35, 56.44s/it] 94%|█████████▍| 1710/1817 [27:05:18<1:41:35, 56.97s/it] {'loss': 1.9151, 'learning_rate': 2e-05, 'epoch': 0.94} 94%|█████████▍| 1710/1817 [27:05:18<1:41:35, 56.97s/it] 94%|█████████▍| 1711/1817 [27:06:14<1:39:55, 56.56s/it] {'loss': 1.8565, 'learning_rate': 2e-05, 'epoch': 0.94} 94%|█████████▍| 1711/1817 [27:06:14<1:39:55, 56.56s/it] 94%|█████████▍| 1712/1817 [27:07:12<1:39:39, 56.95s/it] {'loss': 1.9101, 'learning_rate': 2e-05, 'epoch': 0.94} 94%|█████████▍| 1712/1817 [27:07:11<1:39:39, 56.95s/it] 94%|█████████▍| 1713/1817 [27:08:11<1:39:51, 57.61s/it] {'loss': 1.9192, 'learning_rate': 2e-05, 'epoch': 0.94} 94%|█████████▍| 1713/1817 [27:08:11<1:39:51, 57.61s/it] 94%|█████████▍| 1714/1817 [27:09:08<1:38:48, 57.56s/it] {'loss': 1.8726, 'learning_rate': 2e-05, 'epoch': 0.94} 94%|█████████▍| 1714/1817 [27:09:08<1:38:48, 57.56s/it] 94%|█████████▍| 1715/1817 [27:10:05<1:37:17, 57.23s/it] {'loss': 1.9129, 'learning_rate': 2e-05, 'epoch': 0.94} 94%|█████████▍| 1715/1817 [27:10:05<1:37:17, 57.23s/it] 94%|█████████▍| 1716/1817 [27:11:01<1:35:52, 56.95s/it] {'loss': 1.89, 'learning_rate': 2e-05, 'epoch': 0.94} 94%|█████████▍| 1716/1817 [27:11:01<1:35:52, 56.95s/it] 94%|█████████▍| 1717/1817 [27:11:57<1:34:19, 56.59s/it] {'loss': 1.926, 'learning_rate': 2e-05, 'epoch': 0.94} 94%|█████████▍| 1717/1817 [27:11:57<1:34:19, 56.59s/it] 95%|█████████▍| 1718/1817 [27:12:50<1:31:52, 55.68s/it] {'loss': 1.8981, 'learning_rate': 2e-05, 'epoch': 0.95} 95%|█████████▍| 1718/1817 [27:12:50<1:31:52, 55.68s/it] 95%|█████████▍| 1719/1817 [27:13:48<1:32:04, 56.38s/it] {'loss': 1.8968, 'learning_rate': 2e-05, 'epoch': 0.95} 95%|█████████▍| 1719/1817 [27:13:48<1:32:04, 56.38s/it] 95%|█████████▍| 1720/1817 [27:14:44<1:31:07, 56.37s/it] {'loss': 1.888, 'learning_rate': 2e-05, 'epoch': 0.95} 95%|█████████▍| 1720/1817 [27:14:44<1:31:07, 56.37s/it] 95%|█████████▍| 1721/1817 [27:15:41<1:30:11, 56.36s/it] {'loss': 1.9004, 'learning_rate': 2e-05, 'epoch': 0.95} 95%|█████████▍| 1721/1817 [27:15:41<1:30:11, 56.36s/it] 95%|█████████▍| 1722/1817 [27:16:40<1:30:14, 56.99s/it] {'loss': 1.8645, 'learning_rate': 2e-05, 'epoch': 0.95} 95%|█████████▍| 1722/1817 [27:16:39<1:30:14, 56.99s/it] 95%|█████████▍| 1723/1817 [27:17:36<1:28:58, 56.79s/it] {'loss': 1.8786, 'learning_rate': 2e-05, 'epoch': 0.95} 95%|█████████▍| 1723/1817 [27:17:36<1:28:58, 56.79s/it] 95%|█████████▍| 1724/1817 [27:18:33<1:28:32, 57.12s/it] {'loss': 1.8889, 'learning_rate': 2e-05, 'epoch': 0.95} 95%|█████████▍| 1724/1817 [27:18:33<1:28:32, 57.12s/it] 95%|█████████▍| 1725/1817 [27:19:30<1:27:16, 56.91s/it] {'loss': 1.8953, 'learning_rate': 2e-05, 'epoch': 0.95} 95%|█████████▍| 1725/1817 [27:19:30<1:27:16, 56.91s/it] 95%|█████████▍| 1726/1817 [27:20:27<1:26:06, 56.77s/it] {'loss': 1.8999, 'learning_rate': 2e-05, 'epoch': 0.95} 95%|█████████▍| 1726/1817 [27:20:26<1:26:06, 56.77s/it] 95%|█████████▌| 1727/1817 [27:21:24<1:25:46, 57.18s/it] {'loss': 1.8801, 'learning_rate': 2e-05, 'epoch': 0.95} 95%|█████████▌| 1727/1817 [27:21:25<1:25:46, 57.18s/it] 95%|█████████▌| 1728/1817 [27:22:21<1:24:40, 57.08s/it] {'loss': 1.8641, 'learning_rate': 2e-05, 'epoch': 0.95} 95%|█████████▌| 1728/1817 [27:22:21<1:24:40, 57.08s/it] 95%|█████████▌| 1729/1817 [27:23:22<1:24:57, 57.93s/it] {'loss': 1.895, 'learning_rate': 2e-05, 'epoch': 0.95} 95%|█████████▌| 1729/1817 [27:23:21<1:24:57, 57.93s/it] 95%|█████████▌| 1730/1817 [27:24:20<1:24:10, 58.06s/it] {'loss': 1.8834, 'learning_rate': 2e-05, 'epoch': 0.95} 95%|█████████▌| 1730/1817 [27:24:20<1:24:10, 58.06s/it] 95%|█████████▌| 1731/1817 [27:25:18<1:23:15, 58.09s/it] {'loss': 1.8574, 'learning_rate': 2e-05, 'epoch': 0.95} 95%|█████████▌| 1731/1817 [27:25:18<1:23:15, 58.09s/it] 95%|█████████▌| 1732/1817 [27:26:14<1:21:34, 57.58s/it] {'loss': 1.9208, 'learning_rate': 2e-05, 'epoch': 0.95} 95%|█████████▌| 1732/1817 [27:26:14<1:21:34, 57.58s/it] 95%|█████████▌| 1733/1817 [27:27:11<1:20:06, 57.22s/it] {'loss': 1.8915, 'learning_rate': 2e-05, 'epoch': 0.95} 95%|█████████▌| 1733/1817 [27:27:11<1:20:06, 57.22s/it] 95%|█████████▌| 1734/1817 [27:28:07<1:18:48, 56.97s/it] {'loss': 1.8501, 'learning_rate': 2e-05, 'epoch': 0.95} 95%|█████████▌| 1734/1817 [27:28:07<1:18:48, 56.97s/it] 95%|█████████▌| 1735/1817 [27:29:04<1:17:34, 56.77s/it] {'loss': 1.9098, 'learning_rate': 2e-05, 'epoch': 0.95} 95%|█████████▌| 1735/1817 [27:29:03<1:17:34, 56.77s/it] 96%|█████████▌| 1736/1817 [27:30:00<1:16:26, 56.63s/it] {'loss': 1.9032, 'learning_rate': 2e-05, 'epoch': 0.96} 96%|█████████▌| 1736/1817 [27:30:00<1:16:26, 56.63s/it] 96%|█████████▌| 1737/1817 [27:30:56<1:15:23, 56.55s/it] {'loss': 1.8772, 'learning_rate': 2e-05, 'epoch': 0.96} 96%|█████████▌| 1737/1817 [27:30:56<1:15:23, 56.55s/it] 96%|█████████▌| 1738/1817 [27:31:54<1:14:51, 56.86s/it] {'loss': 1.8999, 'learning_rate': 2e-05, 'epoch': 0.96} 96%|█████████▌| 1738/1817 [27:31:53<1:14:51, 56.86s/it] 96%|█████████▌| 1739/1817 [27:32:52<1:14:16, 57.13s/it] {'loss': 1.9011, 'learning_rate': 2e-05, 'epoch': 0.96} 96%|█████████▌| 1739/1817 [27:32:51<1:14:16, 57.13s/it] 96%|█████████▌| 1740/1817 [27:33:48<1:13:00, 56.89s/it] {'loss': 1.8895, 'learning_rate': 2e-05, 'epoch': 0.96} 96%|█████████▌| 1740/1817 [27:33:48<1:13:00, 56.89s/it] 96%|█████████▌| 1741/1817 [27:34:46<1:12:31, 57.25s/it] {'loss': 1.914, 'learning_rate': 2e-05, 'epoch': 0.96} 96%|█████████▌| 1741/1817 [27:34:46<1:12:31, 57.25s/it] 96%|█████████▌| 1742/1817 [27:35:42<1:11:12, 56.97s/it] {'loss': 1.8789, 'learning_rate': 2e-05, 'epoch': 0.96} 96%|█████████▌| 1742/1817 [27:35:42<1:11:12, 56.97s/it] 96%|█████████▌| 1743/1817 [27:36:38<1:10:01, 56.78s/it] {'loss': 1.8466, 'learning_rate': 2e-05, 'epoch': 0.96} 96%|█████████▌| 1743/1817 [27:36:38<1:10:01, 56.78s/it] 96%|█████████▌| 1744/1817 [27:37:36<1:09:34, 57.18s/it] {'loss': 1.9248, 'learning_rate': 2e-05, 'epoch': 0.96} 96%|█████████▌| 1744/1817 [27:37:36<1:09:34, 57.18s/it] 96%|█████████▌| 1745/1817 [27:38:31<1:07:42, 56.43s/it] {'loss': 1.8849, 'learning_rate': 2e-05, 'epoch': 0.96} 96%|█████████▌| 1745/1817 [27:38:31<1:07:42, 56.43s/it] 96%|█████████▌| 1746/1817 [27:39:29<1:07:08, 56.73s/it] {'loss': 1.8862, 'learning_rate': 2e-05, 'epoch': 0.96} 96%|█████████▌| 1746/1817 [27:39:29<1:07:08, 56.73s/it] 96%|█████████▌| 1747/1817 [27:40:25<1:05:57, 56.53s/it] {'loss': 1.915, 'learning_rate': 2e-05, 'epoch': 0.96} 96%|█████████▌| 1747/1817 [27:40:25<1:05:57, 56.53s/it] 96%|█████████▌| 1748/1817 [27:41:21<1:04:58, 56.51s/it] {'loss': 1.8829, 'learning_rate': 2e-05, 'epoch': 0.96} 96%|█████████▌| 1748/1817 [27:41:21<1:04:58, 56.51s/it] 96%|█████████▋| 1749/1817 [27:42:17<1:03:59, 56.46s/it] {'loss': 1.845, 'learning_rate': 2e-05, 'epoch': 0.96} 96%|█████████▋| 1749/1817 [27:42:17<1:03:59, 56.46s/it] 96%|█████████▋| 1750/1817 [27:43:15<1:03:27, 56.83s/it] {'loss': 1.8824, 'learning_rate': 2e-05, 'epoch': 0.96} 96%|█████████▋| 1750/1817 [27:43:15<1:03:27, 56.83s/it] 96%|█████████▋| 1751/1817 [27:44:12<1:02:19, 56.66s/it] {'loss': 1.8978, 'learning_rate': 2e-05, 'epoch': 0.96} 96%|█████████▋| 1751/1817 [27:44:11<1:02:19, 56.66s/it] 96%|█████████▋| 1752/1817 [27:45:08<1:01:19, 56.61s/it] {'loss': 1.8958, 'learning_rate': 2e-05, 'epoch': 0.96} 96%|█████████▋| 1752/1817 [27:45:08<1:01:19, 56.61s/it] 96%|█████████▋| 1753/1817 [27:46:02<59:38, 55.91s/it] {'loss': 1.8886, 'learning_rate': 2e-05, 'epoch': 0.96} 96%|█████████▋| 1753/1817 [27:46:02<59:38, 55.91s/it] 97%|█████████▋| 1754/1817 [27:46:59<58:50, 56.04s/it] {'loss': 1.899, 'learning_rate': 2e-05, 'epoch': 0.97} 97%|█████████▋| 1754/1817 [27:46:59<58:50, 56.04s/it] 97%|█████████▋| 1755/1817 [27:47:53<57:27, 55.61s/it] {'loss': 1.8896, 'learning_rate': 2e-05, 'epoch': 0.97} 97%|█████████▋| 1755/1817 [27:47:53<57:27, 55.61s/it] 97%|█████████▋| 1756/1817 [27:48:49<56:45, 55.82s/it] {'loss': 1.9216, 'learning_rate': 2e-05, 'epoch': 0.97} 97%|█████████▋| 1756/1817 [27:48:49<56:45, 55.82s/it] 97%|█████████▋| 1757/1817 [27:49:46<55:58, 55.98s/it] {'loss': 1.8871, 'learning_rate': 2e-05, 'epoch': 0.97} 97%|█████████▋| 1757/1817 [27:49:46<55:58, 55.98s/it] 97%|█████████▋| 1758/1817 [27:50:44<55:43, 56.66s/it] {'loss': 1.8926, 'learning_rate': 2e-05, 'epoch': 0.97} 97%|█████████▋| 1758/1817 [27:50:44<55:43, 56.66s/it] 97%|█████████▋| 1759/1817 [27:51:43<55:19, 57.23s/it] {'loss': 1.9059, 'learning_rate': 2e-05, 'epoch': 0.97} 97%|█████████▋| 1759/1817 [27:51:43<55:19, 57.23s/it] 97%|█████████▋| 1760/1817 [27:52:39<54:05, 56.94s/it] {'loss': 1.9089, 'learning_rate': 2e-05, 'epoch': 0.97} 97%|█████████▋| 1760/1817 [27:52:39<54:05, 56.94s/it] 97%|█████████▋| 1761/1817 [27:53:35<53:02, 56.82s/it] {'loss': 1.8936, 'learning_rate': 2e-05, 'epoch': 0.97} 97%|█████████▋| 1761/1817 [27:53:35<53:02, 56.82s/it] 97%|█████████▋| 1762/1817 [27:54:32<51:59, 56.72s/it] {'loss': 1.8901, 'learning_rate': 2e-05, 'epoch': 0.97} 97%|█████████▋| 1762/1817 [27:54:32<51:59, 56.72s/it] 97%|█████████▋| 1763/1817 [27:55:29<51:03, 56.74s/it] {'loss': 1.9, 'learning_rate': 2e-05, 'epoch': 0.97} 97%|█████████▋| 1763/1817 [27:55:29<51:03, 56.74s/it] 97%|█████████▋| 1764/1817 [27:56:25<50:00, 56.61s/it] {'loss': 1.9047, 'learning_rate': 2e-05, 'epoch': 0.97} 97%|█████████▋| 1764/1817 [27:56:25<50:00, 56.61s/it] 97%|█████████▋| 1765/1817 [27:57:21<49:01, 56.56s/it] {'loss': 1.8843, 'learning_rate': 2e-05, 'epoch': 0.97} 97%|█████████▋| 1765/1817 [27:57:21<49:01, 56.56s/it] 97%|█████████▋| 1766/1817 [27:58:19<48:18, 56.82s/it] {'loss': 1.9122, 'learning_rate': 2e-05, 'epoch': 0.97} 97%|█████████▋| 1766/1817 [27:58:19<48:18, 56.82s/it] 97%|█████████▋| 1767/1817 [27:59:17<47:31, 57.02s/it] {'loss': 1.8903, 'learning_rate': 2e-05, 'epoch': 0.97} 97%|█████████▋| 1767/1817 [27:59:16<47:31, 57.03s/it] 97%|█████████▋| 1768/1817 [28:00:16<47:05, 57.67s/it] {'loss': 1.8989, 'learning_rate': 2e-05, 'epoch': 0.97} 97%|█████████▋| 1768/1817 [28:00:16<47:05, 57.67s/it] 97%|█████████▋| 1769/1817 [28:01:13<46:02, 57.54s/it] {'loss': 1.8733, 'learning_rate': 2e-05, 'epoch': 0.97} 97%|█████████▋| 1769/1817 [28:01:13<46:02, 57.54s/it] 97%|█████████▋| 1770/1817 [28:02:09<44:50, 57.25s/it] {'loss': 1.8217, 'learning_rate': 2e-05, 'epoch': 0.97} 97%|█████████▋| 1770/1817 [28:02:09<44:50, 57.25s/it] 97%|█████████▋| 1771/1817 [28:03:07<43:56, 57.32s/it] {'loss': 1.8669, 'learning_rate': 2e-05, 'epoch': 0.97} 97%|█████████▋| 1771/1817 [28:03:07<43:56, 57.32s/it] 98%|█████████▊| 1772/1817 [28:04:03<42:45, 57.02s/it] {'loss': 1.8466, 'learning_rate': 2e-05, 'epoch': 0.97} 98%|█████████▊| 1772/1817 [28:04:03<42:45, 57.02s/it] 98%|█████████▊| 1773/1817 [28:05:01<42:02, 57.32s/it] {'loss': 1.8834, 'learning_rate': 2e-05, 'epoch': 0.98} 98%|█████████▊| 1773/1817 [28:05:01<42:02, 57.32s/it] 98%|█████████▊| 1774/1817 [28:05:59<41:10, 57.44s/it] {'loss': 1.8569, 'learning_rate': 2e-05, 'epoch': 0.98} 98%|█████████▊| 1774/1817 [28:05:59<41:10, 57.44s/it] 98%|█████████▊| 1775/1817 [28:06:56<40:06, 57.31s/it] {'loss': 1.9355, 'learning_rate': 2e-05, 'epoch': 0.98} 98%|█████████▊| 1775/1817 [28:06:56<40:06, 57.31s/it] 98%|█████████▊| 1776/1817 [28:07:52<38:57, 57.00s/it] {'loss': 1.9127, 'learning_rate': 2e-05, 'epoch': 0.98} 98%|█████████▊| 1776/1817 [28:07:52<38:57, 57.00s/it] 98%|█████████▊| 1777/1817 [28:08:50<38:05, 57.14s/it] {'loss': 1.9198, 'learning_rate': 2e-05, 'epoch': 0.98} 98%|█████████▊| 1777/1817 [28:08:50<38:06, 57.15s/it] 98%|█████████▊| 1778/1817 [28:09:49<37:37, 57.88s/it] {'loss': 1.8846, 'learning_rate': 2e-05, 'epoch': 0.98} 98%|█████████▊| 1778/1817 [28:09:49<37:37, 57.88s/it] 98%|█████████▊| 1779/1817 [28:10:46<36:21, 57.41s/it] {'loss': 1.8949, 'learning_rate': 2e-05, 'epoch': 0.98} 98%|█████████▊| 1779/1817 [28:10:46<36:21, 57.41s/it] 98%|█████████▊| 1780/1817 [28:11:42<35:12, 57.10s/it] {'loss': 1.8831, 'learning_rate': 2e-05, 'epoch': 0.98} 98%|█████████▊| 1780/1817 [28:11:42<35:12, 57.10s/it] 98%|█████████▊| 1781/1817 [28:12:38<34:06, 56.85s/it] {'loss': 1.9082, 'learning_rate': 2e-05, 'epoch': 0.98} 98%|█████████▊| 1781/1817 [28:12:38<34:06, 56.85s/it] 98%|█████████▊| 1782/1817 [28:13:34<33:03, 56.68s/it] {'loss': 1.9, 'learning_rate': 2e-05, 'epoch': 0.98} 98%|█████████▊| 1782/1817 [28:13:35<33:03, 56.68s/it] 98%|█████████▊| 1783/1817 [28:14:32<32:16, 56.96s/it] {'loss': 1.8725, 'learning_rate': 2e-05, 'epoch': 0.98} 98%|█████████▊| 1783/1817 [28:14:32<32:16, 56.96s/it] 98%|█████████▊| 1784/1817 [28:15:29<31:15, 56.84s/it] {'loss': 1.9164, 'learning_rate': 2e-05, 'epoch': 0.98} 98%|█████████▊| 1784/1817 [28:15:29<31:15, 56.84s/it] 98%|█████████▊| 1785/1817 [28:16:26<30:27, 57.12s/it] {'loss': 1.8924, 'learning_rate': 2e-05, 'epoch': 0.98} 98%|█████████▊| 1785/1817 [28:16:26<30:27, 57.12s/it] 98%|█████████▊| 1786/1817 [28:17:24<29:38, 57.39s/it] {'loss': 1.9021, 'learning_rate': 2e-05, 'epoch': 0.98} 98%|█████████▊| 1786/1817 [28:17:24<29:38, 57.39s/it] 98%|█████████▊| 1787/1817 [28:18:22<28:47, 57.57s/it] {'loss': 1.8655, 'learning_rate': 2e-05, 'epoch': 0.98} 98%|█████████▊| 1787/1817 [28:18:22<28:47, 57.57s/it] 98%|█████████▊| 1788/1817 [28:19:19<27:39, 57.22s/it] {'loss': 1.8906, 'learning_rate': 2e-05, 'epoch': 0.98} 98%|█████████▊| 1788/1817 [28:19:19<27:39, 57.22s/it] 98%|█████████▊| 1789/1817 [28:20:16<26:44, 57.29s/it] {'loss': 1.9142, 'learning_rate': 2e-05, 'epoch': 0.98} 98%|█████████▊| 1789/1817 [28:20:16<26:44, 57.29s/it] 99%|█████████▊| 1790/1817 [28:21:13<25:40, 57.04s/it] {'loss': 1.8864, 'learning_rate': 2e-05, 'epoch': 0.98} 99%|█████████▊| 1790/1817 [28:21:13<25:40, 57.04s/it] 99%|█████████▊| 1791/1817 [28:22:09<24:38, 56.85s/it] {'loss': 1.8764, 'learning_rate': 2e-05, 'epoch': 0.99} 99%|█████████▊| 1791/1817 [28:22:09<24:38, 56.85s/it] 99%|█████████▊| 1792/1817 [28:23:07<23:48, 57.14s/it] {'loss': 1.8932, 'learning_rate': 2e-05, 'epoch': 0.99} 99%|█████████▊| 1792/1817 [28:23:07<23:48, 57.14s/it] 99%|█████████▊| 1793/1817 [28:24:05<22:56, 57.36s/it] {'loss': 1.8898, 'learning_rate': 2e-05, 'epoch': 0.99} 99%|█████████▊| 1793/1817 [28:24:05<22:56, 57.36s/it] 99%|█████████▊| 1794/1817 [28:25:01<21:49, 56.93s/it] {'loss': 1.8277, 'learning_rate': 2e-05, 'epoch': 0.99} 99%|█████████▊| 1794/1817 [28:25:01<21:49, 56.93s/it] 99%|█████████▉| 1795/1817 [28:25:57<20:48, 56.73s/it] {'loss': 1.8932, 'learning_rate': 2e-05, 'epoch': 0.99} 99%|█████████▉| 1795/1817 [28:25:57<20:48, 56.73s/it] 99%|█████████▉| 1796/1817 [28:26:53<19:48, 56.57s/it] {'loss': 1.8708, 'learning_rate': 2e-05, 'epoch': 0.99} 99%|█████████▉| 1796/1817 [28:26:53<19:48, 56.57s/it] 99%|█████████▉| 1797/1817 [28:27:50<18:49, 56.49s/it] {'loss': 1.8699, 'learning_rate': 2e-05, 'epoch': 0.99} 99%|█████████▉| 1797/1817 [28:27:50<18:49, 56.49s/it] 99%|█████████▉| 1798/1817 [28:28:46<17:52, 56.46s/it] {'loss': 1.8695, 'learning_rate': 2e-05, 'epoch': 0.99} 99%|█████████▉| 1798/1817 [28:28:46<17:52, 56.46s/it] 99%|█████████▉| 1799/1817 [28:29:44<17:07, 57.08s/it] {'loss': 1.8439, 'learning_rate': 2e-05, 'epoch': 0.99} 99%|█████████▉| 1799/1817 [28:29:44<17:07, 57.08s/it] 99%|█████████▉| 1800/1817 [28:30:41<16:07, 56.93s/it] {'loss': 1.8994, 'learning_rate': 2e-05, 'epoch': 0.99} 99%|█████████▉| 1800/1817 [28:30:41<16:07, 56.93s/it] 99%|█████████▉| 1801/1817 [28:31:40<15:21, 57.58s/it] {'loss': 1.8816, 'learning_rate': 2e-05, 'epoch': 0.99} 99%|█████████▉| 1801/1817 [28:31:40<15:21, 57.58s/it] 99%|█████████▉| 1802/1817 [28:32:37<14:18, 57.25s/it] {'loss': 1.8972, 'learning_rate': 2e-05, 'epoch': 0.99} 99%|█████████▉| 1802/1817 [28:32:37<14:18, 57.25s/it] 99%|█████████▉| 1803/1817 [28:33:35<13:25, 57.54s/it] {'loss': 1.9548, 'learning_rate': 2e-05, 'epoch': 0.99} 99%|█████████▉| 1803/1817 [28:33:35<13:25, 57.54s/it] 99%|█████████▉| 1804/1817 [28:34:34<12:32, 57.91s/it] {'loss': 1.9023, 'learning_rate': 2e-05, 'epoch': 0.99} 99%|█████████▉| 1804/1817 [28:34:34<12:32, 57.91s/it] 99%|█████████▉| 1805/1817 [28:35:31<11:31, 57.65s/it] {'loss': 1.8931, 'learning_rate': 2e-05, 'epoch': 0.99} 99%|█████████▉| 1805/1817 [28:35:31<11:31, 57.65s/it] 99%|█████████▉| 1806/1817 [28:36:28<10:31, 57.39s/it] {'loss': 1.8834, 'learning_rate': 2e-05, 'epoch': 0.99} 99%|█████████▉| 1806/1817 [28:36:27<10:31, 57.39s/it] 99%|█████████▉| 1807/1817 [28:37:28<09:43, 58.38s/it] {'loss': 1.9079, 'learning_rate': 2e-05, 'epoch': 0.99} 99%|█████████▉| 1807/1817 [28:37:28<09:43, 58.38s/it] 100%|█████████▉| 1808/1817 [28:38:25<08:40, 57.78s/it] {'loss': 1.9261, 'learning_rate': 2e-05, 'epoch': 0.99} 100%|█████████▉| 1808/1817 [28:38:24<08:40, 57.78s/it] 100%|█████████▉| 1809/1817 [28:39:21<07:39, 57.42s/it] {'loss': 1.8728, 'learning_rate': 2e-05, 'epoch': 1.0} 100%|█████████▉| 1809/1817 [28:39:21<07:39, 57.42s/it] 100%|█████████▉| 1810/1817 [28:40:17<06:39, 57.04s/it] {'loss': 1.8313, 'learning_rate': 2e-05, 'epoch': 1.0} 100%|█████████▉| 1810/1817 [28:40:17<06:39, 57.04s/it] 100%|█████████▉| 1811/1817 [28:41:13<05:40, 56.80s/it] {'loss': 1.9012, 'learning_rate': 2e-05, 'epoch': 1.0} 100%|█████████▉| 1811/1817 [28:41:13<05:40, 56.80s/it] 100%|█████████▉| 1812/1817 [28:42:13<04:48, 57.72s/it] {'loss': 1.9103, 'learning_rate': 2e-05, 'epoch': 1.0} 100%|█████████▉| 1812/1817 [28:42:13<04:48, 57.72s/it] 100%|█████████▉| 1813/1817 [28:43:10<03:49, 57.32s/it] {'loss': 1.9126, 'learning_rate': 2e-05, 'epoch': 1.0} 100%|█████████▉| 1813/1817 [28:43:10<03:49, 57.32s/it] 100%|█████████▉| 1814/1817 [28:44:06<02:50, 56.97s/it] {'loss': 1.9373, 'learning_rate': 2e-05, 'epoch': 1.0} 100%|█████████▉| 1814/1817 [28:44:06<02:50, 56.97s/it] 100%|█████████▉| 1815/1817 [28:45:02<01:53, 56.83s/it] {'loss': 1.8896, 'learning_rate': 2e-05, 'epoch': 1.0} 100%|█████████▉| 1815/1817 [28:45:02<01:53, 56.83s/it] 100%|█████████▉| 1816/1817 [28:45:59<00:56, 56.72s/it] {'loss': 1.9042, 'learning_rate': 2e-05, 'epoch': 1.0} 100%|█████████▉| 1816/1817 [28:45:59<00:56, 56.72s/it] 100%|██████████| 1817/1817 [28:46:55<00:00, 56.62s/it] {'loss': 1.8563, 'learning_rate': 2e-05, 'epoch': 1.0} 100%|██████████| 1817/1817 [28:46:55<00:00, 56.62s/it][INFO|trainer.py:2025] 2024-01-29 09:40:40,846 >> Training completed. Do not forget to share your model on huggingface.co/models =) {'train_runtime': 103616.5298, 'train_samples_per_second': 8.98, 'train_steps_per_second': 0.018, 'train_loss': 1.9468243734877255, 'epoch': 1.0} 100%|██████████| 1817/1817 [28:46:56<00:00, 56.62s/it] 100%|██████████| 1817/1817 [28:46:56<00:00, 57.03s/it] [INFO|trainer.py:2830] 2024-01-29 09:40:40,877 >> Saving model checkpoint to ./checkpoints_ct/ac/allm-ac-7b [INFO|configuration_utils.py:457] 2024-01-29 09:40:40,879 >> Configuration saved in ./checkpoints_ct/ac/allm-ac-7b/config.json [INFO|configuration_utils.py:362] 2024-01-29 09:40:40,885 >> Configuration saved in ./checkpoints_ct/ac/allm-ac-7b/generation_config.json [INFO|modeling_utils.py:1759] 2024-01-29 09:40:40,918 >> Model weights saved in ./checkpoints_ct/ac/allm-ac-7b/pytorch_model.bin [INFO|tokenization_utils_base.py:2164] 2024-01-29 09:40:40,920 >> tokenizer config file saved in ./checkpoints_ct/ac/allm-ac-7b/tokenizer_config.json [INFO|tokenization_utils_base.py:2171] 2024-01-29 09:40:40,923 >> Special tokens file saved in ./checkpoints_ct/ac/allm-ac-7b/special_tokens_map.json [INFO|tokenization_utils_base.py:2221] 2024-01-29 09:40:40,927 >> added tokens file saved in ./checkpoints_ct/ac/allm-ac-7b/added_tokens.json [2024-01-29 09:40:48,710] [INFO] [logging.py:96:log_dist] [Rank 0] [Torch] Checkpoint global_step1817 is about to be saved! [2024-01-29 09:40:48,713] [INFO] [engine.py:3492:save_16bit_model] Saving model weights to ./checkpoints_ct/ac/allm-ac-7b/pytorch_model.bin, tag: global_step1817 [2024-01-29 09:40:48,713] [INFO] [torch_checkpoint_engine.py:21:save] [Torch] Saving ./checkpoints_ct/ac/allm-ac-7b/pytorch_model.bin... [2024-01-29 09:41:05,788] [INFO] [torch_checkpoint_engine.py:23:save] [Torch] Saved ./checkpoints_ct/ac/allm-ac-7b/pytorch_model.bin. [2024-01-29 09:41:05,788] [INFO] [torch_checkpoint_engine.py:33:commit] [Torch] Checkpoint global_step1817 is ready now! ***** train metrics ***** epoch = 1.0 train_loss = 1.9468 train_runtime = 1 day, 4:46:56.52 train_samples = 930514 train_samples_per_second = 8.98 train_steps_per_second = 0.018 [INFO|modelcard.py:451] 2024-01-29 09:41:06,492 >> Dropping the following result as it does not have all the necessary fields: {'task': {'name': 'Causal Language Modeling', 'type': 'text-generation'}, 'dataset': {'name': '/apdcephfs/share_733425/vinnylywang/jianhuipang/datasets/RedPajama-Data-1T-Sample', 'type': '/apdcephfs/share_733425/vinnylywang/jianhuipang/datasets/RedPajama-Data-1T-Sample', 'config': None, 'split': 'None'}}