01/31/2022 07:15:59 - WARNING - __main__ - Process rank: -1, device: cuda:0, n_gpu: 1distributed training: False, 16-bits training: True 01/31/2022 07:15:59 - INFO - __main__ - Training/evaluation parameters TrainingArguments( _n_gpu=1, adafactor=False, adam_beta1=0.9, adam_beta2=0.999, adam_epsilon=1e-08, bf16=False, bf16_full_eval=False, dataloader_drop_last=False, dataloader_num_workers=0, dataloader_pin_memory=True, ddp_bucket_cap_mb=None, ddp_find_unused_parameters=None, debug=[], deepspeed=None, disable_tqdm=False, do_eval=True, do_predict=False, do_train=True, eval_accumulation_steps=None, eval_steps=500, evaluation_strategy=IntervalStrategy.STEPS, fp16=True, fp16_backend=auto, fp16_full_eval=False, fp16_opt_level=O1, gradient_accumulation_steps=4, gradient_checkpointing=True, greater_is_better=None, group_by_length=True, half_precision_backend=auto, hub_model_id=None, hub_strategy=HubStrategy.EVERY_SAVE, hub_token=, ignore_data_skip=False, label_names=None, label_smoothing_factor=0.0, learning_rate=7.5e-05, length_column_name=input_length, load_best_model_at_end=False, local_rank=-1, log_level=-1, log_level_replica=-1, log_on_each_node=True, logging_dir=./runs/Jan31_07-15-59_job-2c68f48a-2d5d-4013-9043-3f2cb25f3ff6, logging_first_step=False, logging_nan_inf_filter=True, logging_steps=100, logging_strategy=IntervalStrategy.STEPS, lr_scheduler_type=SchedulerType.LINEAR, max_grad_norm=1.0, max_steps=-1, metric_for_best_model=None, mp_parameters=, no_cuda=False, num_train_epochs=50.0, optim=OptimizerNames.ADAMW_HF, output_dir=./, overwrite_output_dir=True, past_index=-1, per_device_eval_batch_size=8, per_device_train_batch_size=8, prediction_loss_only=False, push_to_hub=True, push_to_hub_model_id=None, push_to_hub_organization=None, push_to_hub_token=, remove_unused_columns=True, report_to=['tensorboard'], resume_from_checkpoint=None, run_name=./, save_on_each_node=False, save_steps=500, save_strategy=IntervalStrategy.STEPS, save_total_limit=3, seed=42, sharded_ddp=[], skip_memory_metrics=True, tf32=None, tpu_metrics_debug=False, tpu_num_cores=None, use_legacy_prediction_loop=False, warmup_ratio=0.0, warmup_steps=2000, weight_decay=0.0, xpu_backend=None, ) 01/31/2022 07:16:01 - WARNING - datasets.builder - Reusing dataset zeroth_korean_asr (/workspace/.cache/huggingface/datasets/kresnik___zeroth_korean_asr/clean/1.0.1/f6cf96a53d5512525e3113bab8048d36ce268658d6e0c40d45f65dfa3f0bc343) 01/31/2022 07:16:03 - WARNING - datasets.builder - Reusing dataset zeroth_korean_asr (/workspace/.cache/huggingface/datasets/kresnik___zeroth_korean_asr/clean/1.0.1/f6cf96a53d5512525e3113bab8048d36ce268658d6e0c40d45f65dfa3f0bc343) remove special characters from datasets: 0ex [00:00, ?ex/s] remove special characters from datasets: 1000ex [00:00, 9974.66ex/s] remove special characters from datasets: 2055ex [00:00, 10308.75ex/s] remove special characters from datasets: 3199ex [00:00, 10824.47ex/s] remove special characters from datasets: 4345ex [00:00, 11074.98ex/s] remove special characters from datasets: 5495ex [00:00, 11227.17ex/s] remove special characters from datasets: 6633ex [00:00, 11276.25ex/s] remove special characters from datasets: 7761ex [00:00, 11148.49ex/s] remove special characters from datasets: 8877ex [00:00, 10968.51ex/s] remove special characters from datasets: 9975ex [00:00, 10891.69ex/s] remove special characters from datasets: 11065ex [00:01, 10876.53ex/s] remove special characters from datasets: 12204ex [00:01, 11029.97ex/s] remove special characters from datasets: 13308ex [00:01, 10876.84ex/s] remove special characters from datasets: 14405ex [00:01, 10901.35ex/s] remove special characters from datasets: 15496ex [00:01, 10883.42ex/s] remove special characters from datasets: 16585ex [00:01, 10507.14ex/s] remove special characters from datasets: 17639ex [00:01, 10394.81ex/s] remove special characters from datasets: 18681ex [00:01, 10306.72ex/s] remove special characters from datasets: 19750ex [00:01, 10416.89ex/s] remove special characters from datasets: 20893ex [00:01, 10714.37ex/s] remove special characters from datasets: 22000ex [00:02, 10798.35ex/s] remove special characters from datasets: 22263ex [00:02, 10809.37ex/s] remove special characters from datasets: 0ex [00:00, ?ex/s] remove special characters from datasets: 457ex [00:00, 12085.35ex/s] loading configuration file https://huggingface.co/facebook/wav2vec2-xls-r-300m/resolve/main/config.json from cache at /workspace/.cache/huggingface/transformers/dabc27df63e37bd2a7a221c7774e35f36a280fbdf917cf54cadfc7df8c786f6f.a3e4c3c967d9985881e0ae550a5f6f668f897db5ab2e0802f9b97973b15970e6 Model config Wav2Vec2Config { "_name_or_path": "facebook/wav2vec2-xls-r-300m", "activation_dropout": 0.0, "adapter_kernel_size": 3, "adapter_stride": 2, "add_adapter": false, "apply_spec_augment": true, "architectures": [ "Wav2Vec2ForPreTraining" ], "attention_dropout": 0.1, "bos_token_id": 1, "classifier_proj_size": 256, "codevector_dim": 768, "contrastive_logits_temperature": 0.1, "conv_bias": true, "conv_dim": [ 512, 512, 512, 512, 512, 512, 512 ], "conv_kernel": [ 10, 3, 3, 3, 3, 2, 2 ], "conv_stride": [ 5, 2, 2, 2, 2, 2, 2 ], "ctc_loss_reduction": "sum", "ctc_zero_infinity": false, "diversity_loss_weight": 0.1, "do_stable_layer_norm": true, "eos_token_id": 2, "feat_extract_activation": "gelu", "feat_extract_dropout": 0.0, "feat_extract_norm": "layer", "feat_proj_dropout": 0.1, "feat_quantizer_dropout": 0.0, "final_dropout": 0.0, "gradient_checkpointing": false, "hidden_act": "gelu", "hidden_dropout": 0.1, "hidden_size": 1024, "initializer_range": 0.02, "intermediate_size": 4096, "layer_norm_eps": 1e-05, "layerdrop": 0.1, "mask_feature_length": 10, "mask_feature_min_masks": 0, "mask_feature_prob": 0.0, "mask_time_length": 10, "mask_time_min_masks": 2, "mask_time_prob": 0.075, "model_type": "wav2vec2", "num_adapter_layers": 3, "num_attention_heads": 16, "num_codevector_groups": 2, "num_codevectors_per_group": 320, "num_conv_pos_embedding_groups": 16, "num_conv_pos_embeddings": 128, "num_feat_extract_layers": 7, "num_hidden_layers": 24, "num_negatives": 100, "output_hidden_size": 1024, "pad_token_id": 0, "proj_codevector_dim": 768, "tdnn_dilation": [ 1, 2, 3, 1, 1 ], "tdnn_dim": [ 512, 512, 512, 512, 1500 ], "tdnn_kernel": [ 5, 3, 3, 1, 1 ], "torch_dtype": "float32", "transformers_version": "4.17.0.dev0", "use_weighted_layer_sum": false, "vocab_size": 32, "xvector_output_dim": 512 } 0%| | 0/1 [00:00 to the vocabulary Adding to the vocabulary Special tokens have been added in the vocabulary, make sure the associated word embeddings are fine-tuned or trained. loading configuration file https://huggingface.co/facebook/wav2vec2-xls-r-300m/resolve/main/config.json from cache at /workspace/.cache/huggingface/transformers/dabc27df63e37bd2a7a221c7774e35f36a280fbdf917cf54cadfc7df8c786f6f.a3e4c3c967d9985881e0ae550a5f6f668f897db5ab2e0802f9b97973b15970e6 Model config Wav2Vec2Config { "_name_or_path": "facebook/wav2vec2-xls-r-300m", "activation_dropout": 0.0, "adapter_kernel_size": 3, "adapter_stride": 2, "add_adapter": false, "apply_spec_augment": true, "architectures": [ "Wav2Vec2ForPreTraining" ], "attention_dropout": 0.1, "bos_token_id": 1, "classifier_proj_size": 256, "codevector_dim": 768, "contrastive_logits_temperature": 0.1, "conv_bias": true, "conv_dim": [ 512, 512, 512, 512, 512, 512, 512 ], "conv_kernel": [ 10, 3, 3, 3, 3, 2, 2 ], "conv_stride": [ 5, 2, 2, 2, 2, 2, 2 ], "ctc_loss_reduction": "sum", "ctc_zero_infinity": false, "diversity_loss_weight": 0.1, "do_stable_layer_norm": true, "eos_token_id": 2, "feat_extract_activation": "gelu", "feat_extract_dropout": 0.0, "feat_extract_norm": "layer", "feat_proj_dropout": 0.1, "feat_quantizer_dropout": 0.0, "final_dropout": 0.0, "gradient_checkpointing": false, "hidden_act": "gelu", "hidden_dropout": 0.1, "hidden_size": 1024, "initializer_range": 0.02, "intermediate_size": 4096, "layer_norm_eps": 1e-05, "layerdrop": 0.1, "mask_feature_length": 10, "mask_feature_min_masks": 0, "mask_feature_prob": 0.0, "mask_time_length": 10, "mask_time_min_masks": 2, "mask_time_prob": 0.075, "model_type": "wav2vec2", "num_adapter_layers": 3, "num_attention_heads": 16, "num_codevector_groups": 2, "num_codevectors_per_group": 320, "num_conv_pos_embedding_groups": 16, "num_conv_pos_embeddings": 128, "num_feat_extract_layers": 7, "num_hidden_layers": 24, "num_negatives": 100, "output_hidden_size": 1024, "pad_token_id": 0, "proj_codevector_dim": 768, "tdnn_dilation": [ 1, 2, 3, 1, 1 ], "tdnn_dim": [ 512, 512, 512, 512, 1500 ], "tdnn_kernel": [ 5, 3, 3, 1, 1 ], "torch_dtype": "float32", "transformers_version": "4.17.0.dev0", "use_weighted_layer_sum": false, "vocab_size": 32, "xvector_output_dim": 512 } loading feature extractor configuration file https://huggingface.co/facebook/wav2vec2-xls-r-300m/resolve/main/preprocessor_config.json from cache at /workspace/.cache/huggingface/transformers/6fb028b95b394059e7d3b367bbca2382b576c66aebe896f04d2cd34e1b575f5b.d4484dc1c81456a2461485e7168b04347a7b9a4e3b1ef3aba723323b33e12326 Feature extractor Wav2Vec2FeatureExtractor { "do_normalize": true, "feature_extractor_type": "Wav2Vec2FeatureExtractor", "feature_size": 1, "padding_side": "right", "padding_value": 0, "return_attention_mask": true, "sampling_rate": 16000 } loading weights file https://huggingface.co/facebook/wav2vec2-xls-r-300m/resolve/main/pytorch_model.bin from cache at /workspace/.cache/huggingface/transformers/1e6a6507f3b689035cd4b247e2a37c154e27f39143f31357a49b4e38baeccc36.1edb32803799e27ed554eb7dd935f6745b1a0b17b0ea256442fe24db6eb546cd Some weights of the model checkpoint at facebook/wav2vec2-xls-r-300m were not used when initializing Wav2Vec2ForCTC: ['quantizer.weight_proj.bias', 'project_q.bias', 'quantizer.weight_proj.weight', 'project_hid.bias', 'project_q.weight', 'quantizer.codevectors', 'project_hid.weight'] - This IS expected if you are initializing Wav2Vec2ForCTC from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model). - This IS NOT expected if you are initializing Wav2Vec2ForCTC from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model). Some weights of Wav2Vec2ForCTC were not initialized from the model checkpoint at facebook/wav2vec2-xls-r-300m and are newly initialized: ['lm_head.bias', 'lm_head.weight'] You should probably TRAIN this model on a down-stream task to be able to use it for predictions and inference. preprocess datasets: 0ex [00:00, ?ex/s] preprocess datasets: 1ex [00:00, 1.60ex/s] preprocess datasets: 36ex [00:00, 66.01ex/s] preprocess datasets: 67ex [00:00, 116.67ex/s] preprocess datasets: 93ex [00:00, 148.25ex/s] preprocess datasets: 118ex [00:01, 171.71ex/s] preprocess datasets: 143ex [00:01, 190.53ex/s] preprocess datasets: 168ex [00:01, 203.30ex/s] preprocess datasets: 195ex [00:01, 220.45ex/s] preprocess datasets: 220ex [00:01, 225.79ex/s] preprocess datasets: 245ex [00:01, 229.97ex/s] preprocess datasets: 270ex [00:01, 234.06ex/s] preprocess datasets: 295ex [00:01, 230.04ex/s] preprocess datasets: 320ex [00:01, 234.98ex/s] preprocess datasets: 345ex [00:01, 234.81ex/s] preprocess datasets: 369ex [00:02, 233.36ex/s] preprocess datasets: 395ex [00:02, 239.85ex/s] preprocess datasets: 420ex [00:02, 241.40ex/s] preprocess datasets: 447ex [00:02, 247.52ex/s] preprocess datasets: 472ex [00:02, 246.01ex/s] preprocess datasets: 497ex [00:02, 245.10ex/s] preprocess datasets: 522ex [00:02, 240.95ex/s] preprocess datasets: 547ex [00:02, 242.33ex/s] preprocess datasets: 578ex [00:02, 261.68ex/s] preprocess datasets: 611ex [00:03, 281.20ex/s] preprocess datasets: 646ex [00:03, 298.05ex/s] preprocess datasets: 679ex [00:03, 307.13ex/s] preprocess datasets: 711ex [00:03, 310.35ex/s] preprocess datasets: 743ex [00:03, 310.39ex/s] preprocess datasets: 775ex [00:03, 307.19ex/s] preprocess datasets: 809ex [00:03, 315.38ex/s] preprocess datasets: 844ex [00:03, 323.86ex/s] preprocess datasets: 878ex [00:03, 327.28ex/s] preprocess datasets: 911ex [00:03, 325.39ex/s] preprocess datasets: 944ex [00:04, 322.41ex/s] preprocess datasets: 977ex [00:04, 315.13ex/s] preprocess datasets: 1009ex [00:06, 42.75ex/s] preprocess datasets: 1044ex [00:06, 58.86ex/s] preprocess datasets: 1074ex [00:06, 75.68ex/s] preprocess datasets: 1104ex [00:06, 96.01ex/s] preprocess datasets: 1135ex [00:06, 120.33ex/s] preprocess datasets: 1166ex [00:06, 146.37ex/s] preprocess datasets: 1198ex [00:07, 174.27ex/s] preprocess datasets: 1228ex [00:07, 197.23ex/s] preprocess datasets: 1260ex [00:07, 222.15ex/s] preprocess datasets: 1291ex [00:07, 242.20ex/s] preprocess datasets: 1322ex [00:07, 259.01ex/s] preprocess datasets: 1353ex [00:07, 271.02ex/s] preprocess datasets: 1385ex [00:07, 283.17ex/s] preprocess datasets: 1416ex [00:07, 286.25ex/s] preprocess datasets: 1447ex [00:07, 288.57ex/s] preprocess datasets: 1479ex [00:07, 296.07ex/s] preprocess datasets: 1511ex [00:08, 300.63ex/s] preprocess datasets: 1543ex [00:08, 305.36ex/s] preprocess datasets: 1577ex [00:08, 310.98ex/s] preprocess datasets: 1609ex [00:08, 309.74ex/s] preprocess datasets: 1643ex [00:08, 314.49ex/s] preprocess datasets: 1675ex [00:08, 311.95ex/s] preprocess datasets: 1707ex [00:08, 311.51ex/s] preprocess datasets: 1740ex [00:08, 314.82ex/s] preprocess datasets: 1773ex [00:08, 318.97ex/s] preprocess datasets: 1806ex [00:09, 320.06ex/s] preprocess datasets: 1839ex [00:09, 314.34ex/s] preprocess datasets: 1871ex [00:09, 301.36ex/s] preprocess datasets: 1902ex [00:09, 296.06ex/s] preprocess datasets: 1932ex [00:09, 277.26ex/s] preprocess datasets: 1960ex [00:09, 263.74ex/s] preprocess datasets: 1988ex [00:09, 264.25ex/s] preprocess datasets: 2015ex [00:11, 45.72ex/s] preprocess datasets: 2044ex [00:11, 61.21ex/s] preprocess datasets: 2070ex [00:11, 77.71ex/s] preprocess datasets: 2099ex [00:11, 100.01ex/s] preprocess datasets: 2126ex [00:11, 121.94ex/s] preprocess datasets: 2156ex [00:12, 149.32ex/s] preprocess datasets: 2185ex [00:12, 174.20ex/s] preprocess datasets: 2212ex [00:12, 192.99ex/s] preprocess datasets: 2241ex [00:12, 213.70ex/s] preprocess datasets: 2269ex [00:12, 229.04ex/s] preprocess datasets: 2298ex [00:12, 244.49ex/s] preprocess datasets: 2329ex [00:12, 260.35ex/s] preprocess datasets: 2362ex [00:12, 279.07ex/s] preprocess datasets: 2393ex [00:12, 287.10ex/s] preprocess datasets: 2424ex [00:12, 277.86ex/s] preprocess datasets: 2454ex [00:13, 281.97ex/s] preprocess datasets: 2483ex [00:13, 283.44ex/s] preprocess datasets: 2512ex [00:13, 275.14ex/s] preprocess datasets: 2543ex [00:13, 284.24ex/s] preprocess datasets: 2572ex [00:13, 278.29ex/s] preprocess datasets: 2601ex [00:13, 275.43ex/s] preprocess datasets: 2629ex [00:13, 273.17ex/s] preprocess datasets: 2658ex [00:13, 276.24ex/s] preprocess datasets: 2687ex [00:13, 278.92ex/s] preprocess datasets: 2719ex [00:14, 289.97ex/s] preprocess datasets: 2749ex [00:14, 292.75ex/s] preprocess datasets: 2779ex [00:14, 272.36ex/s] preprocess datasets: 2807ex [00:14, 272.20ex/s] preprocess datasets: 2836ex [00:14, 275.88ex/s] preprocess datasets: 2866ex [00:14, 281.08ex/s] preprocess datasets: 2898ex [00:14, 290.71ex/s] preprocess datasets: 2931ex [00:14, 299.46ex/s] preprocess datasets: 2962ex [00:14, 287.98ex/s] preprocess datasets: 2996ex [00:15, 302.71ex/s] preprocess datasets: 3027ex [00:17, 46.29ex/s] preprocess datasets: 3061ex [00:17, 63.54ex/s] preprocess datasets: 3092ex [00:17, 82.51ex/s] preprocess datasets: 3124ex [00:17, 105.96ex/s] preprocess datasets: 3154ex [00:17, 130.06ex/s] preprocess datasets: 3187ex [00:17, 159.24ex/s] preprocess datasets: 3220ex [00:17, 188.16ex/s] preprocess datasets: 3251ex [00:17, 211.99ex/s] preprocess datasets: 3282ex [00:17, 232.25ex/s] preprocess datasets: 3313ex [00:17, 243.74ex/s] preprocess datasets: 3344ex [00:18, 258.48ex/s] preprocess datasets: 3376ex [00:18, 272.95ex/s] preprocess datasets: 3407ex [00:18, 276.12ex/s] preprocess datasets: 3437ex [00:18, 281.73ex/s] preprocess datasets: 3467ex [00:18, 284.33ex/s] preprocess datasets: 3498ex [00:18, 290.61ex/s] preprocess datasets: 3528ex [00:18, 274.40ex/s] preprocess datasets: 3561ex [00:18, 288.99ex/s] preprocess datasets: 3595ex [00:18, 303.08ex/s] preprocess datasets: 3627ex [00:19, 307.21ex/s] preprocess datasets: 3659ex [00:19, 310.82ex/s] preprocess datasets: 3693ex [00:19, 319.31ex/s] preprocess datasets: 3727ex [00:19, 322.34ex/s] preprocess datasets: 3760ex [00:19, 324.49ex/s] preprocess datasets: 3794ex [00:19, 325.65ex/s] preprocess datasets: 3827ex [00:19, 314.29ex/s] preprocess datasets: 3859ex [00:19, 308.29ex/s] preprocess datasets: 3890ex [00:19, 306.32ex/s] preprocess datasets: 3921ex [00:19, 305.89ex/s] preprocess datasets: 3952ex [00:20, 302.62ex/s] preprocess datasets: 3984ex [00:20, 306.19ex/s] preprocess datasets: 4015ex [00:21, 51.31ex/s] preprocess datasets: 4044ex [00:22, 66.78ex/s] preprocess datasets: 4074ex [00:22, 86.36ex/s] preprocess datasets: 4103ex [00:22, 108.32ex/s] preprocess datasets: 4132ex [00:22, 132.21ex/s] preprocess datasets: 4162ex [00:22, 158.77ex/s] preprocess datasets: 4194ex [00:22, 188.70ex/s] preprocess datasets: 4224ex [00:22, 211.53ex/s] preprocess datasets: 4257ex [00:22, 237.07ex/s] preprocess datasets: 4293ex [00:22, 266.30ex/s] preprocess datasets: 4325ex [00:22, 272.82ex/s] preprocess datasets: 4359ex [00:23, 288.89ex/s] preprocess datasets: 4393ex [00:23, 301.53ex/s] preprocess datasets: 4426ex [00:23, 305.39ex/s] preprocess datasets: 4459ex [00:23, 310.75ex/s] preprocess datasets: 4494ex [00:23, 320.61ex/s] preprocess datasets: 4530ex [00:23, 330.70ex/s] preprocess datasets: 4564ex [00:23, 327.90ex/s] preprocess datasets: 4598ex [00:23, 330.52ex/s] preprocess datasets: 4632ex [00:23, 329.84ex/s] preprocess datasets: 4669ex [00:24, 341.53ex/s] preprocess datasets: 4704ex [00:24, 337.52ex/s] preprocess datasets: 4738ex [00:24, 335.59ex/s] preprocess datasets: 4775ex [00:24, 344.96ex/s] preprocess datasets: 4812ex [00:24, 347.26ex/s] preprocess datasets: 4847ex [00:24, 327.83ex/s] preprocess datasets: 4881ex [00:24, 316.49ex/s] preprocess datasets: 4913ex [00:24, 314.28ex/s] preprocess datasets: 4945ex [00:24, 307.85ex/s] preprocess datasets: 4976ex [00:24, 302.74ex/s] preprocess datasets: 5007ex [00:26, 51.78ex/s] preprocess datasets: 5039ex [00:26, 69.03ex/s] preprocess datasets: 5066ex [00:27, 85.84ex/s] preprocess datasets: 5096ex [00:27, 108.64ex/s] preprocess datasets: 5126ex [00:27, 133.73ex/s] preprocess datasets: 5155ex [00:27, 157.94ex/s] preprocess datasets: 5186ex [00:27, 185.94ex/s] preprocess datasets: 5217ex [00:27, 211.38ex/s] preprocess datasets: 5247ex [00:27, 226.22ex/s] preprocess datasets: 5277ex [00:27, 240.99ex/s] preprocess datasets: 5306ex [00:27, 250.77ex/s] preprocess datasets: 5338ex [00:27, 266.91ex/s] preprocess datasets: 5371ex [00:28, 281.71ex/s] preprocess datasets: 5402ex [00:28, 281.00ex/s] preprocess datasets: 5434ex [00:28, 291.27ex/s] preprocess datasets: 5465ex [00:28, 292.64ex/s] preprocess datasets: 5495ex [00:28, 293.26ex/s] preprocess datasets: 5525ex [00:28, 293.28ex/s] preprocess datasets: 5555ex [00:28, 293.80ex/s] preprocess datasets: 5586ex [00:28, 297.35ex/s] preprocess datasets: 5620ex [00:28, 309.12ex/s] preprocess datasets: 5656ex [00:28, 320.84ex/s] preprocess datasets: 5689ex [00:29, 309.93ex/s] preprocess datasets: 5721ex [00:29, 305.20ex/s] preprocess datasets: 5752ex [00:29, 304.29ex/s] preprocess datasets: 5784ex [00:29, 308.76ex/s] preprocess datasets: 5818ex [00:29, 315.31ex/s] preprocess datasets: 5850ex [00:29, 306.49ex/s] preprocess datasets: 5881ex [00:29, 306.87ex/s] preprocess datasets: 5913ex [00:29, 309.23ex/s] preprocess datasets: 5944ex [00:29, 297.97ex/s] preprocess datasets: 5975ex [00:30, 299.32ex/s] preprocess datasets: 6006ex [00:32, 45.95ex/s] preprocess datasets: 6037ex [00:32, 61.51ex/s] preprocess datasets: 6068ex [00:32, 80.83ex/s] preprocess datasets: 6098ex [00:32, 102.50ex/s] preprocess datasets: 6133ex [00:32, 132.99ex/s] preprocess datasets: 6164ex [00:32, 159.01ex/s] preprocess datasets: 6194ex [00:32, 180.80ex/s] preprocess datasets: 6227ex [00:32, 209.72ex/s] preprocess datasets: 6260ex [00:32, 234.15ex/s] preprocess datasets: 6292ex [00:32, 254.27ex/s] preprocess datasets: 6323ex [00:33, 267.92ex/s] preprocess datasets: 6354ex [00:33, 268.61ex/s] preprocess datasets: 6384ex [00:33, 253.57ex/s] preprocess datasets: 6412ex [00:33, 258.91ex/s] preprocess datasets: 6441ex [00:33, 266.53ex/s] preprocess datasets: 6470ex [00:33, 271.90ex/s] preprocess datasets: 6499ex [00:33, 273.94ex/s] preprocess datasets: 6528ex [00:33, 272.57ex/s] preprocess datasets: 6556ex [00:33, 269.96ex/s] preprocess datasets: 6584ex [00:34, 259.44ex/s] preprocess datasets: 6611ex [00:34, 246.96ex/s] preprocess datasets: 6639ex [00:34, 253.41ex/s] preprocess datasets: 6665ex [00:34, 253.86ex/s] preprocess datasets: 6691ex [00:34, 254.97ex/s] preprocess datasets: 6717ex [00:34, 251.22ex/s] preprocess datasets: 6743ex [00:34, 250.24ex/s] preprocess datasets: 6776ex [00:34, 270.96ex/s] preprocess datasets: 6808ex [00:34, 284.91ex/s] preprocess datasets: 6843ex [00:35, 303.07ex/s] preprocess datasets: 6877ex [00:35, 313.17ex/s] preprocess datasets: 6913ex [00:35, 324.13ex/s] preprocess datasets: 6946ex [00:35, 311.10ex/s] preprocess datasets: 6978ex [00:35, 308.62ex/s] preprocess datasets: 7009ex [00:37, 46.18ex/s] preprocess datasets: 7043ex [00:37, 63.08ex/s] preprocess datasets: 7075ex [00:37, 82.57ex/s] preprocess datasets: 7103ex [00:37, 101.61ex/s] preprocess datasets: 7138ex [00:37, 131.69ex/s] preprocess datasets: 7168ex [00:38, 154.05ex/s] preprocess datasets: 7202ex [00:38, 185.32ex/s] preprocess datasets: 7234ex [00:38, 211.57ex/s] preprocess datasets: 7265ex [00:38, 230.77ex/s] preprocess datasets: 7296ex [00:38, 249.46ex/s] preprocess datasets: 7328ex [00:38, 265.93ex/s] preprocess datasets: 7360ex [00:38, 278.55ex/s] preprocess datasets: 7396ex [00:38, 298.96ex/s] preprocess datasets: 7431ex [00:38, 311.60ex/s] preprocess datasets: 7465ex [00:38, 318.25ex/s] preprocess datasets: 7501ex [00:39, 327.83ex/s] preprocess datasets: 7535ex [00:39, 321.00ex/s] preprocess datasets: 7568ex [00:39, 320.58ex/s] preprocess datasets: 7601ex [00:39, 319.38ex/s] preprocess datasets: 7634ex [00:39, 316.44ex/s] preprocess datasets: 7666ex [00:39, 311.52ex/s] preprocess datasets: 7698ex [00:39, 311.51ex/s] preprocess datasets: 7730ex [00:39, 309.54ex/s] preprocess datasets: 7762ex [00:39, 308.64ex/s] preprocess datasets: 7793ex [00:40, 306.67ex/s] preprocess datasets: 7826ex [00:40, 312.76ex/s] preprocess datasets: 7858ex [00:40, 301.64ex/s] preprocess datasets: 7889ex [00:40, 294.02ex/s] preprocess datasets: 7919ex [00:40, 287.35ex/s] preprocess datasets: 7952ex [00:40, 298.09ex/s] preprocess datasets: 7983ex [00:40, 300.50ex/s] preprocess datasets: 8014ex [00:42, 52.49ex/s] preprocess datasets: 8039ex [00:42, 65.64ex/s] preprocess datasets: 8062ex [00:42, 73.14ex/s] preprocess datasets: 8093ex [00:42, 97.26ex/s] preprocess datasets: 8123ex [00:42, 122.71ex/s] preprocess datasets: 8152ex [00:43, 148.14ex/s] preprocess datasets: 8178ex [00:43, 167.79ex/s] preprocess datasets: 8205ex [00:43, 188.27ex/s] preprocess datasets: 8237ex [00:43, 216.77ex/s] preprocess datasets: 8270ex [00:43, 242.33ex/s] preprocess datasets: 8301ex [00:43, 258.55ex/s] preprocess datasets: 8335ex [00:43, 278.28ex/s] preprocess datasets: 8369ex [00:43, 292.63ex/s] preprocess datasets: 8403ex [00:43, 304.53ex/s] preprocess datasets: 8436ex [00:43, 309.82ex/s] preprocess datasets: 8469ex [00:44, 315.52ex/s] preprocess datasets: 8502ex [00:44, 307.40ex/s] preprocess datasets: 8535ex [00:44, 313.20ex/s] preprocess datasets: 8569ex [00:44, 319.53ex/s] preprocess datasets: 8602ex [00:44, 318.68ex/s] preprocess datasets: 8637ex [00:44, 325.76ex/s] preprocess datasets: 8670ex [00:44, 326.03ex/s] preprocess datasets: 8703ex [00:44, 319.03ex/s] preprocess datasets: 8736ex [00:44, 310.26ex/s] preprocess datasets: 8768ex [00:44, 310.73ex/s] preprocess datasets: 8800ex [00:45, 304.55ex/s] preprocess datasets: 8831ex [00:45, 304.29ex/s] preprocess datasets: 8862ex [00:45, 304.28ex/s] preprocess datasets: 8893ex [00:45, 299.46ex/s] preprocess datasets: 8925ex [00:45, 303.87ex/s] preprocess datasets: 8957ex [00:45, 305.80ex/s] preprocess datasets: 8988ex [00:45, 294.20ex/s] preprocess datasets: 9018ex [00:47, 50.69ex/s] preprocess datasets: 9047ex [00:47, 66.23ex/s] preprocess datasets: 9080ex [00:47, 88.41ex/s] preprocess datasets: 9110ex [00:47, 110.77ex/s] preprocess datasets: 9139ex [00:47, 134.40ex/s] preprocess datasets: 9171ex [00:48, 163.59ex/s] preprocess datasets: 9203ex [00:48, 192.03ex/s] preprocess datasets: 9236ex [00:48, 220.46ex/s] preprocess datasets: 9267ex [00:48, 239.74ex/s] preprocess datasets: 9298ex [00:48, 256.54ex/s] preprocess datasets: 9329ex [00:48, 250.74ex/s] preprocess datasets: 9360ex [00:48, 264.99ex/s] preprocess datasets: 9393ex [00:48, 278.82ex/s] preprocess datasets: 9427ex [00:48, 292.96ex/s] preprocess datasets: 9459ex [00:48, 297.91ex/s] preprocess datasets: 9490ex [00:49, 298.10ex/s] preprocess datasets: 9521ex [00:49, 298.68ex/s] preprocess datasets: 9552ex [00:49, 295.42ex/s] preprocess datasets: 9583ex [00:49, 296.45ex/s] preprocess datasets: 9613ex [00:49, 296.23ex/s] preprocess datasets: 9644ex [00:49, 298.02ex/s] preprocess datasets: 9674ex [00:49, 289.51ex/s] preprocess datasets: 9704ex [00:49, 284.92ex/s] preprocess datasets: 9733ex [00:49, 284.90ex/s] preprocess datasets: 9763ex [00:50, 288.86ex/s] preprocess datasets: 9792ex [00:50, 288.40ex/s] preprocess datasets: 9821ex [00:50, 287.53ex/s] preprocess datasets: 9850ex [00:50, 283.62ex/s] preprocess datasets: 9880ex [00:50, 287.00ex/s] preprocess datasets: 9909ex [00:50, 285.40ex/s] preprocess datasets: 9938ex [00:50, 283.95ex/s] preprocess datasets: 9967ex [00:50, 282.40ex/s] preprocess datasets: 9996ex [00:50, 279.64ex/s] preprocess datasets: 10024ex [00:52, 45.32ex/s] preprocess datasets: 10049ex [00:52, 58.25ex/s] preprocess datasets: 10079ex [00:52, 78.23ex/s] preprocess datasets: 10107ex [00:53, 99.53ex/s] preprocess datasets: 10139ex [00:53, 128.08ex/s] preprocess datasets: 10169ex [00:53, 153.88ex/s] preprocess datasets: 10197ex [00:53, 176.77ex/s] preprocess datasets: 10225ex [00:53, 197.28ex/s] preprocess datasets: 10253ex [00:53, 215.16ex/s] preprocess datasets: 10281ex [00:53, 229.12ex/s] preprocess datasets: 10309ex [00:53, 237.01ex/s] preprocess datasets: 10338ex [00:53, 250.26ex/s] preprocess datasets: 10369ex [00:53, 265.62ex/s] preprocess datasets: 10402ex [00:54, 283.35ex/s] preprocess datasets: 10434ex [00:54, 292.86ex/s] preprocess datasets: 10469ex [00:54, 305.95ex/s] preprocess datasets: 10501ex [00:54, 309.07ex/s] preprocess datasets: 10535ex [00:54, 315.58ex/s] preprocess datasets: 10571ex [00:54, 328.08ex/s] preprocess datasets: 10605ex [00:54, 330.50ex/s] preprocess datasets: 10639ex [00:54, 330.63ex/s] preprocess datasets: 10674ex [00:54, 333.39ex/s] preprocess datasets: 10708ex [00:54, 328.04ex/s] preprocess datasets: 10743ex [00:55, 332.73ex/s] preprocess datasets: 10778ex [00:55, 336.79ex/s] preprocess datasets: 10812ex [00:55, 326.75ex/s] preprocess datasets: 10845ex [00:55, 322.82ex/s] preprocess datasets: 10878ex [00:55, 321.68ex/s] preprocess datasets: 10911ex [00:55, 322.99ex/s] preprocess datasets: 10944ex [00:55, 315.94ex/s] preprocess datasets: 10976ex [00:55, 307.28ex/s] preprocess datasets: 11007ex [00:57, 55.97ex/s] preprocess datasets: 11040ex [00:57, 74.74ex/s] preprocess datasets: 11074ex [00:57, 98.33ex/s] preprocess datasets: 11107ex [00:57, 124.25ex/s] preprocess datasets: 11138ex [00:57, 149.40ex/s] preprocess datasets: 11168ex [00:58, 173.03ex/s] preprocess datasets: 11200ex [00:58, 200.62ex/s] preprocess datasets: 11233ex [00:58, 227.46ex/s] preprocess datasets: 11272ex [00:58, 264.28ex/s] preprocess datasets: 11306ex [00:58, 282.74ex/s] preprocess datasets: 11340ex [00:58, 289.26ex/s] preprocess datasets: 11373ex [00:58, 293.03ex/s] preprocess datasets: 11405ex [00:58, 298.36ex/s] preprocess datasets: 11437ex [00:58, 301.73ex/s] preprocess datasets: 11471ex [00:58, 310.73ex/s] preprocess datasets: 11504ex [00:59, 313.18ex/s] preprocess datasets: 11539ex [00:59, 323.78ex/s] preprocess datasets: 11572ex [00:59, 324.46ex/s] preprocess datasets: 11605ex [00:59, 324.18ex/s] preprocess datasets: 11638ex [00:59, 322.25ex/s] preprocess datasets: 11671ex [00:59, 323.50ex/s] preprocess datasets: 11707ex [00:59, 333.53ex/s] preprocess datasets: 11741ex [00:59, 330.61ex/s] preprocess datasets: 11775ex [00:59, 317.48ex/s] preprocess datasets: 11807ex [00:59, 300.94ex/s] preprocess datasets: 11838ex [01:00, 268.24ex/s] preprocess datasets: 11866ex [01:00, 257.55ex/s] preprocess datasets: 11895ex [01:00, 262.52ex/s] preprocess datasets: 11922ex [01:00, 264.01ex/s] preprocess datasets: 11950ex [01:00, 265.70ex/s] preprocess datasets: 11980ex [01:00, 272.72ex/s] preprocess datasets: 12008ex [01:02, 47.10ex/s] preprocess datasets: 12031ex [01:02, 58.84ex/s] preprocess datasets: 12058ex [01:02, 76.75ex/s] preprocess datasets: 12087ex [01:02, 99.64ex/s] preprocess datasets: 12115ex [01:02, 123.14ex/s] preprocess datasets: 12141ex [01:02, 144.57ex/s] preprocess datasets: 12169ex [01:03, 169.45ex/s] preprocess datasets: 12200ex [01:03, 198.15ex/s] preprocess datasets: 12235ex [01:03, 232.59ex/s] preprocess datasets: 12265ex [01:03, 247.50ex/s] preprocess datasets: 12295ex [01:03, 248.18ex/s] preprocess datasets: 12324ex [01:03, 254.47ex/s] preprocess datasets: 12354ex [01:03, 265.78ex/s] preprocess datasets: 12385ex [01:03, 275.65ex/s] preprocess datasets: 12414ex [01:03, 269.04ex/s] preprocess datasets: 12442ex [01:04, 271.50ex/s] preprocess datasets: 12470ex [01:04, 271.76ex/s] preprocess datasets: 12502ex [01:04, 284.87ex/s] preprocess datasets: 12531ex [01:04, 282.81ex/s] preprocess datasets: 12560ex [01:04, 281.54ex/s] preprocess datasets: 12590ex [01:04, 285.61ex/s] preprocess datasets: 12619ex [01:04, 285.52ex/s] preprocess datasets: 12648ex [01:04, 284.87ex/s] preprocess datasets: 12683ex [01:04, 303.59ex/s] preprocess datasets: 12714ex [01:04, 297.64ex/s] preprocess datasets: 12744ex [01:05, 293.07ex/s] preprocess datasets: 12774ex [01:05, 291.56ex/s] preprocess datasets: 12804ex [01:05, 292.43ex/s] preprocess datasets: 12834ex [01:05, 282.24ex/s] preprocess datasets: 12863ex [01:05, 282.06ex/s] preprocess datasets: 12892ex [01:05, 283.27ex/s] preprocess datasets: 12924ex [01:05, 292.32ex/s] preprocess datasets: 12954ex [01:05, 286.20ex/s] preprocess datasets: 12984ex [01:05, 288.39ex/s] preprocess datasets: 13013ex [01:07, 45.72ex/s] preprocess datasets: 13044ex [01:07, 61.96ex/s] preprocess datasets: 13072ex [01:08, 79.55ex/s] preprocess datasets: 13098ex [01:08, 98.13ex/s] preprocess datasets: 13128ex [01:08, 123.85ex/s] preprocess datasets: 13155ex [01:08, 146.15ex/s] preprocess datasets: 13182ex [01:08, 166.63ex/s] preprocess datasets: 13209ex [01:08, 176.75ex/s] preprocess datasets: 13235ex [01:08, 193.22ex/s] preprocess datasets: 13265ex [01:08, 215.91ex/s] preprocess datasets: 13295ex [01:08, 235.54ex/s] preprocess datasets: 13325ex [01:08, 251.21ex/s] preprocess datasets: 13354ex [01:09, 260.84ex/s] preprocess datasets: 13383ex [01:09, 266.40ex/s] preprocess datasets: 13414ex [01:09, 277.64ex/s] preprocess datasets: 13443ex [01:09, 279.48ex/s] preprocess datasets: 13473ex [01:09, 284.40ex/s] preprocess datasets: 13502ex [01:09, 282.08ex/s] preprocess datasets: 13531ex [01:09, 278.02ex/s] preprocess datasets: 13560ex [01:09, 274.56ex/s] preprocess datasets: 13588ex [01:09, 272.83ex/s] preprocess datasets: 13616ex [01:10, 273.54ex/s] preprocess datasets: 13644ex [01:10, 273.06ex/s] preprocess datasets: 13673ex [01:10, 278.00ex/s] preprocess datasets: 13703ex [01:10, 282.71ex/s] preprocess datasets: 13734ex [01:10, 290.54ex/s] preprocess datasets: 13765ex [01:10, 295.71ex/s] preprocess datasets: 13796ex [01:10, 299.17ex/s] preprocess datasets: 13826ex [01:10, 292.47ex/s] preprocess datasets: 13860ex [01:10, 304.65ex/s] preprocess datasets: 13893ex [01:10, 310.45ex/s] preprocess datasets: 13925ex [01:11, 312.19ex/s] preprocess datasets: 13958ex [01:11, 317.28ex/s] preprocess datasets: 13991ex [01:11, 320.19ex/s] preprocess datasets: 14024ex [01:13, 49.64ex/s] preprocess datasets: 14051ex [01:13, 63.02ex/s] preprocess datasets: 14082ex [01:13, 82.78ex/s] preprocess datasets: 14112ex [01:13, 105.07ex/s] preprocess datasets: 14141ex [01:13, 128.49ex/s] preprocess datasets: 14170ex [01:13, 153.32ex/s] preprocess datasets: 14201ex [01:13, 180.99ex/s] preprocess datasets: 14233ex [01:13, 209.20ex/s] preprocess datasets: 14264ex [01:14, 230.81ex/s] preprocess datasets: 14300ex [01:14, 260.30ex/s] preprocess datasets: 14332ex [01:14, 272.46ex/s] preprocess datasets: 14364ex [01:14, 277.77ex/s] preprocess datasets: 14396ex [01:14, 288.54ex/s] preprocess datasets: 14427ex [01:14, 292.23ex/s] preprocess datasets: 14458ex [01:14, 285.99ex/s] preprocess datasets: 14488ex [01:14, 282.19ex/s] preprocess datasets: 14517ex [01:14, 279.45ex/s] preprocess datasets: 14546ex [01:14, 279.33ex/s] preprocess datasets: 14583ex [01:15, 303.29ex/s] preprocess datasets: 14619ex [01:15, 318.63ex/s] preprocess datasets: 14653ex [01:15, 323.43ex/s] preprocess datasets: 14686ex [01:15, 314.64ex/s] preprocess datasets: 14719ex [01:15, 317.41ex/s] preprocess datasets: 14753ex [01:15, 322.27ex/s] preprocess datasets: 14786ex [01:15, 317.82ex/s] preprocess datasets: 14818ex [01:15, 309.31ex/s] preprocess datasets: 14850ex [01:15, 305.63ex/s] preprocess datasets: 14882ex [01:16, 309.52ex/s] preprocess datasets: 14914ex [01:16, 311.02ex/s] preprocess datasets: 14948ex [01:16, 318.59ex/s] preprocess datasets: 14980ex [01:16, 316.26ex/s] preprocess datasets: 15012ex [01:18, 54.42ex/s] preprocess datasets: 15046ex [01:18, 73.47ex/s] preprocess datasets: 15074ex [01:18, 91.41ex/s] preprocess datasets: 15105ex [01:18, 115.38ex/s] preprocess datasets: 15138ex [01:18, 143.94ex/s] preprocess datasets: 15172ex [01:18, 175.53ex/s] preprocess datasets: 15203ex [01:18, 193.96ex/s] preprocess datasets: 15233ex [01:18, 206.67ex/s] preprocess datasets: 15261ex [01:18, 212.24ex/s] preprocess datasets: 15288ex [01:19, 167.23ex/s] preprocess datasets: 15313ex [01:19, 182.18ex/s] preprocess datasets: 15345ex [01:19, 211.26ex/s] preprocess datasets: 15379ex [01:19, 241.18ex/s] preprocess datasets: 15414ex [01:19, 266.29ex/s] preprocess datasets: 15445ex [01:19, 276.73ex/s] preprocess datasets: 15477ex [01:19, 287.02ex/s] preprocess datasets: 15509ex [01:19, 295.75ex/s] preprocess datasets: 15545ex [01:20, 312.73ex/s] preprocess datasets: 15582ex [01:20, 326.48ex/s] preprocess datasets: 15619ex [01:20, 338.60ex/s] preprocess datasets: 15654ex [01:20, 331.92ex/s] preprocess datasets: 15688ex [01:20, 318.62ex/s] preprocess datasets: 15721ex [01:20, 319.31ex/s] preprocess datasets: 15758ex [01:20, 331.85ex/s] preprocess datasets: 15792ex [01:20, 332.85ex/s] preprocess datasets: 15826ex [01:20, 324.52ex/s] preprocess datasets: 15859ex [01:20, 322.02ex/s] preprocess datasets: 15892ex [01:21, 315.15ex/s] preprocess datasets: 15924ex [01:21, 301.49ex/s] preprocess datasets: 15955ex [01:21, 295.13ex/s] preprocess datasets: 15985ex [01:21, 285.01ex/s] preprocess datasets: 16014ex [01:23, 49.16ex/s] preprocess datasets: 16040ex [01:23, 62.56ex/s] preprocess datasets: 16070ex [01:23, 82.10ex/s] preprocess datasets: 16097ex [01:23, 101.64ex/s] preprocess datasets: 16124ex [01:23, 123.20ex/s] preprocess datasets: 16151ex [01:23, 145.66ex/s] preprocess datasets: 16178ex [01:23, 167.61ex/s] preprocess datasets: 16204ex [01:23, 185.88ex/s] preprocess datasets: 16232ex [01:24, 206.35ex/s] preprocess datasets: 16259ex [01:24, 221.12ex/s] preprocess datasets: 16288ex [01:24, 238.53ex/s] preprocess datasets: 16316ex [01:24, 238.59ex/s] preprocess datasets: 16343ex [01:24, 245.05ex/s] preprocess datasets: 16373ex [01:24, 259.10ex/s] preprocess datasets: 16406ex [01:24, 276.47ex/s] preprocess datasets: 16437ex [01:24, 285.72ex/s] preprocess datasets: 16467ex [01:24, 282.74ex/s] preprocess datasets: 16497ex [01:25, 287.16ex/s] preprocess datasets: 16529ex [01:25, 296.02ex/s] preprocess datasets: 16560ex [01:25, 296.51ex/s] preprocess datasets: 16592ex [01:25, 302.86ex/s] preprocess datasets: 16623ex [01:25, 302.52ex/s] preprocess datasets: 16655ex [01:25, 306.74ex/s] preprocess datasets: 16686ex [01:25, 305.17ex/s] preprocess datasets: 16718ex [01:25, 309.10ex/s] preprocess datasets: 16749ex [01:25, 303.99ex/s] preprocess datasets: 16781ex [01:25, 306.58ex/s] preprocess datasets: 16813ex [01:26, 310.10ex/s] preprocess datasets: 16845ex [01:26, 302.58ex/s] preprocess datasets: 16876ex [01:26, 300.93ex/s] preprocess datasets: 16911ex [01:26, 315.14ex/s] preprocess datasets: 16943ex [01:26, 315.14ex/s] preprocess datasets: 16975ex [01:26, 292.29ex/s] preprocess datasets: 17005ex [01:28, 48.66ex/s] preprocess datasets: 17034ex [01:28, 63.59ex/s] preprocess datasets: 17064ex [01:28, 82.73ex/s] preprocess datasets: 17096ex [01:28, 107.20ex/s] preprocess datasets: 17126ex [01:28, 131.50ex/s] preprocess datasets: 17155ex [01:28, 155.68ex/s] preprocess datasets: 17186ex [01:29, 183.45ex/s] preprocess datasets: 17216ex [01:29, 206.34ex/s] preprocess datasets: 17249ex [01:29, 233.54ex/s] preprocess datasets: 17281ex [01:29, 253.45ex/s] preprocess datasets: 17312ex [01:29, 264.00ex/s] preprocess datasets: 17343ex [01:29, 275.30ex/s] preprocess datasets: 17375ex [01:29, 287.15ex/s] preprocess datasets: 17407ex [01:29, 294.31ex/s] preprocess datasets: 17440ex [01:29, 304.43ex/s] preprocess datasets: 17476ex [01:30, 320.29ex/s] preprocess datasets: 17512ex [01:30, 331.81ex/s] preprocess datasets: 17547ex [01:30, 334.78ex/s] preprocess datasets: 17581ex [01:30, 317.65ex/s] preprocess datasets: 17614ex [01:30, 297.17ex/s] preprocess datasets: 17645ex [01:30, 283.71ex/s] preprocess datasets: 17674ex [01:30, 272.41ex/s] preprocess datasets: 17702ex [01:30, 269.32ex/s] preprocess datasets: 17730ex [01:30, 265.58ex/s] preprocess datasets: 17759ex [01:31, 269.42ex/s] preprocess datasets: 17788ex [01:31, 275.09ex/s] preprocess datasets: 17816ex [01:31, 275.23ex/s] preprocess datasets: 17847ex [01:31, 284.55ex/s] preprocess datasets: 17877ex [01:31, 288.18ex/s] preprocess datasets: 17910ex [01:31, 298.60ex/s] preprocess datasets: 17942ex [01:31, 303.31ex/s] preprocess datasets: 17975ex [01:31, 309.90ex/s] preprocess datasets: 18007ex [01:33, 51.68ex/s] preprocess datasets: 18033ex [01:33, 65.15ex/s] preprocess datasets: 18057ex [01:33, 79.89ex/s] preprocess datasets: 18082ex [01:33, 98.00ex/s] preprocess datasets: 18109ex [01:33, 120.19ex/s] preprocess datasets: 18139ex [01:34, 147.70ex/s] preprocess datasets: 18165ex [01:34, 166.85ex/s] preprocess datasets: 18191ex [01:34, 184.93ex/s] preprocess datasets: 18218ex [01:34, 202.36ex/s] preprocess datasets: 18245ex [01:34, 217.13ex/s] preprocess datasets: 18281ex [01:34, 253.49ex/s] preprocess datasets: 18324ex [01:34, 300.72ex/s] preprocess datasets: 18359ex [01:34, 312.48ex/s] preprocess datasets: 18394ex [01:34, 320.36ex/s] preprocess datasets: 18428ex [01:34, 318.83ex/s] preprocess datasets: 18461ex [01:35, 298.73ex/s] preprocess datasets: 18492ex [01:35, 293.26ex/s] preprocess datasets: 18522ex [01:35, 278.66ex/s] preprocess datasets: 18552ex [01:35, 281.59ex/s] preprocess datasets: 18581ex [01:35, 276.73ex/s] preprocess datasets: 18609ex [01:35, 270.26ex/s] preprocess datasets: 18637ex [01:35, 265.17ex/s] preprocess datasets: 18664ex [01:35, 256.66ex/s] preprocess datasets: 18692ex [01:35, 261.69ex/s] preprocess datasets: 18719ex [01:36, 260.40ex/s] preprocess datasets: 18746ex [01:36, 261.57ex/s] preprocess datasets: 18773ex [01:36, 257.82ex/s] preprocess datasets: 18799ex [01:36, 254.85ex/s] preprocess datasets: 18828ex [01:36, 260.96ex/s] preprocess datasets: 18855ex [01:36, 260.98ex/s] preprocess datasets: 18883ex [01:36, 265.77ex/s] preprocess datasets: 18910ex [01:36, 256.27ex/s] preprocess datasets: 18937ex [01:36, 259.33ex/s] preprocess datasets: 18964ex [01:37, 261.09ex/s] preprocess datasets: 18993ex [01:37, 268.40ex/s] preprocess datasets: 19020ex [01:39, 41.16ex/s] preprocess datasets: 19050ex [01:39, 56.70ex/s] preprocess datasets: 19080ex [01:39, 75.93ex/s] preprocess datasets: 19111ex [01:39, 99.54ex/s] preprocess datasets: 19141ex [01:39, 124.67ex/s] preprocess datasets: 19174ex [01:39, 155.81ex/s] preprocess datasets: 19204ex [01:39, 180.80ex/s] preprocess datasets: 19234ex [01:39, 201.48ex/s] preprocess datasets: 19264ex [01:39, 222.44ex/s] preprocess datasets: 19295ex [01:40, 242.07ex/s] preprocess datasets: 19327ex [01:40, 259.34ex/s] preprocess datasets: 19357ex [01:40, 255.26ex/s] preprocess datasets: 19387ex [01:40, 266.17ex/s] preprocess datasets: 19416ex [01:40, 261.21ex/s] preprocess datasets: 19449ex [01:40, 279.18ex/s] preprocess datasets: 19484ex [01:40, 296.64ex/s] preprocess datasets: 19515ex [01:40, 275.35ex/s] preprocess datasets: 19546ex [01:40, 284.52ex/s] preprocess datasets: 19576ex [01:41, 287.28ex/s] preprocess datasets: 19610ex [01:41, 299.60ex/s] preprocess datasets: 19644ex [01:41, 310.30ex/s] preprocess datasets: 19676ex [01:41, 310.03ex/s] preprocess datasets: 19708ex [01:41, 304.07ex/s] preprocess datasets: 19739ex [01:41, 301.14ex/s] preprocess datasets: 19770ex [01:41, 297.09ex/s] preprocess datasets: 19800ex [01:41, 279.97ex/s] preprocess datasets: 19831ex [01:41, 288.19ex/s] preprocess datasets: 19861ex [01:41, 286.77ex/s] preprocess datasets: 19890ex [01:42, 286.92ex/s] preprocess datasets: 19920ex [01:42, 290.31ex/s] preprocess datasets: 19951ex [01:42, 293.92ex/s] preprocess datasets: 19981ex [01:42, 295.64ex/s] preprocess datasets: 20011ex [01:44, 47.70ex/s] preprocess datasets: 20041ex [01:44, 63.51ex/s] preprocess datasets: 20072ex [01:44, 83.87ex/s] preprocess datasets: 20102ex [01:44, 106.38ex/s] preprocess datasets: 20134ex [01:44, 133.89ex/s] preprocess datasets: 20165ex [01:44, 161.23ex/s] preprocess datasets: 20195ex [01:44, 185.99ex/s] preprocess datasets: 20227ex [01:44, 212.29ex/s] preprocess datasets: 20257ex [01:45, 231.20ex/s] preprocess datasets: 20287ex [01:45, 244.32ex/s] preprocess datasets: 20318ex [01:45, 261.04ex/s] preprocess datasets: 20349ex [01:45, 271.94ex/s] preprocess datasets: 20379ex [01:45, 279.48ex/s] preprocess datasets: 20409ex [01:45, 284.95ex/s] preprocess datasets: 20439ex [01:45, 288.62ex/s] preprocess datasets: 20469ex [01:45, 288.42ex/s] preprocess datasets: 20500ex [01:45, 291.90ex/s] preprocess datasets: 20531ex [01:46, 296.08ex/s] preprocess datasets: 20564ex [01:46, 304.48ex/s] preprocess datasets: 20595ex [01:46, 305.94ex/s] preprocess datasets: 20629ex [01:46, 312.52ex/s] preprocess datasets: 20662ex [01:46, 317.04ex/s] preprocess datasets: 20694ex [01:46, 309.67ex/s] preprocess datasets: 20727ex [01:46, 315.17ex/s] preprocess datasets: 20761ex [01:46, 319.17ex/s] preprocess datasets: 20796ex [01:46, 325.56ex/s] preprocess datasets: 20829ex [01:46, 320.66ex/s] preprocess datasets: 20862ex [01:47, 315.76ex/s] preprocess datasets: 20896ex [01:47, 321.95ex/s] preprocess datasets: 20929ex [01:47, 322.03ex/s] preprocess datasets: 20962ex [01:47, 318.80ex/s] preprocess datasets: 20994ex [01:47, 318.04ex/s] preprocess datasets: 21026ex [01:49, 53.44ex/s] preprocess datasets: 21060ex [01:49, 72.13ex/s] preprocess datasets: 21092ex [01:49, 93.17ex/s] preprocess datasets: 21125ex [01:49, 118.92ex/s] preprocess datasets: 21156ex [01:49, 143.87ex/s] preprocess datasets: 21187ex [01:49, 169.80ex/s] preprocess datasets: 21218ex [01:49, 194.78ex/s] preprocess datasets: 21248ex [01:49, 215.16ex/s] preprocess datasets: 21278ex [01:50, 231.15ex/s] preprocess datasets: 21308ex [01:50, 242.31ex/s] preprocess datasets: 21337ex [01:50, 253.36ex/s] preprocess datasets: 21366ex [01:50, 258.00ex/s] preprocess datasets: 21395ex [01:50, 265.98ex/s] preprocess datasets: 21426ex [01:50, 276.44ex/s] preprocess datasets: 21458ex [01:50, 285.19ex/s] preprocess datasets: 21488ex [01:50, 285.75ex/s] preprocess datasets: 21519ex [01:50, 290.83ex/s] preprocess datasets: 21554ex [01:51, 305.68ex/s] preprocess datasets: 21586ex [01:51, 307.38ex/s] preprocess datasets: 21617ex [01:51, 306.34ex/s] preprocess datasets: 21648ex [01:51, 281.49ex/s] preprocess datasets: 21679ex [01:51, 288.23ex/s] preprocess datasets: 21709ex [01:51, 285.05ex/s] preprocess datasets: 21741ex [01:51, 293.75ex/s] preprocess datasets: 21773ex [01:51, 300.94ex/s] preprocess datasets: 21805ex [01:51, 304.94ex/s] preprocess datasets: 21836ex [01:51, 301.58ex/s] preprocess datasets: 21869ex [01:52, 307.54ex/s] preprocess datasets: 21900ex [01:52, 307.65ex/s] preprocess datasets: 21932ex [01:52, 309.77ex/s] preprocess datasets: 21964ex [01:52, 308.80ex/s] preprocess datasets: 21997ex [01:52, 313.99ex/s] preprocess datasets: 22029ex [01:54, 51.76ex/s] preprocess datasets: 22059ex [01:54, 67.67ex/s] preprocess datasets: 22092ex [01:54, 89.71ex/s] preprocess datasets: 22126ex [01:54, 116.28ex/s] preprocess datasets: 22156ex [01:54, 140.21ex/s] preprocess datasets: 22185ex [01:54, 164.02ex/s] preprocess datasets: 22214ex [01:54, 186.88ex/s] preprocess datasets: 22243ex [01:55, 205.15ex/s] preprocess datasets: 22263ex [01:55, 193.45ex/s] preprocess datasets: 0ex [00:00, ?ex/s] preprocess datasets: 16ex [00:00, 157.36ex/s] preprocess datasets: 46ex [00:00, 233.76ex/s] preprocess datasets: 76ex [00:00, 260.10ex/s] preprocess datasets: 102ex [00:00, 259.55ex/s] preprocess datasets: 128ex [00:00, 255.09ex/s] preprocess datasets: 159ex [00:00, 273.13ex/s] preprocess datasets: 187ex [00:00, 266.33ex/s] preprocess datasets: 214ex [00:00, 259.11ex/s] preprocess datasets: 242ex [00:00, 263.55ex/s] preprocess datasets: 272ex [00:01, 272.98ex/s] preprocess datasets: 301ex [00:01, 274.92ex/s] preprocess datasets: 329ex [00:01, 269.00ex/s] preprocess datasets: 357ex [00:01, 269.94ex/s] preprocess datasets: 388ex [00:01, 280.09ex/s] preprocess datasets: 418ex [00:01, 283.82ex/s] preprocess datasets: 447ex [00:01, 280.42ex/s] preprocess datasets: 457ex [00:01, 267.52ex/s] 0%| | 0/23 [00:00 to the vocabulary Adding to the vocabulary /workspace/wav2vec2-xls-r-300m-korean/./ is already a clone of https://huggingface.co/w11wo/wav2vec2-xls-r-300m-korean. Make sure you pull the latest changes with `repo.git_pull()`. 01/31/2022 07:18:18 - WARNING - huggingface_hub.repository - /workspace/wav2vec2-xls-r-300m-korean/./ is already a clone of https://huggingface.co/w11wo/wav2vec2-xls-r-300m-korean. Make sure you pull the latest changes with `repo.git_pull()`. Using amp half precision backend The following columns in the training set don't have a corresponding argument in `Wav2Vec2ForCTC.forward` and have been ignored: input_length. /opt/conda/lib/python3.8/site-packages/transformers/optimization.py:306: FutureWarning: This implementation of AdamW is deprecated and will be removed in a future version. Use thePyTorch implementation torch.optim.AdamW instead, or set `no_deprecation_warning=True` to disable this warning warnings.warn( ***** Running training ***** Num examples = 22262 Num Epochs = 50 Instantaneous batch size per device = 8 Total train batch size (w. parallel, distributed & accumulation) = 32 Gradient Accumulation steps = 4 Total optimization steps = 34750 0%| | 0/34750 [00:00 main 02/01/2022 15:05:46 - WARNING - huggingface_hub.repository - To https://huggingface.co/w11wo/wav2vec2-xls-r-300m-korean 816ebc3..024e2da main -> main Upload file pytorch_model.bin: 100%|██████████| 1.18G/1.18G [00:53<00:00, 23.4MB/s] Upload file pytorch_model.bin: 100%|██████████| 1.18G/1.18G [00:53<00:00, 23.4MB/s] Upload file pytorch_model.bin: 100%|██████████| 1.18G/1.18G [00:53<00:00, 23.7MB/s] Upload file runs/Jan31_07-15-59_job-2c68f48a-2d5d-4013-9043-3f2cb25f3ff6/events.out.tfevents.1643613501.job-2c68f48a-2d5d-4013-9043-3f2cb25f3ff6.1151936.0: 100%|██████████| 83.5k/83.5k [00:53<00:00, 1.53kB/s] Upload file runs/Jan31_07-15-59_job-2c68f48a-2d5d-4013-9043-3f2cb25f3ff6/events.out.tfevents.1643613501.job-2c68f48a-2d5d-4013-9043-3f2cb25f3ff6.1151936.0: 100%|██████████| 83.5k/83.5k [00:53<00:00, 1.53kB/s] Upload file runs/Jan31_07-15-59_job-2c68f48a-2d5d-4013-9043-3f2cb25f3ff6/events.out.tfevents.1643613501.job-2c68f48a-2d5d-4013-9043-3f2cb25f3ff6.1151936.0: 100%|██████████| 83.5k/83.5k [00:53<00:00, 1.53kB/s] Dropping the following result as it does not have all the necessary fields: {'dataset': {'name': 'zeroth_korean_asr', 'type': 'zeroth_korean_asr', 'args': 'clean'}} To https://huggingface.co/w11wo/wav2vec2-xls-r-300m-korean 024e2da..2be5bf6 main -> main 02/01/2022 15:05:53 - WARNING - huggingface_hub.repository - To https://huggingface.co/w11wo/wav2vec2-xls-r-300m-korean 024e2da..2be5bf6 main -> main ***** train metrics ***** epoch = 50.0 train_loss = 2.2317 train_runtime = 1 day, 7:45:11.97 train_samples = 22262 train_samples_per_second = 9.737 train_steps_per_second = 0.304 02/01/2022 15:05:56 - INFO - __main__ - *** Evaluate *** The following columns in the evaluation set don't have a corresponding argument in `Wav2Vec2ForCTC.forward` and have been ignored: input_length. ***** Running Evaluation ***** Num examples = 456 Batch size = 8 0%| | 0/57 [00:00 main ***** eval metrics ***** epoch = 50.0 eval_cer = 0.0953 eval_loss = 0.2089 eval_runtime = 0:00:41.71 eval_samples = 456 eval_samples_per_second = 10.932 eval_steps_per_second = 1.367 eval_wer = 0.2954 02/01/2022 15:07:03 - WARNING - huggingface_hub.repository - To https://huggingface.co/w11wo/wav2vec2-xls-r-300m-korean 2be5bf6..b8d7f07 main -> main Dropping the following result as it does not have all the necessary fields: {'dataset': {'name': 'KRESNIK/ZEROTH_KOREAN - CLEAN', 'type': 'zeroth_korean_asr', 'args': 'Config: clean, Training split: train, Eval split: test'}}