| /opt/conda/lib/python3.12/site-packages/transformers/training_args.py:1474: FutureWarning: `evaluation_strategy` is deprecated and will be removed in version 4.46 of 🤗 Transformers. Use `eval_strategy` instead | |
| warnings.warn( | |
| 05/19/2024 22:08:09 - WARNING - __main__ - Process rank: 0, device: cuda:0, n_gpu: 1, distributed training: False, 16-bits training: True | |
| 05/19/2024 22:08:09 - INFO - __main__ - Training/evaluation parameters TrainingArguments( | |
| _n_gpu=1, | |
| accelerator_config={'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}, | |
| adafactor=False, | |
| adam_beta1=0.9, | |
| adam_beta2=0.999, | |
| adam_epsilon=1e-08, | |
| auto_find_batch_size=False, | |
| batch_eval_metrics=False, | |
| bf16=False, | |
| bf16_full_eval=False, | |
| data_seed=None, | |
| dataloader_drop_last=False, | |
| dataloader_num_workers=0, | |
| dataloader_persistent_workers=False, | |
| dataloader_pin_memory=True, | |
| dataloader_prefetch_factor=None, | |
| ddp_backend=None, | |
| ddp_broadcast_buffers=None, | |
| ddp_bucket_cap_mb=None, | |
| ddp_find_unused_parameters=None, | |
| ddp_timeout=1800, | |
| debug=[], | |
| deepspeed=None, | |
| disable_tqdm=False, | |
| dispatch_batches=None, | |
| do_eval=True, | |
| do_predict=False, | |
| do_train=True, | |
| eval_accumulation_steps=None, | |
| eval_delay=0, | |
| eval_do_concat_batches=True, | |
| eval_steps=100, | |
| eval_strategy=IntervalStrategy.STEPS, | |
| evaluation_strategy=steps, | |
| fp16=True, | |
| fp16_backend=auto, | |
| fp16_full_eval=False, | |
| fp16_opt_level=O1, | |
| fsdp=[], | |
| fsdp_config={'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}, | |
| fsdp_min_num_params=0, | |
| fsdp_transformer_layer_cls_to_wrap=None, | |
| full_determinism=False, | |
| gradient_accumulation_steps=1, | |
| gradient_checkpointing=False, | |
| gradient_checkpointing_kwargs=None, | |
| greater_is_better=None, | |
| group_by_length=True, | |
| half_precision_backend=auto, | |
| hub_always_push=False, | |
| hub_model_id=None, | |
| hub_private_repo=False, | |
| hub_strategy=HubStrategy.EVERY_SAVE, | |
| hub_token=<HUB_TOKEN>, | |
| ignore_data_skip=False, | |
| include_inputs_for_metrics=False, | |
| include_num_input_tokens_seen=False, | |
| include_tokens_per_second=False, | |
| jit_mode_eval=False, | |
| label_names=None, | |
| label_smoothing_factor=0.0, | |
| learning_rate=0.0001, | |
| length_column_name=length, | |
| load_best_model_at_end=False, | |
| local_rank=0, | |
| log_level=passive, | |
| log_level_replica=warning, | |
| log_on_each_node=True, | |
| logging_dir=./wav2vec2-base-timit-fine-tuned/runs/May19_22-08-09_tz579-raptorlake, | |
| logging_first_step=False, | |
| logging_nan_inf_filter=True, | |
| logging_steps=10, | |
| logging_strategy=IntervalStrategy.STEPS, | |
| lr_scheduler_kwargs={}, | |
| lr_scheduler_type=SchedulerType.LINEAR, | |
| max_grad_norm=1.0, | |
| max_steps=-1, | |
| metric_for_best_model=None, | |
| mp_parameters=, | |
| neftune_noise_alpha=None, | |
| no_cuda=False, | |
| num_train_epochs=20.0, | |
| optim=OptimizerNames.ADAMW_TORCH, | |
| optim_args=None, | |
| optim_target_modules=None, | |
| output_dir=./wav2vec2-base-timit-fine-tuned, | |
| overwrite_output_dir=True, | |
| past_index=-1, | |
| per_device_eval_batch_size=1, | |
| per_device_train_batch_size=32, | |
| prediction_loss_only=False, | |
| push_to_hub=True, | |
| push_to_hub_model_id=None, | |
| push_to_hub_organization=None, | |
| push_to_hub_token=<PUSH_TO_HUB_TOKEN>, | |
| ray_scope=last, | |
| remove_unused_columns=True, | |
| report_to=['tensorboard'], | |
| restore_callback_states_from_checkpoint=False, | |
| resume_from_checkpoint=None, | |
| run_name=./wav2vec2-base-timit-fine-tuned, | |
| save_on_each_node=False, | |
| save_only_model=False, | |
| save_safetensors=True, | |
| save_steps=400, | |
| save_strategy=IntervalStrategy.STEPS, | |
| save_total_limit=3, | |
| seed=42, | |
| skip_memory_metrics=True, | |
| split_batches=None, | |
| tf32=None, | |
| torch_compile=False, | |
| torch_compile_backend=None, | |
| torch_compile_mode=None, | |
| torchdynamo=None, | |
| tpu_metrics_debug=False, | |
| tpu_num_cores=None, | |
| use_cpu=False, | |
| use_ipex=False, | |
| use_legacy_prediction_loop=False, | |
| use_mps_device=False, | |
| warmup_ratio=0.0, | |
| warmup_steps=1000, | |
| weight_decay=0.005, | |
| ) | |
| /opt/conda/lib/python3.12/site-packages/datasets/load.py:1486: FutureWarning: The repository for timit_asr contains custom code which must be executed to correctly load the dataset. You can inspect the repository content at https://hf.co/datasets/timit_asr | |
| You can avoid this message in future by passing the argument `trust_remote_code=True`. | |
| Passing `trust_remote_code=True` will be mandatory to load this dataset from the next major release of `datasets`. | |
| warnings.warn( | |
| /opt/conda/lib/python3.12/site-packages/huggingface_hub/file_download.py:1132: FutureWarning: `resume_download` is deprecated and will be removed in version 1.0.0. Downloads always resume when possible. If you want to force a new download, use `force_download=True`. | |
| warnings.warn( | |
| loading configuration file config.json from cache at /root/.cache/huggingface/hub/models--facebook--wav2vec2-base/snapshots/0b5b8e868dd84f03fd87d01f9c4ff0f080fecfe8/config.json | |
| /opt/conda/lib/python3.12/site-packages/transformers/configuration_utils.py:364: UserWarning: Passing `gradient_checkpointing` to a config initialization is deprecated and will be removed in v5 Transformers. Using `model.gradient_checkpointing_enable()` instead, or if you are using the `Trainer` API, pass `gradient_checkpointing=True` in your `TrainingArguments`. | |
| warnings.warn( | |
| Model config Wav2Vec2Config { | |
| "_name_or_path": "facebook/wav2vec2-base", | |
| "activation_dropout": 0.0, | |
| "adapter_attn_dim": null, | |
| "adapter_kernel_size": 3, | |
| "adapter_stride": 2, | |
| "add_adapter": false, | |
| "apply_spec_augment": true, | |
| "architectures": [ | |
| "Wav2Vec2ForPreTraining" | |
| ], | |
| "attention_dropout": 0.1, | |
| "bos_token_id": 1, | |
| "classifier_proj_size": 256, | |
| "codevector_dim": 256, | |
| "contrastive_logits_temperature": 0.1, | |
| "conv_bias": false, | |
| "conv_dim": [ | |
| 512, | |
| 512, | |
| 512, | |
| 512, | |
| 512, | |
| 512, | |
| 512 | |
| ], | |
| "conv_kernel": [ | |
| 10, | |
| 3, | |
| 3, | |
| 3, | |
| 3, | |
| 2, | |
| 2 | |
| ], | |
| "conv_stride": [ | |
| 5, | |
| 2, | |
| 2, | |
| 2, | |
| 2, | |
| 2, | |
| 2 | |
| ], | |
| "ctc_loss_reduction": "sum", | |
| "ctc_zero_infinity": false, | |
| "diversity_loss_weight": 0.1, | |
| "do_stable_layer_norm": false, | |
| "eos_token_id": 2, | |
| "feat_extract_activation": "gelu", | |
| "feat_extract_norm": "group", | |
| "feat_proj_dropout": 0.1, | |
| "feat_quantizer_dropout": 0.0, | |
| "final_dropout": 0.0, | |
| "freeze_feat_extract_train": true, | |
| "gradient_checkpointing": true, | |
| "hidden_act": "gelu", | |
| "hidden_dropout": 0.1, | |
| "hidden_size": 768, | |
| "initializer_range": 0.02, | |
| "intermediate_size": 3072, | |
| "layer_norm_eps": 1e-05, | |
| "layerdrop": 0.0, | |
| "mask_channel_length": 10, | |
| "mask_channel_min_space": 1, | |
| "mask_channel_other": 0.0, | |
| "mask_channel_prob": 0.0, | |
| "mask_channel_selection": "static", | |
| "mask_feature_length": 10, | |
| "mask_feature_min_masks": 0, | |
| "mask_feature_prob": 0.0, | |
| "mask_time_length": 10, | |
| "mask_time_min_masks": 2, | |
| "mask_time_min_space": 1, | |
| "mask_time_other": 0.0, | |
| "mask_time_prob": 0.05, | |
| "mask_time_selection": "static", | |
| "model_type": "wav2vec2", | |
| "no_mask_channel_overlap": false, | |
| "no_mask_time_overlap": false, | |
| "num_adapter_layers": 3, | |
| "num_attention_heads": 12, | |
| "num_codevector_groups": 2, | |
| "num_codevectors_per_group": 320, | |
| "num_conv_pos_embedding_groups": 16, | |
| "num_conv_pos_embeddings": 128, | |
| "num_feat_extract_layers": 7, | |
| "num_hidden_layers": 12, | |
| "num_negatives": 100, | |
| "output_hidden_size": 768, | |
| "pad_token_id": 0, | |
| "proj_codevector_dim": 256, | |
| "tdnn_dilation": [ | |
| 1, | |
| 2, | |
| 3, | |
| 1, | |
| 1 | |
| ], | |
| "tdnn_dim": [ | |
| 512, | |
| 512, | |
| 512, | |
| 512, | |
| 1500 | |
| ], | |
| "tdnn_kernel": [ | |
| 5, | |
| 3, | |
| 3, | |
| 1, | |
| 1 | |
| ], | |
| "transformers_version": "4.42.0.dev0", | |
| "use_weighted_layer_sum": false, | |
| "vocab_size": 32, | |
| "xvector_output_dim": 512 | |
| } | |
| Map: 100%|███████████████████████████████████████████████████████████████████████████████| 3696/3696 [00:00<00:00, 258999.36 examples/s] | |
| Map: 100%|███████████████████████████████████████████████████████████████████████████████| 1344/1344 [00:00<00:00, 582229.35 examples/s] | |
| `use_fast` is set to `True` but the tokenizer class does not have a fast version. Falling back to the slow version. | |
| loading file vocab.json | |
| loading file tokenizer_config.json | |
| loading file added_tokens.json | |
| loading file special_tokens_map.json | |
| loading file tokenizer.json | |
| Special tokens have been added in the vocabulary, make sure the associated word embeddings are fine-tuned or trained. | |
| loading configuration file preprocessor_config.json from cache at /root/.cache/huggingface/hub/models--facebook--wav2vec2-base/snapshots/0b5b8e868dd84f03fd87d01f9c4ff0f080fecfe8/preprocessor_config.json | |
| loading configuration file config.json from cache at /root/.cache/huggingface/hub/models--facebook--wav2vec2-base/snapshots/0b5b8e868dd84f03fd87d01f9c4ff0f080fecfe8/config.json | |
| Model config Wav2Vec2Config { | |
| "_name_or_path": "facebook/wav2vec2-base", | |
| "activation_dropout": 0.0, | |
| "adapter_attn_dim": null, | |
| "adapter_kernel_size": 3, | |
| "adapter_stride": 2, | |
| "add_adapter": false, | |
| "apply_spec_augment": true, | |
| "architectures": [ | |
| "Wav2Vec2ForPreTraining" | |
| ], | |
| "attention_dropout": 0.1, | |
| "bos_token_id": 1, | |
| "classifier_proj_size": 256, | |
| "codevector_dim": 256, | |
| "contrastive_logits_temperature": 0.1, | |
| "conv_bias": false, | |
| "conv_dim": [ | |
| 512, | |
| 512, | |
| 512, | |
| 512, | |
| 512, | |
| 512, | |
| 512 | |
| ], | |
| "conv_kernel": [ | |
| 10, | |
| 3, | |
| 3, | |
| 3, | |
| 3, | |
| 2, | |
| 2 | |
| ], | |
| "conv_stride": [ | |
| 5, | |
| 2, | |
| 2, | |
| 2, | |
| 2, | |
| 2, | |
| 2 | |
| ], | |
| "ctc_loss_reduction": "sum", | |
| "ctc_zero_infinity": false, | |
| "diversity_loss_weight": 0.1, | |
| "do_stable_layer_norm": false, | |
| "eos_token_id": 2, | |
| "feat_extract_activation": "gelu", | |
| "feat_extract_norm": "group", | |
| "feat_proj_dropout": 0.1, | |
| "feat_quantizer_dropout": 0.0, | |
| "final_dropout": 0.0, | |
| "freeze_feat_extract_train": true, | |
| "gradient_checkpointing": true, | |
| "hidden_act": "gelu", | |
| "hidden_dropout": 0.1, | |
| "hidden_size": 768, | |
| "initializer_range": 0.02, | |
| "intermediate_size": 3072, | |
| "layer_norm_eps": 1e-05, | |
| "layerdrop": 0.0, | |
| "mask_channel_length": 10, | |
| "mask_channel_min_space": 1, | |
| "mask_channel_other": 0.0, | |
| "mask_channel_prob": 0.0, | |
| "mask_channel_selection": "static", | |
| "mask_feature_length": 10, | |
| "mask_feature_min_masks": 0, | |
| "mask_feature_prob": 0.0, | |
| "mask_time_length": 10, | |
| "mask_time_min_masks": 2, | |
| "mask_time_min_space": 1, | |
| "mask_time_other": 0.0, | |
| "mask_time_prob": 0.05, | |
| "mask_time_selection": "static", | |
| "model_type": "wav2vec2", | |
| "no_mask_channel_overlap": false, | |
| "no_mask_time_overlap": false, | |
| "num_adapter_layers": 3, | |
| "num_attention_heads": 12, | |
| "num_codevector_groups": 2, | |
| "num_codevectors_per_group": 320, | |
| "num_conv_pos_embedding_groups": 16, | |
| "num_conv_pos_embeddings": 128, | |
| "num_feat_extract_layers": 7, | |
| "num_hidden_layers": 12, | |
| "num_negatives": 100, | |
| "output_hidden_size": 768, | |
| "pad_token_id": 0, | |
| "proj_codevector_dim": 256, | |
| "tdnn_dilation": [ | |
| 1, | |
| 2, | |
| 3, | |
| 1, | |
| 1 | |
| ], | |
| "tdnn_dim": [ | |
| 512, | |
| 512, | |
| 512, | |
| 512, | |
| 1500 | |
| ], | |
| "tdnn_kernel": [ | |
| 5, | |
| 3, | |
| 3, | |
| 1, | |
| 1 | |
| ], | |
| "transformers_version": "4.42.0.dev0", | |
| "use_weighted_layer_sum": false, | |
| "vocab_size": 32, | |
| "xvector_output_dim": 512 | |
| } | |
| Feature extractor Wav2Vec2FeatureExtractor { | |
| "do_normalize": true, | |
| "feature_extractor_type": "Wav2Vec2FeatureExtractor", | |
| "feature_size": 1, | |
| "padding_side": "right", | |
| "padding_value": 0.0, | |
| "return_attention_mask": false, | |
| "sampling_rate": 16000 | |
| } | |
| loading weights file pytorch_model.bin from cache at /root/.cache/huggingface/hub/models--facebook--wav2vec2-base/snapshots/0b5b8e868dd84f03fd87d01f9c4ff0f080fecfe8/pytorch_model.bin | |
| Some weights of the model checkpoint at facebook/wav2vec2-base were not used when initializing Wav2Vec2ForCTC: ['project_hid.bias', 'project_hid.weight', 'project_q.bias', 'project_q.weight', 'quantizer.codevectors', 'quantizer.weight_proj.bias', 'quantizer.weight_proj.weight', 'wav2vec2.encoder.pos_conv_embed.conv.weight_g', 'wav2vec2.encoder.pos_conv_embed.conv.weight_v'] | |
| - This IS expected if you are initializing Wav2Vec2ForCTC from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model). | |
| - This IS NOT expected if you are initializing Wav2Vec2ForCTC from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model). | |
| Some weights of Wav2Vec2ForCTC were not initialized from the model checkpoint at facebook/wav2vec2-base and are newly initialized: ['lm_head.bias', 'lm_head.weight', 'wav2vec2.encoder.pos_conv_embed.conv.parametrizations.weight.original0', 'wav2vec2.encoder.pos_conv_embed.conv.parametrizations.weight.original1'] | |
| You should probably TRAIN this model on a down-stream task to be able to use it for predictions and inference. | |
| Feature extractor saved in ./wav2vec2-base-timit-fine-tuned/preprocessor_config.json | |
| tokenizer config file saved in ./wav2vec2-base-timit-fine-tuned/tokenizer_config.json | |
| Special tokens file saved in ./wav2vec2-base-timit-fine-tuned/special_tokens_map.json | |
| added tokens file saved in ./wav2vec2-base-timit-fine-tuned/added_tokens.json | |
| Configuration saved in ./wav2vec2-base-timit-fine-tuned/config.json | |
| loading configuration file ./wav2vec2-base-timit-fine-tuned/preprocessor_config.json | |
| loading configuration file ./wav2vec2-base-timit-fine-tuned/preprocessor_config.json | |
| loading configuration file ./wav2vec2-base-timit-fine-tuned/config.json | |
| Model config Wav2Vec2Config { | |
| "_name_or_path": "./wav2vec2-base-timit-fine-tuned", | |
| "activation_dropout": 0.0, | |
| "adapter_attn_dim": null, | |
| "adapter_kernel_size": 3, | |
| "adapter_stride": 2, | |
| "add_adapter": false, | |
| "apply_spec_augment": true, | |
| "architectures": [ | |
| "Wav2Vec2ForPreTraining" | |
| ], | |
| "attention_dropout": 0.0, | |
| "bos_token_id": 1, | |
| "classifier_proj_size": 256, | |
| "codevector_dim": 256, | |
| "contrastive_logits_temperature": 0.1, | |
| "conv_bias": false, | |
| "conv_dim": [ | |
| 512, | |
| 512, | |
| 512, | |
| 512, | |
| 512, | |
| 512, | |
| 512 | |
| ], | |
| "conv_kernel": [ | |
| 10, | |
| 3, | |
| 3, | |
| 3, | |
| 3, | |
| 2, | |
| 2 | |
| ], | |
| "conv_stride": [ | |
| 5, | |
| 2, | |
| 2, | |
| 2, | |
| 2, | |
| 2, | |
| 2 | |
| ], | |
| "ctc_loss_reduction": "mean", | |
| "ctc_zero_infinity": false, | |
| "diversity_loss_weight": 0.1, | |
| "do_stable_layer_norm": false, | |
| "eos_token_id": 2, | |
| "feat_extract_activation": "gelu", | |
| "feat_extract_norm": "group", | |
| "feat_proj_dropout": 0.0, | |
| "feat_quantizer_dropout": 0.0, | |
| "final_dropout": 0.0, | |
| "freeze_feat_extract_train": true, | |
| "gradient_checkpointing": false, | |
| "hidden_act": "gelu", | |
| "hidden_dropout": 0.0, | |
| "hidden_size": 768, | |
| "initializer_range": 0.02, | |
| "intermediate_size": 3072, | |
| "layer_norm_eps": 1e-05, | |
| "layerdrop": 0.0, | |
| "mask_channel_length": 10, | |
| "mask_channel_min_space": 1, | |
| "mask_channel_other": 0.0, | |
| "mask_channel_prob": 0.0, | |
| "mask_channel_selection": "static", | |
| "mask_feature_length": 10, | |
| "mask_feature_min_masks": 0, | |
| "mask_feature_prob": 0.0, | |
| "mask_time_length": 10, | |
| "mask_time_min_masks": 2, | |
| "mask_time_min_space": 1, | |
| "mask_time_other": 0.0, | |
| "mask_time_prob": 0.05, | |
| "mask_time_selection": "static", | |
| "model_type": "wav2vec2", | |
| "no_mask_channel_overlap": false, | |
| "no_mask_time_overlap": false, | |
| "num_adapter_layers": 3, | |
| "num_attention_heads": 12, | |
| "num_codevector_groups": 2, | |
| "num_codevectors_per_group": 320, | |
| "num_conv_pos_embedding_groups": 16, | |
| "num_conv_pos_embeddings": 128, | |
| "num_feat_extract_layers": 7, | |
| "num_hidden_layers": 12, | |
| "num_negatives": 100, | |
| "output_hidden_size": 768, | |
| "pad_token_id": 28, | |
| "proj_codevector_dim": 256, | |
| "tdnn_dilation": [ | |
| 1, | |
| 2, | |
| 3, | |
| 1, | |
| 1 | |
| ], | |
| "tdnn_dim": [ | |
| 512, | |
| 512, | |
| 512, | |
| 512, | |
| 1500 | |
| ], | |
| "tdnn_kernel": [ | |
| 5, | |
| 3, | |
| 3, | |
| 1, | |
| 1 | |
| ], | |
| "transformers_version": "4.42.0.dev0", | |
| "use_weighted_layer_sum": false, | |
| "vocab_size": 31, | |
| "xvector_output_dim": 512 | |
| } | |
| loading configuration file ./wav2vec2-base-timit-fine-tuned/preprocessor_config.json | |
| Feature extractor Wav2Vec2FeatureExtractor { | |
| "do_normalize": true, | |
| "feature_extractor_type": "Wav2Vec2FeatureExtractor", | |
| "feature_size": 1, | |
| "padding_side": "right", | |
| "padding_value": 0.0, | |
| "return_attention_mask": false, | |
| "sampling_rate": 16000 | |
| } | |
| loading file vocab.json | |
| loading file tokenizer_config.json | |
| loading file added_tokens.json | |
| loading file special_tokens_map.json | |
| loading file tokenizer.json | |
| Special tokens have been added in the vocabulary, make sure the associated word embeddings are fine-tuned or trained. | |
| Processor Wav2Vec2Processor: | |
| - feature_extractor: Wav2Vec2FeatureExtractor { | |
| "do_normalize": true, | |
| "feature_extractor_type": "Wav2Vec2FeatureExtractor", | |
| "feature_size": 1, | |
| "padding_side": "right", | |
| "padding_value": 0.0, | |
| "return_attention_mask": false, | |
| "sampling_rate": 16000 | |
| } | |
| - tokenizer: Wav2Vec2CTCTokenizer(name_or_path='./wav2vec2-base-timit-fine-tuned', vocab_size=29, model_max_length=1000000000000000019884624838656, is_fast=False, padding_side='right', truncation_side='right', special_tokens={'bos_token': '<s>', 'eos_token': '</s>', 'unk_token': '[UNK]', 'pad_token': '[PAD]'}, clean_up_tokenization_spaces=True), added_tokens_decoder={ | |
| 27: AddedToken("[UNK]", rstrip=True, lstrip=True, single_word=False, normalized=False, special=False), | |
| 28: AddedToken("[PAD]", rstrip=True, lstrip=True, single_word=False, normalized=False, special=False), | |
| 29: AddedToken("<s>", rstrip=False, lstrip=False, single_word=False, normalized=False, special=True), | |
| 30: AddedToken("</s>", rstrip=False, lstrip=False, single_word=False, normalized=False, special=True), | |
| } | |
| { | |
| "processor_class": "Wav2Vec2Processor" | |
| } | |
| Using auto half precision backend | |
| The following columns in the training set don't have a corresponding argument in `Wav2Vec2ForCTC.forward` and have been ignored: input_length. If input_length are not expected by `Wav2Vec2ForCTC.forward`, you can safely ignore this message. | |
| ***** Running training ***** | |
| Num examples = 3,696 | |
| Num Epochs = 20 | |
| Instantaneous batch size per device = 32 | |
| Total train batch size (w. parallel, distributed & accumulation) = 32 | |
| Gradient Accumulation steps = 1 | |
| Total optimization steps = 2,320 | |
| Number of trainable parameters = 90,195,103 | |
| 0%|▎ | 7/2320 [00:10<48:36, 1.26s/it]/opt/conda/lib/python3.12/site-packages/torch/nn/modules/conv.py:306: UserWarning: Plan failed with a cudnnException: CUDNN_BACKEND_EXECUTION_PLAN_DESCRIPTOR: cudnnFinalize Descriptor Failed cudnn_status: CUDNN_STATUS_NOT_SUPPORTED (Triggered internally at /home/conda/feedstock_root/build_artifacts/libtorch_1715567101190/work/aten/src/ATen/native/cudnn/Conv_v8.cpp:919.) | |
| return F.conv1d(input, weight, bias, self.stride, | |
| {'loss': 9.1142, 'grad_norm': 9.595185279846191, 'learning_rate': 9e-07, 'epoch': 0.09} | |
| {'loss': 8.3446, 'grad_norm': 9.732986450195312, 'learning_rate': 1.9e-06, 'epoch': 0.17} | |
| {'loss': 8.6592, 'grad_norm': 14.272214889526367, 'learning_rate': 2.8000000000000003e-06, 'epoch': 0.26} | |
| {'loss': 7.6985, 'grad_norm': 15.0160493850708, 'learning_rate': 3.8e-06, 'epoch': 0.34} | |
| {'loss': 6.9688, 'grad_norm': 16.610979080200195, 'learning_rate': 4.800000000000001e-06, 'epoch': 0.43} | |
| {'loss': 6.232, 'grad_norm': 17.26924705505371, 'learning_rate': 5.8e-06, 'epoch': 0.52} | |
| {'loss': 4.7271, 'grad_norm': 11.347734451293945, 'learning_rate': 6.800000000000001e-06, 'epoch': 0.6} | |
| {'loss': 3.7919, 'grad_norm': 4.237112045288086, 'learning_rate': 7.8e-06, 'epoch': 0.69} | |
| {'loss': 3.3967, 'grad_norm': 1.8833028078079224, 'learning_rate': 8.8e-06, 'epoch': 0.78} | |
| {'loss': 3.1618, 'grad_norm': 1.3788093328475952, 'learning_rate': 9.800000000000001e-06, 'epoch': 0.86} | |
| 4%|████▏ | 100/2320 [01:39<33:07, 1.12it/s]The following columns in the evaluation set don't have a corresponding argument in `Wav2Vec2ForCTC.forward` and have been ignored: input_length. If input_length are not expected by `Wav2Vec2ForCTC.forward`, you can safely ignore this message. | |
| ***** Running Evaluation ***** | |
| Num examples = 1344 | |
| Batch size = 1 | |
| {'eval_loss': 3.1117007732391357, 'eval_wer': 1.0, 'eval_runtime': 40.0512, 'eval_samples_per_second': 33.557, 'eval_steps_per_second': 33.557, 'epoch': 0.86} | |
| {'loss': 3.0865, 'grad_norm': 1.729278802871704, 'learning_rate': 1.08e-05, 'epoch': 0.95} | |
| {'loss': 3.0809, 'grad_norm': 1.905969500541687, 'learning_rate': 1.18e-05, 'epoch': 1.03} | |
| {'loss': 3.0346, 'grad_norm': 0.8360918760299683, 'learning_rate': 1.2800000000000001e-05, 'epoch': 1.12} | |
| {'loss': 3.0106, 'grad_norm': 0.7653716206550598, 'learning_rate': 1.3800000000000002e-05, 'epoch': 1.21} | |
| {'loss': 3.0165, 'grad_norm': 0.94779372215271, 'learning_rate': 1.48e-05, 'epoch': 1.29} | |
| {'loss': 3.0, 'grad_norm': 0.8457741737365723, 'learning_rate': 1.58e-05, 'epoch': 1.38} | |
| {'loss': 2.9903, 'grad_norm': 1.4369837045669556, 'learning_rate': 1.6800000000000002e-05, 'epoch': 1.47} | |
| {'loss': 2.9852, 'grad_norm': 1.8290436267852783, 'learning_rate': 1.78e-05, 'epoch': 1.55} | |
| {'loss': 2.99, 'grad_norm': 1.1530190706253052, 'learning_rate': 1.88e-05, 'epoch': 1.64} | |
| {'loss': 2.9798, 'grad_norm': 1.1261711120605469, 'learning_rate': 1.9800000000000004e-05, 'epoch': 1.72} | |
| 9%|████████▎ | 200/2320 [03:52<24:28, 1.44it/s]The following columns in the evaluation set don't have a corresponding argument in `Wav2Vec2ForCTC.forward` and have been ignored: input_length. If input_length are not expected by `Wav2Vec2ForCTC.forward`, you can safely ignore this message. | |
| ***** Running Evaluation ***** | |
| Num examples = 1344 | |
| Batch size = 1 | |
| {'eval_loss': 2.9736363887786865, 'eval_wer': 1.0, 'eval_runtime': 39.6236, 'eval_samples_per_second': 33.919, 'eval_steps_per_second': 33.919, 'epoch': 1.72} | |
| {'loss': 2.9718, 'grad_norm': 0.903380811214447, 'learning_rate': 2.08e-05, 'epoch': 1.81} | |
| {'loss': 2.9766, 'grad_norm': 0.4889620244503021, 'learning_rate': 2.18e-05, 'epoch': 1.9} | |
| {'loss': 2.9658, 'grad_norm': 1.3861790895462036, 'learning_rate': 2.2800000000000002e-05, 'epoch': 1.98} | |
| {'loss': 2.9588, 'grad_norm': 0.7976490259170532, 'learning_rate': 2.38e-05, 'epoch': 2.07} | |
| {'loss': 2.9523, 'grad_norm': 0.698798418045044, 'learning_rate': 2.48e-05, 'epoch': 2.16} | |
| {'loss': 2.9496, 'grad_norm': 1.0858148336410522, 'learning_rate': 2.58e-05, 'epoch': 2.24} | |
| {'loss': 2.9421, 'grad_norm': 0.5658290386199951, 'learning_rate': 2.6800000000000004e-05, 'epoch': 2.33} | |
| {'loss': 2.9427, 'grad_norm': 0.5713534355163574, 'learning_rate': 2.7800000000000005e-05, 'epoch': 2.41} | |
| {'loss': 2.9228, 'grad_norm': 0.7386118769645691, 'learning_rate': 2.88e-05, 'epoch': 2.5} | |
| {'loss': 2.9144, 'grad_norm': 0.767816960811615, 'learning_rate': 2.98e-05, 'epoch': 2.59} | |
| 13%|████████████▍ | 300/2320 [06:10<33:46, 1.00s/it]The following columns in the evaluation set don't have a corresponding argument in `Wav2Vec2ForCTC.forward` and have been ignored: input_length. If input_length are not expected by `Wav2Vec2ForCTC.forward`, you can safely ignore this message. | |
| ***** Running Evaluation ***** | |
| Num examples = 1344 | |
| Batch size = 1 | |
| {'eval_loss': 2.9074809551239014, 'eval_wer': 1.0, 'eval_runtime': 39.8997, 'eval_samples_per_second': 33.684, 'eval_steps_per_second': 33.684, 'epoch': 2.59} | |
| {'loss': 2.8965, 'grad_norm': 0.8676608204841614, 'learning_rate': 3.08e-05, 'epoch': 2.67} | |
| {'loss': 2.8815, 'grad_norm': 1.6954621076583862, 'learning_rate': 3.18e-05, 'epoch': 2.76} | |
| {'loss': 2.855, 'grad_norm': 1.1631884574890137, 'learning_rate': 3.2800000000000004e-05, 'epoch': 2.84} | |
| {'loss': 2.781, 'grad_norm': 1.625454306602478, 'learning_rate': 3.38e-05, 'epoch': 2.93} | |
| {'loss': 2.7756, 'grad_norm': 2.0763564109802246, 'learning_rate': 3.48e-05, 'epoch': 3.02} | |
| {'loss': 2.6458, 'grad_norm': 2.036031723022461, 'learning_rate': 3.58e-05, 'epoch': 3.1} | |
| {'loss': 2.5189, 'grad_norm': 1.366801142692566, 'learning_rate': 3.68e-05, 'epoch': 3.19} | |
| {'loss': 2.433, 'grad_norm': 2.034527540206909, 'learning_rate': 3.7800000000000004e-05, 'epoch': 3.28} | |
| {'loss': 2.2885, 'grad_norm': 3.8338165283203125, 'learning_rate': 3.88e-05, 'epoch': 3.36} | |
| {'loss': 2.1714, 'grad_norm': 2.3443217277526855, 'learning_rate': 3.9800000000000005e-05, 'epoch': 3.45} | |
| 17%|████████████████▌ | 400/2320 [08:24<23:08, 1.38it/s]The following columns in the evaluation set don't have a corresponding argument in `Wav2Vec2ForCTC.forward` and have been ignored: input_length. If input_length are not expected by `Wav2Vec2ForCTC.forward`, you can safely ignore this message. | |
| ***** Running Evaluation ***** | |
| Num examples = 1344 | |
| Batch size = 1 | |
| {'eval_loss': 2.0944502353668213, 'eval_wer': 1.0325047801147227, 'eval_runtime': 39.7668, 'eval_samples_per_second': 33.797, 'eval_steps_per_second': 33.797, 'epoch': 3.45} | |
| 17%|████████████████▌ | 400/2320 [09:04<23:08, 1.38it/sSaving model checkpoint to ./wav2vec2-base-timit-fine-tuned/checkpoint-400 | |
| Configuration saved in ./wav2vec2-base-timit-fine-tuned/checkpoint-400/config.json | |
| Model weights saved in ./wav2vec2-base-timit-fine-tuned/checkpoint-400/model.safetensors | |
| Feature extractor saved in ./wav2vec2-base-timit-fine-tuned/checkpoint-400/preprocessor_config.json | |
| tokenizer config file saved in ./wav2vec2-base-timit-fine-tuned/checkpoint-400/tokenizer_config.json | |
| Special tokens file saved in ./wav2vec2-base-timit-fine-tuned/checkpoint-400/special_tokens_map.json | |
| added tokens file saved in ./wav2vec2-base-timit-fine-tuned/checkpoint-400/added_tokens.json | |
| Feature extractor saved in ./wav2vec2-base-timit-fine-tuned/preprocessor_config.json | |
| tokenizer config file saved in ./wav2vec2-base-timit-fine-tuned/tokenizer_config.json | |
| Special tokens file saved in ./wav2vec2-base-timit-fine-tuned/special_tokens_map.json | |
| added tokens file saved in ./wav2vec2-base-timit-fine-tuned/added_tokens.json | |
| 17%|████████████████▏ | 401/2320 [09:06<6:52:25, 12.90s/it]/opt/conda/lib/python3.12/site-packages/torch/nn/modules/conv.py:306: UserWarning: Plan failed with a cudnnException: CUDNN_BACKEND_EXECUTION_PLAN_DESCRIPTOR: cudnnFinalize Descriptor Failed cudnn_status: CUDNN_STATUS_NOT_SUPPORTED (Triggered internally at /home/conda/feedstock_root/build_artifacts/libtorch_1715567101190/work/aten/src/ATen/native/cudnn/Conv_v8.cpp:919.) | |
| return F.conv1d(input, weight, bias, self.stride, | |
| {'loss': 2.0881, 'grad_norm': 4.349735260009766, 'learning_rate': 4.08e-05, 'epoch': 3.53} | |
| {'loss': 1.9522, 'grad_norm': 2.450747489929199, 'learning_rate': 4.18e-05, 'epoch': 3.62} | |
| {'loss': 1.8395, 'grad_norm': 2.2519729137420654, 'learning_rate': 4.2800000000000004e-05, 'epoch': 3.71} | |
| {'loss': 1.7525, 'grad_norm': 2.693664789199829, 'learning_rate': 4.38e-05, 'epoch': 3.79} | |
| {'loss': 1.6222, 'grad_norm': 1.9744929075241089, 'learning_rate': 4.4800000000000005e-05, 'epoch': 3.88} | |
| {'loss': 1.5397, 'grad_norm': 3.802494764328003, 'learning_rate': 4.58e-05, 'epoch': 3.97} | |
| {'loss': 1.4376, 'grad_norm': 2.301044225692749, 'learning_rate': 4.6800000000000006e-05, 'epoch': 4.05} | |
| {'loss': 1.2829, 'grad_norm': 2.279372215270996, 'learning_rate': 4.78e-05, 'epoch': 4.14} | |
| {'loss': 1.1976, 'grad_norm': 3.314736843109131, 'learning_rate': 4.88e-05, 'epoch': 4.22} | |
| {'loss': 1.1579, 'grad_norm': 2.434694290161133, 'learning_rate': 4.9800000000000004e-05, 'epoch': 4.31} | |
| 22%|████████████████████▋ | 500/2320 [10:43<34:53, 1.15s/it]The following columns in the evaluation set don't have a corresponding argument in `Wav2Vec2ForCTC.forward` and have been ignored: input_length. If input_length are not expected by `Wav2Vec2ForCTC.forward`, you can safely ignore this message. | |
| ***** Running Evaluation ***** | |
| Num examples = 1344 | |
| Batch size = 1 | |
| {'eval_loss': 1.045101284980774, 'eval_wer': 0.8299189656742239, 'eval_runtime': 39.7455, 'eval_samples_per_second': 33.815, 'eval_steps_per_second': 33.815, 'epoch': 4.31} | |
| {'loss': 1.0684, 'grad_norm': 1.8384031057357788, 'learning_rate': 5.08e-05, 'epoch': 4.4} | |
| {'loss': 1.0319, 'grad_norm': 3.599148988723755, 'learning_rate': 5.1800000000000005e-05, 'epoch': 4.48} | |
| {'loss': 0.9179, 'grad_norm': 2.066476583480835, 'learning_rate': 5.28e-05, 'epoch': 4.57} | |
| {'loss': 0.8838, 'grad_norm': 2.2173750400543213, 'learning_rate': 5.380000000000001e-05, 'epoch': 4.66} | |
| {'loss': 0.8991, 'grad_norm': 2.427091121673584, 'learning_rate': 5.4800000000000004e-05, 'epoch': 4.74} | |
| {'loss': 0.8, 'grad_norm': 2.7432241439819336, 'learning_rate': 5.580000000000001e-05, 'epoch': 4.83} | |
| {'loss': 0.7803, 'grad_norm': 3.254221200942993, 'learning_rate': 5.68e-05, 'epoch': 4.91} | |
| {'loss': 0.8205, 'grad_norm': 4.457448482513428, 'learning_rate': 5.7799999999999995e-05, 'epoch': 5.0} | |
| {'loss': 0.6703, 'grad_norm': 3.1023166179656982, 'learning_rate': 5.88e-05, 'epoch': 5.09} | |
| {'loss': 0.6087, 'grad_norm': 2.5916504859924316, 'learning_rate': 5.9800000000000003e-05, 'epoch': 5.17} | |
| 26%|████████████████████████▊ | 600/2320 [12:58<23:53, 1.20it/s]The following columns in the evaluation set don't have a corresponding argument in `Wav2Vec2ForCTC.forward` and have been ignored: input_length. If input_length are not expected by `Wav2Vec2ForCTC.forward`, you can safely ignore this message. | |
| ***** Running Evaluation ***** | |
| Num examples = 1344 | |
| Batch size = 1 | |
| {'eval_loss': 0.6753795146942139, 'eval_wer': 0.6440863152144223, 'eval_runtime': 39.7485, 'eval_samples_per_second': 33.813, 'eval_steps_per_second': 33.813, 'epoch': 5.17} | |
| {'loss': 0.6569, 'grad_norm': 2.1707613468170166, 'learning_rate': 6.08e-05, 'epoch': 5.26} | |
| {'loss': 0.5627, 'grad_norm': 2.4291555881500244, 'learning_rate': 6.18e-05, 'epoch': 5.34} | |
| {'loss': 0.5381, 'grad_norm': 2.249617338180542, 'learning_rate': 6.280000000000001e-05, 'epoch': 5.43} | |
| {'loss': 0.6338, 'grad_norm': 1.6661946773529053, 'learning_rate': 6.38e-05, 'epoch': 5.52} | |
| {'loss': 0.5181, 'grad_norm': 2.60294771194458, 'learning_rate': 6.48e-05, 'epoch': 5.6} | |
| {'loss': 0.5189, 'grad_norm': 3.3003089427948, 'learning_rate': 6.58e-05, 'epoch': 5.69} | |
| {'loss': 0.564, 'grad_norm': 1.880764126777649, 'learning_rate': 6.680000000000001e-05, 'epoch': 5.78} | |
| {'loss': 0.4729, 'grad_norm': 2.0575127601623535, 'learning_rate': 6.780000000000001e-05, 'epoch': 5.86} | |
| {'loss': 0.4899, 'grad_norm': 2.5159761905670166, 'learning_rate': 6.879999999999999e-05, 'epoch': 5.95} | |
| {'loss': 0.481, 'grad_norm': 1.4463504552841187, 'learning_rate': 6.98e-05, 'epoch': 6.03} | |
| 30%|████████████████████████████▉ | 700/2320 [15:14<36:18, 1.34s/it]The following columns in the evaluation set don't have a corresponding argument in `Wav2Vec2ForCTC.forward` and have been ignored: input_length. If input_length are not expected by `Wav2Vec2ForCTC.forward`, you can safely ignore this message. | |
| ***** Running Evaluation ***** | |
| Num examples = 1344 | |
| Batch size = 1 | |
| {'eval_loss': 0.5275412201881409, 'eval_wer': 0.5760721114449604, 'eval_runtime': 39.9601, 'eval_samples_per_second': 33.634, 'eval_steps_per_second': 33.634, 'epoch': 6.03} | |
| {'loss': 0.3865, 'grad_norm': 1.788765549659729, 'learning_rate': 7.08e-05, 'epoch': 6.12} | |
| {'loss': 0.3726, 'grad_norm': 1.862762212753296, 'learning_rate': 7.18e-05, 'epoch': 6.21} | |
| {'loss': 0.4116, 'grad_norm': 1.6512093544006348, 'learning_rate': 7.280000000000001e-05, 'epoch': 6.29} | |
| {'loss': 0.3779, 'grad_norm': 2.098067045211792, 'learning_rate': 7.38e-05, 'epoch': 6.38} | |
| {'loss': 0.3728, 'grad_norm': 3.3030078411102295, 'learning_rate': 7.48e-05, 'epoch': 6.47} | |
| {'loss': 0.4047, 'grad_norm': 2.1799120903015137, 'learning_rate': 7.58e-05, 'epoch': 6.55} | |
| {'loss': 0.313, 'grad_norm': 1.862434983253479, 'learning_rate': 7.680000000000001e-05, 'epoch': 6.64} | |
| {'loss': 0.4052, 'grad_norm': 6.29113245010376, 'learning_rate': 7.780000000000001e-05, 'epoch': 6.72} | |
| {'loss': 0.3218, 'grad_norm': 1.4220325946807861, 'learning_rate': 7.88e-05, 'epoch': 6.81} | |
| {'loss': 0.3072, 'grad_norm': 2.586819648742676, 'learning_rate': 7.98e-05, 'epoch': 6.9} | |
| 34%|█████████████████████████████████ | 800/2320 [17:30<20:39, 1.23it/s]The following columns in the evaluation set don't have a corresponding argument in `Wav2Vec2ForCTC.forward` and have been ignored: input_length. If input_length are not expected by `Wav2Vec2ForCTC.forward`, you can safely ignore this message. | |
| ***** Running Evaluation ***** | |
| Num examples = 1344 | |
| Batch size = 1 | |
| {'eval_loss': 0.4836220443248749, 'eval_wer': 0.5264499681325685, 'eval_runtime': 39.8762, 'eval_samples_per_second': 33.704, 'eval_steps_per_second': 33.704, 'epoch': 6.9} | |
| 34%|█████████████████████████████████ | 800/2320 [18:10<20:39, 1.23it/sSaving model checkpoint to ./wav2vec2-base-timit-fine-tuned/checkpoint-800 | |
| Configuration saved in ./wav2vec2-base-timit-fine-tuned/checkpoint-800/config.json | |
| Model weights saved in ./wav2vec2-base-timit-fine-tuned/checkpoint-800/model.safetensors | |
| Feature extractor saved in ./wav2vec2-base-timit-fine-tuned/checkpoint-800/preprocessor_config.json | |
| tokenizer config file saved in ./wav2vec2-base-timit-fine-tuned/checkpoint-800/tokenizer_config.json | |
| Special tokens file saved in ./wav2vec2-base-timit-fine-tuned/checkpoint-800/special_tokens_map.json | |
| added tokens file saved in ./wav2vec2-base-timit-fine-tuned/checkpoint-800/added_tokens.json | |
| Feature extractor saved in ./wav2vec2-base-timit-fine-tuned/preprocessor_config.json | |
| tokenizer config file saved in ./wav2vec2-base-timit-fine-tuned/tokenizer_config.json | |
| Special tokens file saved in ./wav2vec2-base-timit-fine-tuned/special_tokens_map.json | |
| added tokens file saved in ./wav2vec2-base-timit-fine-tuned/added_tokens.json | |
| {'loss': 0.3862, 'grad_norm': 1.6589460372924805, 'learning_rate': 8.080000000000001e-05, 'epoch': 6.98} | |
| {'loss': 0.2938, 'grad_norm': 1.7299175262451172, 'learning_rate': 8.18e-05, 'epoch': 7.07} | |
| {'loss': 0.249, 'grad_norm': 2.0545098781585693, 'learning_rate': 8.28e-05, 'epoch': 7.16} | |
| 36%|██████████████████████████████████▋ | 837/2320 [18:46<17:32, 1.41it/s]/opt/conda/lib/python3.12/site-packages/torch/nn/modules/conv.py:306: UserWarning: Plan failed with a cudnnException: CUDNN_BACKEND_EXECUTION_PLAN_DESCRIPTOR: cudnnFinalize Descriptor Failed cudnn_status: CUDNN_STATUS_NOT_SUPPORTED (Triggered internally at /home/conda/feedstock_root/build_artifacts/libtorch_1715567101190/work/aten/src/ATen/native/cudnn/Conv_v8.cpp:919.) | |
| return F.conv1d(input, weight, bias, self.stride, | |
| {'loss': 0.3202, 'grad_norm': 24.935670852661133, 'learning_rate': 8.38e-05, 'epoch': 7.24} | |
| {'loss': 0.2803, 'grad_norm': 2.497840642929077, 'learning_rate': 8.48e-05, 'epoch': 7.33} | |
| {'loss': 0.2473, 'grad_norm': 2.698636531829834, 'learning_rate': 8.58e-05, 'epoch': 7.41} | |
| {'loss': 0.3223, 'grad_norm': 1.4561227560043335, 'learning_rate': 8.680000000000001e-05, 'epoch': 7.5} | |
| {'loss': 0.2481, 'grad_norm': 1.7760556936264038, 'learning_rate': 8.78e-05, 'epoch': 7.59} | |
| {'loss': 0.2545, 'grad_norm': 2.308103084564209, 'learning_rate': 8.88e-05, 'epoch': 7.67} | |
| {'loss': 0.332, 'grad_norm': 1.4128385782241821, 'learning_rate': 8.98e-05, 'epoch': 7.76} | |
| 39%|█████████████████████████████████████▏ | 900/2320 [19:48<29:47, 1.26s/it]The following columns in the evaluation set don't have a corresponding argument in `Wav2Vec2ForCTC.forward` and have been ignored: input_length. If input_length are not expected by `Wav2Vec2ForCTC.forward`, you can safely ignore this message. | |
| ***** Running Evaluation ***** | |
| Num examples = 1344 | |
| Batch size = 1 | |
| {'eval_loss': 0.44030094146728516, 'eval_wer': 0.5233542747883092, 'eval_runtime': 39.9401, 'eval_samples_per_second': 33.65, 'eval_steps_per_second': 33.65, 'epoch': 7.76} | |
| {'loss': 0.2411, 'grad_norm': 1.7903906106948853, 'learning_rate': 9.080000000000001e-05, 'epoch': 7.84} | |
| {'loss': 0.2707, 'grad_norm': 2.0804216861724854, 'learning_rate': 9.180000000000001e-05, 'epoch': 7.93} | |
| {'loss': 0.3186, 'grad_norm': 1.4420605897903442, 'learning_rate': 9.28e-05, 'epoch': 8.02} | |
| {'loss': 0.1937, 'grad_norm': 2.2910854816436768, 'learning_rate': 9.38e-05, 'epoch': 8.1} | |
| {'loss': 0.2321, 'grad_norm': 3.5892796516418457, 'learning_rate': 9.48e-05, 'epoch': 8.19} | |
| {'loss': 0.2868, 'grad_norm': 1.6509956121444702, 'learning_rate': 9.58e-05, 'epoch': 8.28} | |
| {'loss': 0.2004, 'grad_norm': 1.6983604431152344, 'learning_rate': 9.680000000000001e-05, 'epoch': 8.36} | |
| {'loss': 0.2025, 'grad_norm': 2.061176061630249, 'learning_rate': 9.78e-05, 'epoch': 8.45} | |
| {'loss': 0.2598, 'grad_norm': 1.7732270956039429, 'learning_rate': 9.88e-05, 'epoch': 8.53} | |
| {'loss': 0.1876, 'grad_norm': 1.8335466384887695, 'learning_rate': 9.98e-05, 'epoch': 8.62} | |
| 43%|████████████████████████████████████████▉ | 1000/2320 [22:05<20:18, 1.08it/s]The following columns in the evaluation set don't have a corresponding argument in `Wav2Vec2ForCTC.forward` and have been ignored: input_length. If input_length are not expected by `Wav2Vec2ForCTC.forward`, you can safely ignore this message. | |
| ***** Running Evaluation ***** | |
| Num examples = 1344 | |
| Batch size = 1 | |
| {'eval_loss': 0.4757933020591736, 'eval_wer': 0.5221706273331512, 'eval_runtime': 39.8291, 'eval_samples_per_second': 33.744, 'eval_steps_per_second': 33.744, 'epoch': 8.62} | |
| {'loss': 0.2456, 'grad_norm': 2.52902889251709, 'learning_rate': 9.939393939393939e-05, 'epoch': 8.71} | |
| {'loss': 0.2499, 'grad_norm': 1.7294162511825562, 'learning_rate': 9.863636363636364e-05, 'epoch': 8.79} | |
| {'loss': 0.1854, 'grad_norm': 21.9121150970459, 'learning_rate': 9.787878787878789e-05, 'epoch': 8.88} | |
| {'loss': 0.2576, 'grad_norm': 3.9164559841156006, 'learning_rate': 9.712121212121212e-05, 'epoch': 8.97} | |
| {'loss': 0.2118, 'grad_norm': 1.239221215248108, 'learning_rate': 9.636363636363637e-05, 'epoch': 9.05} | |
| {'loss': 0.1577, 'grad_norm': 3.1416544914245605, 'learning_rate': 9.560606060606061e-05, 'epoch': 9.14} | |
| {'loss': 0.2092, 'grad_norm': 2.4253621101379395, 'learning_rate': 9.484848484848486e-05, 'epoch': 9.22} | |
| {'loss': 0.1876, 'grad_norm': 1.194345474243164, 'learning_rate': 9.40909090909091e-05, 'epoch': 9.31} | |
| {'loss': 0.1546, 'grad_norm': 2.411029100418091, 'learning_rate': 9.333333333333334e-05, 'epoch': 9.4} | |
| {'loss': 0.2232, 'grad_norm': 3.246082067489624, 'learning_rate': 9.257575757575758e-05, 'epoch': 9.48} | |
| 47%|█████████████████████████████████████████████ | 1100/2320 [24:18<14:01, 1.45it/s]The following columns in the evaluation set don't have a corresponding argument in `Wav2Vec2ForCTC.forward` and have been ignored: input_length. If input_length are not expected by `Wav2Vec2ForCTC.forward`, you can safely ignore this message. | |
| ***** Running Evaluation ***** | |
| Num examples = 1344 | |
| Batch size = 1 | |
| {'eval_loss': 0.45077577233314514, 'eval_wer': 0.48921059819721385, 'eval_runtime': 39.9221, 'eval_samples_per_second': 33.666, 'eval_steps_per_second': 33.666, 'epoch': 9.48} | |
| {'loss': 0.1777, 'grad_norm': 1.3427454233169556, 'learning_rate': 9.181818181818183e-05, 'epoch': 9.57} | |
| {'loss': 0.1646, 'grad_norm': 1.5090447664260864, 'learning_rate': 9.106060606060606e-05, 'epoch': 9.66} | |
| {'loss': 0.225, 'grad_norm': 1.3060975074768066, 'learning_rate': 9.030303030303031e-05, 'epoch': 9.74} | |
| {'loss': 0.1552, 'grad_norm': 1.3011540174484253, 'learning_rate': 8.954545454545455e-05, 'epoch': 9.83} | |
| {'loss': 0.1715, 'grad_norm': 1.9938538074493408, 'learning_rate': 8.87878787878788e-05, 'epoch': 9.91} | |
| {'loss': 0.2092, 'grad_norm': 3.334385395050049, 'learning_rate': 8.803030303030304e-05, 'epoch': 10.0} | |
| {'loss': 0.14, 'grad_norm': 1.011092185974121, 'learning_rate': 8.727272727272727e-05, 'epoch': 10.09} | |
| {'loss': 0.1512, 'grad_norm': 2.517902135848999, 'learning_rate': 8.651515151515152e-05, 'epoch': 10.17} | |
| {'loss': 0.1846, 'grad_norm': 1.2418378591537476, 'learning_rate': 8.575757575757576e-05, 'epoch': 10.26} | |
| {'loss': 0.1332, 'grad_norm': 1.5885329246520996, 'learning_rate': 8.5e-05, 'epoch': 10.34} | |
| 52%|█████████████████████████████████████████████████▏ | 1200/2320 [26:37<18:40, 1.00s/it]The following columns in the evaluation set don't have a corresponding argument in `Wav2Vec2ForCTC.forward` and have been ignored: input_length. If input_length are not expected by `Wav2Vec2ForCTC.forward`, you can safely ignore this message. | |
| ***** Running Evaluation ***** | |
| Num examples = 1344 | |
| Batch size = 1 | |
| {'eval_loss': 0.4394075274467468, 'eval_wer': 0.4740052808886461, 'eval_runtime': 39.9367, 'eval_samples_per_second': 33.653, 'eval_steps_per_second': 33.653, 'epoch': 10.34} | |
| 52%|█████████████████████████████████████████████████▏ | 1200/2320 [27:17<18:40, 1.00s/itSaving model checkpoint to ./wav2vec2-base-timit-fine-tuned/checkpoint-1200 | |
| Configuration saved in ./wav2vec2-base-timit-fine-tuned/checkpoint-1200/config.json | |
| Model weights saved in ./wav2vec2-base-timit-fine-tuned/checkpoint-1200/model.safetensors | |
| Feature extractor saved in ./wav2vec2-base-timit-fine-tuned/checkpoint-1200/preprocessor_config.json | |
| tokenizer config file saved in ./wav2vec2-base-timit-fine-tuned/checkpoint-1200/tokenizer_config.json | |
| Special tokens file saved in ./wav2vec2-base-timit-fine-tuned/checkpoint-1200/special_tokens_map.json | |
| added tokens file saved in ./wav2vec2-base-timit-fine-tuned/checkpoint-1200/added_tokens.json | |
| Feature extractor saved in ./wav2vec2-base-timit-fine-tuned/preprocessor_config.json | |
| tokenizer config file saved in ./wav2vec2-base-timit-fine-tuned/tokenizer_config.json | |
| Special tokens file saved in ./wav2vec2-base-timit-fine-tuned/special_tokens_map.json | |
| added tokens file saved in ./wav2vec2-base-timit-fine-tuned/added_tokens.json | |
| {'loss': 0.1485, 'grad_norm': 1.2539469003677368, 'learning_rate': 8.424242424242424e-05, 'epoch': 10.43} | |
| {'loss': 0.1988, 'grad_norm': 1.357601284980774, 'learning_rate': 8.348484848484849e-05, 'epoch': 10.52} | |
| 53%|██████████████████████████████████████████████████▏ | 1227/2320 [27:45<19:01, 1.04s/it]/opt/conda/lib/python3.12/site-packages/torch/nn/modules/conv.py:306: UserWarning: Plan failed with a cudnnException: CUDNN_BACKEND_EXECUTION_PLAN_DESCRIPTOR: cudnnFinalize Descriptor Failed cudnn_status: CUDNN_STATUS_NOT_SUPPORTED (Triggered internally at /home/conda/feedstock_root/build_artifacts/libtorch_1715567101190/work/aten/src/ATen/native/cudnn/Conv_v8.cpp:919.) | |
| return F.conv1d(input, weight, bias, self.stride, | |
| {'loss': 0.137, 'grad_norm': 2.0564587116241455, 'learning_rate': 8.272727272727273e-05, 'epoch': 10.6} | |
| {'loss': 0.1245, 'grad_norm': 2.48364520072937, 'learning_rate': 8.196969696969698e-05, 'epoch': 10.69} | |
| {'loss': 0.1602, 'grad_norm': 1.015891671180725, 'learning_rate': 8.121212121212121e-05, 'epoch': 10.78} | |
| {'loss': 0.1215, 'grad_norm': 1.1023950576782227, 'learning_rate': 8.045454545454546e-05, 'epoch': 10.86} | |
| {'loss': 0.1621, 'grad_norm': 2.703427791595459, 'learning_rate': 7.96969696969697e-05, 'epoch': 10.95} | |
| {'loss': 0.1651, 'grad_norm': 1.1821691989898682, 'learning_rate': 7.893939393939395e-05, 'epoch': 11.03} | |
| {'loss': 0.1066, 'grad_norm': 0.930283784866333, 'learning_rate': 7.818181818181818e-05, 'epoch': 11.12} | |
| {'loss': 0.1085, 'grad_norm': 1.6548758745193481, 'learning_rate': 7.742424242424243e-05, 'epoch': 11.21} | |
| 56%|█████████████████████████████████████████████████████▏ | 1300/2320 [28:53<12:42, 1.34it/s]The following columns in the evaluation set don't have a corresponding argument in `Wav2Vec2ForCTC.forward` and have been ignored: input_length. If input_length are not expected by `Wav2Vec2ForCTC.forward`, you can safely ignore this message. | |
| ***** Running Evaluation ***** | |
| Num examples = 1344 | |
| Batch size = 1 | |
| {'eval_loss': 0.4466467499732971, 'eval_wer': 0.46207775653282346, 'eval_runtime': 39.8633, 'eval_samples_per_second': 33.715, 'eval_steps_per_second': 33.715, 'epoch': 11.21} | |
| {'loss': 0.1418, 'grad_norm': 1.1760716438293457, 'learning_rate': 7.666666666666667e-05, 'epoch': 11.29} | |
| {'loss': 0.1133, 'grad_norm': 2.1062755584716797, 'learning_rate': 7.59090909090909e-05, 'epoch': 11.38} | |
| {'loss': 0.1318, 'grad_norm': 2.67399001121521, 'learning_rate': 7.515151515151515e-05, 'epoch': 11.47} | |
| {'loss': 0.1474, 'grad_norm': 1.0049142837524414, 'learning_rate': 7.439393939393939e-05, 'epoch': 11.55} | |
| {'loss': 0.0908, 'grad_norm': 1.586559772491455, 'learning_rate': 7.363636363636364e-05, 'epoch': 11.64} | |
| {'loss': 0.1521, 'grad_norm': 3.784040927886963, 'learning_rate': 7.287878787878788e-05, 'epoch': 11.72} | |
| {'loss': 0.1163, 'grad_norm': 1.125501275062561, 'learning_rate': 7.212121212121213e-05, 'epoch': 11.81} | |
| {'loss': 0.1109, 'grad_norm': 2.1989808082580566, 'learning_rate': 7.136363636363636e-05, 'epoch': 11.9} | |
| {'loss': 0.152, 'grad_norm': 1.1287301778793335, 'learning_rate': 7.060606060606061e-05, 'epoch': 11.98} | |
| {'loss': 0.098, 'grad_norm': 1.538678765296936, 'learning_rate': 6.984848484848485e-05, 'epoch': 12.07} | |
| 60%|█████████████████████████████████████████████████████████▎ | 1400/2320 [31:12<18:06, 1.18s/it]The following columns in the evaluation set don't have a corresponding argument in `Wav2Vec2ForCTC.forward` and have been ignored: input_length. If input_length are not expected by `Wav2Vec2ForCTC.forward`, you can safely ignore this message. | |
| ***** Running Evaluation ***** | |
| Num examples = 1344 | |
| Batch size = 1 | |
| {'eval_loss': 0.42302384972572327, 'eval_wer': 0.44933078393881454, 'eval_runtime': 40.1773, 'eval_samples_per_second': 33.452, 'eval_steps_per_second': 33.452, 'epoch': 12.07} | |
| {'loss': 0.092, 'grad_norm': 1.400772213935852, 'learning_rate': 6.90909090909091e-05, 'epoch': 12.16} | |
| {'loss': 0.1649, 'grad_norm': 3.6780846118927, 'learning_rate': 6.833333333333333e-05, 'epoch': 12.24} | |
| {'loss': 0.091, 'grad_norm': 1.5424057245254517, 'learning_rate': 6.757575757575758e-05, 'epoch': 12.33} | |
| {'loss': 0.0869, 'grad_norm': 1.4868180751800537, 'learning_rate': 6.681818181818183e-05, 'epoch': 12.41} | |
| {'loss': 0.1499, 'grad_norm': 1.1947145462036133, 'learning_rate': 6.606060606060607e-05, 'epoch': 12.5} | |
| {'loss': 0.0954, 'grad_norm': 1.0430784225463867, 'learning_rate': 6.530303030303032e-05, 'epoch': 12.59} | |
| {'loss': 0.1032, 'grad_norm': 2.4261584281921387, 'learning_rate': 6.454545454545455e-05, 'epoch': 12.67} | |
| {'loss': 0.1158, 'grad_norm': 1.033467411994934, 'learning_rate': 6.37878787878788e-05, 'epoch': 12.76} | |
| {'loss': 0.0864, 'grad_norm': 1.1535651683807373, 'learning_rate': 6.303030303030302e-05, 'epoch': 12.84} | |
| {'loss': 0.1219, 'grad_norm': 1.28826105594635, 'learning_rate': 6.227272727272727e-05, 'epoch': 12.93} | |
| 65%|█████████████████████████████████████████████████████████████▍ | 1500/2320 [33:26<10:01, 1.36it/s]The following columns in the evaluation set don't have a corresponding argument in `Wav2Vec2ForCTC.forward` and have been ignored: input_length. If input_length are not expected by `Wav2Vec2ForCTC.forward`, you can safely ignore this message. | |
| ***** Running Evaluation ***** | |
| Num examples = 1344 | |
| Batch size = 1 | |
| {'eval_loss': 0.418023020029068, 'eval_wer': 0.44596194118182647, 'eval_runtime': 40.2192, 'eval_samples_per_second': 33.417, 'eval_steps_per_second': 33.417, 'epoch': 12.93} | |
| {'loss': 0.1289, 'grad_norm': 1.055411458015442, 'learning_rate': 6.151515151515151e-05, 'epoch': 13.02} | |
| {'loss': 0.0776, 'grad_norm': 1.1269094944000244, 'learning_rate': 6.075757575757576e-05, 'epoch': 13.1} | |
| {'loss': 0.0871, 'grad_norm': 1.7149118185043335, 'learning_rate': 6e-05, 'epoch': 13.19} | |
| {'loss': 0.1087, 'grad_norm': 1.7456856966018677, 'learning_rate': 5.9242424242424244e-05, 'epoch': 13.28} | |
| {'loss': 0.0821, 'grad_norm': 1.3434715270996094, 'learning_rate': 5.848484848484849e-05, 'epoch': 13.36} | |
| {'loss': 0.0878, 'grad_norm': 2.103512763977051, 'learning_rate': 5.772727272727273e-05, 'epoch': 13.45} | |
| {'loss': 0.1044, 'grad_norm': 1.240224838256836, 'learning_rate': 5.696969696969697e-05, 'epoch': 13.53} | |
| {'loss': 0.0753, 'grad_norm': 0.7336703538894653, 'learning_rate': 5.6212121212121215e-05, 'epoch': 13.62} | |
| {'loss': 0.1059, 'grad_norm': 2.293342351913452, 'learning_rate': 5.545454545454546e-05, 'epoch': 13.71} | |
| {'loss': 0.1021, 'grad_norm': 1.1853971481323242, 'learning_rate': 5.46969696969697e-05, 'epoch': 13.79} | |
| 69%|█████████████████████████████████████████████████████████████████▌ | 1600/2320 [35:45<13:55, 1.16s/it]The following columns in the evaluation set don't have a corresponding argument in `Wav2Vec2ForCTC.forward` and have been ignored: input_length. If input_length are not expected by `Wav2Vec2ForCTC.forward`, you can safely ignore this message. | |
| ***** Running Evaluation ***** | |
| Num examples = 1344 | |
| Batch size = 1 | |
| {'eval_loss': 0.41785839200019836, 'eval_wer': 0.4405900027314941, 'eval_runtime': 40.2906, 'eval_samples_per_second': 33.358, 'eval_steps_per_second': 33.358, 'epoch': 13.79} | |
| 69%|█████████████████████████████████████████████████████████████████▌ | 1600/2320 [36:25<13:55, 1.16s/itSaving model checkpoint to ./wav2vec2-base-timit-fine-tuned/checkpoint-1600 | |
| Configuration saved in ./wav2vec2-base-timit-fine-tuned/checkpoint-1600/config.json | |
| Model weights saved in ./wav2vec2-base-timit-fine-tuned/checkpoint-1600/model.safetensors | |
| Feature extractor saved in ./wav2vec2-base-timit-fine-tuned/checkpoint-1600/preprocessor_config.json | |
| tokenizer config file saved in ./wav2vec2-base-timit-fine-tuned/checkpoint-1600/tokenizer_config.json | |
| Special tokens file saved in ./wav2vec2-base-timit-fine-tuned/checkpoint-1600/special_tokens_map.json | |
| added tokens file saved in ./wav2vec2-base-timit-fine-tuned/checkpoint-1600/added_tokens.json | |
| Feature extractor saved in ./wav2vec2-base-timit-fine-tuned/preprocessor_config.json | |
| tokenizer config file saved in ./wav2vec2-base-timit-fine-tuned/tokenizer_config.json | |
| Special tokens file saved in ./wav2vec2-base-timit-fine-tuned/special_tokens_map.json | |
| added tokens file saved in ./wav2vec2-base-timit-fine-tuned/added_tokens.json | |
| Deleting older checkpoint [wav2vec2-base-timit-fine-tuned/checkpoint-400] due to args.save_total_limit | |
| {'loss': 0.0648, 'grad_norm': 1.331200361251831, 'learning_rate': 5.393939393939394e-05, 'epoch': 13.88} | |
| {'loss': 0.1121, 'grad_norm': 2.28397536277771, 'learning_rate': 5.3181818181818186e-05, 'epoch': 13.97} | |
| {'loss': 0.0725, 'grad_norm': 0.9436893463134766, 'learning_rate': 5.242424242424243e-05, 'epoch': 14.05} | |
| {'loss': 0.0691, 'grad_norm': 1.6113288402557373, 'learning_rate': 5.166666666666667e-05, 'epoch': 14.14} | |
| {'loss': 0.0979, 'grad_norm': 2.479888439178467, 'learning_rate': 5.090909090909091e-05, 'epoch': 14.22} | |
| {'loss': 0.0909, 'grad_norm': 1.006616473197937, 'learning_rate': 5.015151515151515e-05, 'epoch': 14.31} | |
| 72%|████████████████████████████████████████████████████████████████████ | 1663/2320 [37:27<11:20, 1.04s/it]/opt/conda/lib/python3.12/site-packages/torch/nn/modules/conv.py:306: UserWarning: Plan failed with a cudnnException: CUDNN_BACKEND_EXECUTION_PLAN_DESCRIPTOR: cudnnFinalize Descriptor Failed cudnn_status: CUDNN_STATUS_NOT_SUPPORTED (Triggered internally at /home/conda/feedstock_root/build_artifacts/libtorch_1715567101190/work/aten/src/ATen/native/cudnn/Conv_v8.cpp:919.) | |
| return F.conv1d(input, weight, bias, self.stride, | |
| {'loss': 0.0761, 'grad_norm': 1.4571704864501953, 'learning_rate': 4.93939393939394e-05, 'epoch': 14.4} | |
| {'loss': 0.0862, 'grad_norm': 1.5729875564575195, 'learning_rate': 4.863636363636364e-05, 'epoch': 14.48} | |
| {'loss': 0.0646, 'grad_norm': 1.2180376052856445, 'learning_rate': 4.787878787878788e-05, 'epoch': 14.57} | |
| {'loss': 0.0741, 'grad_norm': 1.7464072704315186, 'learning_rate': 4.712121212121212e-05, 'epoch': 14.66} | |
| 73%|█████████████████████████████████████████████████████████████████████▌ | 1700/2320 [38:02<08:27, 1.22it/s]The following columns in the evaluation set don't have a corresponding argument in `Wav2Vec2ForCTC.forward` and have been ignored: input_length. If input_length are not expected by `Wav2Vec2ForCTC.forward`, you can safely ignore this message. | |
| ***** Running Evaluation ***** | |
| Num examples = 1344 | |
| Batch size = 1 | |
| {'eval_loss': 0.4113341271877289, 'eval_wer': 0.4309387234817445, 'eval_runtime': 40.2841, 'eval_samples_per_second': 33.363, 'eval_steps_per_second': 33.363, 'epoch': 14.66} | |
| {'loss': 0.1315, 'grad_norm': 0.8571386337280273, 'learning_rate': 4.6439393939393944e-05, 'epoch': 14.74} | |
| {'loss': 0.0603, 'grad_norm': 1.331377387046814, 'learning_rate': 4.5681818181818186e-05, 'epoch': 14.83} | |
| {'loss': 0.0796, 'grad_norm': 1.5398732423782349, 'learning_rate': 4.492424242424242e-05, 'epoch': 14.91} | |
| {'loss': 0.085, 'grad_norm': 3.689671754837036, 'learning_rate': 4.4166666666666665e-05, 'epoch': 15.0} | |
| {'loss': 0.0544, 'grad_norm': 1.132613182067871, 'learning_rate': 4.340909090909091e-05, 'epoch': 15.09} | |
| {'loss': 0.0601, 'grad_norm': 1.5951859951019287, 'learning_rate': 4.265151515151515e-05, 'epoch': 15.17} | |
| {'loss': 0.097, 'grad_norm': 0.5179944634437561, 'learning_rate': 4.189393939393939e-05, 'epoch': 15.26} | |
| {'loss': 0.0596, 'grad_norm': 0.9744370579719543, 'learning_rate': 4.113636363636364e-05, 'epoch': 15.34} | |
| {'loss': 0.0677, 'grad_norm': 1.8794275522232056, 'learning_rate': 4.0378787878787885e-05, 'epoch': 15.43} | |
| {'loss': 0.0896, 'grad_norm': 0.748386025428772, 'learning_rate': 3.962121212121213e-05, 'epoch': 15.52} | |
| 78%|█████████████████████████████████████████████████████████████████████████▋ | 1800/2320 [40:18<11:05, 1.28s/it]The following columns in the evaluation set don't have a corresponding argument in `Wav2Vec2ForCTC.forward` and have been ignored: input_length. If input_length are not expected by `Wav2Vec2ForCTC.forward`, you can safely ignore this message. | |
| ***** Running Evaluation ***** | |
| Num examples = 1344 | |
| Batch size = 1 | |
| {'eval_loss': 0.43920788168907166, 'eval_wer': 0.4307566238732587, 'eval_runtime': 40.1997, 'eval_samples_per_second': 33.433, 'eval_steps_per_second': 33.433, 'epoch': 15.52} | |
| {'loss': 0.0604, 'grad_norm': 0.9639837145805359, 'learning_rate': 3.8863636363636364e-05, 'epoch': 15.6} | |
| {'loss': 0.0711, 'grad_norm': 1.9640839099884033, 'learning_rate': 3.810606060606061e-05, 'epoch': 15.69} | |
| {'loss': 0.0867, 'grad_norm': 1.4438735246658325, 'learning_rate': 3.734848484848485e-05, 'epoch': 15.78} | |
| {'loss': 0.0605, 'grad_norm': 1.0062426328659058, 'learning_rate': 3.659090909090909e-05, 'epoch': 15.86} | |
| {'loss': 0.0662, 'grad_norm': 1.6331523656845093, 'learning_rate': 3.5833333333333335e-05, 'epoch': 15.95} | |
| {'loss': 0.0765, 'grad_norm': 0.8070217370986938, 'learning_rate': 3.507575757575758e-05, 'epoch': 16.03} | |
| {'loss': 0.0537, 'grad_norm': 1.4137670993804932, 'learning_rate': 3.431818181818182e-05, 'epoch': 16.12} | |
| {'loss': 0.0684, 'grad_norm': 1.5437769889831543, 'learning_rate': 3.356060606060606e-05, 'epoch': 16.21} | |
| {'loss': 0.0744, 'grad_norm': 0.90281081199646, 'learning_rate': 3.2803030303030305e-05, 'epoch': 16.29} | |
| {'loss': 0.0492, 'grad_norm': 1.139837622642517, 'learning_rate': 3.204545454545455e-05, 'epoch': 16.38} | |
| 82%|█████████████████████████████████████████████████████████████████████████████▊ | 1900/2320 [42:36<06:26, 1.09it/s]The following columns in the evaluation set don't have a corresponding argument in `Wav2Vec2ForCTC.forward` and have been ignored: input_length. If input_length are not expected by `Wav2Vec2ForCTC.forward`, you can safely ignore this message. | |
| ***** Running Evaluation ***** | |
| Num examples = 1344 | |
| Batch size = 1 | |
| {'eval_loss': 0.4201890528202057, 'eval_wer': 0.4313029226987162, 'eval_runtime': 40.1502, 'eval_samples_per_second': 33.474, 'eval_steps_per_second': 33.474, 'epoch': 16.38} | |
| {'loss': 0.0652, 'grad_norm': 1.679457426071167, 'learning_rate': 3.128787878787879e-05, 'epoch': 16.47} | |
| {'loss': 0.0649, 'grad_norm': 0.6661111116409302, 'learning_rate': 3.0530303030303034e-05, 'epoch': 16.55} | |
| {'loss': 0.0469, 'grad_norm': 1.1774355173110962, 'learning_rate': 2.9772727272727273e-05, 'epoch': 16.64} | |
| {'loss': 0.0752, 'grad_norm': 1.783923864364624, 'learning_rate': 2.901515151515152e-05, 'epoch': 16.72} | |
| {'loss': 0.0519, 'grad_norm': 1.176321268081665, 'learning_rate': 2.825757575757576e-05, 'epoch': 16.81} | |
| {'loss': 0.0547, 'grad_norm': 1.3150608539581299, 'learning_rate': 2.7500000000000004e-05, 'epoch': 16.9} | |
| {'loss': 0.0799, 'grad_norm': 0.983769953250885, 'learning_rate': 2.674242424242424e-05, 'epoch': 16.98} | |
| {'loss': 0.0577, 'grad_norm': 0.996890127658844, 'learning_rate': 2.5984848484848483e-05, 'epoch': 17.07} | |
| {'loss': 0.0515, 'grad_norm': 2.3034253120422363, 'learning_rate': 2.5227272727272726e-05, 'epoch': 17.16} | |
| {'loss': 0.0759, 'grad_norm': 3.7528610229492188, 'learning_rate': 2.4469696969696972e-05, 'epoch': 17.24} | |
| 86%|█████████████████████████████████████████████████████████████████████████████████▉ | 2000/2320 [44:50<03:48, 1.40it/s]The following columns in the evaluation set don't have a corresponding argument in `Wav2Vec2ForCTC.forward` and have been ignored: input_length. If input_length are not expected by `Wav2Vec2ForCTC.forward`, you can safely ignore this message. | |
| ***** Running Evaluation ***** | |
| Num examples = 1344 | |
| Batch size = 1 | |
| {'eval_loss': 0.43480169773101807, 'eval_wer': 0.4207411454065374, 'eval_runtime': 40.017, 'eval_samples_per_second': 33.586, 'eval_steps_per_second': 33.586, 'epoch': 17.24} | |
| 86%|█████████████████████████████████████████████████████████████████████████████████▉ | 2000/2320 [45:30<03:48, 1.40it/sSaving model checkpoint to ./wav2vec2-base-timit-fine-tuned/checkpoint-2000 | |
| Configuration saved in ./wav2vec2-base-timit-fine-tuned/checkpoint-2000/config.json | |
| Model weights saved in ./wav2vec2-base-timit-fine-tuned/checkpoint-2000/model.safetensors | |
| Feature extractor saved in ./wav2vec2-base-timit-fine-tuned/checkpoint-2000/preprocessor_config.json | |
| tokenizer config file saved in ./wav2vec2-base-timit-fine-tuned/checkpoint-2000/tokenizer_config.json | |
| Special tokens file saved in ./wav2vec2-base-timit-fine-tuned/checkpoint-2000/special_tokens_map.json | |
| added tokens file saved in ./wav2vec2-base-timit-fine-tuned/checkpoint-2000/added_tokens.json | |
| Feature extractor saved in ./wav2vec2-base-timit-fine-tuned/preprocessor_config.json | |
| tokenizer config file saved in ./wav2vec2-base-timit-fine-tuned/tokenizer_config.json | |
| Special tokens file saved in ./wav2vec2-base-timit-fine-tuned/special_tokens_map.json | |
| added tokens file saved in ./wav2vec2-base-timit-fine-tuned/added_tokens.json | |
| Deleting older checkpoint [wav2vec2-base-timit-fine-tuned/checkpoint-800] due to args.save_total_limit | |
| {'loss': 0.0419, 'grad_norm': 0.6646668314933777, 'learning_rate': 2.3712121212121214e-05, 'epoch': 17.33} | |
| {'loss': 0.0595, 'grad_norm': 1.3250740766525269, 'learning_rate': 2.2954545454545457e-05, 'epoch': 17.41} | |
| {'loss': 0.0691, 'grad_norm': 0.8094995021820068, 'learning_rate': 2.21969696969697e-05, 'epoch': 17.5} | |
| {'loss': 0.052, 'grad_norm': 0.846946120262146, 'learning_rate': 2.143939393939394e-05, 'epoch': 17.59} | |
| {'loss': 0.0565, 'grad_norm': 1.652417540550232, 'learning_rate': 2.0681818181818182e-05, 'epoch': 17.67} | |
| {'loss': 0.0745, 'grad_norm': 1.0080279111862183, 'learning_rate': 1.9924242424242425e-05, 'epoch': 17.76} | |
| 89%|████████████████████████████████████████████████████████████████████████████████████▌ | 2064/2320 [46:36<04:53, 1.15s/it]/opt/conda/lib/python3.12/site-packages/torch/nn/modules/conv.py:306: UserWarning: Plan failed with a cudnnException: CUDNN_BACKEND_EXECUTION_PLAN_DESCRIPTOR: cudnnFinalize Descriptor Failed cudnn_status: CUDNN_STATUS_NOT_SUPPORTED (Triggered internally at /home/conda/feedstock_root/build_artifacts/libtorch_1715567101190/work/aten/src/ATen/native/cudnn/Conv_v8.cpp:919.) | |
| return F.conv1d(input, weight, bias, self.stride, | |
| {'loss': 0.0513, 'grad_norm': 0.7252691388130188, 'learning_rate': 1.9166666666666667e-05, 'epoch': 17.84} | |
| {'loss': 0.055, 'grad_norm': 1.58548903465271, 'learning_rate': 1.840909090909091e-05, 'epoch': 17.93} | |
| {'loss': 0.0658, 'grad_norm': 0.6634634733200073, 'learning_rate': 1.7651515151515153e-05, 'epoch': 18.02} | |
| {'loss': 0.0406, 'grad_norm': 1.1495524644851685, 'learning_rate': 1.6893939393939395e-05, 'epoch': 18.1} | |
| 91%|█████████████████████████████████████████████████████████████████████████████████████▉ | 2100/2320 [47:11<03:46, 1.03s/it]The following columns in the evaluation set don't have a corresponding argument in `Wav2Vec2ForCTC.forward` and have been ignored: input_length. If input_length are not expected by `Wav2Vec2ForCTC.forward`, you can safely ignore this message. | |
| ***** Running Evaluation ***** | |
| Num examples = 1344 | |
| Batch size = 1 | |
| {'eval_loss': 0.44191813468933105, 'eval_wer': 0.42046799599380863, 'eval_runtime': 40.0967, 'eval_samples_per_second': 33.519, 'eval_steps_per_second': 33.519, 'epoch': 18.1} | |
| {'loss': 0.0381, 'grad_norm': 0.9788354635238647, 'learning_rate': 1.6136363636363638e-05, 'epoch': 18.19} | |
| {'loss': 0.071, 'grad_norm': 1.093633770942688, 'learning_rate': 1.5378787878787877e-05, 'epoch': 18.28} | |
| {'loss': 0.0439, 'grad_norm': 0.7164376974105835, 'learning_rate': 1.4621212121212122e-05, 'epoch': 18.36} | |
| {'loss': 0.0481, 'grad_norm': 0.9887032508850098, 'learning_rate': 1.3863636363636364e-05, 'epoch': 18.45} | |
| {'loss': 0.0571, 'grad_norm': 0.45052286982536316, 'learning_rate': 1.3106060606060607e-05, 'epoch': 18.53} | |
| {'loss': 0.0452, 'grad_norm': 1.167181134223938, 'learning_rate': 1.234848484848485e-05, 'epoch': 18.62} | |
| {'loss': 0.0643, 'grad_norm': 1.378661870956421, 'learning_rate': 1.159090909090909e-05, 'epoch': 18.71} | |
| {'loss': 0.0587, 'grad_norm': 0.854932963848114, 'learning_rate': 1.0833333333333334e-05, 'epoch': 18.79} | |
| {'loss': 0.0395, 'grad_norm': 0.8007526397705078, 'learning_rate': 1.0075757575757576e-05, 'epoch': 18.88} | |
| {'loss': 0.074, 'grad_norm': 3.317830801010132, 'learning_rate': 9.318181818181819e-06, 'epoch': 18.97} | |
| 95%|██████████████████████████████████████████████████████████████████████████████████████████ | 2200/2320 [49:24<01:19, 1.51it/s]The following columns in the evaluation set don't have a corresponding argument in `Wav2Vec2ForCTC.forward` and have been ignored: input_length. If input_length are not expected by `Wav2Vec2ForCTC.forward`, you can safely ignore this message. | |
| ***** Running Evaluation ***** | |
| Num examples = 1344 | |
| Batch size = 1 | |
| {'eval_loss': 0.43061742186546326, 'eval_wer': 0.420012746972594, 'eval_runtime': 40.0034, 'eval_samples_per_second': 33.597, 'eval_steps_per_second': 33.597, 'epoch': 18.97} | |
| {'loss': 0.046, 'grad_norm': 0.7710875272750854, 'learning_rate': 8.56060606060606e-06, 'epoch': 19.05} | |
| {'loss': 0.0394, 'grad_norm': 0.5200530886650085, 'learning_rate': 7.803030303030304e-06, 'epoch': 19.14} | |
| {'loss': 0.0582, 'grad_norm': 1.3544327020645142, 'learning_rate': 7.045454545454545e-06, 'epoch': 19.22} | |
| {'loss': 0.0606, 'grad_norm': 0.8653574585914612, 'learning_rate': 6.287878787878789e-06, 'epoch': 19.31} | |
| {'loss': 0.0367, 'grad_norm': 1.5852700471878052, 'learning_rate': 5.530303030303031e-06, 'epoch': 19.4} | |
| {'loss': 0.0782, 'grad_norm': 2.2167246341705322, 'learning_rate': 4.772727272727273e-06, 'epoch': 19.48} | |
| {'loss': 0.0416, 'grad_norm': 0.5891330242156982, 'learning_rate': 4.015151515151515e-06, 'epoch': 19.57} | |
| {'loss': 0.0515, 'grad_norm': 1.1137330532073975, 'learning_rate': 3.257575757575758e-06, 'epoch': 19.66} | |
| {'loss': 0.0512, 'grad_norm': 0.8132285475730896, 'learning_rate': 2.5e-06, 'epoch': 19.74} | |
| {'loss': 0.0378, 'grad_norm': 0.7994781136512756, 'learning_rate': 1.7424242424242427e-06, 'epoch': 19.83} | |
| 99%|██████████████████████████████████████████████████████████████████████████████████████████████▏| 2300/2320 [51:43<00:20, 1.02s/it]The following columns in the evaluation set don't have a corresponding argument in `Wav2Vec2ForCTC.forward` and have been ignored: input_length. If input_length are not expected by `Wav2Vec2ForCTC.forward`, you can safely ignore this message. | |
| ***** Running Evaluation ***** | |
| Num examples = 1344 | |
| Batch size = 1 | |
| {'eval_loss': 0.4273350238800049, 'eval_wer': 0.41728125284530637, 'eval_runtime': 40.0934, 'eval_samples_per_second': 33.522, 'eval_steps_per_second': 33.522, 'epoch': 19.83} | |
| {'loss': 0.0489, 'grad_norm': 0.9775754809379578, 'learning_rate': 9.848484848484847e-07, 'epoch': 19.91} | |
| {'loss': 0.0554, 'grad_norm': 0.8857516050338745, 'learning_rate': 2.2727272727272726e-07, 'epoch': 20.0} | |
| 100%|███████████████████████████████████████████████████████████████████████████████████████████████| 2320/2320 [52:39<00:00, 1.41it/s] | |
| Training completed. Do not forget to share your model on huggingface.co/models =) | |
| {'train_runtime': 3159.4128, 'train_samples_per_second': 23.397, 'train_steps_per_second': 0.734, 'train_loss': 0.8618391515622879, 'epoch': 20.0} | |
| 100%|███████████████████████████████████████████████████████████████████████████████████████████████| 2320/2320 [52:39<00:00, 1.36s/it] | |
| Saving model checkpoint to ./wav2vec2-base-timit-fine-tuned | |
| Configuration saved in ./wav2vec2-base-timit-fine-tuned/config.json | |
| Model weights saved in ./wav2vec2-base-timit-fine-tuned/model.safetensors | |
| Feature extractor saved in ./wav2vec2-base-timit-fine-tuned/preprocessor_config.json | |
| tokenizer config file saved in ./wav2vec2-base-timit-fine-tuned/tokenizer_config.json | |
| Special tokens file saved in ./wav2vec2-base-timit-fine-tuned/special_tokens_map.json | |
| added tokens file saved in ./wav2vec2-base-timit-fine-tuned/added_tokens.json | |
| Saving model checkpoint to ./wav2vec2-base-timit-fine-tuned | |
| Configuration saved in ./wav2vec2-base-timit-fine-tuned/config.json | |
| Model weights saved in ./wav2vec2-base-timit-fine-tuned/model.safetensors | |
| Feature extractor saved in ./wav2vec2-base-timit-fine-tuned/preprocessor_config.json | |
| tokenizer config file saved in ./wav2vec2-base-timit-fine-tuned/tokenizer_config.json | |
| Special tokens file saved in ./wav2vec2-base-timit-fine-tuned/special_tokens_map.json | |
| added tokens file saved in ./wav2vec2-base-timit-fine-tuned/added_tokens.json | |
| events.out.tfevents.1716174523.tz579-raptorlake.65634.0: 100%|██████████████████████████████████████| 63.2k/63.2k [00:00<00:00, 232kB/s] | |
| model.safetensors: 100%|█████████████████████████████████████████████████████████████████████████████| 378M/378M [03:30<00:00, 1.79MB/s] | |
| Upload 2 LFS files: 100%|████████████████████████████████████████████████████████████████████████████████| 2/2 [03:31<00:00, 105.69s/it] | |
| ***** train metrics *****████████████████████████████████████████ | 1/2 [03:31<03:31, 211.39s/it] | |
| epoch = 20.0 | |
| total_flos = 2000175347GF | |
| train_loss = 0.8618 | |
| train_runtime = 0:52:39.41 | |
| train_samples = 3696 | |
| train_samples_per_second = 23.397 | |
| train_steps_per_second = 0.734 | |
| 05/19/2024 23:04:57 - INFO - __main__ - *** Evaluate *** | |
| The following columns in the evaluation set don't have a corresponding argument in `Wav2Vec2ForCTC.forward` and have been ignored: input_length. If input_length are not expected by `Wav2Vec2ForCTC.forward`, you can safely ignore this message. | |
| ***** Running Evaluation ***** | |
| Num examples = 1344 | |
| Batch size = 1 | |
| 100%|███████████████████████████████████████████████████████████████████████████████████████████████| 1344/1344 [00:39<00:00, 34.00it/s] | |
| ***** eval metrics ***** | |
| epoch = 20.0 | |
| eval_loss = 0.4275 | |
| eval_runtime = 0:00:39.60 | |
| eval_samples = 1344 | |
| eval_samples_per_second = 33.935 | |
| eval_steps_per_second = 33.935 | |
| eval_wer = 0.4173 | |
| Saving model checkpoint to ./wav2vec2-base-timit-fine-tuned | |
| Configuration saved in ./wav2vec2-base-timit-fine-tuned/config.json | |
| Model weights saved in ./wav2vec2-base-timit-fine-tuned/model.safetensors | |
| Feature extractor saved in ./wav2vec2-base-timit-fine-tuned/preprocessor_config.json | |
| tokenizer config file saved in ./wav2vec2-base-timit-fine-tuned/tokenizer_config.json | |
| Special tokens file saved in ./wav2vec2-base-timit-fine-tuned/special_tokens_map.json | |
| added tokens file saved in ./wav2vec2-base-timit-fine-tuned/added_tokens.json | |
| events.out.tfevents.1716177937.tz579-raptorlake.65634.1: 100%|███████████████████████████████████████████| 406/406 [00:00<00:00, 884B/s] | |