asr_oxford_frontend_raw_cy_bpe150_sp/train.5.log

# python3 -m espnet2.bin.asr_train --use_preprocessor true --bpemodel data/cy_token_list/bpe_unigram150/bpe.model --token_type bpe --token_list data/cy_token_list/bpe_unigram150/tokens.txt --non_linguistic_symbols none --cleaner none --g2p none --valid_data_path_and_name_and_type dump/raw/dev_cy/wav.scp,speech,sound --valid_data_path_and_name_and_type dump/raw/dev_cy/text,text,text --valid_shape_file exp/asr_stats_raw_cy_bpe150_sp/valid/speech_shape --valid_shape_file exp/asr_stats_raw_cy_bpe150_sp/valid/text_shape.bpe --resume true --init_param --ignore_init_mismatch false --fold_length 80000 --fold_length 150 --output_dir exp/asr_oxford_frontend_raw_cy_bpe150_sp --config conf/tuning/oxford_frontend.yaml --frontend_conf fs=16k --train_data_path_and_name_and_type dump/raw/train_cy_sp/wav.scp,speech,sound --train_data_path_and_name_and_type dump/raw/train_cy_sp/text,text,text --train_shape_file exp/asr_stats_raw_cy_bpe150_sp/train/speech_shape --train_shape_file exp/asr_stats_raw_cy_bpe150_sp/train/text_shape.bpe --ngpu 3 --multiprocessing_distributed True 
# Started at Mon Jun  6 13:35:52 EDT 2022
#
/usr/bin/python3 /project/ocean/junweih/espnet/espnet2/bin/asr_train.py --use_preprocessor true --bpemodel data/cy_token_list/bpe_unigram150/bpe.model --token_type bpe --token_list data/cy_token_list/bpe_unigram150/tokens.txt --non_linguistic_symbols none --cleaner none --g2p none --valid_data_path_and_name_and_type dump/raw/dev_cy/wav.scp,speech,sound --valid_data_path_and_name_and_type dump/raw/dev_cy/text,text,text --valid_shape_file exp/asr_stats_raw_cy_bpe150_sp/valid/speech_shape --valid_shape_file exp/asr_stats_raw_cy_bpe150_sp/valid/text_shape.bpe --resume true --init_param --ignore_init_mismatch false --fold_length 80000 --fold_length 150 --output_dir exp/asr_oxford_frontend_raw_cy_bpe150_sp --config conf/tuning/oxford_frontend.yaml --frontend_conf fs=16k --train_data_path_and_name_and_type dump/raw/train_cy_sp/wav.scp,speech,sound --train_data_path_and_name_and_type dump/raw/train_cy_sp/text,text,text --train_shape_file exp/asr_stats_raw_cy_bpe150_sp/train/speech_shape --train_shape_file exp/asr_stats_raw_cy_bpe150_sp/train/text_shape.bpe --ngpu 3 --multiprocessing_distributed True
[islpc50:0/3] 2022-06-06 13:36:07,422 (distributed_c10d:217) INFO: Added key: store_based_barrier_key:1 to store for rank: 0
[islpc50:0/3] 2022-06-06 13:36:07,423 (distributed_c10d:251) INFO: Rank 0: Completed store-based barrier for key:store_based_barrier_key:1 with 3 nodes.
[islpc50:0/3] 2022-06-06 13:36:07,465 (asr:411) INFO: Vocabulary size: 150
[islpc50:0/3] 2022-06-06 13:36:07,990 (filelock:274) INFO: Lock 140513801612592 acquired on ./hub/s3prl_cache/1c76d6e88090f01736036b28dc995fef583f47f42662d55286332557f957609f.lock
[islpc50:0/3] 2022-06-06 13:36:07,991 (filelock:318) INFO: Lock 140513801612592 released on ./hub/s3prl_cache/1c76d6e88090f01736036b28dc995fef583f47f42662d55286332557f957609f.lock
[s3prl.upstream.experts] Warning: can not import s3prl.upstream.byol_a.expert: No module named 'easydict'. Pass.
[s3prl.hub] Warning: can not import s3prl.upstream.byol_a.hubconf: No module named 'easydict'. Please see upstream/byol_a/README.md
[s3prl.downstream.experts] Warning: can not import s3prl.downstream.quesst14_dtw.expert: No module named 'dtw'. Pass.
[s3prl.downstream.experts] Warning: can not import s3prl.downstream.separation_stft.expert: No module named 'asteroid'. Pass.
[s3prl.downstream.experts] Warning: can not import s3prl.downstream.enhancement_stft.expert: No module named 'asteroid'. Pass.
[s3prl.downstream.experts] Warning: can not import s3prl.downstream.speech_commands.expert: No module named 'catalyst'. Pass.
[s3prl.downstream.experts] Warning: can not import s3prl.downstream.a2a-vc-vctk.expert: No module named 'resemblyzer'. Pass.
[s3prl.downstream.experts] Warning: can not import s3prl.downstream.voxceleb2_ge2e.expert: No module named 'sox'. Pass.
[s3prl.downstream.experts] Warning: can not import s3prl.downstream.sv_voxceleb1.expert: No module named 'sox'. Pass.
Using cache found in ./hub/s3prl_cache/1c76d6e88090f01736036b28dc995fef583f47f42662d55286332557f957609f
for https://dl.fbaipublicfiles.com/fairseq/wav2vec/wav2vec_vox_new.pt
>> inserted adapters to the following layers: 0, 1, 2
  * original model weights: 317,390,592
  * new model weights - all: 319,757,184
  * new model weights - trainable: 2,366,592 ( 0.75% of original model)
[Featurizer] - The selected feature last_hidden_state's downsample rate is 320
Process SpawnProcess-1:
Traceback (most recent call last):
  File "/usr/lib/python3.8/multiprocessing/process.py", line 315, in _bootstrap
    self.run()
  File "/usr/lib/python3.8/multiprocessing/process.py", line 108, in run
    self._target(*self._args, **self._kwargs)
  File "/project/ocean/junweih/espnet/espnet2/tasks/abs_task.py", line 1121, in main_worker
    model = cls.build_model(args=args)
  File "/project/ocean/junweih/espnet/espnet2/tasks/asr.py", line 450, in build_model
    encoder_class = encoder_choices.get_class(args.encoder)
  File "/project/ocean/junweih/espnet/espnet2/train/class_choices.py", line 75, in get_class
    raise ValueError(
ValueError: --encoder must be one of ('conformer', 'transformer', 'contextual_block_transformer', 'contextual_block_conformer', 'vgg_rnn', 'rnn', 'wav2vec2', 'hubert', 'hubert_pretrain', 'longformer'): --encoder linear
[s3prl.upstream.experts] Warning: can not import s3prl.upstream.byol_a.expert: No module named 'easydict'. Pass.
[s3prl.hub] Warning: can not import s3prl.upstream.byol_a.hubconf: No module named 'easydict'. Please see upstream/byol_a/README.md
[s3prl.downstream.experts] Warning: can not import s3prl.downstream.quesst14_dtw.expert: No module named 'dtw'. Pass.
[s3prl.downstream.experts] Warning: can not import s3prl.downstream.separation_stft.expert: No module named 'asteroid'. Pass.
[s3prl.downstream.experts] Warning: can not import s3prl.downstream.enhancement_stft.expert: No module named 'asteroid'. Pass.
[s3prl.downstream.experts] Warning: can not import s3prl.downstream.speech_commands.expert: No module named 'catalyst'. Pass.
[s3prl.downstream.experts] Warning: can not import s3prl.downstream.a2a-vc-vctk.expert: No module named 'resemblyzer'. Pass.
[s3prl.downstream.experts] Warning: can not import s3prl.downstream.voxceleb2_ge2e.expert: No module named 'sox'. Pass.
[s3prl.downstream.experts] Warning: can not import s3prl.downstream.sv_voxceleb1.expert: No module named 'sox'. Pass.
Using cache found in ./hub/s3prl_cache/1c76d6e88090f01736036b28dc995fef583f47f42662d55286332557f957609f
for https://dl.fbaipublicfiles.com/fairseq/wav2vec/wav2vec_vox_new.pt
>> inserted adapters to the following layers: 0, 1, 2
  * original model weights: 317,390,592
  * new model weights - all: 319,757,184
  * new model weights - trainable: 2,366,592 ( 0.75% of original model)
Process SpawnProcess-2:
Traceback (most recent call last):
  File "/usr/lib/python3.8/multiprocessing/process.py", line 315, in _bootstrap
    self.run()
  File "/usr/lib/python3.8/multiprocessing/process.py", line 108, in run
    self._target(*self._args, **self._kwargs)
  File "/project/ocean/junweih/espnet/espnet2/tasks/abs_task.py", line 1121, in main_worker
    model = cls.build_model(args=args)
  File "/project/ocean/junweih/espnet/espnet2/tasks/asr.py", line 450, in build_model
    encoder_class = encoder_choices.get_class(args.encoder)
  File "/project/ocean/junweih/espnet/espnet2/train/class_choices.py", line 75, in get_class
    raise ValueError(
ValueError: --encoder must be one of ('conformer', 'transformer', 'contextual_block_transformer', 'contextual_block_conformer', 'vgg_rnn', 'rnn', 'wav2vec2', 'hubert', 'hubert_pretrain', 'longformer'): --encoder linear
[s3prl.upstream.experts] Warning: can not import s3prl.upstream.byol_a.expert: No module named 'easydict'. Pass.
[s3prl.hub] Warning: can not import s3prl.upstream.byol_a.hubconf: No module named 'easydict'. Please see upstream/byol_a/README.md
[s3prl.downstream.experts] Warning: can not import s3prl.downstream.quesst14_dtw.expert: No module named 'dtw'. Pass.
[s3prl.downstream.experts] Warning: can not import s3prl.downstream.separation_stft.expert: No module named 'asteroid'. Pass.
[s3prl.downstream.experts] Warning: can not import s3prl.downstream.enhancement_stft.expert: No module named 'asteroid'. Pass.
[s3prl.downstream.experts] Warning: can not import s3prl.downstream.speech_commands.expert: No module named 'catalyst'. Pass.
[s3prl.downstream.experts] Warning: can not import s3prl.downstream.a2a-vc-vctk.expert: No module named 'resemblyzer'. Pass.
[s3prl.downstream.experts] Warning: can not import s3prl.downstream.voxceleb2_ge2e.expert: No module named 'sox'. Pass.
[s3prl.downstream.experts] Warning: can not import s3prl.downstream.sv_voxceleb1.expert: No module named 'sox'. Pass.
Using cache found in ./hub/s3prl_cache/1c76d6e88090f01736036b28dc995fef583f47f42662d55286332557f957609f
for https://dl.fbaipublicfiles.com/fairseq/wav2vec/wav2vec_vox_new.pt
>> inserted adapters to the following layers: 0, 1, 2
  * original model weights: 317,390,592
  * new model weights - all: 319,757,184
  * new model weights - trainable: 2,366,592 ( 0.75% of original model)
Process SpawnProcess-3:
Traceback (most recent call last):
  File "/usr/lib/python3.8/multiprocessing/process.py", line 315, in _bootstrap
    self.run()
  File "/usr/lib/python3.8/multiprocessing/process.py", line 108, in run
    self._target(*self._args, **self._kwargs)
  File "/project/ocean/junweih/espnet/espnet2/tasks/abs_task.py", line 1121, in main_worker
    model = cls.build_model(args=args)
  File "/project/ocean/junweih/espnet/espnet2/tasks/asr.py", line 450, in build_model
    encoder_class = encoder_choices.get_class(args.encoder)
  File "/project/ocean/junweih/espnet/espnet2/train/class_choices.py", line 75, in get_class
    raise ValueError(
ValueError: --encoder must be one of ('conformer', 'transformer', 'contextual_block_transformer', 'contextual_block_conformer', 'vgg_rnn', 'rnn', 'wav2vec2', 'hubert', 'hubert_pretrain', 'longformer'): --encoder linear
Traceback (most recent call last):
  File "/usr/lib/python3.8/runpy.py", line 194, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/usr/lib/python3.8/runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "/project/ocean/junweih/espnet/espnet2/bin/asr_train.py", line 23, in <module>
    main()
  File "/project/ocean/junweih/espnet/espnet2/bin/asr_train.py", line 19, in main
    ASRTask.main(cmd=cmd)
  File "/project/ocean/junweih/espnet/espnet2/tasks/abs_task.py", line 1069, in main
    while not ProcessContext(processes, error_queues).join():
  File "/project/ocean/junweih/espnet/tools/python_user_base/lib/python3.8/site-packages/torch/multiprocessing/spawn.py", line 139, in join
    raise ProcessExitedException(
torch.multiprocessing.spawn.ProcessExitedException: process 0 terminated with exit code 1
# Accounting: time=28 threads=1
# Ended (code 1) at Mon Jun  6 13:36:20 EDT 2022, elapsed time 28 seconds