Exception: Caught Exception in DataLoader worker process 0

Guodong_Ma: <h2>I am a fresh of leaning PyTorch. When try to train the TDNN, I got the fellow error and don’t know how to adress it. Thanks in advance.</h2>
<p>File “C:\Users\MGD天真有邪\implement with code\x-vector_pytorch-master\train.py”, line 120, in <br>
Warning, default feature storage mode is now using compression<br>
Warning, default feature storage mode is now using compression<br>
Warning, default feature storage mode is now using compression<br>
Warning, default feature storage mode is now using compression<br>
Warning, default feature storage mode is now using compression<br>
Warning, default feature storage mode is now using compression<br>
Warning, default feature storage mode is now using compression<br>
net = train()<br>
File “C:\Users\MGD天真有邪\implement with code\x-vector_pytorch-master\train.py”, line 73, in train<br>
for i, inputs in enumerate(trainloader , 0):<br>
File “D:\anaconda3\lib\site-packages\torch\utils\data\dataloader.py”, line 819, in <strong>next</strong><br>
Warning, default feature storage mode is now using compression<br>
Warning, default feature storage mode is now using compression<br>
Warning, default feature storage mode is now using compression<br>
return self._process_data(data)<br>
File “D:\anaconda3\lib\site-packages\torch\utils\data\dataloader.py”, line 846, in _process_data<br>
data.reraise()<br>
File “D:\anaconda3\lib\site-packages\torch_utils.py”, line 369, in reraise<br>
raise self.exc_type(msg)<br>
Exception: Caught Exception in DataLoader worker process 0.</p>

ptrblck: <p>Could you set <code>num_workers=0</code>, rerun the code again, and post the error message here?<br>
Using the main thread should give a better message.</p>

Guodong_Ma: <p>Thank you very much first, i get the reason that my dataLoader is error for my Dataset class error. I‘m sorry for my late reply. Thank you again, if i have new problem and expect your guidance.</p>

Kuldeep_Rana: <p>hello, I am facing an issue when I am running the ./train.sh command. Error is displaying not file/directory found but my wave is inside the same directory. This is my command<br>
python main/nnet/trainnew.py --gpus 0 --epochs 50 --checkpoint /media/speech70809/Data02/MS_R1_SL2_Wparameter_update_saved_models --batch-size 4 --num-workers 0</p>
<p>How can I solve this error? thank you so much</p>
<p>2021-10-31 07:16:04 [/media/speech70809/Data01/speech_donoiser_new/main/nnet/trainnew.py:87 - INFO ] Arguments in command:<br>
{‘batch_size’: 4,<br>
‘checkpoint’: ‘/media/speech70809/Data02/MS_R1_SL2_Wparameter_update_saved_models’,<br>
‘epochs’: 50,<br>
‘gpus’: ‘0’,<br>
‘num_workers’: 0,<br>
‘resume’: ‘’}<br>
no_impr:  100<br>
patience:  2<br>
start eval<br>
Traceback (most recent call last):<br>
File “/media/speech70809/Data01/speech_donoiser_new/main/nnet/trainnew.py”, line 91, in <br>
run(args)<br>
File “/media/speech70809/Data01/speech_donoiser_new/main/nnet/trainnew.py”, line 53, in run<br>
trainer.run(train_loader, dev_loader, num_epochs=args.epochs)<br>
File “/media/speech70809/Data01/speech_donoiser_new/main/nnet/libs/trainer.py”, line 221, in run<br>
cv = self.eval(dev_loader)<br>
File “/media/speech70809/Data01/speech_donoiser_new/main/nnet/libs/trainer.py”, line 208, in eval<br>
for egs in data_loader:<br>
File “/media/speech70809/Data01/speech_donoiser_new/main/nnet/libs/dataset.py”, line 143, in <strong>iter</strong><br>
for chunks in self.eg_loader:<br>
File “/home/speech70809/anaconda3/envs/speech_denoiser/lib/python3.9/site-packages/torch/utils/data/dataloader.py”, line 521, in <strong>next</strong><br>
data = self._next_data()<br>
File “/home/speech70809/anaconda3/envs/speech_denoiser/lib/python3.9/site-packages/torch/utils/data/dataloader.py”, line 561, in _next_data<br>
data = self._dataset_fetcher.fetch(index)  # may raise StopIteration<br>
File “/home/speech70809/anaconda3/envs/speech_denoiser/lib/python3.9/site-packages/torch/utils/data/_utils/fetch.py”, line 44, in fetch<br>
data = [self.dataset[idx] for idx in possibly_batched_index]<br>
File “/home/speech70809/anaconda3/envs/speech_denoiser/lib/python3.9/site-packages/torch/utils/data/_utils/fetch.py”, line 44, in <br>
data = [self.dataset[idx] for idx in possibly_batched_index]<br>
File “/media/speech70809/Data01/speech_donoiser_new/main/nnet/libs/dataset.py”, line 41, in <strong>getitem</strong><br>
mix = self.mix[key]<br>
File “/media/speech70809/Data01/speech_donoiser_new/main/nnet/libs/audio.py”, line 120, in <strong>getitem</strong><br>
return self._load(index)<br>
File “/media/speech70809/Data01/speech_donoiser_new/main/nnet/libs/audio.py”, line 138, in _load<br>
samp_rate, samps = read_wav(<br>
File “/media/speech70809/Data01/speech_donoiser_new/main/nnet/libs/audio.py”, line 38, in read_wav<br>
samp_rate, samps_int16 = wf.read(fname)<br>
File “/home/speech70809/anaconda3/envs/speech_denoiser/lib/python3.9/site-packages/scipy/io/wavfile.py”, line 647, in read<br>
fid = open(filename, ‘rb’)<br>
FileNotFoundError: [Errno 2] No such file or directory: ‘/media/lab70809/Data01/speech_donoiser_new/datasets/ner-300hr/cv/mix/BG_20170420_133-JZ_20160301_040_LTL7zFtz8UQ_snr6_fileid_528.wav’</p>

ptrblck: <p>Make sure the file is indeed in this location by running <code>ls</code> on it in a terminal:</p>
<pre><code class="lang-python">~$: ls /media/lab70809/Data01/speech_donoiser_new/datasets/ner-300hr/cv/mix/BG_20170420_133-JZ_20160301_040_LTL7zFtz8UQ_snr6_fileid_528.wav
</code></pre>
<p>of this returns valid file information, I would guess that the mount is somehow dropped during the script execution.</p>

Kuldeep_Rana: <p>ls: cannot access ‘/media/lab70809/Data01/speech_donoiser_new/datasets/ner-300hr/cv/mix/BG_20170420_133-JZ_20160301_040_LTL7zFtz8UQ_snr6_fileid_528.wav’: No such file or directory</p>
<p>after using the ls command it says “cannot access”.<br>
how I can solve this one?</p>

ptrblck: <p>This would mean that the file doesn’t exist and the <code>Dataset</code> raises the error as expected so make sure to load existing files.</p>

Kuldeep_Rana: <p>I have solved this problem. Thank you so much</p>

Jesse_Clark: <p>Kuldeep - would you mind sharing your solution? I am having intermittent errors of the same type.</p>

Kuldeep_Rana: <p>Check your dataset path and the other remaining paths.</p>