Model inference inside custom dataset class

Sadegh_Aliakbarian: <p>Hi,</p>
<p>As part of my code, my model needs features of another pre-trained model (call it feature-net) as input. To this end, I fine-tuned feature-net, and now I want to do the inference inside my custom dataset class. Inside the custom dataset class, I get some images, do the inference, and return extracted features in <strong>getitem</strong>() function. However, when I use my custom dataset, I receive this error:</p>
<pre><code class="lang-auto">Traceback (most recent call last):
  File "demo.py", line 133, in &lt;module&gt;
    model  = train_model(model, criterion, optimizer_ft, exp_lr_scheduler, dataloaders, use_gpu, dataset_sizes, num_epochs=25)
  File "demo.py", line 46, in train_model
    for data in dataloaders[phase]:
  File "some/dir/.local/lib/python2.7/site-packages/torch/utils/data/dataloader.py", line 281, in __next__
    return self._process_next_batch(batch)
  File "some/dir/.local/lib/python2.7/site-packages/torch/utils/data/dataloader.py", line 301, in _process_next_batch
    raise batch.exc_type(batch.exc_msg)
RuntimeError: Traceback (most recent call last):
  File "some/dir/.local/lib/python2.7/site-packages/torch/utils/data/dataloader.py", line 55, in _worker_loop
    samples = collate_fn([dataset[i] for i in batch_indices])
  File "some/dir/pytorch/pytorch_example/video_decoder.py", line 156, in __getitem__
    features = extract_ResNet18_feature(self.feature_extractor, Variable(frames.cuda()))
  File "some/dir/.local/lib/python2.7/site-packages/torch/_utils.py", line 69, in _cuda
    return new_type(self.size()).copy_(self, async)
  File "some/dir/.local/lib/python2.7/site-packages/torch/cuda/__init__.py", line 384, in _lazy_new
    _lazy_init()
  File "some/dir/.local/lib/python2.7/site-packages/torch/cuda/__init__.py", line 140, in _lazy_init
    "Cannot re-initialize CUDA in forked subprocess. " + msg)
RuntimeError: Cannot re-initialize CUDA in forked subprocess. To use CUDA with multiprocessing, you must use Python 3.4+ and the 'spawn' start method

</code></pre>
<p>BTW, I should use Python 2.x. Does anyone have an idea how to address this?</p>
<p>Cheers.</p>

ptrblck: <p>Did you use <code>num_workers</code> &gt; 0 and did you use your model on the GPU?<br>
If so, it might be a bit problematic, since the <code>DataLoader</code> wants to use multi-processing, which apparently clashes with the CUDA calls.</p>
<p>With <code>num_workers=0</code> it’ll work, but I’m not sure, if it will be the bottleneck in your application, since the main thread will be used to load the data.</p>
<p>Another approach would be to store all processed features as a <code>Tensor</code> beforehand and then just load it in a <code>Dataset</code>.</p>

Sadegh_Aliakbarian: <p>Thank you very much for your reply. Setting num_workers=0 solved the problem. I see if it becomes too slow, I will stored the computed features as Tensor.</p>
<p>Thanks</p>

mhubii: <p><a class="mention" href="/u/haydnspass">@Haydnspass</a> proposal: you might alternatively make the feature extractor part of your <code>Module</code>, rather than calling it in the <code>Dataset</code> or pre-computing it</p>