ValueError: signal number 32 out of range when loading data with `num_worker>0`

MariosOreo: <p>Hi guys,</p>
<p>When I use <code>Dataloader</code> to load data with <code>num_worker&gt;0</code>, I got the following error:</p>
<pre><code class="lang-auto">Exception in thread Thread-1:
Traceback (most recent call last):
  File "/home/Maro/anaconda3/envs/pytorch/lib/python3.5/threading.py", line 914, in _bootstrap_inner
    self.run()
  File "/home/Maro/anaconda3/envs/pytorch/lib/python3.5/threading.py", line 862, in run
    self._target(*self._args, **self._kwargs)
  File "/home/Maro/anaconda3/envs/pytorch/lib/python3.5/multiprocessing/resource_sharer.py", line 139, in _serve
    signal.pthread_sigmask(signal.SIG_BLOCK, range(1, signal.NSIG))
  File "/home/Maro/anaconda3/envs/pytorch/lib/python3.5/signal.py", line 60, in pthread_sigmask
    sigs_set = _signal.pthread_sigmask(how, mask)
ValueError: signal number 32 out of range

</code></pre>
<p>And I must set <code>num_workers</code> to 0, but it cost too much time in loading data in each iteration.<br>
Is there any good way to solve this problem? Any advice will be highly appreciated.</p>
<p>Thanks in advance.</p>

MariosOreo: <p>Does the error related to the os configuration?</p>
<p>I have run the same code on two server with the same os (Ubuntu16.04.1) but different gpu. And set <code>num_workers=12</code> in DataLoader for both. One can run normally but another raise the ValueError mentioned above.</p>

ptrblck: <p>I’ve never seen this error on my machine, but searching for it it looks like Python 3.7 could fix this issue.<br>
Would it be possible for you to create a new conda environment and try the code with Python 3.7?<br>
If not, could you post a code snippet so that we can reproduce this error?</p>

MariosOreo: <p>Oh, <a class="mention" href="/u/ptrblck">@ptrblck</a> thank you very much, your help was greatly appreciated!</p>
<p>It resolved by updating Python3.5-&gt;Python3.7.<a href="https://bugs.python.org/issue33329" rel="nofollow noopener">https://bugs.python.org/issue33329</a></p>
<p>Another question, though it does not relate to this topic:<br>
If I have two cards ( card0 and card1 ) and train my network just on card1, when I set <code>pin_memory=True</code> in <code>DataLoader</code> it will occupy a portion of card0’s memory ( it seems to be the default  ), is there any way to assign memory consumption to card1?</p>
<p>Thanks a lot.<img src="https://discuss.pytorch.org/images/emoji/apple/wink.png?v=6" title=":wink:" class="emoji" alt=":wink:"></p>
<p>PS: In another machine it works fine with Python3.5, it is so strange and I dont know what makes the mistake.</p>

ptrblck: <p>I assume the little memory will be allocated on GPU0.<br>
If that’s the case, you can run your script using:</p>
<pre><code class="lang-auto">CUDA_VISIBLE_DEVICES=1 python script.py args
</code></pre>
<p>to hide GPU0 in your script. GPU1 will then be remapped to GPU0, so that you would have to use <code>'cuda:0'</code>or just <code>'cuda'</code> in your script.</p>
<p>Which PyTorch version are you using btw? I thought this issue was solved recently.</p>
<p>I haven’t seen this issue before, but apparently it’s related to some <code>multiprocessing</code> functions in Python.</p>

MariosOreo: <p>Thanks a lot.</p>
<p>I will have a try on it.<br>
My PyTorch version is 1.0.1.post2, and on another machine is 1.0.0 which worked fine before.</p>

Joker: <p>Yes  , it is        a   python  bug  , due  to  this  isseu    <a href="https://bugs.python.org/issue33329" rel="nofollow noopener">https://bugs.python.org/issue33329</a> ,  it got fixed in 3.6.6</p>

drscotthawley: <p>I get this error when running python 3.7.7</p>