Device-side assert triggered after mask softmax output

username: Jie_Han_Chen
cooked: <p>I’ve implemented a reinforcement learning algorithm, A3C, which shared the model parameters between global model and local models.</p>
<p>A3C samples the action from softmax output like: <code>policy_head.multinomial()</code>. In my scenario, I need to mask the output of not available actions in current status. The mask variable contains FloatTensor which the value of elements is either 0.0 or 1.0.  The masking code is like:</p>
<pre><code class="lang-auto">selection_mask = torch.from_numpy((state.observation['screen'][_SCREEN_PLAYER_RELATIVE] == 1).astype('float32'))
selection_mask = Variable(selection_mask.view(1, -1), requires_grad=False).cuda(args['gpu'])
select_spatial_action_prob = select_spatial_action_prob * selection_mask
select_action = select_spatial_action_prob.multinomial()
print("select action:", select_action)
select_entropy = - (log_select_spatial_action_prob * select_spatial_action_prob).sum(1)
</code></pre>
<p>After training few steps (around 160K steps), some error occurred. When I dived into the variable of masked <code>select_spatial_action_prob</code>, I found all the value in that variable is <strong>nan</strong>, and I don’t know how to fix it.</p>
<p>Here are my code of training part and model part.<br>
training code: <a href="https://gist.github.com/JIElite/a2352517b985b1d9102091ac843aff9d" class="inline-onebox" rel="noopener nofollow ugc">train.py · GitHub</a><br>
model: <a href="https://gist.github.com/JIElite/174450bb9aff72a92ec698a85e66b6fc" class="inline-onebox" rel="noopener nofollow ugc">model.py · GitHub</a></p>
<p>The error message is as following:</p>
<blockquote>
<p>Traceback (most recent call last):<br>
File “/home/jielite/anaconda3/lib/python3.6/multiprocessing/process.py”, line 258, in _bootstrap<br>
self.run()<br>
File “/home/jielite/anaconda3/lib/python3.6/multiprocessing/process.py”, line 93, in run<br>
self._target(*self._args, **self._kwargs)<br>
File “/home/jielite/pysc2-A3C/sc2/train_hierarchical.py”, line 85, in train_master<br>
print(“select action:”, select_action)<br>
File “/home/jielite/anaconda3/lib/python3.6/site-packages/torch/autograd/variable.py”, line 119, in <strong>repr</strong><br>
return ‘Variable containing:’ + self.data.<strong>repr</strong>()<br>
File “/home/jielite/anaconda3/lib/python3.6/site-packages/torch/tensor.py”, line 144, in <strong>repr</strong><br>
return str(self)<br>
File “/home/jielite/anaconda3/lib/python3.6/site-packages/torch/tensor.py”, line 151, in <strong>str</strong><br>
return _tensor_str._str(self)<br>
File “/home/jielite/anaconda3/lib/python3.6/site-packages/torch/_tensor_str.py”, line 297, in _str<br>
strt = _matrix_str(self)<br>
File “/home/jielite/anaconda3/lib/python3.6/site-packages/torch/_tensor_str.py”, line 216, in _matrix_str<br>
min_sz=5 if not print_full_mat else 0)<br>
File “/home/jielite/anaconda3/lib/python3.6/site-packages/torch/_tensor_str.py”, line 79, in _number_format</p>
</blockquote>
<blockquote>
<p>RuntimeError: cuda runtime error (59) : device-side assert triggered at /opt/conda/conda-bld/pytorch_1518244421288/work/torch/lib/THC/generic/THCTensorCopy.c:70<br>
/opt/conda/conda-bld/pytorch_1518244421288/work/torch/lib/THC/THCTensorRandom.cuh:179: void sampleMultinomialOnce(long *, long, int, T *, T *) [with T = float, AccT = float]: block: [0,0,0], thread: [704,0,0] Assertion <code>THCNumerics&lt;T&gt;::ge(val, zero)</code> failed.<br>
/opt/conda/conda-bld/pytorch_1518244421288/work/torch/lib/THC/THCTensorRandom.cuh:179: void sampleMultinomialOnce(long *, long, int, T *, T *) [with T = float, AccT = float]: block: [0,0,0], thread: [705,0,0] Assertion <code>THCNumerics&lt;T&gt;::ge(val, zero)</code> failed.<br>
/opt/conda/conda-bld/pytorch_1518244421288/work/torch/lib/THC/THCTensorRandom.cuh:179: void sampleMultinomialOnce(long *, long, int, T *, T *) [with T = float, AccT = float]: block: [0,0,0], thread: [706,0,0] Assertion <code>THCNumerics&lt;T&gt;::ge(val, zero)</code> failed.<br>
/opt/conda/conda-bld/pytorch_1518244421288/work/torch/lib/THC/THCTensorRandom.cuh:179: void sampleMultinomialOnce(long *, long, int, T *, T *) [with T = float, AccT = float]: block: [0,0,0], thread: [707,0,0] Assertion <code>THCNumerics&lt;T&gt;::ge(val, zero)</code> failed.<br>
/opt/conda/conda-bld/pytorch_1518244421288/work/torch/lib/THC/THCTensorRandom.cuh:179: void sampleMultinomialOnce(long *, long, int, T *, T *) [with T = float, AccT = float]: block: [0,0,0], thread: [708,0,0] Assertion <code>THCNumerics&lt;T&gt;::ge(val, zero)</code> failed.</p>
</blockquote>

username: richard
cooked: <p>What version of PyTorch are you using? I know there were a few bugs around multinomial before that should be fixed on master.</p>

username: Jie_Han_Chen
cooked: <p>I use pytorch 0.3.1 installed by conda with CUDA 9.0</p>

username: Ricardo_Gama
cooked: <p>Hello. I’m facing a similar problem.<br>
Have you found a workaround?<br>
Thanks.</p>

username: Jie_Han_Chen
cooked: <p>This error in my case is from zero-division error, which means the sum of exp(output_i) is 0.<br>
You could try to think how to avoid zero-division error in your problem, if your error was also caused by zero-division error.</p>

username: TonyS
cooked: <p>Hi, I have a similar issue. I didn’t add a Relu() layer before Softmax(), so sometimes the sum of the previously FC output is zero, which will cause the error. Hope this will help you guys get out of this annoying bug quickly. <img src="https://discuss.pytorch.org/images/emoji/apple/crazy_face.png?v=9" title=":crazy_face:" class="emoji" alt=":crazy_face:"></p>