Hello,I have this problem in running code！

Bcw_93: <p>I have this error in running this code.Anyone can help me?<br>
Traceback (most recent call last):<br>
File “d:\VsCode Project\nlp-tutorial-master\5-2.BERT\BERT-Torch.py”, line 229, in <br>
logits_lm, logits_clsf = model(input_ids.to(device), segment_ids.to(device), masked_pos.to(device))<br>
File “C:\Users\Administrator\AppData\Local\Programs\Python\Python37\lib\site-packages\torch\nn\modules\module.py”, line 550, in <strong>call</strong><br>
result = self.forward(*input, **kwargs)<br>
File “d:\VsCode Project\nlp-tutorial-master\5-2.BERT\BERT-Torch.py”, line 202, in forward<br>
output, enc_self_attn = layer(output.to(device), enc_self_attn_mask.to(device))<br>
File “C:\Users\Administrator\AppData\Local\Programs\Python\Python37\lib\site-packages\torch\nn\modules\module.py”, line 550, in <strong>call</strong><br>
result = self.forward(*input, **kwargs)<br>
File “d:\VsCode Project\nlp-tutorial-master\5-2.BERT\BERT-Torch.py”, line 177, in forward<br>
enc_outputs = self.pos_ffn(enc_outputs) # enc_outputs: [batch_size x len_q x d_model]<br>
File “C:\Users\Administrator\AppData\Local\Programs\Python\Python37\lib\site-packages\torch\nn\modules\module.py”, line 550, in <strong>call</strong><br>
result = self.forward(*input, **kwargs)<br>
File “d:\VsCode Project\nlp-tutorial-master\5-2.BERT\BERT-Torch.py”, line 166, in forward<br>
return self.fc2(gelu(self.fc1(x)))<br>
File “d:\VsCode Project\nlp-tutorial-master\5-2.BERT\BERT-Torch.py”, line 102, in gelu<br>
return x * 0.5 * (1.0 + torch.erf(x / math.sqrt(2.0)))<br>
RuntimeError: CUDA error: an illegal memory access was encountered</p>

ptrblck: <p>Is your code running fine on the CPU?<br>
This would probably give you a better error message than the illegal memory access.</p>
<p>Also, if you are using PyTorch <code>1.5.0</code>, I would strongly recommend to update to <code>1.5.1</code>, as <code>1.5.0</code> has a bug, where internal CUDA assert statements are skipped and will not raise a proper error but will run into other potentially issues (such as an illegal memory access).</p>
<p>If the code is running fine on the CPU and you are already using <code>1.5.1</code>, rerun the code with</p>
<pre><code class="lang-auto">CUDA_LAUNCH_BLOCKING=1 python script.py args
</code></pre>
<p>and post the stack trace here, please.</p>

Sagnik_Mukherjee: <p>I also ran into this issue. And found the solution here,<br>
</p><aside class="onebox githubissue">
  <header class="source">
      <a href="https://github.com/pytorch/pytorch/issues/37449" target="_blank" rel="nofollow noopener">github.com/pytorch/pytorch</a>
  </header>
  <article class="onebox-body">
    <div class="github-row">
  <div class="github-icon-container" title="Issue">
	  <svg width="60" height="60" class="github-icon" viewBox="0 0 14 16" aria-hidden="true"><path d="M7 2.3c3.14 0 5.7 2.56 5.7 5.7s-2.56 5.7-5.7 5.7A5.71 5.71 0 0 1 1.3 8c0-3.14 2.56-5.7 5.7-5.7zM7 1C3.14 1 0 4.14 0 8s3.14 7 7 7 7-3.14 7-7-3.14-7-7-7zm1 3H6v5h2V4zm0 6H6v2h2v-2z"></path></svg>
  </div>

  <div class="github-info-container">
    <h4>
      <a href="https://github.com/pytorch/pytorch/issues/37449" target="_blank" rel="nofollow noopener">RuntimeError: CUDA error: an illegal memory access was encountered with channels_last</a>
    </h4>

    <div class="github-info">
      <div class="date">
        opened <span class="discourse-local-date" data-format="ll" data-date="2020-04-28" data-time="19:35:49" data-timezone="UTC">07:35PM - 28 Apr 20 UTC</span>
      </div>


      <div class="user">
        <a href="https://github.com/tstandley" target="_blank" rel="nofollow noopener">
          <img alt="tstandley" src="https://avatars2.githubusercontent.com/u/17170654?v=4" class="onebox-avatar-inline" width="20" height="20">
          tstandley
        </a>
      </div>
    </div>
  </div>
</div>

<div class="github-row">
  <p class="github-content">I get an illegal memory access when trying to train mnasnet (any version) with apex (O1) and channels_last
To Reproduce
Steps to reproduce...</p>
</div>

<div class="labels">
    <span style="display:inline-block;margin-top:2px;background-color: #B8B8B8;padding: 2px;border-radius: 4px;color: #fff;margin-left: 3px;">high priority</span>
    <span style="display:inline-block;margin-top:2px;background-color: #B8B8B8;padding: 2px;border-radius: 4px;color: #fff;margin-left: 3px;">module: binaries</span>
    <span style="display:inline-block;margin-top:2px;background-color: #B8B8B8;padding: 2px;border-radius: 4px;color: #fff;margin-left: 3px;">module: cuda</span>
    <span style="display:inline-block;margin-top:2px;background-color: #B8B8B8;padding: 2px;border-radius: 4px;color: #fff;margin-left: 3px;">module: cudnn</span>
    <span style="display:inline-block;margin-top:2px;background-color: #B8B8B8;padding: 2px;border-radius: 4px;color: #fff;margin-left: 3px;">topic: dependency bug</span>
    <span style="display:inline-block;margin-top:2px;background-color: #B8B8B8;padding: 2px;border-radius: 4px;color: #fff;margin-left: 3px;">triaged</span>
</div>

  </article>
  <div class="onebox-metadata">
    
    
  </div>
  <div style="clear: both"></div>
</aside>
<br>
Apparently the latest cudnn v8 has it fixed.