RuntimeError: diff_view_meta->output_nr_ == 0 only happen in GPU

username: Boazrciasn
cooked: <p>Hello the following code snippet gives the error in the title. Snippet works on cpu but fails on 2 GPU environment, I haven’t checked with a single GPU. Code particularly fails on w_theta_sin line.</p>
<pre><code>def forward(self, x, a):
    with torch.no_grad():
        self.W_hat.div_(torch.norm(self.W_hat, dim=5, keepdim=True))
        self.W_theta.fmod_(math.pi)
    W_theta_sin = torch.sin(self.W_theta) + eps
</code></pre>
<p>RuntimeError: diff_view_meta-&gt;output_nr_ == 0 ASSERT FAILED at /opt/conda/conda-bld/pytorch_1556653215914/work/torch/csrc/autograd/variable.cpp:209, please report a bug to PyTorch.</p>
<p>Searched the same error, I know that there is an open issue, but I did not understand much of it tbh:) is there a workaround for this?</p>
<p>Thanks in advance.</p>

username: ptrblck
cooked: <p>Are you using <code>nn.DataParallel</code>?<br>
If so, could you post the shapes you’ve used (parameters and inputs) to raise this error?</p>

username: Boazrciasn
cooked: <p>Yes I am using nn.DataParallel , sorry for sharing small snippets, it is a fairly large script and my research.</p>
<p>self.W_theta = nn.Parameter(torch.rand(1, 1, 1,  in_channels, out_channels, 1, 1, 1))<br>
self.W_hat = nn.Parameter(torch.rand(1, 1, 1,  in_channels, out_channels, 3, 1, 1))</p>
<p>Input is a custom dataset with 2 channels. Everything is fine except this section I have done many experiments before adding this part.</p>

username: ptrblck
cooked: <p>Were you able to isolate this issue further or do you have a reproducible code snippet or is this issue reproducible using just the provided information regarding <code>W_theta</code> and <code>W_hat</code>?</p>
<p>Also, could you share the shapes of <code>x</code> and <code>a</code>?</p>

username: Boazrciasn
cooked: <p>I had to drop the reassignment in order to avoid the error in my further experiments. I am sure that error happens at the following lines.</p>
<pre><code>with torch.no_grad():
    self.W_hat.div_(torch.norm(self.W_hat, dim=5, keepdim=True))
    self.W_theta.fmod_(math.pi)
</code></pre>
<p>I will try to write a small snippet for reproduction in coming weeks, I only have one computer with gpu and that is working atm <img src="https://discuss.pytorch.org/images/emoji/apple/slight_smile.png?v=9" title=":slight_smile:" class="emoji" alt=":slight_smile:"> Sharing a and x would not help you, there are several steps before I use w_theta_sin I need to simplify it to make it easier for you to trace it.</p>
<p>I think this issue is also related, but I am using 1.1.0:<br>
<aside class="onebox githubissue">
  <header class="source">
      <a href="https://github.com/pytorch/pytorch/issues/14365" target="_blank" rel="nofollow noopener">github.com/pytorch/pytorch</a>
  </header>
  <article class="onebox-body">
    <a href="https://github.com/neocheema" rel="nofollow noopener">
<img src="https://avatars1.githubusercontent.com/u/427941?v=2&amp;s=96" class="thumbnail onebox-avatar" width="60" height="60">
</a>

<h4><a href="https://github.com/pytorch/pytorch/issues/14365" target="_blank" rel="nofollow noopener">Issue: Assertion fails when using DataParallel with two nn.Embedding </a></h4>

<div class="date" style="margin-top:10px;">
	<div class="user" style="margin-top:10px;">
	opened by <a href="https://github.com/neocheema" target="_blank" rel="nofollow noopener">neocheema</a>
	on <a href="https://github.com/pytorch/pytorch/issues/14365" target="_blank" rel="nofollow noopener">2018-11-26</a>
	</div>
	<div class="user">
	closed by <a href="https://github.com/yf225" target="_blank" rel="nofollow noopener">yf225</a>
	on <a href="https://github.com/pytorch/pytorch/issues/14365" target="_blank" rel="nofollow noopener"></a>
	</div>
</div>

<pre class="content" style="white-space: pre-wrap;">🐛 Bug
I'm using the nightly build: 1.0.0.dev20181123. This issue is very similar to #13569 . When I instantiate two nn.Embedding, with...</pre>

<div class="labels">
 	<span style="display:inline-block;margin-top:2px;background-color: #B8B8B8;padding: 2px;border-radius: 4px;color: #fff;margin-left: 3px;">high priority</span>
 	<span style="display:inline-block;margin-top:2px;background-color: #B8B8B8;padding: 2px;border-radius: 4px;color: #fff;margin-left: 3px;">module: autograd</span>
 	<span style="display:inline-block;margin-top:2px;background-color: #B8B8B8;padding: 2px;border-radius: 4px;color: #fff;margin-left: 3px;">triaged</span>
</div>

  </article>
  <div class="onebox-metadata">
    
    
  </div>
  <div style="clear: both"></div>
</aside>
</p>
<p>Thanks for asking,</p>