<UnsafeViewBackward> in grad_fn

b_tavlet: <p>Hi,<br>
While tracking down my gradient functions, I came accross this label  . As my model fails to learn, I was wondering whether that could be related to that.<br>
And more generally speaking, I havent found much information related to that, what does this “Unsafe” account for ?<br>
Thanks</p>

CarlosMendoza: <p>I also realized I have a variable with grad_fn=UnsafeViewBackward after using torchviz. I think that this is related to the function <code>_unsafe_view</code>, which is defined in  <a href="https://github.com/pytorch/pytorch/blob/master/aten/src/ATen/native/TensorShape.cpp" rel="nofollow noopener">https://github.com/pytorch/pytorch/blob/master/aten/src/ATen/native/TensorShape.cpp</a>. There is a comment just before the definition that might be helpful. In my case this happened to the variable <code>x_hat</code>, which is created using the following line</p>
<p><code>x_hat = F.linear(Z.view(batch_size,1,-1),self.D)</code></p>
<p>I found that _unsafe_view () is called inside matmul(), which is in turn called by F.linear, being <code>F</code> just <code>torch.nn.functional</code>. See here: <a href="https://github.com/pytorch/pytorch/blob/master/aten/src/ATen/native/LinearAlgebra.cpp" rel="nofollow noopener">https://github.com/pytorch/pytorch/blob/master/aten/src/ATen/native/LinearAlgebra.cpp</a></p>

ptrblck: <p>From the comment in the <a href="https://github.com/pytorch/pytorch/blob/af64789cfa3aef37926fb3d9601d5e47406098c8/aten/src/ATen/native/TensorShape.cpp#L959" rel="nofollow noopener">code</a>:</p>
<pre><code class="lang-auto">// _unsafe_view() differs from view() in that the returned tensor isn't treated
// as a view for the purposes of automatic differentiation. (It's not listed in
// VIEW_FUNCTIONS in gen_autograd.py).  It's only safe to use if the `self` tensor
// is temporary. For example, the viewed tensor here (a + b) is discarded immediately
// after viewing:
//
//  res = at::_unsafe_view(a + b, size);
//
// This is a hack because in-place operations on tensors treated like views
// can be much more expensive than the same operations on non-view tensors.
</code></pre>
<p>Used e.g. in <a href="https://github.com/pytorch/pytorch/blob/caed485873a750fa698c402e7d64f9b1c386cca9/aten/src/ATen/native/LinearAlgebra.cpp#L378" rel="nofollow noopener">matmul</a>.</p>

BarCodeReader: <p>Hi <a class="mention" href="/u/ptrblck">@ptrblck</a>,</p>
<p>I just created a very simple snippet like below:</p>
<pre><code class="lang-auto">p = torch.ones((3,4), requires_grad = True)

p1 = p.permute(1,0).flatten()
p2 = p.transpose(1,0).flatten()
p3 = p.T.reshape(-1)

p4 = torch.empty(12)
for i in range(4):
    for j in range(3):
        p4[i*3 + j] = p[j,i]

print(p1, p2, p3, p4)

output
----------------------
tensor([1., 1., ... 1.], grad_fn=&lt;UnsafeViewBackward&gt;)
tensor([1., 1., ... 1.], grad_fn=&lt;UnsafeViewBackward&gt;)
tensor([1., 1., ... 1.], grad_fn=&lt;UnsafeViewBackward&gt;)
tensor([1., 1., ... 1.], grad_fn=&lt;CopySlices&gt;)
</code></pre>
<p>seems if we change the shape of original tensor then apply flatten, the grad_fn will become <br>
so if I am going to use p1~p3 as convolution weight and apply F.conv2d to a tensor x, which p should I use?</p>
<p><code>F.conv2d(x, p1)</code>, or <code>F.conv2d(x, p2)</code>, or <code>F.conv2d(x, p3)</code> or <code>F.conv2d(x, p4)</code>???</p>

ptrblck: <p>You as a user shouldn’t see any difference, so you could use any <code>pX</code> tensor for your operation.<br>
Anyway, let us know, if you see any unexpected behavior or errors.</p>
<p>Of course the shape doesn’t fit the expected weight shape for <code>F.conv2d</code>, but that is another issue.</p>

BarCodeReader: <p>ok thank you for the explaination.</p>

Razi_Bayati: <p>Hi,<br>
I receive an error while using this function for reshaping my tensor</p>
<pre><code class="lang-auto">def reshape_fortran(x, shape):
    if len(x.shape) &gt; 0:
        x = x.permute(*reversed(range(len(x.shape))))
    return x.reshape(*reversed(shape)).permute(*reversed(range(len(shape))))
</code></pre>
<p>and it says:</p>
<pre><code class="lang-auto">RuntimeError: _unsafe_view does not support automatic differentiation for outputs with complex dtype.
</code></pre>
<p>it is weird cause, sometime I receive it and sometime it works fine!</p>
<p>what is this unsafe_view?</p>

ptrblck: <p>Could you post the type and shape of <code>x</code> as well as the input argument <code>shape</code>, so that we could debug this issue?<br>
This error points towards the <code>complex</code> types so are you using this tensor type or might the error message be misleading?</p>

Razi_Bayati: <p>Thanks for your response.<br>
Yes, I am working with complex data and I use this function to implement F order reshape in tensors. (There is no built-in function for Fortran reshape of tensors)<br>
The input is a (4,4,2) complex64 valued matrix and the shape value is (8,4).<br>
I was able to fix the problem by using this function instead, but I have no idea why it was not working and why it is working now:</p>
<pre><code class="lang-auto">def convert_output84_torch(input):
  shape84 = (8,4)
  T1 = torch.transpose(input,0,2).permute(0,1,2).contiguous()
  T2 = T1.view(shape84[::-1])
  output = torch.transpose(T2,0,1)
  return output
</code></pre>
<p>I replaced reshape with view, and it works fine (for now). But as my code was working fine before, I am not sure if it is a long-term solution as I am not aware of the main source.</p>

ptrblck: <p>Thanks for the information.<br>
I can reproduce this error in PyTorch <code>1.7.0</code> and get a different error message in <code>1.8.0.dev20201022</code>:</p>
<pre><code class="lang-python">def reshape_fortran(x, shape):
    if len(x.shape) &gt; 0:
        x = x.permute(*reversed(range(len(x.shape))))
    return x.reshape(*reversed(shape)).permute(*reversed(range(len(shape))))

x = torch.randn(4, 4, 2, dtype=torch.complex64, requires_grad=True)
shape = (8, 4)
out = reshape_fortran(x, shape)
out.mean().backward()
&gt; RuntimeError: mean does not support automatic differentiation for outputs with complex dtype.
</code></pre>
<p>Note that complex support is not fully implemented yet, so I would recommend to verify the results of your second approach using some reference values.</p>