Can't call numpy() on Variable that requires grad

Qiang_Sun: <ul>
<li>Error informaiton :</li>
</ul>
<p>There is only one GPU<br>
Traceback (most recent call last):<br>
File “train.py”, line 123, in <br>
test_train.start_train(epoch=30,batch_display=2000,batch_size=50,save_freq=1)<br>
File “train.py”, line 96, in start_train<br>
pcc = scipy.stats.pearsonr(gtlabel, output.squeeze(-1))[0]<br>
File “/home/simon/anaconda3/lib/python3.6/site-packages/scipy/stats/stats.py”, line 3003, in pearsonr<br>
y = np.asarray(y)<br>
File “/home/simon/anaconda3/lib/python3.6/site-packages/numpy/core/numeric.py”, line 492, in asarray<br>
return array(a, dtype, copy=False, order=order)<br>
File “/home/simon/anaconda3/lib/python3.6/site-packages/torch/tensor.py”, line 376, in <strong>array</strong><br>
return self.cpu().numpy()</p>
<ul>
<li>Codes involved:<br>
labels_batch = torch.FloatTensor(labels_batch.view(-1,self.num_classes).detach().numpy())</li>
</ul>

albanD: <p>Hi,</p>
<p>This is expected behavior because moving to numpy will break the graph and so no gradient will be computed.<br>
If you don’t actually need gradients, then you can explicitly <code>.detach()</code> the Tensor that requires grad to get a tensor with the same content that does not require grad. This other Tensor can then be converted to a numpy array.</p>

Qiang_Sun: <p>Hi albanD, I am a newer to the PyTorch, can you please provide me with more informaiton? Cheers!</p>

Qiang_Sun: <p>I have tried the line as<br>
labels_batch = torch.FloatTensor(labels_batch.wiew(-1,self.num_classes).numpy(), requires_grad=True)</p>
<p>but, there is the problem as follows</p>
<p>There is only one GPU<br>
Traceback (most recent call last):<br>
File “train.py”, line 123, in <br>
test_train.start_train(epoch=30,batch_display=2000,batch_size=50,save_freq=1)<br>
File “train.py”, line 85, in start_train<br>
labels_batch = torch.FloatTensor(labels_batch.view(-1,self.num_classes).numpy(), requires_grad=True)<br>
TypeError: new() received an invalid combination of arguments - got (numpy.ndarray, requires_grad=bool), but expected one of:</p>
<ul>
<li>(torch.device device)</li>
<li>(tuple of ints size, torch.device device)<br>
didn’t match because some of the keywords were incorrect: requires_grad</li>
<li>(torch.Storage storage)</li>
<li>(Tensor other)</li>
<li>(object data, torch.device device)<br>
didn’t match because some of the keywords were incorrect: requires_grad</li>
</ul>
<p>Can you plz help me solve it?</p>

albanD: <p>I don’t understand what do you try to achieve here?</p>

Qiang_Sun: <p>After I deteleted the code of ‘requires_grad=True’, similar error occured as</p>
<p>There is only one GPU<br>
Traceback (most recent call last):<br>
File “train.py”, line 123, in <br>
test_train.start_train(epoch=30,batch_display=2000,batch_size=50,save_freq=1)<br>
File “train.py”, line 96, in start_train<br>
pcc = scipy.stats.pearsonr(gtlabel, output.squeeze(-1))[0]<br>
File “/home/simon/anaconda3/lib/python3.6/site-packages/scipy/stats/stats.py”, line 3003, in pearsonr<br>
y = np.asarray(y)<br>
File “/home/simon/anaconda3/lib/python3.6/site-packages/numpy/core/numeric.py”, line 492, in asarray<br>
return array(a, dtype, copy=False, order=order)<br>
File “/home/simon/anaconda3/lib/python3.6/site-packages/torch/tensor.py”, line 376, in <strong>array</strong><br>
return self.cpu().numpy()<br>
RuntimeError: Can’t call numpy() on Variable that requires grad. Use var.detach().numpy() instead.</p>

albanD: <p>If you’re trying to compute your loss with <code>pcc = scipy.stats.pearsonr(gtlabel, output.squeeze(-1))[0]</code>. Then this is not possible. The only operations that are allowed to be able to differentiate them are pytorch operations.<br>
If you’re using <code>pcc = scipy.stats.pearsonr(gtlabel, output.squeeze(-1))[0]</code> just to get some metric but <strong>don’t need to backprop</strong> through it, then you need to <code>detach()</code> your output before giving it to this function as <code>output.detach().squeeze(-1)</code>.</p>

Qiang_Sun: <p>The story behind this error can be described as follows:</p>
<p>At first, I used the MSE-based loss function in the python file as<br>
loss_function = nn.MSELoss().cuda()</p>
<p>the loss was input as<br>
loss = loss_function(output, gtlabel)</p>
<p>in addition, thee line of  ‘labels_batch=…’ was input as<br>
labels_batch = torch.FloatTensor(labels_batch.wiew(-1,self.num_classes).numpy())</p>
<p>These codes was run smoothly!</p>
<hr>
<p>At this time, I want to use another loss function, i.e., 1-CCC,<br>
and the codes are changed as</p>
<pre><code>            pcc = scipy.stats.pearsonr(gtlabel, output.squeeze(-1))[0]
            loss = 1-2*pcc*np.std(gtlabel)*np.std(output)/((np.std(gtlabel))**2+(np.std(output))**2+(np.mean(output)-np.mean(gtlabel))**2)
</code></pre>
<p>When I use the line of " labels_batch = torch.FloatTensor(labels_batch.wiew(-1,self.num_classes).numpy())", there is the error listed in the previous posts,</p>
<p>So my colleague suggested me to add ‘requires_grad=True’ to this line, but there is the new error as listed in the previous post.<br>
That’s all!</p>

albanD: <p>As I stated above, you <strong>cannot</strong> use numpy functions to compute your loss. Even you hack it to do so, gradient won’t be computed.<br>
You have to use pytorch’s function <strong>only</strong> to be able to get gradients.</p>

Qiang_Sun: <p>I have got it and solved my problem. Cheers, man!</p>

amir_omidi: <p>Hi I have the same error in new version of torch (1.5). in previous version (1.4) my program worked with no error. what do you suggest to do to solve the error.</p>

albanD: <p>Hi,</p>
<p>Do you have a code sample that work with 1.4 and fails now please?</p>

amir_omidi: <p>Hi, Thank you for your reply. I found out that it is related to following part related to plotting my results.<br>
I commented this part and solved the problem. since it was not so important, I ignored that part.</p>
<pre><code class="lang-auto"># def plot_constellation(

#     ax,

#     constellation,

#     channel,

#     decoder,

#     grid_step=0.05,

#     noise_samples=1000):

#     """

#     Plot a constellation with its decoder and channel noise.

#     :param ax: Matplotlib axes to plot on.

#     :param constellation: Constellation to plot.

#     :param channel: Channel model to use for generating noise.

#     :param decoder: Decoder function able to map the constellation points back

#     to the original messages.

#     :param grid_step: Grid step used for drawing the decision regions,

#     expressed as percentage of the total plot width (or equivalently height).

#     Lower steps makes more precise grids but takes more time to compute.

#     :param noise_samples: Number of noisy points to sample and plot.

#     """

#     ax.grid()

#     order = len(constellation)

#     color_map = ListedColormap(seaborn.color_palette('husl', n_colors=order))

#     color_norm = matplotlib.colors.BoundaryNorm(range(order + 1), order)

#     # Extend axes symmetrically around zero so that they fit data

#     axis_extent = max(

#         abs(constellation.min()),

#         abs(constellation.max())

#     ) * 1.05

#     ax.set_xlim(-axis_extent, axis_extent)

#     ax.set_ylim(-axis_extent, axis_extent)

#     # Hide borders but keep ticks

#     for direction in ['left', 'bottom', 'right', 'top']:

#         ax.axis[direction].line.set_color('#00000000')

#     # Show zero-centered axes without ticks

#     for direction in ['xzero', 'yzero']:

#         axis = ax.axis[direction]

#         axis.set_visible(True)

#         axis.set_axisline_style('-|&gt;')

#         axis.major_ticklabels.set_visible(False)

#     # Add axis names

#     ax.annotate(

#         'I', (1, 0.5), xycoords='axes fraction',

#         xytext=(25, 0), textcoords='offset points',

#         va='center', ha='right'

#     )

#     ax.annotate(

#         'Q', (0.5, 1), xycoords='axes fraction',

#         xytext=(0, 25), textcoords='offset points',

#         va='center', ha='center'

#     )

#     # Plot decision regions

#     regions_extent = 2 * axis_extent

#     step = grid_step * regions_extent

#     grid_range = torch.arange(-regions_extent, regions_extent, step)

#     grid_y, grid_x = torch.meshgrid(grid_range, grid_range)

#     grid_points = torch.stack((grid_x, grid_y), dim=-1).flatten(end_dim=1)

#     grid_images = decoder(grid_points).argmax(dim=-1).reshape(grid_x.shape)

#     ax.imshow(

#         grid_images,

#         extent=(

#             -regions_extent, regions_extent,

#             -regions_extent, regions_extent

#         ),

#         aspect='auto',

#         origin='lower',

#         cmap=color_map,

#         norm=color_norm,

#         alpha=0.2

#     )

#     # Plot constellation

#     ax.scatter(

#         *zip(*constellation.tolist()),

#         zorder=10,

#         s=60,

#         c=range(order),

#         edgecolor='black',

#         cmap=color_map,

#         norm=color_norm,

#     )

#     # Plot constellation center

#     center = constellation.sum(dim=0) / order

#     ax.scatter(

#         center[0], center[1],

#         marker='X',

#     )

#     # Plot channel noise

#     if noise_samples &gt; 0:

#         noisy_vectors = channel(constellation.repeat(noise_samples, 1))

#         ax.scatter(

#             *zip(*noisy_vectors.tolist()),

#             marker='.',

#             s=5,

#             c=list(range(order)) * noise_samples,

#             cmap=color_map,

#             norm=color_norm,

#             alpha=0.3,

#             zorder=8

#         )
</code></pre>

albanD: <p>You are most likely missing a detach in there if this is just plotting code.</p>