How to use numpy and opencv without breaking computation graph?

username: AshviniKSharma
cooked: <p>I am working on a custom loss function, as the output of network I am getting x and y coordinates of the image. I am trying to put a constraint that predicted coordinates and ground truth should be close to each other in the 2D domain. But generating an image from coordinates requires me to work in NumPy and OpenCV. This break the computation graph, can somebody help me to get through this:</p>
<p><div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/1/5/1522912744698071be904643e653dff2dc11ceea.png" data-download-href="https://discuss.pytorch.org/uploads/default/1522912744698071be904643e653dff2dc11ceea" title="Screenshot from 2020-01-29 14-39-22"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/1/5/1522912744698071be904643e653dff2dc11ceea_2_538x500.png" alt="Screenshot from 2020-01-29 14-39-22" data-base62-sha1="30Y5lUU3Ks1WaaEuC6o1GIpKe8y" width="538" height="500" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/1/5/1522912744698071be904643e653dff2dc11ceea_2_538x500.png, https://discuss.pytorch.org/uploads/default/original/3X/1/5/1522912744698071be904643e653dff2dc11ceea.png 1.5x, https://discuss.pytorch.org/uploads/default/original/3X/1/5/1522912744698071be904643e653dff2dc11ceea.png 2x" data-dominant-color="F2F1F4"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">Screenshot from 2020-01-29 14-39-22</span><span class="informations">699×649 91.7 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>
<p>vis_loss1 is my main function which returns loss and visualize is the function which convert coordinates to an image(this is where I use numpy and openCV). using l1 I want to <strong>adjust gt and pred</strong> , but this is not happening. Can someone help me?</p>

username: Parskatt
cooked: <pre><code class="lang-python">gt_coord = torch.empty(34,requires_grad=False)
gt_coord = gt[i,:,j]
</code></pre>
<p>Here you first create a tensor that doesnt require gradients, and then you overwrite it with a tensor that does require gradient. I think just writing:</p>
<pre><code class="lang-python">gt_coord = gt[i,:,j].detach().numpy()
</code></pre>
<p>should work (<a href="https://pytorch.org/docs/stable/autograd.html#torch.Tensor.detach" rel="nofollow noopener">https://pytorch.org/docs/stable/autograd.html#torch.Tensor.detach</a>)</p>

username: AshviniKSharma
cooked: <p>Thankyou for your reply. I will try and comback in case of issue.</p>

username: AshviniKSharma
cooked: <p>I did as you suggested but the network is still not learning, loss is still constant. I am trying to make pred as close as possible to gt , by constraining them in 2D for that I need numpy and opencv, therefore I need a custom loss function. But my loss is not decreasing , its a constant throughout learning.</p>

username: Parskatt
cooked: <p>Looking at this a bit more your code seems problematic. You can’t just detach and then reattach tensors to the computational graph, sorry that I didn’t really read carefully before. Can you explain why you don’t just use (gt_coord-pred_coord)**2 ?<br>
Is it because you somehow don’t care about the ordering of the coordinates, just that some of them overlap?</p>
<p>cv2.circle is not differentiable in pytorch, so I think it’s not possible to use it, however there might be some easy way to implement it in pytorch instead.</p>

username: AshviniKSharma
cooked: <p>I want to constraint them in 2 dimension making prediction image as close as possible to ground truth image. That’s why I am not using simple MSE loss , I want to do it in 2 dimension.</p>

username: Parskatt
cooked: <p>But x,y are already 2d coordinates, why not just use the coordinate-wise distance, i.e. (x_gt-x_pred,y_gt-y_pred)**2 as the loss function? I understand that you want the circles to overlap, but this loss function would (from what I can tell), minimize what you want.</p>

username: AshviniKSharma
cooked: <p>Ohh now I realized that even I project them in 2D, most of the values will be still zero, therefore it doesn’t matter I do it in 2D or 1D.<br>
Thankyou very much for your answers, it helped me alot. Thankyou:)</p>