Autograd inside torchscript from java (libtorch)

mhermon: <p>Hello, I am trying to use autograd to get gradients inside a torchscript file I am calling from java using libtorch but am running into an error “RuntimeError: element 0 of tensors does not require grad and does not have a grad_fn”</p>
<p>My torchscript model:</p>
<pre><code class="lang-auto">class MyModel(torch.nn.Module):
    def __init__(self):
        super().__init__()
    
    def forward(self, x):
        # Require a gradient
        x.requires_grad_()
        y = x.pow(2).sum()
        print(f'y.requires_grad: {y.requires_grad}')
        # Check if x requires a gradient
        print(f'x.requires_grad: {x.requires_grad}')
        dy_dx = torch.autograd.grad([y], [x])[0]
        return dy_dx
</code></pre>
<p>Saving with:</p>
<pre><code class="lang-auto">model_scripted = torch.jit.script(MyModel())
model_scripted.save("model.pt")
</code></pre>
<p>This throws no errors and returns the expected values when I reload and run the model inside python with:</p>
<pre><code class="lang-auto">model_loaded = torch.jit.load("model.pt")
dy_dx = model_loaded(x)
print(dy_dx)
</code></pre>
<p>However this fails and throws an error when I load inside java and try to run the model:</p>
<pre><code class="lang-auto">Traceback of TorchScript, original code (most recent call last):
  File "/var/folders/g9/ykm_xvjx25s8bq5bkfjqy5240000gn/T/ipykernel_39052/954115032.py", line 12, in forward
        # Check if x requires a gradient
        print(f'x.requires_grad: {x.requires_grad}')
        dy_dx = torch.autograd.grad([y], [x])[0]
                ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ &lt;--- HERE
        return dy_dx
RuntimeError: element 0 of tensors does not require grad and does not have a grad_fn
</code></pre>
<p>Not sure why this is a problem when loading the torchscript into java but not a problem when loading into python.</p>

ptrblck: <p>I guess you are trying to deploy the model onto a mobile platform?<br>
If so, I think gradient calculation is disabled by default as its main purpose is inference.</p>

mhermon: <p>Hey <a class="mention" href="/u/ptrblck">@ptrblck</a> thanks for the response. No, this is not to a mobile platform. Just trying to run the torchscript model inside a java application.</p>

ptrblck: <p>Ah OK, I’m not familiar with this workflow and don’t know what the Java env might set and if it would disable gradient calculation.</p>

saudet: <p>Does it do the same if you try to load it with the JavaCPP Presets for PyTorch?</p>
<aside class="onebox githubfolder" data-onebox-src="https://github.com/bytedeco/javacpp-presets/tree/master/pytorch">
  <header class="source">
      <img src="https://github.githubassets.com/favicons/favicon.svg" class="site-icon" width="32" height="32">

      <a href="https://github.com/bytedeco/javacpp-presets/tree/master/pytorch" target="_blank" rel="noopener nofollow ugc">github.com</a>
  </header>

  <article class="onebox-body">
    <h3><a href="https://github.com/bytedeco/javacpp-presets/tree/master/pytorch" target="_blank" rel="noopener nofollow ugc">javacpp-presets/pytorch at master · bytedeco/javacpp-presets</a></h3>

  <p><a href="https://github.com/bytedeco/javacpp-presets/tree/master/pytorch" target="_blank" rel="noopener nofollow ugc">master/pytorch</a></p>

  <p><span class="label1">The missing Java distribution of native C++ libraries</span></p>

  </article>

  <div class="onebox-metadata">
    
    
  </div>

  <div style="clear: both"></div>
</aside>
<aside class="onebox githubblob" data-onebox-src="https://github.com/bytedeco/javacpp-presets/blob/master/pytorch/samples/ExampleApp.java">
  <header class="source">

      <a href="https://github.com/bytedeco/javacpp-presets/blob/master/pytorch/samples/ExampleApp.java" target="_blank" rel="noopener nofollow ugc">github.com</a>
  </header>

  <article class="onebox-body">
    <h4><a href="https://github.com/bytedeco/javacpp-presets/blob/master/pytorch/samples/ExampleApp.java" target="_blank" rel="noopener nofollow ugc">bytedeco/javacpp-presets/blob/master/pytorch/samples/ExampleApp.java</a></h4>


      <pre><code class="lang-java">// © Copyright 2021, PyTorch.

import org.bytedeco.javacpp.*;
import org.bytedeco.pytorch.*;
import static org.bytedeco.pytorch.global.torch.*;

// Ported from C++ code at https://pytorch.org/tutorials/advanced/cpp_export.html
public class ExampleApp {
    public static void main(String[] args) throws Exception {
        /* try to use MKL when available */
        System.setProperty("org.bytedeco.openblas.load", "mkl");

        if (args.length != 1) {
            System.err.println("usage: java ExampleApp &lt;path-to-exported-script-module&gt;");
            System.exit(-1);
        }

        JitModule module;
        try {
            // Deserialize the ScriptModule from a file using torch::jit::load().
</code></pre>



  This file has been truncated. <a href="https://github.com/bytedeco/javacpp-presets/blob/master/pytorch/samples/ExampleApp.java" target="_blank" rel="noopener nofollow ugc">show original</a>

  </article>

  <div class="onebox-metadata">
    
    
  </div>

  <div style="clear: both"></div>
</aside>


mhermon: <p>Attempting to use the JavaCPP Presets I run into an issue with JNITorch:</p>
<pre><code class="lang-auto">Warning: Version of org.bytedeco:pytorch could not be found.
Warning: Version of org.bytedeco:openblas could not be found.
 Uncaught exception: exiting with status code 1
java.lang.UnsatisfiedLinkError: no jnitorch in java.library.path
</code></pre>

saudet: <p>If you’re trying this on Android, we’d first need to create builds for that platform.</p>

dabs: <p><a class="mention" href="/u/mhermon">@mhermon</a> I am running into a similar issue. Did you find a solution?</p>