LibTorch(C++) => TorchScript => PyTorch(Python) example

abhinavkulkarni: <p>Hi,</p>
<p>Are there examples to go from C++ to PyTorch in terms of model saving and loading?</p>
<p>All the examples online are for the reverse case (prototyping in Python =&gt; TorchScript =&gt; serving the model in C++).</p>
<p>I couldn’t figure a way to trace a model in C++ and save it to the disk.</p>
<p>Thanks.</p>

glaringlee: <p><a class="mention" href="/u/abhinavkulkarni">@abhinavkulkarni</a><br>
See my post before.<br>
‘<a href="https://discuss.pytorch.org/t/load-saved-file-from-libtorch-to-pytorch-failed/93099/5" class="inline-onebox">Load saved file from LibTorch to PyTorch failed</a>’</p>

botelho: <p><a class="mention" href="/u/glaringlee">@glaringlee</a>: Can you please explain how you save a <code>nn::Module</code> in libtorch to file? And how to load that file in Python? When I try your suggestion and do</p>
<pre><code class="lang-auto">torch::save(model, "model.pt");
</code></pre>
<p>I get errors of the type</p>
<pre><code class="lang-auto">‘torch::serialize::OutputArchive’ is not derived from ‘std::basic_ostream&lt;_CharT, _Traits&gt;’
   archive &lt;&lt; value;
</code></pre>

glaringlee: <p>Hey, can you provide more info?<br>
What’s your pytorch version?<br>
Can you provide more code that enough for me to test on my end?</p>

botelho: <p>Hi, I’m using libtorch 1.7.1, but that shouldn’t really matter. As for code, you can take this simple <a href="https://github.com/pytorch/examples/blob/master/cpp/mnist/mnist.cpp" rel="noopener nofollow ugc">MNIST example</a> and add the line of code above to save the model. It won’t compile.<br>
In any case, that’s not the point of the ticket. I know how to export a model in libtorch (serializing to OutputArchive, etc), but I cannot load that model in Python. The problem I’m trying to solve is how to export a model in libtorch that can be loaded in Python.</p>

glaringlee: <p><a class="mention" href="/u/botelho">@botelho</a> version is really matter to us since the serialize code keep changing between versions.<br>
The example was written for libtorch 1.4, I think we need to update it.<br>
There is a post here which can solve your problem. Please take a look at.</p><aside class="quote quote-modified" data-post="1" data-topic="34234">
  <div class="title">
    <div class="quote-controls"></div>
    <img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/h/d26b3c/48.png" class="avatar">
    <a href="https://discuss.pytorch.org/t/libtorch-how-to-save-model-in-mnist-cpp-example/34234">(libtorch) How to save model in MNIST cpp example?</a> <a class="badge-category__wrapper " href="/c/c/11"><span data-category-id="11" data-drop-close="true" class="badge-category " title="Topics related to the C++ Frontend, C++ API or C++ Extensions"><span class="badge-category__name">C++</span></span></a>
  </div>
  <blockquote>
    I’m running mnist <a href="https://github.com/goldsborough/examples/tree/cpp/cpp/mnist" rel="noopener nofollow ugc">example</a> and try to save trained model to disk: 
torch::save(model, "model.pt")   # save model using torch::save

Then got error as: 
In file included from /home/christding/env/libtorch/include/torch/csrc/api/include/torch/all.h:8:0,
                 from /home/christding/env/libtorch/include/torch/csrc/api/include/torch/torch.h:3,
                 from /home/christding/Desktop/workspace/TorchLearning/C++/mnist-custom/src/mnist.cpp:1:
/home/christding/env/libtorch/include/torch/…
  </blockquote>
</aside>


botelho: <p>Hi <a class="mention" href="/u/glaringlee">@glaringlee</a>. I don’t think you read my last comment until the end. I already know how to export a model and read it back in libtorch. What I cannot do is to take a model saved by libtorch and <strong>load it in python</strong>. In fact, this is the subject of this ticket: LibTorch(C++) =&gt; TorchScript =&gt; PyTorch(Python).</p>

glaringlee: <p><a class="mention" href="/u/botelho">@botelho</a> Sry, but I got confused.<br>
You original said <code>torch::save(model, "model.pt");</code> got compile error. If you solved this problem, then you should be able to load your saved model.pt in python using torch.jit.load which is within the link that I pasted earlier in this post, no?</p>
<p>here is the link ‘<a href="https://discuss.pytorch.org/t/load-saved-file-from-libtorch-to-pytorch-failed/93099/5" class="inline-onebox">Load saved file from LibTorch to PyTorch failed - #5 by glaringlee</a>’</p>
<p>Please make sure the version of libtorch and pytorch must match.</p>

botelho: <p>Hi <a class="mention" href="/u/glaringlee">@glaringlee</a>. Actually, doing</p>
<pre><code class="lang-auto">module = torch.jit.load('model.pt')
</code></pre>
<p>does <strong>not</strong> work. That is, the <code>load</code> call itself returns without errors, but if you then try to do a forward pass on the model, you get something like this:</p>
<pre><code class="lang-auto">Traceback (most recent call last):
  File "./load.py", line 63, in &lt;module&gt;
    main()
  File "./load.py", line 59, in main
    output = model(data)
  File "/home/botelho/.local/lib/python3.6/site-packages/torch/nn/modules/module.py", line 727, in _call_impl
    result = self.forward(*input, **kwargs)
  File "/home/botelho/.local/lib/python3.6/site-packages/torch/jit/_script.py", line 558, in __getattr__
    return super(RecursiveScriptModule, self).__getattr__(attr)
  File "/home/botelho/.local/lib/python3.6/site-packages/torch/jit/_script.py", line 288, in __getattr__
    return super(ScriptModule, self).__getattr__(attr)
  File "/home/botelho/.local/lib/python3.6/site-packages/torch/nn/modules/module.py", line 779, in __getattr__
    type(self).__name__, name))
torch.nn.modules.module.ModuleAttributeError: 'RecursiveScriptModule' object has no attribute 'forward'
</code></pre>
<p>And yes, both my libtorch and pytorch versions are 1.7.0.</p>

glaringlee: <p><a class="mention" href="/u/botelho">@botelho</a><br>
I see the difference now. I am sry, my bad. I had a wrong impression of my previous post. The save/load from libtorch to pytorch direction supports saving tensor only at this point.</p>

botelho: <p>Hi <a class="mention" href="/u/glaringlee">@glaringlee</a>. Sounds good, thanks for confirming.</p>