How to save torchscript model using cpp?

username: SixerWang
cooked: <p>I know detail about how to save torchscript model using python, and load it with c++ from the doc: <a href="https://pytorch.org/tutorials/advanced/cpp_export.html#step-5-getting-help-and-exploring-the-api" rel="nofollow noopener">link</a>.</p>
<p>But how to save torchscript using cpp？I should train my model using c++, but I don’t find any detail about it in document.<br>
Thanks!</p>

username: 21fa417e3fb06e56040c
cooked: <p>Hi.</p>
<p>You can refer to the repository below to do what you want.</p>
<ol>
<li>First create a C ++ model with the same structure as the Python model</li>
<li>Save your Python model as a JIT script.</li>
<li>Instead of loading the model into a JIT script, you use torch :: load to load it.</li>
</ol>
<p>Created by referring torchvision.<br>
<aside class="onebox githubblob">
  <header class="source">
      <a href="https://github.com/pytorch/vision/blob/cd1748443afb1ccd168a3b3da29317ca5d4cd253/test/test_models.cpp#L10" target="_blank" rel="nofollow noopener">github.com</a>
  </header>
  <article class="onebox-body">
    <h4><a href="https://github.com/pytorch/vision/blob/cd1748443afb1ccd168a3b3da29317ca5d4cd253/test/test_models.cpp#L10" target="_blank" rel="nofollow noopener">pytorch/vision/blob/cd1748443afb1ccd168a3b3da29317ca5d4cd253/test/test_models.cpp#L10</a></h4>
<pre class="onebox"><code class="lang-cpp"><ol class="start lines" start="1" style="counter-reset: li-counter 0 ;">
<li>#include &lt;torch/script.h&gt;</li>
<li>#include &lt;torch/torch.h&gt;</li>
<li>#include &lt;iostream&gt;</li>
<li>
</li>
<li>#include "../torchvision/csrc/models/models.h"</li>
<li>
</li>
<li>using namespace vision::models;</li>
<li>
</li>
<li>template &lt;typename Model&gt;</li>
<li class="selected">torch::Tensor forward_model(const std::string&amp; input_path, torch::Tensor x) {</li>
<li>  Model network;</li>
<li>  torch::load(network, input_path);</li>
<li>  network-&gt;eval();</li>
<li>  return network-&gt;forward(x);</li>
<li>}</li>
<li>
</li>
<li>torch::Tensor forward_alexnet(const std::string&amp; input_path, torch::Tensor x) {</li>
<li>  return forward_model&lt;AlexNet&gt;(input_path, x);</li>
<li>}</li>
<li>
</li>
</ol></code></pre>


  </article>
  <div class="onebox-metadata">
    
    
  </div>
  <div style="clear: both"></div>
</aside>
</p>
<aside class="onebox whitelistedgeneric">
  <header class="source">
      <img src="https://github.githubassets.com/favicon.ico" class="site-icon" width="16" height="16">
      <a href="https://github.com/kerry-Cho/transfer-learning-Libtorch" target="_blank" rel="nofollow noopener">GitHub</a>
  </header>
  <article class="onebox-body">
    <img src="https://avatars0.githubusercontent.com/u/20694053?s=400&amp;v=4" class="thumbnail onebox-avatar" width="60" height="60">

<h3><a href="https://github.com/kerry-Cho/transfer-learning-Libtorch" target="_blank" rel="nofollow noopener">kerry-Cho/transfer-learning-Libtorch</a></h3>

<p>LibTorch Example of transfer-learning. Contribute to kerry-Cho/transfer-learning-Libtorch development by creating an account on GitHub.</p>


  </article>
  <div class="onebox-metadata">
    
    
  </div>
  <div style="clear: both"></div>
</aside>


username: SixerWang
cooked: <p>Hi, Thanks for your reply.<br>
But I should train model with C++ and save the model as JIT script. The reason why I should save model as JIT script is that the inference code can’t access to model structure(the model defination class). Thus, I should find a way to save model as JIT script using C++.<br>
However, the code you provide above should pass <code>Model</code> instance as parameter, so It doesn’t meet my need.</p>

username: 21fa417e3fb06e56040c
cooked: <p>I don’t know<br>
Is it to avoid exposing the model structure in inference?</p>

username: SixerWang
cooked: <p>It is one aspect. On the other hand, the Script Model has high-performance.</p>

username: abtion
cooked: <p>Hi, have you solved it? I have the same problem, and I try to trian the script model in C++, but it not works, could you please give me some help while you have train it successfully?thanks</p>

username: abtion
cooked: <p>Hi, I found that both <code>torchscript</code> and <code>torch::nn::Module</code> model file can be loaded with <code>torch::load(model, "model_file.pt");</code>. It isn’t nesscery to save torchscript model in cpp while you load torchscript model to nn::Module and train it.</p>

username: SixerWang
cooked: <p><div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/2/d/2d26477540b108cc9dd6a41f56f5ba08ea87ed74.png" data-download-href="https://discuss.pytorch.org/uploads/default/2d26477540b108cc9dd6a41f56f5ba08ea87ed74" title="image"><img src="https://discuss.pytorch.org/uploads/default/original/3X/2/d/2d26477540b108cc9dd6a41f56f5ba08ea87ed74.png" alt="image" data-base62-sha1="6rpuVw9m4Dx3IMkXmaKzQPGOyQA" width="690" height="352" data-dominant-color="F0F0F0"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">image</span><span class="informations">775×396 32.3 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>
<p>link : <a href="https://discuss.pytorch.org/t/libtorch-save-mnist-c-examples-trained-model-into-a-file-and-load-in-from-another-c-file-to-use-for-prediction/51681/9" class="inline-onebox">(libtorch) Save MNIST c++ example's trained model into a file, and load in from another c++ file to use for prediction? - #9 by Ziri_Ziri</a></p>

username: SixerWang
cooked: <p><a class="mention" href="/u/yf225">@yf225</a> I am sorry to disturb, Could u give me a little hint about whether libtorch could save torchscript model using cpp or not. Thank you so much!</p>

username: yf225
cooked: <p><a class="mention" href="/u/sixerwang">@SixerWang</a> By “saving a TorchScript model in C++”, do you mean one of the following:</p>
<ol>
<li>You want to embed Python/TorchScript code in your C++ code, and then save that portion of code to a TorchScript model file<br>
Answer: This is not supported right now, but it is possible to implement / expose this API. I’d suggest opening an issue in <a href="https://github.com/pytorch/pytorch/issues" rel="nofollow noopener">https://github.com/pytorch/pytorch/issues</a> and we can discuss the use cases.</li>
<li>You want to save a model written in C++ frontend to a TorchScript model file<br>
Answer: This is not supported and likely won’t be supported in the short term, because implementing this mechanism is a huge effort, and we will need more use cases to actually establish this as a project to work on.</li>
</ol>

username: SixerWang
cooked: <p>Thank you very much for your reply. My situation is the second one. I will change my training process.Thank you!</p>

username: freezek
cooked: <p>Hi, have you solved the problem?</p>