Cross-platform PyTorch stand-alone executables

Deeply: <p>At this stage of my project, I need to compile my code to binaries that work under Windows and / or Ubuntu. I know that <code>jit</code> is a great mile stone in <code>PyTorch</code>, but, I am not sure if this is the correct approach that I can use to achieve my objectives; not to mention the complexity and effort needed to use <code>jit</code>; or <strong>PyTorch v1.0</strong>  C++ Front-End. That said, there are some tools to convert Python to binaries, for example, <a href="https://www.pyinstaller.org/" rel="nofollow noopener">PyInstaller</a> , which are not compatible with <code>PyTorch</code>. Are there any other alternatives to do <code>PyTorch</code> code compilation?</p>
<p>Thanks in advance</p>

driazati: <p>What difficulties are you seeing with TorchScript (aka jit)? We are trying to make it as easy to use as possible for models defined in Python with PyTorch, so if you’re having issues please let us know. It appears that PyInstaller includes a Python runtime inside the binary, whereas TorchScript and the C++ frontend both have no dependency on Python and can be run in multithreaded environments with no GIL.</p>
<p>As for <a href="https://pytorch.org/tutorials/advanced/cpp_export.html" rel="nofollow noopener">TorchScript</a> vs. the <a href="https://pytorch.org/tutorials/advanced/cpp_frontend.html" rel="nofollow noopener">C++ frontend</a>, that’s a more personal choice as they both rely on <code>libtorch</code> and the difference mostly comes down to how you want to define your model (in Python + TorchScript vs in C++), so it’s hard to say without more information on your use case.</p>

Deeply: <p>Thanks for your answer, which saved me putting lots of sparse effort.</p>
<p>To be honest, I have not tried <code>jit</code> yet; but I need to know which path to take before going forward with it.</p>
<p>From your answer I can say, as I am using PyTorch, then, it’s better to use <code>jit</code>.</p>

soldierofhell: <p><a class="mention" href="/u/driazati">@driazati</a>, but if one needs to (1) train C++ and then (2) serialize model and then (3) load in C++ on the production, do you have to go through TorchScript in (2)? (Ok, I know that there’s ONYX)</p>

driazati: <aside class="quote no-group" data-username="soldierofhell" data-post="4" data-topic="49475">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/soldierofhell/48/14074_2.png" class="avatar"> soldierofhell:</div>
<blockquote>
<p>one needs to (1) train C++ and then (2) serialize model and then (3) load in C++ on the production, do you have to</p>
</blockquote>
</aside>
<p>The C++ frontend is a high level API that provides a similar experience as using PyTorch’s Python API, as such it has its own serialization mechanisms, so you can iterate on your model all in C++, <a href="https://pytorch.org/cppdocs/frontend.html#end-to-end-example">this example shows how</a>.</p>
<p>TorchScript lets you go from PyTorch models coded in Python to something that you can load in C++, so it’s not necessary for that case.</p>

soldierofhell: <p>Thank you <a class="mention" href="/u/driazati">@driazati</a>, but it’s not clear what  torch::save do. From API doc seems like full serialization of Module or Tensor, but the comment in example states “checkpoint” (state_dict)? What is the method of serialization, is it something similar to Tensorflow ProtoBuf? Are there aby limitations?</p>

driazati: <p>We have many serialization formats and they’re all different and easy to mix up. We’re working on ways to fix the UX here but for now:</p>
<ul>
<li>
<code>torch.save()</code> in eager mode Python lets you save models so they can be loaded in Python</li>
<li>
<code>torch::save()</code> in the C++ API lets you save models so they can be loaded in C++ with <code>torch::load()</code>
</li>
<li>
<code>torch.jit.save()</code> in eager mode Python lets you save models that have been compiled with TorchScript so they can be loaded in Python with <code>torch.jit.load()</code> and in C++ with <code>torch::jit::load()</code>
</li>
</ul>

soldierofhell: <p><a class="mention" href="/u/driazati">@driazati</a>, thanks, but from your answer it’s still unclear what torch::save() saves: only “weights” or whole model (Module).<br>
Definitely at least doc should be more precise. And to be honest this jit thing is also a little confusing, because this is kind of another world. E.g. in Tensorflow if you load .pb you end with normal tf.Graph not something like tf.pb.Graph <img src="https://discuss.pytorch.org/images/emoji/apple/slight_smile.png?v=9" title=":slight_smile:" class="emoji" alt=":slight_smile:"></p>