Porting saved neural nets from c++ GPU to other places

Willem: <p>So for reasons that I shall not discuss here in detail (e.g. better multi-node support on supercomputers), I adopted the c++ framework for actually training a neural network. I used the GPU-enabled c++ framework installed using libtorch. (I cannot port this program to Python because too much code is involved. )</p>
<p>Now, after training a network using pure c++ that was created by inheriting torch::nn::module and training it, it was saved using torch::save.  I have been able to again load the saved module without problems in the same c++ program.</p>
<p>My problem is that as soon as I try to run the saved module elsewhere, I run into troubles. E.g. loading the net into c++ libtorch without CPU support causes errors. Same for loading it into Python (either with or without GPU support). My end-goal is running inference with the neural net in places without GPU support.</p>
<p>I read this:<br>
</p><aside class="onebox allowlistedgeneric">
  <header class="source">
      <a href="https://pytorch.org/tutorials/recipes/recipes/save_load_across_devices.html" target="_blank" rel="noopener nofollow ugc">pytorch.org</a>
  </header>
  <article class="onebox-body">
    <img src="" class="thumbnail" width="60" height="60">

<h3><a href="https://pytorch.org/tutorials/recipes/recipes/save_load_across_devices.html" target="_blank" rel="noopener nofollow ugc">Saving and loading models across devices in PyTorch — PyTorch Tutorials 1.7.0...</a></h3>



  </article>
  <div class="onebox-metadata">
    
    
  </div>
  <div style="clear: both"></div>
</aside>
<br>
and<br>
<aside class="onebox stackexchange">
  <header class="source">
      <a href="https://stackoverflow.com/questions/42703500/best-way-to-save-a-trained-model-in-pytorch" target="_blank" rel="noopener nofollow ugc">stackoverflow.com</a>
  </header>
  <article class="onebox-body">
      <a href="https://stackoverflow.com/users/5352399/wasi-ahmad" target="_blank" rel="noopener nofollow ugc">
    <img alt="Wasi Ahmad" src="https://lh3.googleusercontent.com/-Rk_dLwN-xPY/AAAAAAAAAAI/AAAAAAAAAC8/DtCya1W9lX4/photo.jpg?sz=128" class="thumbnail onebox-avatar" width="60" height="60">
  </a>
<h4>
  <a href="https://stackoverflow.com/questions/42703500/best-way-to-save-a-trained-model-in-pytorch" target="_blank" rel="noopener nofollow ugc">Best way to save a trained model in PyTorch?</a>
</h4>

<div class="tags">
  <strong>python, serialization, deep-learning, pytorch</strong>
</div>

<div class="date">
  asked by
  
  <a href="https://stackoverflow.com/users/5352399/wasi-ahmad" target="_blank" rel="noopener nofollow ugc">
    Wasi Ahmad
  </a>
  on <a href="https://stackoverflow.com/questions/42703500/best-way-to-save-a-trained-model-in-pytorch" target="_blank" rel="noopener nofollow ugc">07:06PM - 09 Mar 17 UTC</a>
</div>

  </article>
  <div class="onebox-metadata">
    
    
  </div>
  <div style="clear: both"></div>
</aside>

<p>However, I am mainly interested in starting from C++. Is what I want to do supported, or not? If yes, how? ( I could of course manually save all weights to a file, and load again from other platform, but obviously I would rather not go that route.)</p>