CPP Simple TorchVision example

andreasceid: <p>Hello,</p>
<p>I am trying to compile something that simply initializes a random <code>torchvision.models</code> model, like ResNet50, and then just passes a simple random tensor through. All the examples that I have seen so far involve loading the model from the disk rather than being able to just initialize it out of the box like in Python. Specifically, in C++ the examples so far instruct the following:</p>
<pre><code class="lang-auto">#include &lt;torch/script.h&gt; // One-stop header.

#include &lt;iostream&gt;
#include &lt;memory&gt;

int main(int argc, const char* argv[]) {
  if (argc != 2) {
    std::cerr &lt;&lt; "usage: example-app &lt;path-to-exported-script-module&gt;\n";
    return -1;
  }

  // Deserialize the ScriptModule from a file using torch::jit::load().
  std::shared_ptr&lt;torch::jit::script::Module&gt; module = torch::jit::load(argv[1]);

  assert(module != nullptr);
  std::cout &lt;&lt; "ok\n";
}
</code></pre>
<p>However, in Python I can just do this:</p>
<pre><code class="lang-auto">import torch
from torchvision import models

resnet50 = models.resnet50(pretrained=False)
</code></pre>
<p>Hence, in Python, there was no need to load any file from the disk to initialize a simple ResNet model. I am also looking for a way to initialize a model in C++ similarly. Does anyone know anyway (even with third-party tools that are compatible with PyTorch C++ API)?</p>

ptrblck: <p><a class="mention" href="/u/krshrimali">@krshrimali</a> published a few blog posts <a href="https://discuss.pytorch.org/t/a-series-of-blogs-on-pytorch-c-api-transfer-learning-jupyter-notebook-with-libtorch-xeus-cling-and-more/54628">here</a> showing how the C++ API can be used. I don’t know if these tutorials are still up to date but they could be a good starter.</p>

andreasceid: <p>Thank you, Patric, for your prompt response. I visited the links. Even though at first they seem to be down, one can quickly search through the older post of the creator and find the tutorials. In the creator’s tutorials, the model classes (e.g., VGG-16) are created from scratch in CPP. In a different tutorial, the user also loads the model from a pre-built <code>torch.jit</code> checkpoint. Furthermore, I have experienced cases where <code>torch.jit</code> does not compile the model properly. For example, in the case of AlexNet, if one builds the model using <code>torch.jit</code> and then attempts to perform inference, they will observe that when the parser reaches the linear layers, forward passing will fail because the <code>flatten()</code> operation is not embedded in the <code>jit</code> checkpoint. Another interesting addition to this topic is that this solution is not scalable if one targets workloads of multiple DNNs running concurrently. I also posted a <a href="https://github.com/pytorch/vision/issues/8327" rel="noopener nofollow ugc">question on GitHub</a>, and it seems that, for some reason, the native PyTorch CPP implementations were deprecated. I also expressed my opinion on why it is useful for many use cases to reinstate the native CPP model implementations (at list for <code>torchvision.models</code>).</p>

andreasceid: <p>Making minor corrections to my response. Jit does successfully parse the hidden tensor operations (operations that are not <code>nn.Module()</code> classes like <code>a.flatten()</code>). I think my intel was outdated, and now that I have run some tests again, it seems to work alright. Still, I think the original issue is unresolved.</p>