Passing Python torch.nn.Module to c++ torch::nn::Module

cpp_for_torch: <p>Hi guys, I want to do forward pass of my pytorch models that is defined in python with C++ and pybind. I followed the tutorial on cuda/C++ extension <a href="https://pytorch.org/tutorials/advanced/cpp_extension" rel="noopener nofollow ugc">here</a>.  With that I was able to:</p>
<ol>
<li>Wrote ext.cpp where the C++ function for forward pass resides</li>
</ol>
<blockquote>
<h3>ext.cpp</h3>
<p><span class="hashtag-raw">#include</span> &lt;torch/extension.h&gt; // One-stop header.<br>
<span class="hashtag-raw">#include</span> <br>
<span class="hashtag-raw">#include</span> <br>
<span class="hashtag-raw">#include</span> </p>
<p>at::Tensor extforward(torch::nn::Module *model, torch::Tensor x)<br>
{<br>
for(const auto submod : model-&gt;modules())<br>
//Looping through the network’s every submodule and applying forward to input<br>
{<br>
x = submod-&gt;as<a>torch::nn::AnyModule</a>()-&gt;forward(x);<br>
}<br>
return x;<br>
}</p>
<p>PYBIND11_MODULE(TORCH_EXTENSION_NAME, m){<br>
m.def(“forw”, &amp;extforward, “CPP forward”);<br>
}</p>
</blockquote>
<ol start="2">
<li>Build a setup.py that compiles the C++ function and make an API sucessfully</li>
</ol>
<blockquote>
<h3>setup.py</h3>
<p>from setuptools import setup, Extension<br>
from torch.utils import cpp_extension</p>
<p>libname = ‘ext’<br>
setup(name=libname,<br>
ext_modules=[cpp_extension.CppExtension(libname, [‘ext.cpp’])],<br>
cmdclass={‘build_ext’: cpp_extension.BuildExtension})</p>
<p>print(cpp_extension.BuildExtension)</p>
</blockquote>
<ol start="3">
<li>Wrote test.py that builds a simple network and try to do forward pass with c++ function</li>
</ol>
<blockquote>
<h3>test.py</h3>
<p>import torch<br>
import torch.nn as nn<br>
import torch.nn.functional as F<br>
import ext as ext</p>
<p>class Net(nn.Module):</p>
<pre><code>def __init__(self):
    super(Net, self).__init__()
    # 1 input image channel, 6 output channels, 3x3 square convolution
    # kernel
    self.conv1 = nn.Conv2d(1, 6, 3)
    self.conv2 = nn.Conv2d(6, 16, 3)
    # an affine operation: y = Wx + b
    self.fc1 = nn.Linear(16 * 6 * 6, 120)  # 6*6 from image dimension
    self.fc2 = nn.Linear(120, 84)
    self.fc3 = nn.Linear(84, 10)

def forward(self, x):
    # Max pooling over a (2, 2) window
    x = F.max_pool2d(F.relu(self.conv1(x)), (2, 2))
    # If the size is a square you can only specify a single number
    x = F.max_pool2d(F.relu(self.conv2(x)), 2)
    x = x.view(-1, self.num_flat_features(x))
    x = F.relu(self.fc1(x))
    x = F.relu(self.fc2(x))
    x = self.fc3(x)
    return x

def num_flat_features(self, x):
    size = x.size()[1:]  # all dimensions except the batch dimension
    num_features = 1
    for s in size:
        num_features *= s
    return num_features
</code></pre>
<p>if <strong>name</strong> == “<strong>main</strong>”:<br>
net = Net()<br>
x = torch.randn(1, 1, 32, 32)<br>
ext.forw(net,x)</p>
</blockquote>
<p>Running Instruction:</p>
<ol>
<li>python setup.py --install</li>
<li>python test.py</li>
</ol>
<p>Issue:</p>
<ol>
<li>The C++ API does not recognize python’s custom network’s type as seen below:<br>
<div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/c/4/c482bb55468ca64c2f25bc1a46dd6701e3c0715b.png" data-download-href="https://discuss.pytorch.org/uploads/default/c482bb55468ca64c2f25bc1a46dd6701e3c0715b" title="Screenshot from 2020-07-18 01-38-13"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/c/4/c482bb55468ca64c2f25bc1a46dd6701e3c0715b_2_690x256.png" alt="Screenshot from 2020-07-18 01-38-13" data-base62-sha1="s2pFTwbiD1FLigepPmH8fzjFaVt" width="690" height="256" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/c/4/c482bb55468ca64c2f25bc1a46dd6701e3c0715b_2_690x256.png, https://discuss.pytorch.org/uploads/default/optimized/3X/c/4/c482bb55468ca64c2f25bc1a46dd6701e3c0715b_2_1035x384.png 1.5x, https://discuss.pytorch.org/uploads/default/optimized/3X/c/4/c482bb55468ca64c2f25bc1a46dd6701e3c0715b_2_1380x512.png 2x" data-dominant-color="353431"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">Screenshot from 2020-07-18 01-38-13</span><span class="informations">1826×678 123 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></li>
</ol>
<p>Other Details:</p>
<ol>
<li>System: Ubuntu 18.04</li>
<li>Environment: virtual env with python3.6</li>
<li>Pytorch version: 1.5.1 stable</li>
</ol>
<p>I know we can use torch jit script, but I feel like torch::nn::Module has much capability that torch::script::module do not have. Would there be anyways to sort of extract all the information from torch.nn.Module into torch::nn::Module? Thanks in advance for all your feedback guys! <img src="https://discuss.pytorch.org/images/emoji/apple/slight_smile.png?v=12" title=":slight_smile:" class="emoji" alt=":slight_smile:" loading="lazy" width="20" height="20"></p>

tom: <p>Hi,</p>
<p>no. torch.nn.Module and torch::nn::Module are completely independent. While we aim for parity between the APIs, you cannot convert from one to another.<br>
The deeper reason is that it would give up quite bit of “Pythonic” to move the modular interface into C++.<br>
So indeed, I think JIT might be the only way to use the same module on both sides of C++ / Python (unless you want to call back into Python from C++).</p>
<p>Best regards</p>
<p>Thomas</p>

Iaroslav_Elistratov: <p>Hi,</p>
<p><a class="mention" href="/u/tom">@tom</a> Could you please clarify, is it still true that we cannot convert from one API to another as of now (a couple years later)?</p>
<p>Best regards</p>

tom: <p>“cannot convert” is decidedly too strong.<br>
A lot of progress has been made.<br>
You still need to do a bit of a dance because the Python API for nn.Module itself does not translate to C++ in a completely “just add semicolons and a few auto typing declarations”-way (e.g. forward is actually tricky because of the signatures).</p>
<p>Best regards</p>
<p>Thomas</p>

charliebudd: <p>Some details on how to achieve this would be helpful.</p>