Running cpp torch::jit::Graph

Omer_B: <p>Hey, I’m quite stuck on trying to run torch::jit::Graph with cpp API, I have already tried to search for docs or implementations but failed.<br>
my main purpose to do so is to divide a JIT Graph into subgraphs (I would like to control which nodes go to each subgraph). My plan is to parse a graph into nodes and then create new graphs from each sequence of nodes (with my own decision) and then run each graph one by one.</p>
<p>this is what I have already tried:</p>
<pre><code class="lang-cpp">#include &lt;torch/script.h&gt;
#include &lt;iostream&gt;

int main() {
    torch::jit::script::Module my_model = torch::jit::load("&lt;MY_PATH&gt;/model.pkl");

    torch::jit::script::Method m = my_model.get_method("forward");
    auto g = m.graph();

    auto cu = std::make_shared&lt;torch::jit::script::CompilationUnit&gt;();
    c10::QualifiedName name("forward");
    torch::Function *fn = cu-&gt;create_function(std::move(name), g);

    std::vector&lt;torch::jit::IValue&gt; inputs;
    inputs.push_back(torch::rand({1, 1, 3, 3}));

    auto output = fn-&gt;operator()(inputs);
    std::cout &lt;&lt; output &lt;&lt; std::endl;
}
</code></pre>
<p>based on <code>model.pkl</code> that was produced by the python code below:</p>
<pre><code class="lang-python">import torch
import torch.nn as nn

class Net(nn.Module):
    def __init__(self):
        super(Net, self).__init__()
        self.conv = nn.Conv2d(1, 1, 3)

    def forward(self, x):
        return self.conv(x)

scripted_foo = torch.jit.script(Net())
scripted_foo.save('model.pkl')
</code></pre>
<p>this cpp approch produces the following error:</p>
<pre><code class="lang-auto">libc++abi.dylib: terminating with uncaught exception of type c10::Error: forward() Expected a value of type '__torch__.Net' for argument 'self' but instead found type 'Tensor'.
Position: 0
Declaration: forward(__torch__.Net self, Tensor x) -&gt; (Tensor) (checkArg at ../aten/src/ATen/core/function_schema_inl.h:194)
frame #0: c10::Error::Error(c10::SourceLocation, std::__1::basic_string&lt;char, std::__1::char_traits&lt;char&gt;, std::__1::allocator&lt;char&gt; &gt; const&amp;) + 135 (0x1038ce787 in libc10.dylib)
frame #1: c10::FunctionSchema::checkArg(c10::IValue const&amp;, c10::Argument const&amp;, c10::optional&lt;unsigned long&gt;) const + 719 (0x11285ee6f in libtorch.dylib)
frame #2: c10::FunctionSchema::checkAndNormalizeInputs(std::__1::vector&lt;c10::IValue, std::__1::allocator&lt;c10::IValue&gt; &gt;&amp;, std::__1::unordered_map&lt;std::__1::basic_string&lt;char, std::__1::char_traits&lt;char&gt;, std::__1::allocator&lt;char&gt; &gt;, c10::IValue, std::__1::hash&lt;std::__1::basic_string&lt;char, std::__1::char_traits&lt;char&gt;, std::__1::allocator&lt;char&gt; &gt; &gt;, std::__1::equal_to&lt;std::__1::basic_string&lt;char, std::__1::char_traits&lt;char&gt;, std::__1::allocator&lt;char&gt; &gt; &gt;, std::__1::allocator&lt;std::__1::pair&lt;std::__1::basic_string&lt;char, std::__1::char_traits&lt;char&gt;, std::__1::allocator&lt;char&gt; &gt; const, c10::IValue&gt; &gt; &gt; const&amp;) const + 228 (0x11285ddf4 in libtorch.dylib)
frame #3: torch::jit::Function::operator()(std::__1::vector&lt;c10::IValue, std::__1::allocator&lt;c10::IValue&gt; &gt;, std::__1::unordered_map&lt;std::__1::basic_string&lt;char, std::__1::char_traits&lt;char&gt;, std::__1::allocator&lt;char&gt; &gt;, c10::IValue, std::__1::hash&lt;std::__1::basic_string&lt;char, std::__1::char_traits&lt;char&gt;, std::__1::allocator&lt;char&gt; &gt; &gt;, std::__1::equal_to&lt;std::__1::basic_string&lt;char, std::__1::char_traits&lt;char&gt;, std::__1::allocator&lt;char&gt; &gt; &gt;, std::__1::allocator&lt;std::__1::pair&lt;std::__1::basic_string&lt;char, std::__1::char_traits&lt;char&gt;, std::__1::allocator&lt;char&gt; &gt; const, c10::IValue&gt; &gt; &gt; const&amp;) + 49 (0x11285d221 in libtorch.dylib)
frame #4: main + 686 (0x1037eb59e in deep_project)
frame #5: start + 1 (0x7fff700f0cc9 in libdyld.dylib)
frame #6: 0x0 + 1 (0x1 in ???)

Process finished with exit code 6
</code></pre>
<p>I assume that I need to push <code>self</code> of type <code>__torch__.Net</code> somehow but I dont know how, I will be happy for some help.<br>
thanks a lot.</p>

glaringlee: <p>Can you post this under ‘jit’ category?</p>

Michael_Suo: <p>For methods like, as in Python, the first argument to the graph is <code>self</code>, which represents the module object instance. The <code>Module</code> API takes care of adding <code>self</code> to the stack, see <a href="https://github.com/pytorch/pytorch/blob/master/torch/csrc/jit/api/module.cpp#L115" rel="nofollow noopener">here</a>.</p>

Omer_B: <p>thanks a lot, it works - I have another question though, do you know what is the most clean way to create a graph from a couple of nodes ?</p>

Michael_Suo: <p>Probably using the base Graph API for node creation and insertion is the cleanest. For an example of how to take “foreign” <code>Node*</code>s not owned by a graph and copy them into the graph, the <code>Graph::copy</code> method is a useful example: <a href="https://github.com/pytorch/pytorch/blob/master/torch/csrc/jit/ir/ir.cpp#L680" rel="nofollow noopener">https://github.com/pytorch/pytorch/blob/master/torch/csrc/jit/ir/ir.cpp#L680</a></p>

squareroot: <p>Could someone to update solution for new versions ?</p>