How can I feed the output of one module to another in C++?

username: Shisho_Sama
cooked: <p>Hello everyone. I’m trying a very basic example, in which I’m loading an MTCNN model (consists of 3 submodules onet, pnet and rnet) and trying to feed forward them. This is the code :</p>
<pre><code class="lang-cpp">#include &lt;torch/torch.h&gt;
#include &lt;torch/script.h&gt; // One-stop header.
#include &lt;iostream&gt;
#include &lt;memory&gt;
#include &lt;cstdlib&gt;

int main(int argc, const char* argv[]) 
{

    // Create a vector of inputs.
    std::vector&lt;torch::jit::IValue&gt; inputs;
    inputs.push_back(torch::ones({ 1, 3, 112, 112 }));

    torch::jit::script::Module onet;
    torch::jit::script::Module pnet;
    torch::jit::script::Module rnet;
    try 
    {
        std::string mtcnn_onet_path = "G:\\Codes\\python\\MTCNN\\weights_mtcnn_onet.pt";
        std::string mtcnn_pnet_path = "G:\\Codes\\python\\MTCNN\\weights_mtcnn_pnet.pt";
        std::string mtcnn_rnet_path = "G:\\Codes\\python\\MTCNN\\weights_mtcnn_rnet.pt";

        // Deserialize the ScriptModule from a file using torch::jit::load().
        onet = torch::jit::load(mtcnn_onet_path);
        pnet = torch::jit::load(mtcnn_pnet_path);
        rnet = torch::jit::load(mtcnn_rnet_path);

        // Execute the model and turn its output into a tensor.
        at::Tensor output_o = onet.forward(inputs).toTensor();
        at::Tensor output_p = pnet.forward(output_o).toTensor();
        at::Tensor output = rnet.forward(output_p).toTensor();

        std::cout &lt;&lt; output.sizes() &lt;&lt; std::endl;
        std::cout &lt;&lt; "output: " &lt;&lt; output[0] &lt;&lt; std::endl;
    
    }
    catch (const c10::Error&amp; e) 
    {
        std::cerr &lt;&lt; "error loading the model\n";
        return -1;
    }

    std::cout &lt;&lt; "ok\n";
    std::system("pause");
    return 0;
}
</code></pre>
<p>but I get :</p>
<pre><code class="lang-cpp">Severity	Code	Description	Project	File	Line	Suppression State
Error	C2664	'c10::IValue torch::jit::Module::forward(std::vector&lt;c10::IValue,std::allocator&lt;c10::IValue&gt;&gt;)': cannot convert argument 1 from 'at::Tensor' to 'std::vector&lt;c10::IValue,std::allocator&lt;c10::IValue&gt;&gt;'	LibtorchPort	G:\Codes\cpp\port\LibtorchPort\LibtorchPort\LibtorchPort.cpp	54	

Severity	Code	Description	Project	File	Line	Suppression State
Error	C2664	'c10::IValue torch::jit::Module::forward(std::vector&lt;c10::IValue,std::allocator&lt;c10::IValue&gt;&gt;)': cannot convert argument 1 from 'at::Tensor' to 'std::vector&lt;c10::IValue,std::allocator&lt;c10::IValue&gt;&gt;'	LibtorchPort	G:\Codes\cpp\port\LibtorchPort\LibtorchPort\LibtorchPort.cpp	55	

</code></pre>
<p>Shouldnt we pass inputs as Tensors like in Python?<br>
I couldnt find any documentation that properly explains this . any help in this regard is appreciated.</p>
<p>By the way these are the modules : <a href="https://gofile.io/d/DyeriF" rel="nofollow noopener">https://gofile.io/d/DyeriF</a></p>

username: chetan_patil
cooked: <p>Hi, the <code>output_o</code> had been converted back to an <code>at::Tensor</code>, and this is now placed inside <code>pnet.forward()</code>.<br>
And error is thrown at this line.<br>
<code>output_o</code> needs to be converted to a <code>torch::jit::IValue</code> an then put inside <code>pnet.forward()</code>.<br>
The same goes to <code>rnet.forward()</code> as well.</p>

username: Shisho_Sama
cooked: <p>Thanks, really appreciate it, but what Should I be setting as the type for the new outputs?<br>
using auto also fails! :</p>
<pre><code class="lang-cpp">auto output_o = onet.forward(inputs);
auto output_p = pnet.forward(output_o);
</code></pre>
<p>while I expected this to work!<br>
also these failed as well :</p>
<pre><code class="lang-cpp">std::vector&lt;c10::IValue, std::allocator&lt;c10::IValue&gt;&gt; output_o = onet.forward(inputs);
std::vector&lt;c10::IValue, std::allocator&lt;c10::IValue&gt;&gt; output_p = pnet.forward(output_o);
</code></pre>
<p>and also this :</p>
<pre><code class="lang-cpp">std::vector&lt;c10::IValue&gt; output_o = onet.forward(inputs);
std::vector&lt;c10::IValue&gt;  output_p = pnet.forward(output_o);
</code></pre>
<p>results in :</p>
<pre><code class="lang-cpp">Severity	Code	Description	Project	File	Line	Suppression State
Error	C2440	'initializing': cannot convert from 'c10::IValue' to 'std::vector&lt;c10::IValue,std::allocator&lt;c10::IValue&gt;&gt;'	LibtorchPort	G:\Codes\cpp\port\LibtorchPort\LibtorchPort\LibtorchPort.cpp	52	
Error	C2440	'initializing': cannot convert from 'c10::IValue' to 'std::vector&lt;c10::IValue,std::allocator&lt;c10::IValue&gt;&gt;'	LibtorchPort	G:\Codes\cpp\port\LibtorchPort\LibtorchPort\LibtorchPort.cpp	53	

</code></pre>
<p>and also this:</p>
<pre><code class="lang-cpp">torch::jit::IValue output_o = onet.forward(inputs);
torch::jit::IValue output_p = pnet.forward(output_o);
</code></pre>
<p>results in :</p>
<pre><code class="lang-cpp">Severity	Code	Description	Project	File	Line	Suppression State
Error	C2664	'c10::IValue torch::jit::Module::forward(std::vector&lt;c10::IValue,std::allocator&lt;c10::IValue&gt;&gt;)': cannot convert argument 1 from 'c10::IValue' to 'std::vector&lt;c10::IValue,std::allocator&lt;c10::IValue&gt;&gt;'	LibtorchPort	G:\Codes\cpp\port\LibtorchPort\LibtorchPort\LibtorchPort.cpp	53	
Error	C2664	'c10::IValue torch::jit::Module::forward(std::vector&lt;c10::IValue,std::allocator&lt;c10::IValue&gt;&gt;)': cannot convert argument 1 from 'c10::IValue' to 'std::vector&lt;c10::IValue,std::allocator&lt;c10::IValue&gt;&gt;'	LibtorchPort	G:\Codes\cpp\port\LibtorchPort\LibtorchPort\LibtorchPort.cpp	54	

</code></pre>
<p>and</p>
<pre><code class="lang-cpp">std::vector&lt;torch::jit::IValue&gt; output_o = onet.forward(inputs);
std::vector&lt;torch::jit::IValue&gt; output_p = pnet.forward(output_o);
</code></pre>
<p>results in :</p>
<pre><code class="lang-cpp">Severity	Code	Description	Project	File	Line	Suppression State
Error	C2440	'initializing': cannot convert from 'c10::IValue' to 'std::vector&lt;c10::IValue,std::allocator&lt;c10::IValue&gt;&gt;'	LibtorchPort	G:\Codes\cpp\port\LibtorchPort\LibtorchPort\LibtorchPort.cpp	52	

Severity	Code	Description	Project	File	Line	Suppression State
Error	C2440	'initializing': cannot convert from 'c10::IValue' to 'std::vector&lt;c10::IValue,std::allocator&lt;c10::IValue&gt;&gt;'	LibtorchPort	G:\Codes\cpp\port\LibtorchPort\LibtorchPort\LibtorchPort.cpp	53	
</code></pre>

username: chetan_patil
cooked: <p>Looking at the errors,<br>
the output from the <code>model.forward()</code> will be a<br>
<code>torch::jit::IValue</code>.<br>
However the input to a <code>model.forward()</code> will be a <code>std::vector&lt;torch::jit::IValue&gt;</code></p>
<p>Can you try this ?</p>
<pre><code class="lang-auto">auto output_o = onet.forward(inputs);

std::vector&lt;torch::jit::IValue&gt;temp_op;

temp_op.push_back(output_o);

auto output_p = pnet.forward(temp_op);
</code></pre>

username: Shisho_Sama
cooked: <p>Thanks I no more get that error instead I get an excpetion :</p>
<pre><code class="lang-auto">Unhandled exception at 0x00007FFF8523A308 in LibtorchPort.exe: Microsoft C++ exception: std::runtime_error at memory location 0x0000002FB98F7830.
</code></pre>
<p>pointing at the very beginning meaning <code>onet.forward</code>:</p>
<pre><code class="lang-cpp">auto output_o = onet.forward(inputs);
auto  output_p = pnet.forward(std::vector&lt;torch::jit::IValue&gt;{ output_o });
at::Tensor output = rnet.forward(std::vector&lt;torch::jit::IValue&gt;{ output_p }).toTensor();
</code></pre>
<p>I also uploaded the mtcnn modules in case they are needed : <a href="https://gofile.io/d/DyeriF" rel="nofollow noopener">https://gofile.io/d/DyeriF</a></p>

username: chetan_patil
cooked: <p>Thanks I’ll get back to you if any progress from my side <img src="https://discuss.pytorch.org/images/emoji/apple/slight_smile.png?v=9" title=":slight_smile:" class="emoji" alt=":slight_smile:"></p>

username: Shisho_Sama
cooked: <p>thanks a lot. really appreciate it:)</p>