Question on how to interpret the output of FasterRCNN in C++ with the GenericDict type

fstrati70: <p>I’m trying to interpret the output of FasterRCNN in C++ and I’m fighting with the GenericDict type.</p>
<p>My code is as follows:</p>
<pre><code class="lang-auto">#include &lt;opencv4/opencv2/opencv.hpp&gt;
#include &lt;opencv4/opencv2/shape.hpp&gt;
#include &lt;opencv4/opencv2/imgcodecs.hpp&gt;
#include &lt;opencv4/opencv2/highgui.hpp&gt;
#include &lt;opencv4/opencv2/imgproc.hpp&gt;
#include &lt;opencv4/opencv2/core/utility.hpp&gt;
#include &lt;opencv4/opencv2/core/mat.hpp&gt;

#include &lt;c10/cuda/CUDAStream.h&gt;
#include &lt;torch/csrc/autograd/grad_mode.h&gt;

#include &lt;torch/csrc/api/include/torch/torch.h&gt;
#include &lt;torch/script.h&gt;
#include &lt;torchvision/vision.h&gt;
#include &lt;torchvision/nms.h&gt;

#include &lt;iostream&gt;
#include &lt;memory&gt;
#include &lt;string&gt;

int main(int argc, const char* argv[])
{
    if (argc != 3)
    {
        printf("usage: %s &lt;path-to-exported-script-module&gt; &lt;image_to_test&gt;\n",argv[0]);
        return -1;
    }

    std::string module_filename = argv[1];
    std::string image_file = argv[2];

    try
    {
        cv::Mat input_img = cv::imread(image_file, cv::IMREAD_GRAYSCALE);

        torch::autograd::AutoGradMode guard(false);
        // Deserialize the ScriptModule from a file using torch::jit::load().
        torch::jit::script::Module module = torch::jit::load(module_filename);

        assert(module.buffers().size() &gt; 0);

        module.eval();

        // Assume that the entire model is on the same device.
        // We just put input to this device.
        auto device = (*std::begin(module.buffers())).device();

        const int height = input_img.rows;
        const int width  = input_img.cols;
        const int channels = 1;

        auto input = torch::from_blob(input_img.data, {height, width, channels}, torch::kUInt8);
        // HWC to CHW
        // input = input.to(device, torch::kFloat).permute({2, 0, 1}).contiguous();
        input = input.to(device, torch::kFloat).permute({2, 0, 1}).contiguous();

        // run the network
        std::vector&lt;at::Tensor&gt; inputs;
        inputs.push_back(input);
        auto output = module.forward({inputs});
        if (device.is_cuda())
            c10::cuda::getCurrentCUDAStream().synchronize();

        std::cout &lt;&lt; "output: " &lt;&lt; output &lt;&lt; std::endl;

        auto outputs = output.toTuple()-&gt;elements();

        std::cout &lt;&lt; "outputs: " &lt;&lt; outputs &lt;&lt; std::endl;

        for( auto&amp; elem : outputs )
        {
            std::cout &lt;&lt; "elem: " &lt;&lt; elem &lt;&lt; std::endl;
            if( elem.isGenericDict() )
            {
                std::cout &lt;&lt; "elem is generic dict: " &lt;&lt; elem &lt;&lt; std::endl;
                c10::Dict&lt;c10::IValue, c10::IValue&gt; dict = elem.toGenericDict();

                auto elem_vector_0 = dict.at(c10::IValue("scores")).toIntVector();
                auto elem_vector_1 = dict.at(c10::IValue("boxes")).toIntVector();
                auto elem_vector_2 = dict.at(c10::IValue("labels")).toIntVector();

                for( auto&amp; ee0 : elem_vector_0 )
                {
                    std::cout &lt;&lt; "elem_vector_0" &lt;&lt; ee0 &lt;&lt; std::endl;
                }
                for( auto&amp; ee0 : elem_vector_1 )
                {
                    std::cout &lt;&lt; "elem_vector_1" &lt;&lt; ee0 &lt;&lt; std::endl;
                }
                for( auto&amp; ee0 : elem_vector_2 )
                {
                    std::cout &lt;&lt; "elem_vector_2" &lt;&lt; ee0 &lt;&lt; std::endl;
                }
            }
        }

        cv::namedWindow("Display Image", cv::WINDOW_AUTOSIZE );
        cv::imshow("Display Image", input_img);
        cv::waitKey(0);
    }
    catch(const c10::Error&amp; e)
    {
        std::cerr &lt;&lt; e.what() &lt;&lt; std::endl;
        return -1;
    }
    catch(const cv::Exception&amp; e)
    {
        std::cerr &lt;&lt; e.what() &lt;&lt; std::endl;
        return -1;
    }
    catch(const std::exception&amp; e)
    {
        std::cerr &lt;&lt; e.what() &lt;&lt; std::endl;
        return -1;
    }
    catch(...)
    {
        std::cerr &lt;&lt; "Unknown error" &lt;&lt; std::endl;
        return -1;
    }

    std::cout &lt;&lt; "ok\n";
    return 0;
}
</code></pre>
<p>and the output is:</p>
<pre><code class="lang-auto">(base) fstrati@fstrati-desktop:~/libtorch_shared_cuda_10.1/load_and_run_model/Release$ ./load_and_run_model ./torch_script_v0.2.pt test_img.png 
[W faster_rcnn.py:95] Warning: RCNN always returns a (Losses, Detections) tuple in scripting (function )
output: ({}, [{boxes: [ CPUFloatType{0,4} ], labels: [ CPULongType{0} ], scores: [ CPUFloatType{0} ]}])
outputs: {} [{boxes: [ CPUFloatType{0,4} ], labels: [ CPULongType{0} ], scores: [ CPUFloatType{0} ]}]
elem: {}
elem is generic dict: {}
Argument passed to at() was not in the map.
</code></pre>
<p>I’m struggling to find a way to extract the boxes, labels and scores from the dictionary GenericDict.</p>
<p>This map is strange, I cannot iterate on it and I cannot access first and second types… with it-&gt;first it-&gt;second</p>
<p>Any ideas ?</p>
<p>Thanks in advance</p>

ms28: <p>Hi, I am having the same problem. Have you found a solution ?</p>

Mikhail_Miha: <p>Hi! I’ve got the same. Did you overcame that obstacle? Could you advise something?</p>

KouWeiBin: <p>you can try following code:</p>
<pre><code class="lang-auto">
    auto val_out = out[1].toList().get(0).toGenericDict();
    auto boxes = val_out.at("boxes");
    auto scores = val_out.at("scores");


    at::Tensor bbox = boxes.toTensor().data();
    at::Tensor bscore = scores.toTensor().data();
</code></pre>

KouWeiBin: <p>Can you share me your code which is used to convert the model trained by python to Torchscript model? Actually I used my code to convert. but the torchscipt model always occur issue in forward function.</p>

Mikhail_Miha: <p>Yes it works!!! <img src="https://discuss.pytorch.org/images/emoji/apple/+1.png?v=9" title=":+1:" class="emoji" alt=":+1:"></p>

KouWeiBin: <p>I am very glad that it can benefit for you.<br>
can you share me your code which is used to convert the the trained model by python to Torchscript?</p>

Mikhail_Miha: <p>Hello<br>
Where to send  code?</p>

KouWeiBin: <p>Thanks for your reply!</p>
<p>you can send the code to <a href="mailto:qvb751201402@163.com">qvb751201402@163.com</a>; or you can pose the code between <code> </code>  here!</p>