Question about working with the result of the torch :: jit :: module :: forward () function

DimKaKiber: <p>Good day!<br>
I’m just starting to get used to PyTorch and I just can’t figure out how to work with the result of the generated model.<br>
Task: using C / C ++ and openCV to search for key points on faces. To find faces, I use the dnn module of the opencv library with the caffe model.</p>
<pre><code class="lang-auto">std::vector&lt;cv::Point&gt; MainClass::face_detector(cv::Mat src, cv::dnn::Net &amp;net)
{
    std::vector &lt;cv::Point&gt; res;
    if (src.empty() || net.empty())
    {
        std::cerr &lt;&lt; "No image or model empty!" &lt;&lt; std::endl;
        return res;
    }

    cv::Mat blob = cv::dnn::blobFromImage(src,1.0,cv::Size(300,300),cv::Scalar(104.0,177.0,123.0),false,false);
    net.setInput(blob,"data");
    cv::Mat detection = net.forward("detection_out");
    cv::Mat detection_matrix(detection.size[2], detection.size[3], CV_32F, detection.ptr&lt;float&gt;());

    for (int i = 0; i &lt; detection_matrix.rows; i++)
    {
        double confidence = detection_matrix.at&lt;float&gt;(i, 2);

        if (confidence &lt; 0.7) continue;

        int left = static_cast&lt;int&gt;(detection_matrix.at&lt;float&gt;(i, 3) * src.cols);
        int top = static_cast&lt;int&gt;(detection_matrix.at&lt;float&gt;(i, 4) * src.rows);
        res.push_back(cv::Point(left,top));

        int right = static_cast&lt;int&gt;(detection_matrix.at&lt;float&gt;(i, 5) * src.cols);
        int bottom = static_cast&lt;int&gt;(detection_matrix.at&lt;float&gt;(i, 6) * src.rows);
        res.push_back(cv::Point(right,bottom));
    }

    return res;
}

bool MainClass::prepare(std::string path2img,cv::Mat &amp;img, cv::dnn::Net &amp;net,std::vector&lt;cv::Point&gt; &amp;face_areas)
{
    img = cv::imread(path2img);
    if (img.empty())
    {
        std::cerr &lt;&lt; "Image not open!" &lt;&lt; std::endl;
        return false;
    }

    if (!fillCaffe(net,"..//models//caffe_faces//faces_deploy_caffe.prototxt","..//models//caffe_faces//faces_caffe.caffemodel")) return false;
    face_areas.clear();
    unsigned long start = clock();
    face_areas = face_detector(img,net);
    int size = face_areas.size();
    if (size &lt; 1)
    {
        std::cerr &lt;&lt; "No faces on photo" &lt;&lt; std::endl;
        return false;
    }

    std::cout &lt;&lt; "Time of PreProcessing" &lt;&lt; (clock() - start)/(double)(CLOCKS_PER_SEC) &lt;&lt; std::endl;
    return true;
}
</code></pre>
<p>Then I use the MobileNetV2 (56x56) model from here (<a href="https://github.com/cunjian/pytorch_face_landmark" rel="noopener nofollow ugc">https://github.com/cunjian/pytorch_face_landmark</a>) to find keypoints on the found face. For porting to C, I prepared it in advance and got a 5.2 MB * .pt fIle:</p>
<pre><code class="lang-auto">void MainClass::on_pushButton_3_clicked()
https://github.com/cunjian/pytorch_face_landmark
{
    cv::Mat img;
    cv::dnn::Net net;
    std::vector&lt;cv::Point&gt; face_areas;
    if (!prepare("photo.jpg",img,net,face_areas)) return;

    torch::jit::script::Module model = torch::jit::load("../models/MobileFaceNet.pt");
    cv::Point *area_ptr = face_areas.data();
    int size = face_areas.size();

    torch::NoGradGuard no_grad;
    for (int i = 0; i &lt; size; i+=2,area_ptr+=2)
    {
        unsigned long start_image = clock();

        cv::Mat cropped_face = getCroppedFace_mobileNet(img,area_ptr[0],area_ptr[1],112);
        cv::imwrite("cropped"+std::to_string(i)+".png",cropped_face);
        if (cropped_face.empty()) continue;

        unsigned long start_net = clock();
        //from: https://discuss.pytorch.org/t/how-to-convert-an-opencv-image-into-libtorch-tensor/90818/2
        at::Tensor tensor = ToTensor(cropped_face).cpu();
        tensor = tensor.clamp_max(c10::Scalar(50));
        tensor = tensor.toType(c10::kFloat).div(255.0);
        tensor = tensor.permute({ 2,0,1 } );
        tensor = tensor.unsqueeze_(0);
        std::vector&lt;torch::jit::IValue&gt; input;
        input.push_back(tensor);

        auto output = model.forward(input).toTuple(); //https://github.com/pytorch/pytorch/issues/22440

        at::Tensor out_tensor = output-&gt;elements()[0].toTensor();
     }
}
</code></pre>
<p>As a result, I get a tensor, the second dimension of which is 3 (tensor .size (2)). How do I get 68 dots from this tensor? Am I doing everything right?</p>

ptrblck: <p>I’m unsure which model you are using from the linked repository (or rather which submodules are used in your model), but would assume that e.g. <a href="https://github.com/cunjian/pytorch_face_landmark/blob/f575be168a24af6f4807c852173fdfedf6d2c67d/test_batch_detections.py#L141-L176">this code</a> is performing the inference in Python?<br>
If so, you could check which utility methods are needed and try to adapt them in <code>libtorch</code>.</p>

DimKaKiber: <p>I have traced <a href="https://github.com/cunjian/pytorch_face_landmark/blob/f575be168a24af6f4807c852173fdfedf6d2c67d/models/mobilefacenet.py" rel="noopener nofollow ugc">this model</a>. I transferred the code that you resulted (Python) to C ++ until the result is displayed on the image. For this I am using this example <a href="https://github.com/cunjian/pytorch_face_landmark/blob/master/test_camera_mobilefacenet_openvino.py" rel="noopener nofollow ugc">the same</a> repository. But this example uses the openVino library, which outputs a NumPy array to the function output (<code>outputs = exec_net.infer(inputs={input_blob: test_face})</code>).<br>
Without openVino, using a serialized model (from its Python code), I get a tuple, the first part of which (output) I take and don’t understand how to get the result of the model from it. In the case of the Numpy array, everything is clear. But what about the resulting tensor? How do I get the results? I suppose that at least one of its dimensions should become equal to 68 (by the number of points sought)?</p>