Repository: opencv
Issue: Run tensorflow model through inference engine on OpenCVDNN
Labels: category: dnn

##### System information (version)

- OpenCV => 4.5.5 with OpenVINO 2021.4.689
- Operating System / Platform => Windows 64 Bit
- Compiler =>Visual Studio 2017

##### Detailed description

I'm trying to load and infer using a Tensorflow model but using the Inference Engine instead of the OpenCV, you can check the code down bellow:

```
int Setup()
{
    std::cout << CV_VERSION << std::endl;

    unsigned int numInferences = 1000;
    cv::Size inputSizeNewBarracuda = cv::Size(256, 256);

    std::string imagefilename = "C:/Lixo/alreadyTested/1.jpeg";
    std::string newBarracudaTF = "C:/Users/cesar.gouveia/Downloads/saved_model_openvino/model_float32.pb";

    std::cout << "Load model..." << std::endl;
    cv::dnn::Net net = cv::dnn::readNetFromTensorflow(newBarracudaTF);
    
    net.setPreferableBackend(cv::dnn::DNN_BACKEND_INFERENCE_ENGINE);
    net.setPreferableTarget(cv::dnn::DNN_TARGET_CPU);

    std::cout << "Image read..." << std::endl;
    cv::Mat img = cv::imread(imagefilename);
    cv::Mat resized;
    cv::resize(img, resized, inputSizeNewBarracuda);

    std::vector<cv::Mat> imgBatch = { resized };
    bool swapRBChannels = false;
    cv::Mat blob = cv::dnn::blobFromImages(imgBatch, 1.0, cv::Size(), cv::Scalar(), swapRBChannels, false, CV_32F);
    net.setInput(blob);

    std::vector<cv::Mat> outputs;
    outputs.clear();

    std::vector<cv::String> unconnectedOutLayerNames = net.getUnconnectedOutLayersNames();

    std::cout << "Load model and forward first time..." << std::endl;
    net.forward(outputs, unconnectedOutLayerNames);

    std::cout << "Forward..." << std::endl;
    try
    {
        for (size_t i = 0; i < numInferences; i++)
        {
            net.forward(outputs, unconnectedOutLayerNames);
        }
    }
    catch (std::exception& ex)
    {
        std::cout << ex.what() << std::endl;
    }
    
    const cv::Mat& targetMat = outputs[0];
    float* targetBuffer = (float*)targetMat.data;

    // Creating new 2d mat so that we can img show
    cv::Mat newMat = cv::Mat(inputSizeNewBarracuda, CV_32F, targetBuffer);

    cv::imshow("test", newMat * 255);
    cv::waitKey(1);
    std::cout << "Finished" << std::endl;
}

int main()
{
    while (1)
    {
        Setup();
    }
}

```

This is the output of the program:

```
4.5.5
Load model...
Image read...
Load model and forward first time...
Forward...
Finished
4.5.5
Load model...
Image read...
Load model and forward first time...

C:\Lixo\bin\TestBarracuda.exe (process 34652) exited with code -1073741819.
Press any key to close this window . . .
```

Basically I have two questions, the first one is can I use OpenVINO Inference Engine through OpenCVDNN with a tensorflow model (by calling ReadNetworkFromTensorflow())? It does not throw any exception so I think it's possible but I really want to confirm that.

The second question is, why I'm getting this strange behavior? If you look at the output I can load and infer with the tf model using OpenVINO inference engine the first time, but then when I call Setup() function a second time I wasnt able to infer again (it crashes on the first forward).

I tried to run the script multiple times, and it has a really strange behavior because sometimes I'm able to call two setups without the program crashing but the output of the inference is all 0's.

I really don't know why this is happening, If you can give me some input on this I would be very grateful.

Thanks,
César.