GitHub issue: How to convert the read images to openCV ? 
State: closed

Hello, 
I'm trying to set the received images in the same format they are if I read them with cv::imread ( is with that format that I train my models). Unfortunately, I'm unable to get correct images, when I save them, they make no sense. The code I'm trying with inside the 'createTemplate' function is the following:

int depth = faces[0].depth;
int width = faces[0].width;
int height = faces[0].height;
int channel = depth/3;

uchar* img_data = faces[0].data.get();

cv::frame = cv::Mat(height, width, CV_8UC3, img_data, channel*width);

By doing channel = depth/8, it works.

Hi @wfcb-85, when I compile and run with :

 cv::Mat(height, width, CV_8UC3, img_data, channel*width);

it gives the error:

error: (-215:Assertion failed) _step >= minstep in function 'Mat'

I am new to C++, can you or someone help me?

Why is channel divided by 3?

I have managed to solve the above issue with the following codes:        

```
for (auto const& face : faces){
                int depth = face.depth;
                int width = face.width;
                int height = face.height;
                int channel = depth/8;
                uchar* img_data = face.data.get();
                auto mat = cv::Mat(height, width, CV_8UC3, img_data);
                cout << typeid(mat).name() << endl;

        }
```

Because I would need to feed a vector into my embedded python program (to run template inference), how can I convert the Mat object to 3D Array or Vector of uint8 elements?


Surely it's easier and faster not to use Python?

A Mat is already a 3d array of uint8.
`uint8_t* = mat.data();`

If you want to split a Mat object in to separate channels you simply use:
```
std::vector<cv::Mat> rgb(3);
cv::split(mat, rgb);
```

Anyway if you need OpenCV support, I'd recommend reading the docs first or asking on their GitHub. This project has nothing to do with OpenCV.

My approach is to embed my Python codes in C++, so that the program can execute in Python. Otherwise I wouldn't know how to execute the Python models, which are written in Tensorflow and Pytorch.

I'd imagine that scenario isn't very common. Tensorflow and Pytorch are actually written in C++ first and have Python as a wrapper. You can also port the model to ONNX and run it in any framework for inference.

For instance, if you typically use PyTorch, you can grab the libtorch include/libs, link to it and then run your model in that. It's far easier and will prevent any troubles with the NIST environment!

It also gives you a lot more control to limit the threading in the way they expect.


Here is an actual complete example of code required to load, run, execute and retrieve the result of a PyTorch model (eg. Face recognition) in C++
```
#include <torch/script.h>
// Only do this once:
auto model = torch::jit::load("filename.pt");
constexpr size_t featureSize = 512; // I assume

// For each image:
for (auto const& f : faces)
{
    auto tensorImage = torch::from_blob(f.data.get(), {(int64_t)f.height, (int64_t)f.width, (int64_t)f.depth / 8});
    tensorImage = tensorImage.permute({2, 0, 1}).unsqueeze(0); // WHC -> NCWH
    auto o = model({tensorImage});
    std::vector<float> feature{o.data_ptr<float>(), o.data_ptr<float>() + featureSize};
}
```

@xsacha Thanks for the tip! Though I am still not sure purely C++ is the way to go. I have model for face detection in pytorch, which means I have to apply functions/attributes to C++ to obtain x-max, x-min, y-max, y-min coordinates. The template inference model is in tensorflow, which i believe can be converted to ONNX.

For now, I am just trying out embedding Python in C++ and hopefully it pass the speed test.

Another C++ question here: Because this is probably linked to data type in face.data.get()

```
        Py_Initialize();
        PyObject *pName, *pModule, *pFunc, *pArgs, *pValue;
        pName = PyUnicode_FromString((char*)"extract_template");
        pModule = PyImport_Import(pName);
        if (pModule==nullptr){
                PyErr_Print();
        }
        pFunc = PyObject_GetAttrString(pModule, (char*)"extract");


        for (auto const& face : faces){
                int depth = face.depth;
                int width = face.width;
                int height = face.height;
                int channel = depth/8;
                int sizes[3] = {height,width,channel};
                npy_intp dims[3] = {height,width,channel};
                uchar* img_data = face.data.get();
                auto mat = cv::Mat(3, sizes, CV_8UC3, img_data);
                pArgs = PyTuple_Pack(1,PyArray_SimpleNewFromData(3,dims,NPY_UINT8,reinterpret_cast<void*>(mat)));
                pValue = PyObject_CallObject(pFunc, pArgs); 
```

I get the following error:
`error: invalid cast from type ‘cv::Mat’ to type ‘void*’`
How can I take in data from face.data.get() and pass into Python as numpy array?




Just do: mat.data if you want the pointer.
Although I'm really confused why you guys are creating a cv::Mat from a pointer, only to retrieve the pointer again.

Huge hint: you do not need OpenCV at all.

```
for (const auto&f : faces)
{
    pArgs = PyArray_SimpleNewFromData(3,{f.height, f.width, f.depth / 8}, NPY_UINT8, (void*)f.data.get());
    pValue = PyObject_CallObject(pFunc, PyTuple_Pack(1, pArgs));
}
```