Repository: pytorch
Issue: [Caffe2] Errors occured when running Cpp Predictor
Labels: caffe2

Hello,

I am writing a predictor written in cpp so I can use it to predict the output class using the mnist data set. I managed to make it work completely in python but my goal is to use the trained net from python to predict in cpp. However, I am encountering some errors when I run the predictor:
```
carlos@carlos-ubuntu:~/Documents/git/Caffe2_scripts/03_cpp_forward$ mkdir build && cd build
carlos@carlos-ubuntu:~/Documents/git/Caffe2_scripts/03_cpp_forward/build$ cmake ../
-- The C compiler identification is GNU 5.4.0
-- The CXX compiler identification is GNU 5.4.0
-- Check for working C compiler: /usr/bin/cc
-- Check for working C compiler: /usr/bin/cc -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Detecting C compile features
-- Detecting C compile features - done
-- Check for working CXX compiler: /usr/bin/c++
-- Check for working CXX compiler: /usr/bin/c++ -- works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Found Caffe2: /usr/local/lib/libcaffe2.so  
-- Looking for pthread.h
-- Looking for pthread.h - found
-- Looking for pthread_create
-- Looking for pthread_create - not found
-- Looking for pthread_create in pthreads
-- Looking for pthread_create in pthreads - not found
-- Looking for pthread_create in pthread
-- Looking for pthread_create in pthread - found
-- Found Threads: TRUE  
-- Found CUDA: /usr/local/cuda (found version "8.0") 
Project_include_path:/usr/local/include/usr/local/include/eigen3/usr/local/cuda/include/usr/include/opencv/usr/include
-- Configuring done
-- Generating done
-- Build files have been written to: /home/carlos/Documents/git/Caffe2_scripts/03_cpp_forward/build
carlos@carlos-ubuntu:~/Documents/git/Caffe2_scripts/03_cpp_forward/build$ make
Scanning dependencies of target classifier
[ 50%] Building CXX object CMakeFiles/classifier.dir/main.cpp.o
[100%] Linking CXX executable ../classifier
[100%] Built target classifier
carlos@carlos-ubuntu:~/Documents/git/Caffe2_scripts/03_cpp_forward/build$ cd ../
carlos@carlos-ubuntu:~/Documents/git/Caffe2_scripts/03_cpp_forward$ ./classifier --file ./test_img/3.jpg
E0919 11:13:08.691625 11152 init_intrinsics_check.cc:43] CPU feature avx is present on your machine, but the Caffe2 binary is not compiled with it. It means you may not get the full speed of your CPU.
E0919 11:13:08.692265 11152 init_intrinsics_check.cc:43] CPU feature avx2 is present on your machine, but the Caffe2 binary is not compiled with it. It means you may not get the full speed of your CPU.
E0919 11:13:08.692294 11152 init_intrinsics_check.cc:43] CPU feature fma is present on your machine, but the Caffe2 binary is not compiled with it. It means you may not get the full speed of your CPU.
== GPU processing selected  ==
== network loaded  ==
== image size: [42 x 41] ==
== simply resize: [28 x 28] ==
== Tensor  ==
== Blob got  ==
== Data copied  ==
*** Aborted at 1537348389 (unix time) try "date -d @1537348389" if you are using GNU date ***
PC: @     0x7f1db78d381d getenv
*** SIGSEGV (@0x0) received by PID 11152 (TID 0x7f1dc0900000) from PID 0; stack trace: ***
    @     0x7f1db78cf4b0 (unknown)
    @     0x7f1db78d381d getenv
    @     0x7f1dbdc40ea4 caffe2::NumCudaDevices()
    @     0x7f1dbdc41efd caffe2::GetDeviceProperty()
    @     0x7f1dbdcc4c05 caffe2::CudnnConvOpBase::DetermineComputeTypeFromInput<>()
    @     0x7f1dbdcce146 caffe2::CudnnConvOp::DoRunWithType<>()
    @     0x7f1dbdcc21f8 caffe2::CudnnConvOp::RunOnDevice()
    @     0x7f1dbdc0606b caffe2::Operator<>::Run()
    @     0x7f1dbfc1f75b caffe2::SimpleNet::Run()
    @     0x7f1dbfb9b76a caffe2::Workspace::RunNet()
    @           0x42a2e0 caffe2::run()
    @ 0xbfd4bd36bfd893b0 (unknown)
Segmentation fault (core dumped)
```

Since I don't know how I can debbug using cmake, I put some printouts to be able to see where the issue occurs. It seems that error occurs in:
```
// forward
workSpace.RunNet(predictNet.name());
```
This is code of the cpp predictor:
```
/*******************************************************
 * Copyright (C) 2018-2019 bigballon <fm.bigballon@gmail.com>
 * 
 * This file is a caffe2 C++ image classification test 
 * by using pre-trained cifar10 model.
 *
 * Feel free to modify if you need.
 *******************************************************/
#include "caffe2/core/common.h"
#include "caffe2/utils/proto_utils.h"
#include "caffe2/core/workspace.h"
#include "caffe2/core/tensor.h"
#include "caffe2/core/init.h"


// feel free to define USE_GPU if you want to use gpu

#define USE_GPU


#ifdef USE_GPU
#include "caffe2/core/context_gpu.h"
#endif

// headers for opencv 
#include <opencv2/highgui/highgui.hpp>
#include <opencv2/imgproc/imgproc.hpp>

#include <string>
#include <iostream>
#include <map>

// define flags
CAFFE2_DEFINE_string(init_net, "./init_net.pb",
                     "The given path to the init protobuffer.");
CAFFE2_DEFINE_string(predict_net, "./predict_net.pb",
                     "The given path to the predict protobuffer.");
CAFFE2_DEFINE_string(file, "./image_file.jpg", "The image file.");


namespace caffe2{

void loadImage(std::string file_name, float* imgArray){

    auto image = cv::imread(file_name);    // CV_8UC3
    std::cout << "== image size: " << image.size()
              << " ==" << std::endl;

    // scale image to fit
    cv::Size scale(28,28);
    cv::resize(image, image, scale);
    std::cout << "== simply resize: " << image.size() 
              << " ==" << std::endl;
    
    // convert [unsigned int] to [float]
    image.convertTo(image, CV_32FC3);

    // convert NHWC to NCHW
    std::vector<cv::Mat> channels(3);
    cv::split(image, channels);
    std::vector<float> data;
    for (auto &c : channels) {
        data.insert(data.end(), (float *)c.datastart, (float *)c.dataend);
    }
    
    // do normalization & copy to imgArray
    int dim = 0;
    float image_mean[3] = {113.865, 122.95, 125.307};
    float image_std[3] = {66.7048, 62.0887, 62.9932};
        
    for(auto i = 0; i < data.size();++i){
        if(i > 0 && i % (32*32) == 0) dim++;
        imgArray[i] = (data[i] - image_mean[dim]) / image_std[dim];
        // std::cout << imgArray[i] << std::endl;
    }
}

void run(){

    // define a caffe2 Workspace
    Workspace workSpace;

    // define initNet and predictNet
    NetDef initNet, predictNet;

    // read protobuf
    CAFFE_ENFORCE(ReadProtoFromFile(FLAGS_init_net, &initNet));
    CAFFE_ENFORCE(ReadProtoFromFile(FLAGS_predict_net, &predictNet));

    // set device type
#ifdef USE_GPU
    predictNet.mutable_device_option()->set_device_type(CUDA);
    initNet.mutable_device_option()->set_device_type(CUDA);
    std::cout << "== GPU processing selected " << " ==" << std::endl;
#else
    predictNet.mutable_device_option()->set_device_type(CPU);
    initNet.mutable_device_option()->set_device_type(CPU);

    for(int i = 0; i < predictNet.op_size(); ++i){
        predictNet.mutable_op(i)->mutable_device_option()->set_device_type(CPU);
    }
    for(int i = 0; i < initNet.op_size(); ++i){
        initNet.mutable_op(i)->mutable_device_option()->set_device_type(CPU);
    }
#endif

    // load network
    CAFFE_ENFORCE(workSpace.RunNetOnce(initNet));
    CAFFE_ENFORCE(workSpace.CreateNet(predictNet));
    std::cout << "== network loaded " << " ==" << std::endl;

    // load image from file, then convert it to float array.
    float imgArray[1 * 28 * 28];
    loadImage(FLAGS_file, imgArray);

    // define a Tensor which is used to stone input data
    std::cout << "== Tensor " << " ==" << std::endl;
    TensorCPU input;
    input.Resize(std::vector<TIndex>({1, 1, 28, 28}));
    input.ShareExternalPointer(imgArray);

    // get "data" blob
#ifdef USE_GPU
    auto data = workSpace.GetBlob("data")->GetMutable<TensorCUDA>();
#else
    auto data = workSpace.GetBlob("data")->GetMutable<TensorCPU>();
#endif
    std::cout << "== Blob got " << " ==" << std::endl;

    // copy from input data
    data->CopyFrom(input);
    std::cout << "== Data copied " << " ==" << std::endl;

    // forward
    workSpace.RunNet(predictNet.name());
    std::cout << "== Net was run " << " ==" << std::endl;

    // get predictions blob and show the results
    std::vector<std::string> labelName = {"0","1","2","3","4",
        "5","6","7","8","9"};

#ifdef USE_GPU
    auto predictions = TensorCPU(workSpace.GetBlob("predictions")->Get<TensorCUDA>());
#else
    auto predictions = workSpace.GetBlob("predictions")->Get<TensorCPU>();
#endif
    std::cout << "== predictions got " << " ==" << std::endl;

    std::vector<float> probs(predictions.data<float>(),
        predictions.data<float>() + predictions.size());

    auto max = std::max_element(probs.begin(), probs.end());
    auto index = std::distance(probs.begin(), max);
    std::cout << "== predicted label: " << labelName[index]
              << " ==\n== with probability: " << (*max * 100)
              << "% ==" << std::endl;
}

}    // namespace caffe2

// main function
int main(int argc, char** argv) {
    caffe2::GlobalInit(&argc, &argv);
    caffe2::run();
    google::protobuf::ShutdownProtobufLibrary();
    return 0;
}
```

This is the structure of my project:
![image](https://user-images.githubusercontent.com/25825048/45745633-8e1c2b00-bc01-11e8-8ffc-fac066470584.png)

Attached my init_net.pb and predict_net.pb in `.pb` and `pbtxt` format. I had to chnage their extensions to .txt to be able to upload them.
[init_net.txt](https://github.com/BIGBALLON/Caffe2_Demo/files/2396478/init_net.txt)
[predict_net.txt](https://github.com/BIGBALLON/Caffe2_Demo/files/2396479/predict_net.txt)
[init_net_pbtxt.txt](https://github.com/BIGBALLON/Caffe2_Demo/files/2396599/init_net_pbtxt.txt)
[predict_net_pbtxt.txt](https://github.com/BIGBALLON/Caffe2_Demo/files/2396600/predict_net_pbtxt.txt)


**Do you have an idea what could be the reason for the error?**

**My configuration:
-  Caffe2 tag v0.4.0
- CUDA/cuDNN version: 8.0/7.0.5
- GPU models and configuration: GTX 1050
- CMake version:**