Repository: pytorch
Issue: Loading pretrained model
Labels: caffe2

Hi, 

Recently i have converted a detectron model using the tools/convert_pkl_to_pb.py from Detetron repo which results in model_init.pb,  model.pb and model.pbtxt. Then, i try to load the pretrained model following this tutorial (https://github.com/facebookarchive/tutorials/blob/master/Loading_Pretrained_Models.ipynb). By, following through the tutorial, i was able to make predictions using the provided model. However, when i try with the converted model, the following error occurs. 

```
[E operator.cc:203] Cannot find operator schema for BatchPermutation. Will skip schema checking.
Traceback (most recent call last):
  File "infer.py", line 153, in <module>
    p = workspace.Predictor(init_net, predict_net)
  File "/opt/conda/lib/python3.6/site-packages/caffe2/python/workspace.py", line 186, in Predictor
    return C.Predictor(StringifyProto(init_net), StringifyProto(predict_net))
RuntimeError: [enforce fail at operator.cc:273] op. Cannot create operator of type 'BatchPermutation' on the device 'CUDA'. Verify that implementation for the corresponding device exist. It might also happen if the binary is not linked with the operator implementation code. If Python frontend is used it might happen if dyndep.InitOpsLibrary call is missing. Operator def: input: "roi_feat_shuffled" input: "rois_idx_restore_int32_gpu_0" output: "roi_feat" name: "" type: "BatchPermutation" device_option { device_type: 1 device_id: 0 }
frame #0: c10::ThrowEnforceNotMet(char const*, int, char const*, std::string const&, void const*) + 0x5b (0x7fd70bf1341b in /opt/conda/lib/python3.6/site-packages/caffe2/python/../../torch/lib/libc10.so)
frame #1: caffe2::CreateOperator(caffe2::OperatorDef const&, caffe2::Workspace*, int) + 0xc14 (0x7fd70eaec084 in /opt/conda/lib/python3.6/site-packages/caffe2/python/../../torch/lib/libtorch.so)
frame #2: caffe2::SimpleNet::SimpleNet(std::shared_ptr<caffe2::NetDef const> const&, caffe2::Workspace*) + 0x2e0 (0x7fd70eae4040 in /opt/conda/lib/python3.6/site-packages/caffe2/python/../../torch/lib/libtorch.so)
frame #3: <unknown function> + 0x278338e (0x7fd70eae638e in /opt/conda/lib/python3.6/site-packages/caffe2/python/../../torch/lib/libtorch.so)
frame #4: std::_Function_handler<std::unique_ptr<caffe2::NetBase, std::default_delete<caffe2::NetBase> > (std::shared_ptr<caffe2::NetDef const> const&, caffe2::Workspace*), std::unique_ptr<caffe2::NetBase, std::default_delete<caffe2::NetBase> > (*)(std::shared_ptr<caffe2::NetDef const> const&, caffe2::Workspace*)>::_M_invoke(std::_Any_data const&, std::shared_ptr<caffe2::NetDef const> const&, caffe2::Workspace*&&) + 0xf (0x7fd70eabaf7f in /opt/conda/lib/python3.6/site-packages/caffe2/python/../../torch/lib/libtorch.so)
frame #5: caffe2::CreateNet(std::shared_ptr<caffe2::NetDef const> const&, caffe2::Workspace*) + 0x464 (0x7fd70eaacb74 in /opt/conda/lib/python3.6/site-packages/caffe2/python/../../torch/lib/libtorch.so)
frame #6: caffe2::Workspace::CreateNet(std::shared_ptr<caffe2::NetDef const> const&, bool) + 0xfe (0x7fd70eb297ee in /opt/conda/lib/python3.6/site-packages/caffe2/python/../../torch/lib/libtorch.so)
frame #7: caffe2::Predictor::Predictor(caffe2::PredictorConfig) + 0x323 (0x7fd70edbb853 in /opt/conda/lib/python3.6/site-packages/caffe2/python/../../torch/lib/libtorch.so)
frame #8: <unknown function> + 0x4f8a5 (0x7fd73e1298a5 in /opt/conda/lib/python3.6/site-packages/caffe2/python/caffe2_pybind11_state_gpu.cpython-36m-x86_64-linux-gnu.so)
frame #9: <unknown function> + 0x4fb77 (0x7fd73e129b77 in /opt/conda/lib/python3.6/site-packages/caffe2/python/caffe2_pybind11_state_gpu.cpython-36m-x86_64-linux-gnu.so)
frame #10: <unknown function> + 0x95696 (0x7fd73e16f696 in /opt/conda/lib/python3.6/site-packages/caffe2/python/caffe2_pybind11_state_gpu.cpython-36m-x86_64-linux-gnu.so)
<omitting python frames>
frame #30: __libc_start_main + 0xf0 (0x7fd76da8e830 in /lib/x86_64-linux-gnu/libc.so.6)
```
Anyone knows why this happen?

Thanks