Repository: pytorch
Issue: Caffe2 GPU Inference not working on pytorch nightly version
Labels: caffe2,triaged

## 🐛 Bug

I have exported detectron2 FRCNN model to caffe2 format(model_init.pb,model.pb) on GPU, then trying to infer on GPU using exported model. But it's not working. Inference is working with stable version of Pytorch but not on nightly version. CPU Inference is also working

## To Reproduce

Steps to reproduce the behavior:

1. Export FRCNN model
1. Infer on exported model.

<!-- If you have a code sample, error messages, stack traces, please provide it here as well -->
```
[E operator.cc:172] Cannot find operator schema for CollectRpnProposals. Will skip schema checking.
Original python traceback for operator `291` in network `torch-jit-export_predict` in exception above (most recent call last):
Traceback (most recent call last):
  File "caffe2_prediction_final.py", line 223, in <module>
    pred = Caffe2Predictor(init_net_path,predict_net_path,init_ops_lib_path,device='GPU')
  File "caffe2_prediction_final.py", line 130, in __init__
    self.predictor = ProtobufModel(init_net_path,predict_net_path,init_ops_lib_path,device=device)
  File "caffe2_prediction_final.py", line 70, in __init__
    ws.CreateNet(self.net)
  File "/home/ubuntu/anaconda3/envs/cf2_nght_gpu4/lib/python3.6/site-packages/caffe2/python/workspace.py", line 172, in CreateNet
    StringifyProto(net), overwrite,
  File "/home/ubuntu/anaconda3/envs/cf2_nght_gpu4/lib/python3.6/site-packages/caffe2/python/workspace.py", line 198, in CallWithExceptionIntercept
    return func(*args, **kwargs)
RuntimeError: [enforce fail at operator.cc:242] op. Cannot create operator of type 'CollectRpnProposals' on the device 'CPU'. Verify that implementation for the corresponding device exist. It might also happen if the binary is not linked with the operator implementation code. If Python frontend is used it might happen if dyndep.InitOpsLibrary call is missing. Operator def: input: "952" input: "953" input: "954" input: "955" input: "956" input: "957" input: "958" input: "959" input: "960" input: "961" output: "rpn_rois.1" name: "" type: "CollectRpnProposals" arg { name: "rpn_max_level" i: 6 } arg { name: "rpn_min_level" i: 2 } arg { name: "rpn_post_nms_topN" i: 1000 } device_option { device_type: 0 device_id: 0 }
frame #0: c10::ThrowEnforceNotMet(char const*, int, char const*, std::string const&, void const*) + 0x59 (0x7f07ce144ed9 in /home/ubuntu/anaconda3/envs/cf2_nght_gpu4/lib/python3.6/site-packages/caffe2/python/../../torch/lib/libc10.so)
frame #1: caffe2::CreateOperator(caffe2::OperatorDef const&, caffe2::Workspace*, int) + 0xa95 (0x7f07f6065c85 in /home/ubuntu/anaconda3/envs/cf2_nght_gpu4/lib/python3.6/site-packages/caffe2/python/../../torch/lib/libcaffe2.so)
frame #2: caffe2::SimpleNet::SimpleNet(std::shared_ptr<caffe2::NetDef const> const&, caffe2::Workspace*) + 0x14f (0x7f07f605ec0f in /home/ubuntu/anaconda3/envs/cf2_nght_gpu4/lib/python3.6/site-packages/caffe2/python/../../torch/lib/libcaffe2.so)
frame #3: <unknown function> + 0x166abbe (0x7f07f6060bbe in /home/ubuntu/anaconda3/envs/cf2_nght_gpu4/lib/python3.6/site-packages/caffe2/python/../../torch/lib/libcaffe2.so)
frame #4: <unknown function> + 0x165147f (0x7f07f604747f in /home/ubuntu/anaconda3/envs/cf2_nght_gpu4/lib/python3.6/site-packages/caffe2/python/../../torch/lib/libcaffe2.so)
frame #5: caffe2::CreateNet(std::shared_ptr<caffe2::NetDef const> const&, caffe2::Workspace*) + 0x659 (0x7f07f603b9a9 in /home/ubuntu/anaconda3/envs/cf2_nght_gpu4/lib/python3.6/site-packages/caffe2/python/../../torch/lib/libcaffe2.so)
frame #6: caffe2::Workspace::CreateNet(std::shared_ptr<caffe2::NetDef const> const&, bool) + 0xe4 (0x7f07f6093834 in /home/ubuntu/anaconda3/envs/cf2_nght_gpu4/lib/python3.6/site-packages/caffe2/python/../../torch/lib/libcaffe2.so)
frame #7: caffe2::Workspace::CreateNet(caffe2::NetDef const&, bool) + 0x7f (0x7f07f6093aef in /home/ubuntu/anaconda3/envs/cf2_nght_gpu4/lib/python3.6/site-packages/caffe2/python/../../torch/lib/libcaffe2.so)
frame #8: <unknown function> + 0x51992 (0x7f07f7c4f992 in /home/ubuntu/anaconda3/envs/cf2_nght_gpu4/lib/python3.6/site-packages/caffe2/python/caffe2_pybind11_state_gpu.cpython-36m-x86_64-linux-gnu.so)
frame #9: <unknown function> + 0x8ade8 (0x7f07f7c88de8 in /home/ubuntu/anaconda3/envs/cf2_nght_gpu4/lib/python3.6/site-packages/caffe2/python/caffe2_pybind11_state_gpu.cpython-36m-x86_64-linux-gnu.so)
<omitting python frames>
frame #49: __libc_start_main + 0xe7 (0x7f0811784b97 in /lib/x86_64-linux-gnu/libc.so.6)
```

## Expected behavior

It is supposed to work as it is working CPU or as stable version . Is there any workaround to solve this?

## Environment

Please copy and paste the output from our
[environment collection script](https://raw.githubusercontent.com/pytorch/pytorch/master/torch/utils/collect_env.py)
(or fill out the checklist below manually).

You can get the script and run it with:
```
wget https://raw.githubusercontent.com/pytorch/pytorch/master/torch/utils/collect_env.py
# For security purposes, please check the contents of collect_env.py before running it.
python collect_env.py
```

```
PyTorch version: 1.0.0.dev20190328
Is debug build: No
CUDA used to build PyTorch: 10.0.130

OS: Ubuntu 18.04.4 LTS
GCC version: (Ubuntu 7.5.0-3ubuntu1~18.04) 7.5.0
CMake version: version 3.13.3

Python version: 3.6
Is CUDA available: Yes
CUDA runtime version: 10.0.130
GPU models and configuration: GPU 0: Tesla V100-SXM2-16GB
Nvidia driver version: 440.33.01
cuDNN version: Probably one of the following:
/usr/local/cuda-10.1/targets/x86_64-linux/lib/libcudnn.so.7.6.5
/usr/local/cuda-10.2/targets/x86_64-linux/lib/libcudnn.so.7.6.5

Versions of relevant libraries:
[pip3] numpy==1.18.5
[pip3] torch==1.0.0.dev20190328
[conda] blas                      1.0                         mkl  
[conda] mkl                       2020.1                      217  
[conda] mkl-service               2.3.0            py36he904b0f_0  
[conda] mkl_fft                   1.1.0            py36h23d657b_0  
[conda] mkl_random                1.1.1            py36h0573a6f_0  
[conda] pytorch-nightly           1.0.0.dev20190328 py3.6_cuda10.0.130_cudnn7.4.2_0    pytorch

```

## Additional context

Installed using ``` conda install pytorch-nightly -c pytorch ```.
Have checked core. _GetRegisteredOperators() . "CollectRpnProposals" is not present in nightly build version but present in release version.
[caff2_nightly.txt](https://github.com/pytorch/pytorch/files/4954733/caff2_nightly.txt)
[pytroch_release.txt](https://github.com/pytorch/pytorch/files/4954734/pytroch_release.txt)