Repository: pytorch
Issue: How to load PyTorch model with LSTM using C++ api
Labels: module: cpp,triaged

## 🐛 Bug

<!-- A clear and concise description of what the bug is. -->

## To Reproduce

Steps to reproduce the behavior:

1. Establish a PyTorch model with LSTM module using python, and store the script module after using torch.jit.trace. Python code like this:

```python
class MyModule(nn.Module):
    def __init__(self, N, M):
        super(MyModule, self).__init__()
        self.lstm = nn.LSTM(M, M, batch_first=True)
        self.linear = nn.Linear(M, 1)

    def forward(self, inputs, h0, c0):

        output, (_, _) = self.lstm(inputs, h0, c0)
        output, _ = torch.max(output, dim=1)
        # output, _ = torch.max(inputs, dim=1)
        output = self.linear(output)
        return output

batch_size = 8
h = 33
w = 45
model =  MyModule(h, w)
data = np.random.normal(1, 1, size=(batch_size, h, w))
data = torch.Tensor(data)
h0, c0 = torch.zeros(1, batch_size, w), torch.zeros(1, batch_size, w)

traced_script_module = torch.jit.trace(model, (data, h0,c0))
traced_script_module.save('model.pt')
```



2. Load the model and move the model to GPU, then when the script exit, there is a core dump. However, If we don't move the model to gpu, the cpp script exits normally.My cpp script like this:

```c++
int main(int argc, const char* argv[]) {
  if (argc != 2) {
    std::cerr << "usage: example-app <path-to-exported-script-module>\n";
    return -1;
  }

  // Deserialize the ScriptModule from a file using torch::jit::load().
  std::shared_ptr<torch::jit::script::Module> module = torch::jit::load(argv[1]);

  assert(module != nullptr);
  std::cout << "ok\n";
  this->module->to(at::Device("cuda:0"))

  vector<torch::jit::IValue> inputs;
  int b = 2, h = 33, w = 45;
  vector<float> data(b*h*w, 1.0);
  torch::Tensor data_tensor = torch::from_blob(data.data(), {b, h, w}.to(at::Device("cuda:0"));
  torch::Tensor h0 = torch::from_blob(vector<float>(1*b*w, 0.0), {b, h, w}).to(at::Device("cuda:0"));
  torch::Tensor c0 = torch::from_blob(vector<float>(1*b*w, 0.0), {b, h, w}).to(at::Device("cuda:0"));
  inputs.push_back(data_tensor);
  inputs.push_back(h0);
  inputs.push(c0);
  torch::Tensor output = module->forward(inputs).toTensor().cpu();
  auto accessor = output.accessor<float, 2>();
  vector<float> answer(b);
  for (int i=0; i<accessor.size(0); ++i){
        answer[i] = accessor[i][0];
  }
  cout << "predict ok" << endl;
}
```

> Note: There is a bug to move init hidden state tensor of lstm to gpu [link](https://github.com/pytorch/pytorch/issues/15272) I use two methods to solve this problem, one is to specify the device in python model using hard code, another is to pass init hidden state as input parameter of forward in cpp script, which may cause a warning [link](https://discuss.pytorch.org/t/rnn-module-weights-are-not-part-of-single-contiguous-chunk-of-memory/6011/14)

the gdb trace info like this:

```shell
(gdb) where
#0  0x00007ffff61ca9fe in ?? () from /usr/local/cuda/lib64/libcudart.so.10.0
#1  0x00007ffff61cf96b in ?? () from /usr/local/cuda/lib64/libcudart.so.10.0
#2  0x00007ffff61e4be2 in cudaDeviceSynchronize () from /usr/local/cuda/lib64/libcudart.so.10.0
#3  0x00007fffb945dcf4 in cudnnDestroy () from repo/pytorch_cpp/libtorch/lib/libcaffe2_gpu.so
#4  0x00007fffb4fca17d in std::unordered_map<int, std::vector<at::native::(anonymous namespace)::Handle, std::allocator<at::native::(anonymous namespace)::Handle> >, std::hash<int>, std::equal_to<int>, std::allocator<std::pair<int const, std::vector<at::native::(anonymous namespace)::Handle, std::allocator<at::native::(anonymous namespace)::Handle> > > > >::~unordered_map() () from repo/pytorch_cpp/libtorch/lib/libcaffe2_gpu.so
#5  0x00007fffb31fe615 in __cxa_finalize (d=0x7fffe8519680) at cxa_finalize.c:83
#6  0x00007fffb4dd3ac3 in __do_global_dtors_aux () from repo/pytorch_cpp/libtorch/lib/libcaffe2_gpu.so
#7  0x00007fffffffe010 in ?? ()
#8  0x00007ffff7de5b73 in _dl_fini () at dl-fini.c:138
Backtrace stopped: frame did not save the PC

```

3. When I remove the LSTM in python model, then the cpp script exits normally.

4. I guess the hidden state of LSTM cause the core dump, maybe relate to the release the init hidden state memory?



<!-- If you have a code sample, error messages, stack traces, please provide it here as well -->

## Expected behavior

So, When I want to load a model with LSTM using c++, how to deal with the hidden state, and how to avoid core dump?

## Environment

PyTorch version: 1.0.1.post2
Is debug build: No
CUDA used to build PyTorch: 10.0.130

OS: Ubuntu 18.04.2 LTS
GCC version: (Ubuntu 7.3.0-27ubuntu1~18.04) 7.3.0
CMake version: version 3.10.2

Python version: 3.6
Is CUDA available: Yes
CUDA runtime version: 10.0.130
GPU models and configuration:
GPU 0: GeForce RTX 2080 Ti
GPU 1: GeForce RTX 2080 Ti

Nvidia driver version: 410.48
cuDNN version: Could not collect

Versions of relevant libraries:
[pip3] numpy==1.16.0
[pip3] numpydoc==0.8.0
[pip3] torch==1.0.1.post2
[pip3] torchvision==0.2.2.post3

## Additional context

<!-- Add any other context about the problem here. -->