Repository: pytorch
Issue: Error in torchScript based on Pytorch tutorial
Labels: oncall: jit

I ’m new to Pytorch and its C++ API. I followed [this tutorial](https://pytorch.org/tutorials/advanced/cpp_export.html) in order to learn how to load and process a .pt file in C++.

Unfortunately, while the code is built correctly, and loads the pt file without errors, when I try to execute something from the saved module, it raises the following error:

    terminate called after throwing an instance of 'std::runtime_error'
      what():  The following operation failed in the TorchScript interpreter.
    Traceback of TorchScript, serialized code (most recent call last):
      File "code/__torch__.py", line 11, in forward
        if bool(torch.gt(torch.sum(input), 0)):
          weight = self.weight
          output = torch.mv(weight, input)
                   ~~~~~~~~ <--- HERE
        else:
      weight0 = self.weight

    Traceback of TorchScript, original code (most recent call last):
    File "torchScript.py", line 10, in forward
    def forward(self, input):
        if input.sum() > 0:
            output = self.weight.mv(input)
                     ~~~~~~~~~~~~~~ <--- HERE
        else:
            output = self.weight + input
    RuntimeError: vector + matrix @ vector expected, got 1, 2, 4

    Aborted (core dumped)

I followed the tutorial without any change. I converted the torch script via annotation and not via tracing.
My torchscript:

    import torch

    class MyModule(torch.nn.Module):
        def __init__(self, N, M):
            super(MyModule, self).__init__()
            self.weight = torch.nn.Parameter(torch.rand(N, M))

        def forward(self, input):
            if input.sum() > 0:
                output = self.weight.mv(input)
            else:
                output = self.weight + input
            return output

    my_module = MyModule(10,20)
    sm = torch.jit.script(my_module)

    sm.save("my_module_model.pt")

My cpp code:

    #include <torch/script.h> // One-stop header.

     #include <iostream>
     #include <memory>

    int main(int argc, const char* argv[]) {
      if (argc != 2) {
        std::cerr << "usage: example-app <path-to-exported-script-module>\n";
        return -1;
      }
      torch::jit::script::Module module;
      try {
        // Deserialize the ScriptModule from a file using torch::jit::load().
        module = torch::jit::load(argv[1]);
      }
      catch (const c10::Error& e) {
        std::cerr << "error loading the model\n";
        return -1;
      }

      // Create a vector of inputs.
      std::vector<torch::jit::IValue> inputs;

My system based on collect_env.py:

    Collecting environment information...
    PyTorch version: 1.9.0+cu102
    Is debug build: False
    CUDA used to build PyTorch: 10.2
    ROCM used to build PyTorch: N/A

    OS: Ubuntu 18.04.5 LTS (x86_64)
    GCC version: (Ubuntu 8.4.0-1ubuntu1~18.04) 8.4.0
    Clang version: Could not collect
    CMake version: version 3.19.6
    Libc version: glibc-2.17

    Python version: 3.6.13 |Anaconda, Inc.| (default, Jun  4 2021, 14:25:59)  [GCC 7.5.0] (64-bit runtime)
    Python platform: Linux-5.4.0-81-generic-x86_64-with-debian-buster-sid
    Is CUDA available: True
    CUDA runtime version: Could not collect
    GPU models and configuration: GPU 0: GeForce MX130
    Nvidia driver version: 440.33.01
    cuDNN version: Probably one of the following:
    /usr/local/cuda-10.2/targets/x86_64-linux/lib/libcudnn.so.8.2.2
    /usr/local/cuda-10.2/targets/x86_64-linux/lib/libcudnn_adv_infer.so.8.2.2
    /usr/local/cuda-10.2/targets/x86_64-linux/lib/libcudnn_adv_train.so.8.2.2
    /usr/local/cuda-10.2/targets/x86_64-linux/lib/libcudnn_cnn_infer.so.8.2.2
    /usr/local/cuda-10.2/targets/x86_64-linux/lib/libcudnn_cnn_train.so.8.2.2
    /usr/local/cuda-10.2/targets/x86_64-linux/lib/libcudnn_ops_infer.so.8.2.2
    /usr/local/cuda-10.2/targets/x86_64-linux/lib/libcudnn_ops_train.so.8.2.2
    HIP runtime version: N/A
    MIOpen runtime version: N/A

    Versions of relevant libraries:
    [pip3] numpy==1.17.0
    [pip3] torch==1.9.0
    [pip3] torchaudio==0.9.0
    [pip3] torchvision==0.10.0
    [conda] mkl                       2021.3.0           h06a4308_520  
    [conda] mkl-include               2021.3.0           h06a4308_520  
    [conda] numpy                     1.17.0           py36h99e49ec_0  
    [conda] numpy-base                1.17.0           py36h2f8d375_0  
    [conda] torch                     1.9.0                    pypi_0    pypi
    [conda] torchaudio                0.9.0                    pypi_0    pypi
    [conda] torchvision               0.10.0                   pypi_0    pypi
`

It's worth to mention that I built the libtorch from source using tools/build_libtorch.py (version 1.9.0) script and after that I installed Pytorch 1.9.0 via pip in an activated anaconda environment in order to be able to run the torchScript.py file and create the .pt file.