Repository: pytorch
Issue: pytorch-crf model to onnx conversion
Labels: module: onnx,triaged

## 🐛 Bug

<!-- A clear and concise description of what the bug is. -->

## To Reproduce

i'd like to report an onnx conversion issue when  the`pytorch-crf` library is used.
( pytorch-crf : https://github.com/kmkurn/pytorch-crf/blob/master/torchcrf/__init__.py )

**1. model i used**
- https://github.com/dsindex/ntagger/blob/master/model.py#L225
  - GloVe + LSTM + CRF

**2. onnx conversion code**
- https://github.com/dsindex/ntagger/blob/master/evaluate.py#L279
- using this code, the conversion has done with no error. but some warning raise.

**3. warning**
```
/data/private/ntagger/model.py:353: TracerWarning: torch.as_tensor results are registered as constants in the trace. You can safely ignore this warning if you use this function to create tensors out of constant variables that would be the same every time you call this function. In any other case, this might cause the trace to be incorrect.
  prediction = torch.as_tensor(prediction, dtype=torch.long)

/data/private/anaconda3/envs/conda_torch/lib/python3.7/site-packages/torch/onnx/symbolic_opset9.py:1577: UserWarning: Exporting a model to ONNX with a batch_size other than 1, with a variable length with LSTM can cause an error when running the ONNX model with a different batch size. Make sure to save the model with a batch size of 1, or define the initial states (h0/c0) as inputs of the model.
  "or define the initial states (h0/c0) as inputs of the model. ")
```

**4. weird same prediction for every test examples using onnxruntime**
```
1, 2, 7, 7, 7, 7, 7, ...
1, 2, 7, 7, 7, 7, 7, ...
1, 2, 7, 7, 7, 7, 7, ...
...
```

**5. code position which the warning points out**
  - https://github.com/dsindex/ntagger/blob/master/model.py#L353
```
  # 4. Output
        logits = self.linear(mha_out)
        # logits : [batch_size, seq_size, label_size]
        if not self.use_crf: return logits
        prediction = self.crf.decode(logits)
        prediction = torch.as_tensor(prediction, dtype=torch.long) <-- here!
        # prediction : [batch_size, seq_size]
        return logits, prediction
```

**6. onnx graph position i suspect**
```
...
%345 : Tensor = onnx::Reshape(%343, %344) # /usr/local/lib/python3.6/dist-packages/torch/nn/modules/rnn.py:585:0
  %346 : Float(1:400, 180:400, 400:1, requires_grad=0, device=cuda:0) = onnx::Transpose[perm=[1, 0, 2]](%345) # /usr/local/lib/python3.6/dist-packages/torch/nn/utils/rnn.py:310:0
  %347 : Float(1:72000, 180:400, 400:1, requires_grad=0, device=cuda:0) = onnx::Gather[axis=0](%346, %49) # /usr/local/lib/python3.6/dist-packages/torch/nn/functional.py:983:0
  %349 : Float(1:1800, 180:10, 10:1, requires_grad=0, device=cuda:0) = onnx::MatMul(%347, %446) # /usr/local/lib/python3.6/dist-packages/torch/nn/functional.py:1692:0
  %logits : Float(1:1800, 180:10, 10:1, requires_grad=0, device=cuda:0) = onnx::Add(%349, %linear.bias)
  %prediction : Long(1:180, 180:1, requires_grad=0, device=cpu) = onnx::Constant[value=<Tensor>]() <-- here!
  return (%logits, %prediction)
```

since `self.crf.decode()` returns `List[List[int]]`, we should use `torch.as_tensor()` in the last place of forward() method.
but, `torch.onnx.export()` indicates above warning and produces same (useless) prediction results.

what should i do for it to fix ? 
any idea will be appreciated.


## Expected behavior

<!-- A clear and concise description of what you expected to happen. -->

## Environment

 - PyTorch Version (e.g., 1.0): train = 1.7.1+cu101, inference/onnx conversion = 1.7.0
 - OS (e.g., Linux): Ubuntu 18.04
 - How you installed PyTorch (`conda`, `pip`, source): pip
 - Build command you used (if compiling from source):
 - Python version: 3.6.9
 - CUDA/cuDNN version: 10.1
 - GPU models and configuration: Tesla V100
 - Any other relevant information: onnx = 1.7.0, onnxruntime = 1.4.0

## Additional context

<!-- Add any other context about the problem here. -->

* i modified this issue up-to-date.



cc @houseroad @spandantiwari @lara-hdr @BowenBao @neginraoof