Repository: pytorch
Issue: BUG torch.jit.annotate on List + torch.stack give wrong DTYPE
Labels: oncall: jit

### 🐛 Describe the bug

Hi, we are switching to pytorch2.0; we have some internal LSTM implem with JIT and are using `torch.jit.annotate`; here is a small example to reproduce the error; 

`torch.jit.annote` are giving a value of 6 in the dtype after torch.stack the list.


Reproduce bug:

```
from typing import List
import torch
from torch import Tensor


class Example(torch.nn.Module):
    def __init__(self,i_size) -> None:
        super().__init__()
        self.layer_norm = torch.nn.LayerNorm(i_size)

    
    @torch.jit.script
    def do_stuff_with_jit_annotate(input):
        # pad output x times
        outputs = torch.jit.annotate(List[Tensor], [])
        for i in range(5):
            outputs = outputs.append(input)
        print(torch.stack(outputs).dtype)
        # It print always value 6.
        # This make the onnx fail when this value is used by other modules.
        # I solve it in our JITLSTM by 
        # torch.stack(outputs).float()
        return torch.stack(outputs)
    
    def forward(self,input):
        output = self.do_stuff_with_jit_annotate(input)
        return self.layer_norm(output)

model = Example(10)

torch.onnx.export(model,
                  torch.randn(1,10),
                  "model.onnx",
                  input_names=["input"],
                  output_names=["output"],
                  do_constant_folding=True,
                  verbose=True,
                  opset_version=11,
                  )
```
Error:
```
Traceback (most recent call last):
  File "example.py", line 30, in <module>
    torch.onnx.export(model,
  File "/opt/conda/envs/ptca/lib/python3.8/site-packages/torch/onnx/utils.py", line 506, in export
    _export(
  File "/opt/conda/envs/ptca/lib/python3.8/site-packages/torch/onnx/utils.py", line 1548, in _export
    graph, params_dict, torch_out = _model_to_graph(
  File "/opt/conda/envs/ptca/lib/python3.8/site-packages/torch/onnx/utils.py", line 1117, in _model_to_graph
    graph = _optimize_graph(
  File "/opt/conda/envs/ptca/lib/python3.8/site-packages/torch/onnx/utils.py", line 665, in _optimize_graph
    graph = _C._jit_pass_onnx(graph, operator_export_type)
  File "/opt/conda/envs/ptca/lib/python3.8/site-packages/torch/onnx/utils.py", line 1891, in _run_symbolic_function
    return symbolic_fn(graph_context, *inputs, **attrs)
  File "/opt/conda/envs/ptca/lib/python3.8/site-packages/torch/onnx/symbolic_helper.py", line 392, in wrapper
    return fn(g, *args, **kwargs)
  File "/opt/conda/envs/ptca/lib/python3.8/site-packages/torch/onnx/symbolic_helper.py", line 306, in wrapper
    return fn(g, *args, **kwargs)
  File "/opt/conda/envs/ptca/lib/python3.8/site-packages/torch/onnx/symbolic_opset9.py", line 2837, in layer_norm
    normalized, _, _ = native_layer_norm(g, input, normalized_shape, weight, bias, eps)
  File "/opt/conda/envs/ptca/lib/python3.8/site-packages/torch/onnx/symbolic_helper.py", line 392, in wrapper
    return fn(g, *args, **kwargs)
  File "/opt/conda/envs/ptca/lib/python3.8/site-packages/torch/onnx/symbolic_helper.py", line 306, in wrapper
    return fn(g, *args, **kwargs)
  File "/opt/conda/envs/ptca/lib/python3.8/site-packages/torch/onnx/symbolic_opset9.py", line 2774, in native_layer_norm
    _type_utils.JitScalarType.from_value(numerator)
  File "/opt/conda/envs/ptca/lib/python3.8/site-packages/torch/onnx/_type_utils.py", line 197, in from_value
    raise errors.SymbolicValueError(
torch.onnx.errors.SymbolicValueError: Cannot determine scalar type for this '<class 'torch.TensorType'>' instance and a default value was not provided.  [Caused by the value '21 defined in (%21 : Tensor = onnx::Sub(%input, %20), scope: __main__.Example::/torch.nn.modules.normalization.LayerNorm::layer_norm
)' (type 'Tensor') in the TorchScript graph. The containing node has kind 'onnx::Sub'.]

    Inputs:
        #0: input defined in (%input : Tensor = onnx::ConcatFromSequence[axis=0, new_axis=1](%outputs) # example.py:22:15
    )  (type 'Tensor')
        #1: 20 defined in (%20 : Tensor = onnx::ReduceMean[axes=[-1]](%input), scope: __main__.Example::/torch.nn.modules.normalization.LayerNorm::layer_norm
    )  (type 'Tensor')
    Outputs:
        #0: 21 defined in (%21 : Tensor = onnx::Sub(%input, %20), scope: __main__.Example::/torch.nn.modules.normalization.LayerNorm::layer_norm
    )  (type 'Tensor')
```

### Versions

```
[pip3] flake8==4.0.1
[pip3] mypy-extensions==1.0.0
[pip3] numpy==1.22.4
[pip3] pytorch-lightning==1.9.3
[pip3] torch==2.0.0
[pip3] torch-nebula==0.16.2
[pip3] torch-ort==1.14.0
[pip3] torch-tb-profiler==0.4.0
[pip3] torchmetrics==0.11.3
[pip3] torchsnapshot==0.1.0
[pip3] torchvision==0.15.1+cu117
[pip3] triton==2.0.0
[conda] magma-cuda117             2.6.1                         1    pytorch
[conda] mkl                       2023.0.0                 pypi_0    pypi
[conda] mkl-fft                   1.3.1                    pypi_0    pypi
[conda] mkl-include               2021.4.0                 pypi_0    pypi
[conda] numpy                     1.22.4                   pypi_0    pypi
[conda] pytorch-lightning         1.9.3                    pypi_0    pypi
[conda] torch                     2.0.0                    pypi_0    pypi
[conda] torch-nebula              0.16.2                   pypi_0    pypi
[conda] torch-ort                 1.14.0                   pypi_0    pypi
[conda] torch-tb-profiler         0.4.0                    pypi_0    pypi
[conda] torchmetrics              0.11.3                   pypi_0    pypi
[conda] torchsnapshot             0.1.0                    pypi_0    pypi
[conda] torchvision               0.15.1+cu117             pypi_0    pypi
[conda] triton                    2.0.0                    pypi_0    pypi
```

cc @EikanWang @jgong5 @wenzhe-nrv @sanchitintel