Repository: pytorch
Issue: The conversion of Inplace aten::append to outplace is incomplete, creating incorrect subgraphs
Labels: oncall: jit,triaged

# Description
aten::append is inplace. This creates issues for downstream torchscript users that do not have the inplace semantics, such as ONNX. 
For example, when used inside a loop, the output of aten::append is not captured in the loop output. Another example is that wherever the appended list is used as input, it is always using the original (empty) list, instead of the output of latest aten::append.
We have tried using the latest `remove_mutation` pass, and the issue remains as the graph is unchanged.

Below is an example with using append in a loop.
Note that output(%10) of aten::append is not captured in the loop output.
And the aten::cat node outside of loop is using the original list value(%res.1) as input.

As a side note, it is not clear what should be captured as loop output as it is not mentioned https://github.com/pytorch/pytorch/blob/master/torch/csrc/jit/OVERVIEW.md#loops.
From looking at the `If` definition above, I'm assuming that loop would have similar definition for its output as `the outputs are the new values of variables in the outer block whose values were altered in an if-statement`. If the assumption is correct, then aten::append output should also be part of loop output.

# Code to repro
```python
import torch

class AppendModel(torch.nn.Module):
    def __init__(self):
        super().__init__()
        self.linear = torch.nn.Linear(2, 2)
    def forward(self, x):
        res = []
        for i in range(x.size(0)):
            res.append(self.linear(x[i]))
        return torch.cat(res, dim=0)

m = torch.jit.script(AppendModel())
m.eval()

m._c.dump()

# freezing

m_freezed = torch._C._freeze_module(m._c)

m_freezed.dump()

# remove mutation
graph = m_freezed._get_method('forward').graph
torch._C._jit_pass_remove_mutation(graph)
print(graph)

"""
Output graph. Note that output(%10) of aten::append is not captured in the loop output.
And the aten::cat node outside of loop is using the original list value(%res.1) as input.

graph(%self : __torch__.___torch_mangle_0.AppendModel,
      %x.1 : Tensor):
  %27 : Float(2:1, 2:2) = prim::Constant[value=-0.1994  0.3128 -0.4812  0.3948 [ CPUFloatType{2,2} ]]()
  %13 : int = prim::Constant[value=2]() # /home/bowbao/repos/pytorch/torch/nn/functional.py:1672:22
  %12 : int = prim::Constant[value=1]()
  %3 : int = prim::Constant[value=0]() # repro_append.py:10:30
  %2 : bool = prim::Constant[value=1]() # repro_append.py:10:8
  %self.linear.bias : Float(2:1) = prim::Constant[value=-0.4483  0.6543 [ CPUFloatType{2} ]]()
  %res.1 : Tensor[] = prim::ListConstruct()
  %5 : int = aten::size(%x.1, %3) # repro_append.py:10:23
   = prim::Loop(%5, %2) # repro_append.py:10:8
    block0(%i.1 : int):
      %8 : Tensor = aten::select(%x.1, %3, %i.1) # repro_append.py:11:35
      %16 : int = aten::dim(%8) # /home/bowbao/repos/pytorch/torch/nn/functional.py:1672:7
      %17 : bool = aten::eq(%16, %13) # /home/bowbao/repos/pytorch/torch/nn/functional.py:1672:7
      %ret : Tensor = prim::If(%17) # /home/bowbao/repos/pytorch/torch/nn/functional.py:1672:4
        block0():
          %ret.1 : Tensor = aten::addmm(%self.linear.bias, %8, %27, %12, %12) # /home/bowbao/repos/pytorch/torch/nn/functional.py:1674:14
          -> (%ret.1)
        block1():
          %output.1 : Tensor = aten::matmul(%8, %27) # /home/bowbao/repos/pytorch/torch/nn/functional.py:1676:17
          %output.3 : Tensor = aten::add_(%output.1, %self.linear.bias, %12) # /home/bowbao/repos/pytorch/torch/nn/functional.py:1678:12
          -> (%output.3)
      %10 : Tensor[] = aten::append(%res.1, %ret) # repro_append.py:11:12
      -> (%2)
  %11 : Tensor = aten::cat(%res.1, %3) # repro_append.py:12:15
  return (%11)
"""
```

cc @suo