Repository: pytorch
Issue: In-place operations on `.data` or `.detach()` of sparse tensor doesn't update the original tensor
Labels: module: sparse,triaged

Previously, if we change the tensor metadata (e.g. sizes / strides / storage / storage_offset) of a derived tensor (i.e. tensors created from Python `tensor.data` or Python/C++ `tensor.detach()`), those metadata in the original tensor will also be updated. However, after https://github.com/pytorch/pytorch/pull/13827 is merged, the new behavior is that those metadata changes to a derived tensor will not update the original tensor anymore, and we created the `allow_tensor_metadata_change_` flag to make such changes explicitly illegal, to prevent users from changing metadata of the derived tensor and expecting the original tensor to also be updated.

In particular, this affects in-place operations (e.g. `zero_()` / `copy_()` / `add_()`) on `.data` or `.detach()` of sparse tensors, because for sparse tensors, `indices` and `values` are also considered as a tensor's metadata, and changing those two fields on a derived tensor will not update the original tensor.

Examples:

`zero_()`
```python
>>> import torch
>>> i = torch.tensor([[0, 1, 1], [2, 0, 2]])
>>> v = torch.tensor([3., 4., 5.])
>>> st = torch.sparse.FloatTensor(i, v)
>>> dst = st.data

# Without the allow_tensor_metadata_change_ flag:
>>> dst.zero_()   # Note that `dst` and `st` have different values now
>>> dst
tensor(indices=tensor([], size=(2, 0)), values=tensor([], size=(0,)), size=(2, 3), nnz=0, layout=torch.sparse_coo)
>>> st
tensor(indices=tensor([[0, 1, 1], [2, 0, 2]]), values=tensor([3., 4., 5.]), size=(2, 3), nnz=3, layout=torch.sparse_coo)

# With the allow_tensor_metadata_change_ flag:
>>> dst.zero_()   # This operation is disallowed, because it breaks consistency between `dst` and `st`
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
RuntimeError: resize_and_clear_ is not allowed on Tensor created from .data or .detach()
```

`copy_()`
```python
>>> import torch
>>> i = torch.tensor([[0, 1, 1], [2, 0, 2]])
>>> v = torch.tensor([3., 4., 5.])
>>> st = torch.sparse.FloatTensor(i, v)
>>> dst = st.data

# Without the allow_tensor_metadata_change_ flag:
>>> i2 = torch.tensor([[0, 1, 1], [2, 0, 2]])
>>> v2 = torch.tensor([10., 10., 10.])
>>> st2 = torch.sparse.FloatTensor(i2, v2)
>>> dst.copy_(st2)  # Note that `dst` and `st` have different values now
>>> dst
tensor(indices=tensor([[0, 1, 1], [2, 0, 2]]), values=tensor([10., 10., 10.]), size=(2, 3), nnz=3, layout=torch.sparse_coo)
>>> st
tensor(indices=tensor([[0, 1, 1], [2, 0, 2]]), values=tensor([3., 4., 5.]), size=(2, 3), nnz=3, layout=torch.sparse_coo)

# With the allow_tensor_metadata_change_ flag:
>>> dst.copy_(st2)   # This operation is disallowed, because it breaks consistency between `dst` and `st`
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
RuntimeError: resize_ is not allowed on Tensor created from .data or .detach()
```

`add_()`
```python
>>> import torch
>>> i = torch.tensor([[0, 1, 1], [2, 0, 2]])
>>> v = torch.tensor([3., 4., 5.])
>>> st = torch.sparse.FloatTensor(i, v)
>>> dst = st.data
>>> st2 = torch.sparse.FloatTensor(torch.tensor([[0, 1, 1], [2, 0, 2]]), torch.tensor([10., 10., 10.]))

# Without the allow_tensor_metadata_change_ flag:
>>> dst.add_(st2)   # Note that `dst` and `st` have different values now
>>> dst
tensor(indices=tensor([[0, 1, 1], [2, 0, 2]]), values=tensor([13., 14., 15.]), size=(2, 3), nnz=3, layout=torch.sparse_coo)
>>> st
tensor(indices=tensor([[0, 1, 1], [2, 0, 2]]), values=tensor([3., 4., 5.]), size=(2, 3), nnz=3, layout=torch.sparse_coo)

# With the allow_tensor_metadata_change_ flag:
>>> dst.add_(st2)   # This operation is disallowed, because it breaks consistency between `dst` and `st`
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
RuntimeError: set_indices_and_values_unsafe is not allowed on Tensor created from .data or .detach()
```

The current solution to this problem is to always perform in-place operations on the original sparse tensor and not the derived tensor (i.e. tensor obtained from `.data` or `.detach()`).

However, if in the future we do want to have `zero_()` / `copy_()` / `add_()` on `.data` or `.detach()` of sparse tensor work, we can consider adding one more level of indirection to how we store the `indices` and `values` of a sparse tensor (e.g. by having a `SparseStorageImpl` that stores those two fields), and we make the derived tensor from `.data` and `.detach()` share the same storage pointer with the original tensor, so that any changes to `indices` and `values` will always be reflected back to the original tensor.