Repository: pytorch
Issue: [doc] view appears to mean different things, `view/reshape` vs `transpose/permute`.
Labels: module: docs,triaged,module: numpy,module: partial aliasing,module: viewing and reshaping,topic: docs

### 📚 The doc issue

From the docs it might appear that "view" and "shares the same storage" are interchangeable concepts. Unfortunately, this is not the case and this might cause some confusion.

For example, the doc for `transpose` is clear in saying that the returned tensor shares the same storage with the original input.
However, `permute` reads `Returns a view of the original tensor input with its dimensions permuted`. If we check the doc for `reshape`, it reads `When possible, the returned tensor will be a view of input. Otherwise, it will be a copy`, and references the `view` operation. The doc for `view`  defines when a tensor could be "viewed", and provides a relation between strides and shapes. Note, however, that `permute` is stride-agnostic. So there we have our first contradiction, when "view", `view` and "shares the same storage are not equivalent" when restricted to at least `view` and `permute`.

To illustrate, let's have a look at some examples:
```python
>>> import torch
>>> x = torch.rand(4, 3, 2) # contiguous
>>> y = x.reshape(2, 3, 4) # same storage
>>> z = x.permute((2, 1, 0)) # same storage
>>> x[0, 0, 0] = -100
>>> y[0, 0, 0]
tensor(-100.)
>>> z[0, 0, 0]
tensor(-100.)
>>> x = torch.rand(4, 2, 3).transpose(-1, -2) # NON-contiguous!!!
>>> y = x.reshape(2, 3, 4) # same storage???
>>> z = x.permute((2, 1, 0)) # same storage
>>> x[0, 0, 0] = -100
>>> y[0, 0, 0]
tensor(0.2377)
>>> z[0, 0, 0]
tensor(-100.)
```

We can see that `reshape` handles permutations of dimensions in the contiguous case, but makes a copy in the non-contiguous.
The user, if interested in memory efficiency, now has to differentiate between contiguous/non-contiguous cases and depending on that use either `permute` or `reshape`.

### Suggest a potential alternative/fix

"Shares the same storage" and a "view" are not the same in some cases with non-contiguous inputs. It would be great to point that out in the docs.

Also, the documentation of `view` is not informative. It is really hard to grasp the underlying contiguity-like condition, as it is not clear what `d`, `k` even are and where they come from. This is also true for `strides`. Which `strides`? Of the input or of the output? If the input is contiguous, and it is reshaped into a permutation of the original size, the strides will be different and then the contiguity-like condition will fail yet a "view" on the same storage is going to get created.

I think there is a much easier description of when `view/reshape` can operate over the input's storage, for example:
```
1. view/reshape will use the same storage if the input is contiguous,
    no matter what the shape is (as long as numel is preserved).

2. view/reshape will use the same storage if the input is non-contiguous,
   and its shape (d_1, ..., d_k) is reshaped into
   (d_{11}, ... d_{1 l_1}, ..., d_{k1}, ..., d_{k l_k}) as long as
   d_i = prod(d_{i1}, ..., d_{i l_i}) for each i in {1, ..., k}.

3. view/reshape will use the same storage if the input is non-contiguous,
   and its shape (..., d_{i1}, ... d_{ik}, ...) with strides (..., s_{i1}, ..., s_{ik}...)
   is reshaped into (..., d_i, ...) with strides (..., s_{ik}, ...) as long as
   d_i = prod(d_{i1}, ..., d_{ik}) and
   s_{il} = s_{i l+1} * d{i l+1} for each l in {1, ..., k-1}.
   This essentially means that a tensor of shape (d_{i1}, ..., d_{ik})
   with strides (s_{i1}, ..., s_{ik}) could be squashed into a single
   tensor of length d_i = prod(d_{i1}, ..., d_{ik}) with stride s_{ik},
   which is like reversing paragraph 2 while making sure that the dims
   come from the same dimension "bucket" (no combination of divisors of 
   adjacent dims), i.e. they are the same stride apart.

4. view/reshape will use the same storage if any 1-size dims
    are added to and/or removed from the original tensor's shape.
```

Some examples testing this description.

Non-contiguous case:
```python
>>> x = torch.rand(12, 8, 15).transpose(-1, -2)
>>> x.shape
torch.Size([12, 15, 8])
>>> y = x.view(3, 4, 15, 8) # split 12=3*4
>>> y = x.view(12, 3, 5, 8) # split 15=3*5
>>> y = x.view(12, 15, 2, 4) # split 8=2*4
>>> y = x.view(3, 2, 2, 5, 3, 2, 2, 2) # split into all prime factors
>>> y = x.view(3, 1, 2, 1, 2, 1, 5, 1, 3, 1, 2, 1, 2, 1, 2, 1) # split into all prime factors and insert fake 1-size dims
>>> y = x.view(12, 8, 15) # this should fail as it contradicts paragraph 2 of the no-copy description.
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
RuntimeError: view size is not compatible with input tensor's size and stride (at least one dimension spans across two contiguous subspaces). Use .reshape(...) instead.

```

Contiguous case:
```python
>>> x = torch.rand(12, 15, 8)
>>> y = x.view(8, 15, 12) # any shape will do as long as numel is preserved.
```

NumPy handles reshape similar and could also benefit from the doc fix.
```python
>>> x = np.moveaxis(np.random.rand(12, 15, 8), -1, -2)
>>> x.shape
(12, 8, 15)
>>> y = x.reshape(15, 8, 12) # a copy as we permute dims
>>> x[0, 0, 0] = -100
>>> y[0, 0, 0]
0.1404786289109895

>>> x = np.moveaxis(np.random.rand(12, 15, 8), -1, -2)
>>> y = x.reshape(2, 2, 3, 2, 2, 2, 3, 5) # no copy as dim order is preserved and we just split them
>>> x[0, 0, 0] = -100
>>> y[0, 0, 0, 0, 0, 0, 0, 0]
-100.0

```

cc @brianjo @mruberry @rgommers