Repository: pytorch/vision

Issue:
torchvision/utils.py modify grad_fn of the tensor, throw exception "Output X of UnbindBackward is a view and is being modified inplace"
## 🐛 Bug

<!-- A clear and concise description of what the bug is. -->

In pytorch1.7,  Lib/site-packages/torchvision/utils.py **line 74** ( for t in tensor ) ， this code will modify the grad_fn  of the tensor and become UnbindBackward， and then, it will be throw exception by "RuntimeError: Output 0 of UnbindBackward is a view and is being modified inplace. This view is the output of a function that returns multiple views. Such functions do not allow the output views to be modified inplace. You should replace the inplace operation by an out-of-place one."

## To Reproduce


```
import torch.nn as nn
import torchvision.utils as vutils
import torchvision.models as models

alexnet = models.alexnet()

for sub_module in alexnet.modules():
    if isinstance(sub_module, nn.Conv2d):
        kernels = sub_module.weight
        c_out, c_int, k_w, k_h = tuple(kernels.shape)
        for o_idx in range(c_out):
            kernel_idx = kernels[o_idx, :, :, :].unsqueeze(1)  # kernel_idx.grad_fn == UnsqueezeBackward0
            img_grid = vutils.make_grid(kernel_idx, normalize=True, scale_each=True, nrow=3)

```

Steps to reproduce the behavior:

1. 
1.
1.

<!-- If you have a code sample, error messages, stack traces, please provide it here as well -->

Traceback (most recent call last):
  File "/path/to/fiel/***.py", line 20, in <module>
    img_grid = vutils.make_grid(kernel_idx, normalize=True, scale_each=True, nrow=3)
  File "D:\Anaconda_data\envs\pytorch_1.7_cpu\lib\site-packages\torchvision\utils.py", line 77, in make_grid
    norm_range(t, range)
  File "D:\Anaconda_data\envs\pytorch_1.7_cpu\lib\site-packages\torchvision\utils.py", line 71, in norm_range
    norm_ip(t, float(t.min()), float(t.max()))
  File "D:\Anaconda_data\envs\pytorch_1.7_cpu\lib\site-packages\torchvision\utils.py", line 64, in norm_ip
    img.clamp_(min=min, max=max)
RuntimeError: Output 0 of UnbindBackward is a view and is being modified inplace. This view is the output of a function that returns multiple views. Such functions do not allow the output views to be modified inplace. You should replace the inplace operation by an out-of-place one.

Process finished with exit code 1

## Expected behavior

<!-- A clear and concise description of what you expected to happen. -->

## Environment

Collecting environment information...
PyTorch version: 1.7.0+cpu
Is debug build: True
CUDA used to build PyTorch: Could not collect
ROCM used to build PyTorch: N/A

OS: Microsoft Windows 10 家庭中文版
GCC version: (MinGW.org GCC-6.3.0-1) 6.3.0
Clang version: Could not collect
CMake version: Could not collect

Python version: 3.6 (64-bit runtime)
Is CUDA available: False
CUDA runtime version: 10.1.105
GPU models and configuration: Could not collect
Nvidia driver version: Could not collect
cuDNN version: C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v10.1\bin\cudnn64_7.dll
HIP runtime version: N/A
MIOpen runtime version: N/A

Versions of relevant libraries:
[pip] numpy==1.18.1
[pip] numpydoc==0.9.2
[conda] blas                      1.0                         mkl    defaults
[conda] mkl                       2020.0                      166    defaults
[conda] mkl-service               2.3.0            py37hb782905_0    defaults
[conda] mkl_fft                   1.0.15           py37h14836fe_0    defaults
[conda] mkl_random                1.1.0            py37h675688f_0    defaults
[conda] numpy                     1.18.1           py37h93ca92e_0    defaults
[conda] numpy-base                1.18.1           py37hc3f5095_1    defaults
[conda] numpydoc                  0.9.2                      py_0    defaults





Discussion:
That is linked to the change to `for el in tensor` behavior wrt to inplace ops.
In particular, this line https://github.com/pytorch/vision/blob/8c281757a0daf3a8e92cbb4bded0e5e6b389a375/torchvision/utils.py#L74 should be changed to 
```
for i in range(tensor.size(0)):  # loop over mini-batch dimension
    t = tensor[i]
```
We can fix this in torchvision following @albanD comment, but note that you are backpropagating through visualization code, which is probably not what you want to do, and you should instead do a `with torch.no_grad()` in your visualization code otherwise you could have unexpected behaviors

We should probably add a `@torch.no_grad()` to the visualization functions to avoid this happening in user code in the future
Wrapping the whole function in `torch.no_grad()` will fix the issue as well.
I assumed it could be differentiated throw since I didn't saw one.
  @albanD Thanks your idea. It can be solved this problem theoretically. 
```
for i in range(tensor.size(0)):  # loop over mini-batch dimension
    t = tensor[i]
```

In addition, this line

> https://github.com/pytorch/vision/blob/8c281757a0daf3a8e92cbb4bded0e5e6b389a375/torchvision/utils.py#L14

change "range" to a local variable, so that it doesn't work.  It can be changed to 
```
for idx, _ in enumerate([jj for jj in tensor]):
    t = tensor[idx]
```

Moreover, it's better not to use "range" as a local variable. @fmassa 

@TingsongYu I agree, we should not be using `range` as an argument here, this is a bad name for the function.

I would be happy merging a PR that fixes both issues (with proper deprecation warning that `range` is deprecated, via `**kwargs` unpacking).

Resolving patch:
diff --git a/torchvision/utils.py b/torchvision/utils.py
--- a/torchvision/utils.py
+++ b/torchvision/utils.py
@@ -2,23 +2,24 @@
 import pathlib
 import torch
 import math
+import warnings
 import numpy as np
 from PIL import Image, ImageDraw
 from PIL import ImageFont
 
 __all__ = ["make_grid", "save_image", "draw_bounding_boxes"]
 
-irange = range
-
 
+@torch.no_grad()
 def make_grid(
     tensor: Union[torch.Tensor, List[torch.Tensor]],
     nrow: int = 8,
     padding: int = 2,
     normalize: bool = False,
-    range: Optional[Tuple[int, int]] = None,
+    value_range: Optional[Tuple[int, int]] = None,
     scale_each: bool = False,
     pad_value: int = 0,
+    **kwargs
 ) -> torch.Tensor:
     """Make a grid of images.
 
@@ -30,7 +31,7 @@ def make_grid(
         padding (int, optional): amount of padding. Default: ``2``.
         normalize (bool, optional): If True, shift the image to the range (0, 1),
             by the min and max values specified by :attr:`range`. Default: ``False``.
-        range (tuple, optional): tuple (min, max) where min and max are numbers,
+        value_range (tuple, optional): tuple (min, max) where min and max are numbers,
             then these numbers are used to normalize the image. By default, min and max
             are computed from the tensor.
         scale_each (bool, optional): If ``True``, scale each image in the batch of
@@ -43,7 +44,12 @@ def make_grid(
     """
     if not (torch.is_tensor(tensor) or
             (isinstance(tensor, list) and all(torch.is_tensor(t) for t in tensor))):
-        raise TypeError('tensor or list of tensors expected, got {}'.format(type(tensor)))
+        raise TypeError(f'tensor or list of tensors expected, got {type(tensor)}')
+
+    if "range" in kwargs.keys():
+        warning = "range will be deprecated, please use value_range instead."
+        warnings.warn(warning)
+        value_range = kwargs["range"]
 
     # if list of tensors, convert to a 4D mini-batch Tensor
     if isinstance(tensor, list):
@@ -61,25 +67,25 @@ def make_grid(
 
     if normalize is True:
         tensor = tensor.clone()  # avoid modifying tensor in-place
-        if range is not None:
-            assert isinstance(range, tuple), \
-                "range has to be a tuple (min, max) if specified. min and max are numbers"
+        if value_range is not None:
+            assert isinstance(value_range, tuple), \
+                "value_range has to be a tuple (min, max) if specified. min and max are numbers"
 
         def norm_ip(img, low, high):
             img.clamp_(min=low, max=high)
             img.sub_(low).div_(max(high - low, 1e-5))
 
-        def norm_range(t, range):
-            if range is not None:
-                norm_ip(t, range[0], range[1])
+        def norm_range(t, value_range):
+            if value_range is not None:
+                norm_ip(t, value_range[0], value_range[1])
             else:
                 norm_ip(t, float(t.min()), float(t.max()))
 
         if scale_each is True:
             for t in tensor:  # loop over mini-batch dimension
-                norm_range(t, range)
+                norm_range(t, value_range)
         else:
-            norm_range(tensor, range)
+            norm_range(tensor, value_range)
 
     if tensor.size(0) == 1:
         return tensor.squeeze(0)
@@ -92,8 +98,8 @@ def norm_range(t, range):
     num_channels = tensor.size(1)
     grid = tensor.new_full((num_channels, height * ymaps + padding, width * xmaps + padding), pad_value)
     k = 0
-    for y in irange(ymaps):
-        for x in irange(xmaps):
+    for y in range(ymaps):
+        for x in range(xmaps):
             if k >= nmaps:
                 break
             # Tensor.copy_() is a valid method but seems to be missing from the stubs
@@ -105,16 +111,13 @@ def norm_range(t, range):
     return grid
 
 
+@torch.no_grad()
 def save_image(
     tensor: Union[torch.Tensor, List[torch.Tensor]],
     fp: Union[Text, pathlib.Path, BinaryIO],
-    nrow: int = 8,
-    padding: int = 2,
     normalize: bool = False,
-    range: Optional[Tuple[int, int]] = None,
-    scale_each: bool = False,
-    pad_value: int = 0,
     format: Optional[str] = None,
+    **kwargs
 ) -> None:
     """Save a given Tensor into an image file.
 
@@ -126,8 +129,8 @@ def save_image(
             If a file object was used instead of a filename, this parameter should always be used.
         **kwargs: Other arguments are documented in ``make_grid``.
     """
-    grid = make_grid(tensor, nrow=nrow, padding=padding, pad_value=pad_value,
-                     normalize=normalize, range=range, scale_each=scale_each)
+
+    grid = make_grid(tensor, **kwargs)
     # Add 0.5 after unnormalizing to [0, 255] to round to nearest integer
     ndarr = grid.mul(255).add_(0.5).clamp_(0, 255).permute(1, 2, 0).to('cpu', torch.uint8).numpy()
     im = Image.fromarray(ndarr)


Test patch: