Repository: pytorch/vision

Issue:
[transforms] Troubles with padding for PIL image in mode P
### 🐛 Describe the bug

Hello there :wave: 

I was running some training for semantic segmentation using the implementation of `RandomCrop` provided in the [references](https://github.com/pytorch/vision/blob/main/references/segmentation/transforms.py#L9) folder. And apparently there is an issue with the padding function for PIL images. Please note that this was not happening a few months back.

The following snippet
```python
from PIL import Image
from torchvision.transforms.functional import pad

img = Image.open('/path/to/any/img.jpg').convert('P')
padded = pad(img, (0, 0, 5, 10), fill=255)
```
produces
```
---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-14-e151dc3a245c> in <module>
----> 1 padded = pad(img, (0, 0, 5, 10), fill=255)

~/miniconda3/lib/python3.8/site-packages/torchvision/transforms/functional.py in pad(img, padding, fill, padding_mode)
    470     """
    471     if not isinstance(img, torch.Tensor):
--> 472         return F_pil.pad(img, padding=padding, fill=fill, padding_mode=padding_mode)
    473 
    474     return F_t.pad(img, padding=padding, fill=fill, padding_mode=padding_mode)

~/miniconda3/lib/python3.8/site-packages/torchvision/transforms/functional_pil.py in pad(img, padding, fill, padding_mode)
    162         if img.mode == "P":
    163             palette = img.getpalette()
--> 164             image = ImageOps.expand(img, border=padding, **opts)
    165             image.putpalette(palette)
    166             return image

~/miniconda3/lib/python3.8/site-packages/PIL/ImageOps.py in expand(image, border, fill)
    401 
    402         draw = ImageDraw.Draw(out)
--> 403         draw.rectangle((0, 0, width - 1, height - 1), outline=color, width=border)
    404     else:
    405         out = Image.new(image.mode, (width, height), color)

~/miniconda3/lib/python3.8/site-packages/PIL/ImageDraw.py in rectangle(self, xy, fill, outline, width)
    257             self.draw.draw_rectangle(xy, fill, 1)
    258         if ink is not None and ink != fill and width != 0:
--> 259             self.draw.draw_rectangle(xy, ink, 0, width)
    260 
    261     def rounded_rectangle(self, xy, radius=0, fill=None, outline=None, width=1):

TypeError: an integer is required (got type tuple)
```

I investigated a bit on the `main` branch, and I believe the problem comes from this section: https://github.com/pytorch/vision/blob/main/torchvision/transforms/functional_pil.py#L163

With other padding modes or image modes, extra steps are taken to format the padding but here it's not working. Regarding the PIL documentation, [PIL.ImageOps.expand](https://pillow.readthedocs.io/en/stable/reference/ImageOps.html#PIL.ImageOps.expand) does not support tuple as a border argument. However there is the [PIL.ImageOps.pad](https://pillow.readthedocs.io/en/stable/reference/ImageOps.html#PIL.ImageOps.pad) which does. I haven't experimented with that function so I don't know if that's an easy fix :man_shrugging: 

Since this was not happening earlier, I checked the recent changes on PIL side, and their 8.3.0 release changed the `expand` function: https://github.com/python-pillow/Pillow/blob/8.3.0/src/PIL/ImageOps.py#L396-L402
So perhaps the quickfix would be to update the version constraint on PIL (https://github.com/pytorch/vision/blob/main/setup.py#L66) or perform a conditional operation depending on PIL's version!

Happy to help, with some guidance :)


### Versions
```
PyTorch version: 1.10.0
Is debug build: False
CUDA used to build PyTorch: 11.3
ROCM used to build PyTorch: N/A

OS: Ubuntu 20.04.3 LTS (x86_64)
GCC version: (Ubuntu 9.3.0-17ubuntu1~20.04) 9.3.0
Clang version: Could not collect
CMake version: Could not collect
Libc version: glibc-2.31

Python version: 3.8.10 (default, Jun  4 2021, 15:09:15)  [GCC 7.5.0] (64-bit runtime)
Python platform: Linux-5.11.0-40-generic-x86_64-with-glibc2.17
Is CUDA available: True
CUDA runtime version: 11.4.100
GPU models and configuration: GPU 0: NVIDIA GeForce RTX 2070 with Max-Q Design
Nvidia driver version: 470.57.02
cuDNN version: Probably one of the following:
/usr/lib/x86_64-linux-gnu/libcudnn.so.8.2.2
/usr/lib/x86_64-linux-gnu/libcudnn_adv_infer.so.8.2.2
/usr/lib/x86_64-linux-gnu/libcudnn_adv_train.so.8.2.2
/usr/lib/x86_64-linux-gnu/libcudnn_cnn_infer.so.8.2.2
/usr/lib/x86_64-linux-gnu/libcudnn_cnn_train.so.8.2.2
/usr/lib/x86_64-linux-gnu/libcudnn_ops_infer.so.8.2.2
/usr/lib/x86_64-linux-gnu/libcudnn_ops_train.so.8.2.2
HIP runtime version: N/A
MIOpen runtime version: N/A

Versions of relevant libraries:
[pip3] mypy==0.910
[pip3] mypy-extensions==0.4.3
[pip3] numpy==1.19.5
[pip3] torch==1.10.0
[pip3] torchcam==0.3.2.dev0+35e4e67
[pip3] torchvision==0.11.1
[conda] blas                      1.0                         mkl  
[conda] cudatoolkit               11.3.1               ha36c431_9    nvidia
[conda] mkl                       2021.3.0           h06a4308_520  
[conda] mkl-service               2.4.0            py38h7f8727e_0  
[conda] mkl_fft                   1.3.1            py38hd3c417c_0  
[conda] mkl_random                1.2.2            py38h51133e4_0  
[conda] mypy                      0.910                    pypi_0    pypi
[conda] mypy-extensions           0.4.3                    pypi_0    pypi
[conda] numpy                     1.19.5                   pypi_0    pypi
[conda] pytorch                   1.10.0          py3.8_cuda11.3_cudnn8.2.0_0    pytorch
[conda] pytorch-mutex             1.0                        cuda    pytorch
[conda] torchcam                  0.3.2.dev0+35e4e67           dev_0    <develop>
[conda] torchvision               0.10.1                   pypi_0    pypi
```


Discussion:
I can confirm this comes from Pillow:
- any version `>=8.3.0,<8.4.0` has this issue
- installing `>=8.4.0` fixes the problem (cf. https://github.com/python-pillow/Pillow/blob/8.4.0/src/PIL/ImageOps.py#L384-L408)

I suggest to update the version constraints on PIL in torchvision :+1: 
Hi @frgfm 

torchvision has excluded the Pillow versions before because of such issues.

https://github.com/pytorch/vision/blob/main/setup.py#L66

Can you also have a look at #4310 #4146. I guess it is duplicate of same. 

https://github.com/pytorch/vision/issues/4146#issuecomment-875287090
> We can close this now. Unless someone explicitly specifies Pillow==8.3.0, pip install torch torchvision will now install a compatible Pillow version.


Hi @oke-aditya :wave: 

Yes I saw that it excluded 8.3.0, but as mentioned in my previous comment, this issue is present for all `8.3.*` :sweat_smile:  Here is the implementation (with the bug) in 8.3.2: https://github.com/python-pillow/Pillow/blob/8.3.2/src/PIL/ImageOps.py#L384-L407

which is in every point identical to the one in 8.3.0: https://github.com/python-pillow/Pillow/blob/8.3.0/src/PIL/ImageOps.py#L383-L406

It has been fixed in `>=8.4.0`, so I think we should also exclude all the `8.3.*` :) 
Nice catch @frgfm can you send a PR to patch it as you suggested?
Sure thing @oke-aditya :)

Resolving patch:
diff --git a/setup.py b/setup.py
--- a/setup.py
+++ b/setup.py
@@ -62,8 +62,8 @@ def write_version_file():
     pytorch_dep,
 ]
 
-# Excluding 8.3.0 because of https://github.com/pytorch/vision/issues/4146
-pillow_ver = " >= 5.3.0, !=8.3.0"
+# Excluding 8.3.* because of https://github.com/pytorch/vision/issues/4934
+pillow_ver = " >= 5.3.0, !=8.3.*"
 pillow_req = "pillow-simd" if get_dist("pillow-simd") is not None else "pillow"
 requirements.append(pillow_req + pillow_ver)
 


Test patch: