Repository: pytorch/vision

Issue:
Loading 16bit png images
ported here from https://github.com/pytorch/pytorch/issues/32971

Original description:

When I was trying  to load 16 bit .png grayscale image with torchvision.datasets.imagefolder ,it is loading every image as white only. 
I solved this issue by doing transformation operations outside Compose function.


cc @pmeier @wanifarooq @choidongyeon 


Discussion:
What was the solution for this issue ? 

@shubhamsidhwa If you don't set a `loader` for `datasets.ImageFolder`, internally 

https://github.com/pytorch/vision/blob/90a6206c5ca2ab26190cb89c2e42d95e0ece8c90/torchvision/datasets/folder.py#L264-L269

is used. Since `accimage` is an extra installation and activation step, I'm guessing it is falling back to `Pillow` in your case:

https://github.com/pytorch/vision/blob/90a6206c5ca2ab26190cb89c2e42d95e0ece8c90/torchvision/datasets/folder.py#L247-L251

Their documentation states:

> Pillow also provides limited support for a few special modes, including:

> - [...]
> - `I;16` (16-bit unsigned integer pixels)
> - `I;16L` (16-bit little endian unsigned integer pixels)
> - `I;16B` (16-bit big endian unsigned integer pixels)
> - `I;16N` (16-bit native endian unsigned integer pixels)
> - [...]

My guess would be that you are hitting a case where `Pillow` does not have adequate support. Could you share an image so we can confirm hunch?

---

To overcome this you can set a custom `loader` that is able to handle your images. It should take a path as argument and should return the image in whatever format you need it to be.

Resolving patch:
diff --git a/torchvision/io/image.py b/torchvision/io/image.py
--- a/torchvision/io/image.py
+++ b/torchvision/io/image.py
@@ -61,7 +61,12 @@ def decode_png(input: torch.Tensor, mode: ImageReadMode = ImageReadMode.UNCHANGE
     """
     Decodes a PNG image into a 3 dimensional RGB or grayscale Tensor.
     Optionally converts the image to the desired format.
-    The values of the output tensor are uint8 between 0 and 255.
+    The values of the output tensor are uint8 in [0, 255], except for
+    16-bits pngs which are int32 tensors in [0, 65535].
+
+    .. warning::
+        Should pytorch ever support the uint16 dtype natively, the dtype of the
+        output for 16-bits pngs will be updated from int32 to uint16.
 
     Args:
         input (Tensor[1]): a one dimensional uint8 tensor containing
@@ -188,7 +193,8 @@ def decode_image(input: torch.Tensor, mode: ImageReadMode = ImageReadMode.UNCHAN
     operation to decode the image into a 3 dimensional RGB or grayscale Tensor.
 
     Optionally converts the image to the desired format.
-    The values of the output tensor are uint8 between 0 and 255.
+    The values of the output tensor are uint8 in [0, 255], except for
+    16-bits pngs which are int32 tensors in [0, 65535].
 
     Args:
         input (Tensor): a one dimensional uint8 tensor containing the raw bytes of the
@@ -209,7 +215,8 @@ def read_image(path: str, mode: ImageReadMode = ImageReadMode.UNCHANGED) -> torc
     """
     Reads a JPEG or PNG image into a 3 dimensional RGB or grayscale Tensor.
     Optionally converts the image to the desired format.
-    The values of the output tensor are uint8 between 0 and 255.
+    The values of the output tensor are uint8 in [0, 255], except for
+    16-bits pngs which are int32 tensors in [0, 65535].
 
     Args:
         path (str): path of the JPEG or PNG image.


Test patch:
diff --git a/test/assets/fakedata/logos/rgb_pytorch16.png b/test/assets/fakedata/logos/rgb_pytorch16.png
new file mode 100644
Binary files /dev/null and b/test/assets/fakedata/logos/rgb_pytorch16.png differ
diff --git a/test/assets/fakedata/logos/rgbalpha_pytorch16.png b/test/assets/fakedata/logos/rgbalpha_pytorch16.png
new file mode 100644
Binary files /dev/null and b/test/assets/fakedata/logos/rgbalpha_pytorch16.png differ
diff --git a/test/test_image.py b/test/test_image.py
--- a/test/test_image.py
+++ b/test/test_image.py
@@ -168,6 +168,11 @@ def test_decode_png(img_path, pil_mode, mode):
         # TODO: remove once fix is released in PIL. Should be > 8.3.1.
         img_lpng, img_pil = img_lpng[0], img_pil[0]
 
+    if "16" in img_path:
+        # PIL converts 16 bits pngs in uint8
+        assert img_lpng.dtype == torch.int32
+        img_lpng = torch.round(img_lpng / (2 ** 16 - 1) * 255).to(torch.uint8)
+
     torch.testing.assert_close(img_lpng, img_pil, atol=tol, rtol=0)