Torchvision.transforms.v2 does nothing / fails silently?

turbotimon: <p>I want to transform a PIL image or np.ndarray, but it in both cases, the transform does nothing to the image.</p>
<p><strong>Minimal reproducable example:</strong> As you can see, the mean does not change</p>
<pre data-code-wrap="python"><code class="lang-python">import torch
import numpy as np
import torchvision.transforms.v2 as v2
import matplotlib.pyplot as plt
from PIL import Image

## np.array (does nothing / fails silently)
img_np = np.ones((100,100,3))
img_np_transformed = v2.Pad(50)(img_np)
print(np.mean(img_np_transformed)) # still 1.0
plt.imshow(img_np_transformed)

## PIL image (does nothing / fails silently)
img_pil = Image.fromarray((img_np * 255).astype('uint8'))
img_pil_transformed = v2.Pad(50)(img_np) # still 1.0
print(np.mean(img_pil_transformed))

## Tensor (works)
tensor_img = torch.tensor(np.ones((1,3,100,100)))
tensor_img_transformed = v2.Pad(50)(tensor_img)
print(float(tensor_img_transformed.mean())) # correct 0.25
plt.figure()
plt.imshow(np.transpose(tensor_img_transformed.squeeze(), (2, 1, 0)))
</code></pre>
<p>What do I do wrong or is this a bug? According to the <a href="https://pytorch.org/vision/stable/transforms.html#supported-input-types-and-conventions" rel="noopener nofollow ugc">docs</a>, PIL Images should work. And at least, I would expect an Exception if the input is invalid.</p>

Gillian: <aside class="quote no-group" data-username="turbotimon" data-post="1" data-topic="208500">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/turbotimon/48/69503_2.png" class="avatar"> turbotimon:</div>
<blockquote>
<p><code>img_pil_transformed = v2.Pad(50)(img_np)</code></p>
</blockquote>
</aside>
<p>for PIL you have a typo:<br>
img_pil_transformed = v2.Pad(50)(<strong>img_np</strong>)</p>

turbotimon: <p>Yes, thats true, thanks. What still bothers me, is that it fails silently with numpy.ndarray. Do you think that is word a issue github? Or is this intended?</p>

Gillian: <p>Yes, you’re right. I also think the silent failure is a problem.</p>

ptrblck: <p>Yes, I the transformation should not fail silently. CC <a class="mention" href="/u/nicolashug">@NicolasHug</a> would you know if we are missing to raise an error or a warning here?</p>

turbotimon: <p><a class="mention" href="/u/ptrblck">@ptrblck</a> thanks for picking this up!</p>
<p>I made an issue in the mean time <a href="https://github.com/pytorch/vision/issues/8662" class="inline-onebox" rel="noopener nofollow ugc">Torchvision.transforms.v2 does nothing / fails silently with numpy array · Issue #8662 · pytorch/vision · GitHub</a></p>
<p>Let me know if i can help something</p>

ptrblck: <p>Thanks for creating this issue!</p>

turbotimon: <p>Apparently this not a bug but a feature for passing arbitrary data structures. However, this is (at this time) not very well described in the documents and is only described here <a href="https://pytorch.org/vision/stable/auto_examples/transforms/plot_transforms_getting_started.html#what-do-i-pass-as-input" class="inline-onebox" rel="noopener nofollow ugc">Getting started with transforms v2 — Torchvision 0.20 documentation</a> but e.g. not <a href="https://pytorch.org/vision/stable/transforms.html" rel="noopener nofollow ugc">here</a></p>
<p>See also <a href="https://github.com/pytorch/vision/issues/8662" rel="noopener nofollow ugc">#8662</a></p>