Premultiplied alpha

rdgq: <p>When you use <code>torchvision.io.decode_image(..., mode=ImageReadMode.UNCHANGED)</code>, if the image has premultiplied alpha (ie. <a href="https://pkg.go.dev/image/color#NRGBA" rel="noopener nofollow ugc">NRGBA in go’s image/color</a>) does it automatically convert to non-premultiplied alpha?</p>

ajayrkumar: <p>I believe you question is basically to know how images with premultiplied alpha is handled during image decoding.</p>
<p>When you do <code>torchvision.io.decode_image('/path/to/image', mode='UNCHANGED')</code>, PyTorch ensures that it loads the image as is, i.e. the data and channels are preserved. Hence, if you load an image with pre-multiplied alpha, it won’t be automatically converted into non-pre-multiplied alpha.</p>
<p>For example,</p>
<pre><code class="lang-auto"># Loading images
img_nrgba = decode_image("./data/example_without_premup_alpha.png", mode='UNCHANGED')
img_rgba = decode_image("./data/example_with_alpha.png", mode='UNCHANGED')
</code></pre>
<p><div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/0/6/06390099a7ea040a895bdad89958be1a585e06b2.png" data-download-href="https://discuss.pytorch.org/uploads/default/06390099a7ea040a895bdad89958be1a585e06b2" title="img_alpha"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/0/6/06390099a7ea040a895bdad89958be1a585e06b2_2_517x96.png" alt="img_alpha" data-base62-sha1="T2ZupBlxeko5weUOvVf0K42Ncm" width="517" height="96" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/0/6/06390099a7ea040a895bdad89958be1a585e06b2_2_517x96.png, https://discuss.pytorch.org/uploads/default/optimized/3X/0/6/06390099a7ea040a895bdad89958be1a585e06b2_2_775x144.png 1.5x, https://discuss.pytorch.org/uploads/default/original/3X/0/6/06390099a7ea040a895bdad89958be1a585e06b2.png 2x" data-dominant-color="8C8D8F"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">img_alpha</span><span class="informations">989×186 202 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>
<p>TLDR:<br>
Images with pre-multiplied alpha are not converted to non-pre-multiplied alpha automatically.</p>

rdgq: <p>Is there a way I can tell whether the image used premultiplied alpha with torchvision so that I can normalize my inputs to be of the same format? Also will setting mode=RGB force non-premultiplied?</p>

ajayrkumar: <p>Short answer, you cannot.</p>
<p>Since pre-multiplied alpha cannot be reliably identified in PNGs, most datasets provide images with non-premultiplied-alpha, where the alpha channel is simply the fourth channel in the RGBA PNG image.</p>
<p>Hence, if you load your image with <code>decode_image(..., mode=UNCHANGED)</code> , and analyze the loaded tensor’s shape, an original RGB image will have 3 channels, and an original RGBA image (non-pre-multiplied-alpha) will have 4 channels.</p>
<p>With that said, setting <code>mode=RGB</code> for a non-pre-multiplied-alpha image would give you a RGB image, with the alpha channel discarded.</p>