Feed RAW or PNG images to neural network

Py_en: <p>Hi,<br>
I have RAW images from a DSLR camera Canon (image.CR2). How can I feed my image into a neural network with input raw images of size 192X192x4 perhaps in .PNG format?</p>
<p>My images are 6000X4000, RGB, RAW file (.CR2).</p>
<p>Of course I need to convert my image in .PNG format, and I did, I tried to resize in 192X192 (but i don’t want to do that to not loose information, so I think  would be better to split one image into severale parts of 192X192 size).</p>
<p>But, in any case, when I feed a 192X192 image, the error is this:</p>
<p>“Traceback (most recent call last):<br>
File “C:\Users\ingen\Documents\raw2rgbnet\test-full.py”, line 91, in <br>
output = infer(img)<br>
File “C:\Users\ingen\Documents\raw2rgbnet\test-full.py”, line 71, in infer<br>
output = model(im_pad_th)<br>
File “C:\Users\ingen\AppData\Local\Programs\Python\Python37\lib\site-packages\torch\nn\modules\module.py”, line 489, in <strong>call</strong><br>
result = self.forward(*input, **kwargs)<br>
File “C:\Users\ingen\Documents\raw2rgbnet\models\full_mix3_deep_encoder_decoder.py”, line 141, in forward<br>
fix_s = self.fix_path(fix_s)<br>
File “C:\Users\ingen\AppData\Local\Programs\Python\Python37\lib\site-packages\torch\nn\modules\module.py”, line 489, in <strong>call</strong><br>
result = self.forward(*input, **kwargs)<br>
File “C:\Users\ingen\AppData\Local\Programs\Python\Python37\lib\site-packages\torch\nn\modules\container.py”, line 92, in forward<br>
input = module(input)<br>
File “C:\Users\ingen\AppData\Local\Programs\Python\Python37\lib\site-packages\torch\nn\modules\module.py”, line 489, in <strong>call</strong><br>
result = self.forward(*input, **kwargs)<br>
File “C:\Users\ingen\AppData\Local\Programs\Python\Python37\lib\site-packages\torch\nn\modules\conv.py”, line 320, in forward<br>
self.padding, self.dilation, self.groups)<br>
<strong>RuntimeError: Given groups=1, weight of size [32, 4, 3, 3], expected input[1, 3, 192, 192] to have 4 channels, but got 3 channels instead</strong>”</p>
<p>Maybe the problem is the 4th channel?<br>
If you need some code, ask me.</p>
<p>Thanks.</p>

ptrblck: <p>You could try to use e.g. <code>rawpy</code> to open the RAW images without transforming them into a particular (lossy) image format.</p>
<p>The error is raised since your conv layer expects an input image with 4 channels, while your input has 3 channels.<br>
I guess you might expect the image files to have an additional alpha channel?<br>
If that’s the case you should take a look at the image loading and processing and make sure this channel is also loaded.<br>
Alternatively, change <code>in_channels</code> to <code>3</code> in the conv layer if you want to work with your current input tensors.</p>

Py_en: <p>Thanks. Actually in this model I don’t know the nature of the additional channel, but in another model where I got nearly the same error, the owner answered me:“<strong>input raw images should be 4-channel Bayer images ([R, Gr, B, Gb]), not the 3-channel</strong>”, this is criptyc for me.</p>
<p>So, the BAYER CHANNEL is an alpha channel? I mean, if I add an alpha channel and the training is performed with the bayer channel, is it the same?</p>
<p>P.S. I tried to add the alpha channel, but the result (this is an enhancement image code) is an image with a huge PURPLE  dominant.</p>

ptrblck: <p>There shouldn’t be a Bayer “channel”, but the Bayer filter mosaic is an arrangement of color filters used in digital image sensors.<br>
It should contain a pattern of 50% green, 25% red and 25% blue pixels, thus the color format would be BGGR, RGBG, GRGB, or RGGB.<br>
In that case 4 input channels might be expected. However, your current images seem to only contain 3 channels so you would have to check, if you are really reading the RAW images in this format.</p>

Py_en: <p>I transformed with GIMP the raw format .CR2 in .PNG.<br>
I think the node is here: the bayer channel, so I’m getting some informations about my camera (Canon EOS 2000D) and the bayer. I also thinking to use rawpy to read the images… I keep you update and very thanks for your answer!</p>