Grayscale images in u-net

username: Diauro
cooked: <p>Hello,</p>
<p>I was looking to the following example of U-net for multiclass segmentation <a href="https://github.com/usuyama/pytorch-unet/blob/master/pytorch_unet.ipynb" rel="nofollow noopener">Unet</a>.<br>
Maybe is a silly question but is it possible to use u-net with grayscale images?<br>
In principle I would like to “re-use” the example in the link (if possible). Otherwise if somebody could address me to some examples it would be also greatly appreciated!</p>
<p>Thanks!</p>

username: RyanPatterson
cooked: <p>I am far from an expert but I am using UNetMini from <a href="https://towardsdatascience.com/train-a-lines-segmentation-model-using-pytorch-34d4adab8296" rel="nofollow noopener">this example</a>. <a href="https://discuss.pytorch.org/t/classes-accuracy-and-loss-in-pixel-wise-using-masks/81385/4">This</a> is a post with some of my code where I got some help not to long ago. Might find it somewhat useful.</p>

username: Diauro
cooked: <p>Hi Ryan,</p>
<p>Thanks for your reply! I was trying to look to your code and the example you linked. I’ve been looking to your post here (I quote the original)</p>
<aside class="quote no-group" data-username="KFrank" data-post="4" data-topic="81385">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/k/ecb155/48.png" class="avatar"><a href="https://discuss.pytorch.org/t/classes-accuracy-and-loss-in-pixel-wise-using-masks/81385/4"># Classes, Accuracy, and Loss in Pixel-wise using Masks</a></div>
<blockquote>
<p>Since you’re using a version of “UNet” and the first layer of your<br>
model is:</p>
<pre><code class="lang-auto">Conv2d(1, 32, kernel_size=k_size, padding=pad)
</code></pre>
</blockquote>
</aside>
<p>So I tried to modify also the first layer in the Unet I’m using by writing:<br>
<code>self.dconv_down1 = double_conv(1, 64)</code></p>
<p>Before there was 3 being the three channels RGB I guess.</p>
<p>I’ve tried to run the training and I get this error now</p>
<blockquote>
<p>invalid argument 0: Sizes of tensors must match except in dimension 1. Got 263 and 272 in dimension 2 at C:/w/1/s/tmp_conda_3.7_055457/conda/conda-bld/pytorch_1565416617654/work/aten/src\THC/generic/THCTensorMath.cu:71</p>
</blockquote>
<p>I’ve checked all the images they are 1024x1024 png grayscale images but I’m not sure how to solve this issue.</p>
<p>Best</p>

username: ashj
cooked: <p>By seeing the image sizes 263, a guess would be to check the padding values that you have set.</p>
<p>Also, could you share some more information about your log message?</p>

username: Diauro
cooked: <p>Hi ! thanks for you reply!</p>
<p>Currently i’m getting this:</p>
<blockquote>
<p>RuntimeError                              Traceback (most recent call last)<br>
 in <br>
17 exp_lr_scheduler = lr_scheduler.StepLR(optimizer_ft, step_size=75, gamma=0.1)<br>
18<br>
—&gt; 19 model = train_model(model, optimizer_ft, exp_lr_scheduler, num_epochs=100)</p>
<p> in train_model(model, optimizer, scheduler, num_epochs)<br>
62                 # track history if only in train<br>
63                 with torch.set_grad_enabled(phase == ‘train’):<br>
—&gt; 64                     outputs = model(inputs)<br>
65                     loss = calc_loss(outputs, labels, metrics)<br>
66</p>
<p>F:\aaa\Anaconda3\lib\site-packages\torch\nn\modules\module.py in <strong>call</strong>(self, *input, **kwargs)<br>
545             result = self._slow_forward(*input, **kwargs)<br>
546         else:<br>
 → 547             result = self.forward(*input, **kwargs)<br>
548         for hook in self._forward_hooks.values():<br>
549             hook_result = hook(self, input, result)</p>
<p>~\Desktop\graph_rec_png\pytorch-unet\pytorch_unet_cable.py in forward(self, x)<br>
44<br>
45         x = self.upsample(x)<br>
—&gt; 46         x = torch.cat([x, conv3], dim=1)<br>
47<br>
48         x = self.dconv_up3(x)</p>
<p>RuntimeError: invalid argument 0: Sizes of tensors must match except in dimension 1. Got 23 and 32 in dimension 2 at C:/w/1/s/tmp_conda_3.7_055457/conda/conda-bld/pytorch_1565416617654/work/aten/src\THC/generic/THCTensorMath.cu:71</p>
</blockquote>
<p>I’m using as the example  to load my images 1024x1024 gay scale images in png  format.  The idea is to feed them into the Unet here in the example <a href="https://github.com/usuyama/pytorch-unet/blob/master/pytorch_unet.ipynb" rel="noopener nofollow ugc">Unet</a></p>
<p>from torch.utils.data import Dataset, DataLoader<br>
from torchvision import transforms, datasets, models</p>
<p>class SimDataset(Dataset):<br>
def <strong>init</strong>(self, image_paths, mask_paths , transform=None):<br>
self.image_paths = image_paths<br>
self.mask_paths = mask_paths</p>
<pre><code>def transforms(self, image, mask):
    #img = img.resize((wsize, baseheight), PIL.Image.ANTIALIAS)

    image = image.resize((64, 64), PIL.Image.NEAREST)
    mask = mask.resize((64, 64), PIL.Image.NEAREST)        
    image = TF.to_tensor(image)
    mask = TF.to_tensor(mask)
    #mask = np.array(mask)
    #mask = np.moveaxis(mask,2,0)
    #mask = torch.from_numpy(np.array(mask))
    #mask = mask.unsqueeze(0)
    #mask = mask.unsqueeze(0)
    
    return [image, mask]





def __getitem__(self, index):
    image = Image.open(self.image_paths[index])
    mask = Image.open(self.mask_paths[index])
    
    x, y = self.transforms(image, mask)
    return x, y
    


def __len__(self):
    return len(self.image_paths)
</code></pre>
<p>The training loop would be as the one in that example (I used this model to train successfully a multi-class segmentation problem)</p>
<p>Thanks for any suggestion !</p>

username: ashj
cooked: <p>Thanks for the error logs.</p>
<p>From line no 45-48, I believe, it must be the invalid padding setting that is consuming one pixel at each layer.<br>
Could you check the padding values once, I guess your issue is something similar to one addressed in <a href="https://discuss.pytorch.org/t/linear-layer-input-neurons-number-calculation-after-conv2d/28659/2">this answer.</a></p>

username: Diauro
cooked: <p>Currently i’m using padding = 1 and before I did not have any issue. Not sure why now is giving me this problem.</p>
<p>I’ve checked the link but I could not solve the issue. Do you have any idea of how I could solve this problem or things which I could look at?</p>
<p>Best</p>