RuntimeError: Given groups=1, weight of size [3, 3, 1, 1], expected input[4, 1, 64, 64] to have 3 channels, but got 1 channels instead

username: PPB
cooked: <p>I am working on the following code:</p><aside class="onebox allowlistedgeneric" data-onebox-src="https://github.com/hellloxiaotian/CFSRCNN">
  <header class="source">
      <img src="https://github.githubassets.com/favicons/favicon.svg" class="site-icon" width="32" height="32">

      <a href="https://github.com/hellloxiaotian/CFSRCNN" target="_blank" rel="noopener nofollow ugc">GitHub</a>
  </header>

  <article class="onebox-body">
    <div class="aspect-image" style="--aspect-ratio:690/345;"><img src="https://opengraph.githubassets.com/da9d18e066fb317f2ba5f9d2996dff44952302c9e174a842f4dbc3d242e2fbbd/hellloxiaotian/CFSRCNN" class="thumbnail" width="690" height="345"></div>

<h3><a href="https://github.com/hellloxiaotian/CFSRCNN" target="_blank" rel="noopener nofollow ugc">GitHub - hellloxiaotian/CFSRCNN: Coarse-to-Fine CNN for Image...</a></h3>

  <p>Coarse-to-Fine CNN for Image Super-Resolution (IEEE Transactions on Multimedia,2021) - GitHub - hellloxiaotian/CFSRCNN: Coarse-to-Fine CNN for Image Super-Resolution (IEEE Transactions on Multimedi...</p>


  </article>

  <div class="onebox-metadata">
    
    
  </div>

  <div style="clear: both"></div>
</aside>

<p>The code is written for RGB images (3 channels), but the dataset I’m working has only single channel. I have changed the line numbers 108 and 109 of  <strong>dataset.py</strong> file to process the single channel.</p>
<p>But I got the following error:</p>
<pre><code class="lang-auto">Traceback (most recent call last):
  File "train.py", line 68, in &lt;module&gt;
    main(cfg)
  File "train.py", line 63, in main
    solver.fit()
  File "F:\CFSRCNN\cfsrcnn_x2\solver.py", line 89, in fit
    sr = refiner(lr, scale)
  File "C:\Users\anaconda3\envs\CFSRCNN\lib\site-packages\torch\nn\modules\module.py", line 727, in _call_impl
    result = self.forward(*input, **kwargs)
  File "C:\Users\anaconda3\envs\CFSRCNN\lib\site-packages\torch\nn\parallel\data_parallel.py", line 159, in forward
    return self.module(*inputs[0], **kwargs[0])
  File "C:\Users\anaconda3\envs\CFSRCNN\lib\site-packages\torch\nn\modules\module.py", line 727, in _call_impl
    result = self.forward(*input, **kwargs)
  File "F:\CFSRCNN\cfsrcnn_x2\model\cfsrcnn.py", line 74, in forward
    x0 = self.sub_mean(x)
  File "C:\Users\anaconda3\envs\CFSRCNN\lib\site-packages\torch\nn\modules\module.py", line 727, in _call_impl
    result = self.forward(*input, **kwargs)
  File "F:\CFSRCNN\cfsrcnn_x2\model\ops.py", line 29, in forward
    x = self.shifter(x)
  File "C:\Users\anaconda3\envs\CFSRCNN\lib\site-packages\torch\nn\modules\module.py", line 727, in _call_impl
    result = self.forward(*input, **kwargs)
  File "C:\Users\anaconda3\envs\CFSRCNN\lib\site-packages\torch\nn\modules\conv.py", line 423, in forward
    return self._conv_forward(input, self.weight)
  File "C:\Users\anaconda3\envs\CFSRCNN\lib\site-packages\torch\nn\modules\conv.py", line 420, in _conv_forward
    self.padding, self.dilation, self.groups)
RuntimeError: Given groups=1, weight of size [3, 3, 1, 1], expected input[4, 1, 64, 64] to have 3 channels, but got 1 channels instead
</code></pre>
<p>I need to change the <strong>model/cfsrcnn.py</strong> file to fed the single channel. How to change <strong>MeanShift</strong>. function to process the single channel. Please help me to change this model code so that I can fed the single channel. The <strong>MeanShift</strong> function is given below:</p>
<pre><code class="lang-auto">class MeanShift(nn.Module):
    def __init__(self, mean_rgb, sub):
        super(MeanShift, self).__init__()

        sign = -1 if sub else 1
        r = mean_rgb[0] * sign
        g = mean_rgb[1] * sign
        b = mean_rgb[2] * sign

        self.shifter = nn.Conv2d(3, 3, 1, 1, 0) #3 is size of output, 3 is size of input, 1 is kernel 1 is padding, 0 is group 
        self.shifter.weight.data = torch.eye(3).view(3, 3, 1, 1) # view(3,3,1,1) convert a shape into (3,3,1,1) eye(3) is a 3x3 matrix and diagonal is 1.
        self.shifter.bias.data   = torch.Tensor([r, g, b])
        #in_channels, out_channels,ksize=3, stride=1, pad=1
        # Freeze the mean shift layer
        for params in self.shifter.parameters():
            params.requires_grad = False

    def forward(self, x):
        x = self.shifter(x)
        return x
</code></pre>

username: ptrblck
cooked: <p>Change the <code>in_channels</code> argument of the <code>self.shilfter</code> conv layer to <code>1</code> and it should work.</p>

username: PPB
cooked: <p>I have changed the in_channels argument of the <code>self.shilfter</code> conv layer to <code>1</code> in the following line ( cfsrcnn/<strong>model/ops.py</strong>: <strong>line 20</strong>):<br>
<strong>self.shifter = nn.Conv2d(3, 1, 1, 1, 0)</strong></p>
<p>But, the error  still persists.</p>
<p>In  <code>MeanShift</code> function, they have used <code>mean_rgb</code> variable to calculate r, g, b, but in my case, I have gray images, so how to modify the code according to my dataset.</p>

username: ptrblck
cooked: <p>You’ve changed the <code>out_channels</code> to <code>1</code> and kept <code>in_channels=3</code>.</p>

username: PPB
cooked: <p>I have corrected this. But now I’m getting this  error:</p>
<blockquote>
<p>Traceback (most recent call last):<br>
File “F:\CFSRCNN\cfsrcnn_x2\model\cfsrcnn.py”, line 75, in forward<br>
x1 = self.conv1(x0)<br>
RuntimeError: Given groups=1, weight of size [64, 1, 1, 1], expected input[4, 3, 64, 64] to have 1 channels, but got 3 channels instead</p>
</blockquote>

username: PPB
cooked: <p><a class="mention" href="/u/ptrblck">@ptrblck</a>  Thnak you so much for your solution. The previous problem is solved now. But I have got a new error:</p>
<blockquote>
<p>RuntimeError: The size of tensor a (128) must match the size of tensor b (168) at non-singleton dimension 2<br>
The input LR image size is <strong>128x128</strong> and output should be <strong>256x256</strong>. The <code>evaluate</code> function returns the sizes of <strong>lr</strong>, <strong>lr_ptach</strong>, <strong>sr</strong> and <strong>results</strong> are as follows:</p>
</blockquote>
<p><strong>lr</strong>=<strong>torch.Size([1, 128, 128])</strong><br>
<strong>lr_ptach=torch.Size([4, 1, 84, 84])</strong> #shave=20<br>
<strong>sr=torch.Size([4, 3, 168, 168])</strong><br>
<strong>results=torch.Size([1, 256, 256])</strong><br>
How to solve this problem?</p>
<pre><code class="lang-auto">def evaluate(self, test_data_dir, scale=2, num_step=0):
        global mean_psnr, mean_psnr1, mean_ssim
        mean_psnr = 0
        mean_psnr1 = 0
        mean_ssim = 0
        #print 'sdfs'
        cfg = self.cfg
        self.refiner.eval()
        test_data   = TestDataset(test_data_dir, scale=scale)
        test_loader = DataLoader(test_data,
                                 batch_size=1,
                                 num_workers=0,
                                 shuffle=False)

        for step, inputs in enumerate(test_loader): #step is the number of test images
            hr = inputs[0].squeeze(0) #reduce the first dimension
            lr = inputs[1].squeeze(0)
            print(lr.size())
            name = inputs[2][0]
            h, w = lr.size()[1:]
            #print lr.size()[1:]
            h_half, w_half = int(h/2), int(w/2)
            h_chop, w_chop = h_half + cfg.shave, w_half + cfg.shave

            # split large image to 4 patch to avoid OOM error
            lr_patch = torch.FloatTensor(4, 1, h_chop, w_chop)
            #rint(lr_patch.size())
            lr_patch[0].copy_(lr[:, 0:h_chop, 0:w_chop])
            lr_patch[1].copy_(lr[:, 0:h_chop, w-w_chop:w])
            lr_patch[2].copy_(lr[:, h-h_chop:h, 0:w_chop])
            lr_patch[3].copy_(lr[:, h-h_chop:h, w-w_chop:w])
            lr_patch = lr_patch.to(self.device)
            print(lr_patch.size())

            # run refine process in here!

            with torch.no_grad():
                sr = self.refiner(lr_patch, scale).data
                print(sr.size())
            h, h_half, h_chop = h*scale, h_half*scale, h_chop*scale
            w, w_half, w_chop = w*scale, w_half*scale, w_chop*scale
            
            # merge splited patch images
            result = torch.FloatTensor(1, h, w).to(self.device)
            print(result.size())
            result[:, 0:h_half, 0:w_half].copy_(sr[0,0:h_half, 0:w_half])
            result[:, 0:h_half, w_half:w].copy_(sr[1,0:h_half, w_chop-w+w_half:w_chop])
            result[:, h_half:h, 0:w_half].copy_(sr[2,h_chop-h+h_half:h_chop, 0:w_half])
            result[:, h_half:h, w_half:w].copy_(sr[3,h_chop-h+h_half:h_chop, w_chop-w+w_half:w_chop])
            sr = result
            hr = hr.cpu().mul(255).clamp(0, 255).byte().permute(1, 2, 0).numpy() #(644.1024,3) is the same dimensional with the size of input test image in dataset.py.
            sr = sr.cpu().mul(255).clamp(0, 255).byte().permute(1, 2, 0).numpy()
           
            # evaluate PSNR
            # this evaluation is different to MATLAB version
            # we evaluate PSNR in single channel
            bnd = scale
            im1 = hr[bnd:-bnd, bnd:-bnd]
            im2 = sr[bnd:-bnd, bnd:-bnd]
            mean_psnr += psnr(im1, im2) / len(test_data)
            mean_ssim += calculate_ssim(im1,im2)/len(test_data)
        print('epochs is %d, mean_psnr is %f, mean_ssim is %f' %(self.step,mean_psnr,mean_ssim))
        return mean_psnr
</code></pre>