How to change no of input channels to a pretrained model?

username: avijit_dasgupta
cooked: <p>Hi,</p>
<p>I have loaded the pre-trained AlexNet model in Pytorch. Now, I have images of 6 channels. So, I want to change the num_of_input_channels of the first convolutional layer to 6 from 3. How can I do that? Also, I want to use the pre-trained weights for the RGB channels and for other 3 I want to initialize the weights with random numbers.</p>
<p>Any help will be appreciated. Thanks.</p>

username: crcrpar
cooked: <p>Hi,</p>
<aside class="quote no-group" data-username="avijit_dasgupta" data-post="1" data-topic="19379">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/avijit_dasgupta/48/2541_2.png" class="avatar"> avijit_dasgupta:</div>
<blockquote>
<p>Now, I have images of 6 channels. So, I want to change the num_of_input_channels of the first convolutional layer to 6 from 3. How can I do that?</p>
</blockquote>
</aside>
<p>If you want to pass 6-channel images to AlexNet directly, there will be a need to replace the 1st convolution layer of AlexNet.<br>
Because the model is defined using <code>nn.Sequential</code> <a href="https://github.com/pytorch/vision/blob/master/torchvision/models/alexnet.py#L13" rel="noopener nofollow ugc">code</a>, it’s a bit confusing to replacing the specific layer.</p>
<aside class="quote no-group" data-username="avijit_dasgupta" data-post="1" data-topic="19379">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/avijit_dasgupta/48/2541_2.png" class="avatar"> avijit_dasgupta:</div>
<blockquote>
<p>I want to use the pre-trained weights for the RGB channels and for other 3 I want to initialize the weights with random numbers.</p>
</blockquote>
</aside>
<p>You can prepare the pretrained &amp; not trained AlexNet by <code>torchvision.models.alexnet(pretrained=True)</code> and <code>torchvision.models.alexnet(pretrained=False)</code> respectively. Then, split the 6-channel images into 2 3-channel images before pass them to the 2 alexnets.</p>

username: avijit_dasgupta
cooked: <p>I am using the following code for a 4 channel input-</p>
<pre><code class="lang-auto">
class RGBMaskEncoderCNN(nn.Module):
	def __init__(self):
		super(RGBMaskEncoderCNN, self).__init__()

		self.alexnet = models.alexnet(pretrained=True)
		# get the pre-trained weights of the first layer
		pretrained_weights = self.alexnet.features[0].weight
		new_features = nn.Sequential(*list(self.alexnet.features.children()))
		new_features[0] = nn.Conv2d(4, 64, kernel_size=11, stride=4, padding=2)

		# For M-channel weight should randomly initialized with Gaussian
		new_feaures_[0].weight.data.normal_(0, 0.001)
		# For RGB it should be copied from pretrained weights
		new_features[0].weight.data[:, :3, :, :] = pretrained_weights

		self.alexnet.features = new_features

		
		

	def forward(self, images):
		"""Extract Feature Vector from Input Images"""
		features = self.alexnet(images)
		return features
</code></pre>
<p>Don’t know if this is the correct way to do it or not.</p>

username: Mohamed_Hegazy
cooked: <p>Hi,</p>
<p>Can you please specify if this solution did work out correctly for you?</p>

username: josephj1o4e1
cooked: <p>This actually leads to an error of ‘Sequential’ object does not support item assignment.<br>
Is there any solution to this?</p>
<p>I was thinking to use a for loop to rebuild the whole object starting from the new feature, and followed by the rest of the alexnet.</p>

username: avijit_dasgupta
cooked: <aside class="onebox githubblob">
  <header class="source">
      <a href="https://github.com/avijit9/forces/blob/master/model.py" target="_blank" rel="nofollow noopener">github.com</a>
  </header>
  <article class="onebox-body">
    <h4><a href="https://github.com/avijit9/forces/blob/master/model.py" target="_blank" rel="nofollow noopener">avijit9/forces/blob/master/model.py</a></h4>
<pre><code class="lang-py">import torch
import torch.nn as nn
import torch.nn.functional as F
import torchvision.models as models
import pdb

class RGBMaskEncoderCNN(nn.Module):
	def __init__(self):
		super(RGBMaskEncoderCNN, self).__init__()

		self.alexnet = models.alexnet(pretrained=True)
		new_classifier = nn.Sequential(*list(self.alexnet.classifier.children())[:-1])
		self.alexnet.classifier = new_classifier

		# get the pre-trained weights of the first layer
		pretrained_weights = self.alexnet.features[0].weight
		new_features = nn.Sequential(*list(self.alexnet.features.children()))
		new_features[0] = nn.Conv2d(4, 64, kernel_size=11, stride=4, padding=2)
		# For M-channel weight should randomly initialized with Gaussian
		new_features[0].weight.data.normal_(0, 0.001)
</code></pre>

  This file has been truncated. <a href="https://github.com/avijit9/forces/blob/master/model.py" target="_blank" rel="nofollow noopener">show original</a>

  </article>
  <div class="onebox-metadata">
    
    
  </div>
  <div style="clear: both"></div>
</aside>


username: haile493
cooked: <p>This code works for me.</p>