How to change parameters of first layer in pretrained network by copying its own weight iteratively

username: Jayasree_Saha
cooked: <p>I want to use VGG network for a dataset whose input size is 12X64x64. Also I want to initialize first 12 layers with pretrained weights of its first layer. How can I do that?</p>

username: ptrblck
cooked: <p>To change the number of input channels in the first layer, you could just assign a new <code>nn.Conv2d</code> layer:</p>
<pre><code class="lang-python">model = models.vgg16()
model.features[0] = nn.Conv2d(12, 64, 3, 1, 1)
x = torch.randn(1, 12, 64, 64)
output = model(x)
</code></pre>
<p>If you want to copy the pretrained weights from the original first layer, you could do so after reassigning the new layer.</p>
<p>I’m not sure, how you would want to initialize the first 12 layers with the weights of the first one, as the shapes of the kernels are different. Could you explain your use case a bit more?</p>

username: Jayasree_Saha
cooked: <p>I tried the following. But it is not working.</p>
<p>vgg = models.vgg16_bn(pretrained=True)<br>
temp= models.vgg16_bn(pretrained=True)</p>
<p>vgg.features[0] = nn.Conv2d(12, 64, 3, 1, 1)<br>
params1 = temp.features[0].state_dict()<br>
params2 = vgg.features[0].state_dict()</p>
<p>params2[‘weight’][:,:3,:,:]=params1[‘weight’][:,:,:,:]<br>
params2[‘weight’][:,3:6,:,:]=params1[‘weight’][:,:,:,:]<br>
params2[‘weight’][:,6:9,:,:]=params1[‘weight’][:,:,:,:]<br>
params2[‘weight’][:,9:12,:,:]=params1[‘weight’][:,:,:,:]<br>
params2[‘bias’]=params1[‘bias’]<br>
vgg.features[0].load_state_dict(params2)</p>
<p>Can you please help?</p>

username: ptrblck
cooked: <p><a href="https://discuss.pytorch.org/t/how-to-transfer-the-pretrained-weights-for-a-standard-resnet50-to-a-4-channel/52252/2">This post</a> gives you a small example.</p>

username: Jayasree_Saha
cooked: <p>I am pasting my code here. I am not able to execute the following code</p>
<p>model = models.vgg16_bn(pretrained=True)<br>
weight = model.features[0].weight.clone()<br>
model.features[0] = nn.Conv2d(12, 64, kernel_size=3, stride=1, padding=1, bias=False)<br>
with torch.no_grad():<br>
model.features[0].weight[:, :3] = weight<br>
model.features[0].weight[:, 3:6] = weight<br>
model.features[0].weight[:, 6:9] = weight<br>
model.features[0].weight[:, 9:12] = weight</p>
<p>features=list(model.features.children())[:-20]<br>
model.features=nn.Sequential(*features)<br>
avgpool=list(model.avgpool.children())[:-1]<br>
model.avgpool=nn.Sequential(*avgpool)<br>
CLASS_L1=10<br>
features_classifier = list(model.classifier.children())[:-8]<br>
features_classifier.extend(nn.Sequential(nn.Linear(256, CLASS_L1),nn.BatchNorm1d(CLASS_L1),nn.ReLU(inplace=True)))<br>
model.classifier = nn.Sequential(*features_classifier)</p>
<p>X=np.load(“file1.npy”)<br>
train_loader=DataLoader(dataset=X,batch_size=10, shuffle=True,num_workers=4, pin_memory=True)</p>
<p>for epoch in range(num_epochs):<br>
print(“Epoch {}/{}”.format(epoch, num_epochs))<br>
print(’-*’ * 20)</p>
<pre><code>loss_train = 0
model.train(True)

for i,input in enumerate(train_loader):


inputs=input.float()
print("shape:",inputs.shape)
   	with torch.no_grad():
    	if use_gpu:
     		inputs = Variable(inputs.cuda())
        	else:
            	inputs = Variable(inputs)
    optimizer.zero_grad()
    
    
    outputs = model(inputs)
    
    _, preds = torch.max(outputs.data, 1)

    

    loss=new_Loss(outputs).cuda()
    loss_train += loss.data

    print("minibatch loss:",loss)


    loss.backward()
    optimizer.step()
   
	
    del  inputs, outputs
    torch.cuda.empty_cache()
   
    print('*'*25)
</code></pre>
<p>ERROR MESSAGE:</p>
<p>Epoch 0/250<br>
-<em>-</em>-<em>-</em>-<em>-</em>-<em>-</em>-<em>-</em>-<em>-</em>-<em>-</em>-<em>-</em>-<em>-</em>-<em>-</em><br>
(‘shape:’, (10, 12, 8, 8))<br>
THCudaCheck FAIL file=/pytorch/aten/src/THC/THCGeneral.cpp line=383 error=11 : invalid argument<br>
vgg_new.py:69: UserWarning: Implicit dimension choice for softmax has been deprecated. Change the call to include dim=X as an argument.<br>
temp=(torch.sum(softmax(input), dim=0))<br>
vgg_new.py:71: UserWarning: Implicit dimension choice for softmax has been deprecated. Change the call to include dim=X as an argument.<br>
ce=(torch.sum(softmax(input)*f.log_softmax(input, 1)*x))/len(input)<br>
(‘classification entropy:’, tensor(6.8673, device=‘cuda:0’, grad_fn=))<br>
(‘class entropy:’, tensor(9.9495, device=‘cuda:0’, grad_fn=))<br>
vgg_new.py:84: UserWarning: Implicit dimension choice for softmax has been deprecated. Change the call to include dim=X as an argument.<br>
return ((torch.sum(softmax(input)*f.log_softmax(input, 1)<em>x))/len(input))-torch.sum(temp</em>torch.log(temp)*x[0])<br>
(‘minibatch loss:’, tensor(-0.0291, device=‘cuda:0’, grad_fn=))<br>
Traceback (most recent call last):<br>
File “vgg_new.py”, line 141, in <br>
loss.backward()<br>
File “/home/students/.local/lib/python2.7/site-packages/torch/tensor.py”, line 107, in backward<br>
torch.autograd.backward(self, gradient, retain_graph, create_graph)<br>
File “/home/students/.local/lib/python2.7/site-packages/torch/autograd/<strong>init</strong>.py”, line 91, in backward<br>
Variable._execution_engine.run_backward(tensors, grad_tensors, retain_graph, create_graph, allow_unreachable=True)<br>
RuntimeError: cuDNN error: CUDNN_STATUS_EXECUTION_FAILED</p>
<p>Kindly advice how to fix it.</p>