TypeError: cannot assign 'torch.FloatTensor' as parameter 'weight' (torch.nn.Parameter or None expected)

students: <p>Hello, I want to changed layer replace to this layer but I get  this error! I want to shift one hidden layer in row and column… please help me.</p>
<pre><code class="lang-auto">import numpy as np
import torch
import torch.nn as nn
import torch.nn.functional as F
import torch.optim as optim
import matplotlib.pyplot as plt
from torch.utils.data.sampler import SubsetRandomSampler
from torch.utils.data import DataLoader
from torchvision import datasets, transforms

#%%

#Converting data to torch.FloatTensor
transform = transforms.ToTensor()


# Download the training and test datasets
train_data = datasets.MNIST(root='data', train=True, download=True, transform=transform)

test_data = datasets.MNIST(root='data', train=False, download=True, transform=transform)

train_loader = torch.utils.data.DataLoader(train_data, batch_size=32, num_workers=0)
test_loader = torch.utils.data.DataLoader(test_data, batch_size=32, num_workers=0)

#train_loader=torch.to

#%%

#Define the Convolutional Autoencoder
class ConvAutoencoder(nn.Module):
    def __init__(self):
        super(ConvAutoencoder, self).__init__()
       
        #Encoder
        self.conv1 = nn.Conv2d(1, 16, 3, stride=2, padding=1)
        self.conv2 = nn.Conv2d(16, 8, 3, stride=2, padding=1)
        self.conv3 = nn.Conv2d(8,8,3)
    
        #Decoder
        self.conv4 = nn.ConvTranspose2d(8, 8, 3)
        self.conv5 = nn.ConvTranspose2d(8, 16, 3, stride=2, padding=1, output_padding=1)
        self.conv6 = nn.ConvTranspose2d(16, 1, 3, stride=2, padding=1, output_padding=1)

    def forward(self, x):
        x = F.relu(self.conv1(x))      
        x = F.relu(self.conv2(x))
        x = F.relu(self.conv3(x))  
        #mask = torch.cat([torch.ones([8,8,2,3]), torch.zeros([8,8,1,3])], 2)
        #x=torch.multiply(x,mask)
       # x[7,7,:,:]=x[6,6,:,:]
        x = F.relu(self.conv4(x))
        x = F.relu(self.conv5(x))
        x = F.relu(self.conv6(x))

        return x

#Instantiate the model
model = ConvAutoencoder()
print(model)

#%%
def train(model, num_epochs=20, batch_size=64, learning_rate=1e-3):
    torch.manual_seed(42)
    criterion = nn.MSELoss() # mean square error loss
    optimizer = torch.optim.Adam(model.parameters(),
                                 lr=learning_rate, 
                                 weight_decay=1e-5) # &lt;--
   # train_loader =train_loader;

    outputs = []
    for epoch in range(num_epochs):
        for data in train_loader:
            img, _ = data
            recon = model(img)
            loss = criterion(recon, img)
            loss.backward()
            optimizer.step()
            optimizer.zero_grad()

        print('Epoch:{}, Loss:{:.4f}'.format(epoch+1, float(loss)))
        outputs.append((epoch, img, recon),)
    return outputs
#%%

#test_image = test_loader.open(test_image_name).convert('RGB')

model =  ConvAutoencoder()
max_epochs =20
outputs = train(model, num_epochs=max_epochs)
#%%

for k in range(0, max_epochs, 9):
    plt.figure(figsize=(9, 2))
    imgs = outputs[k][1].detach().numpy()
    recon = outputs[k][2].detach().numpy()
    for i, item in enumerate(imgs):
        if i &gt;= 9: break
        plt.subplot(2, 9, i+1)
        plt.imshow(item[0])
        
    for i, item in enumerate(recon):
        if i &gt;= 9: break
        plt.subplot(2, 9, 9+i+1)
        plt.imshow(item[0])
        
        #%%
   
        
a=(ConvAutoencoder().conv3.weight)
a0=a[0,0,:,:]
a1=a[1,1,:,:]
a2=a[2,2,:,:]
a3=a[3,3,:,:]
a4=a[4,4,:,:]
a5=a[5,5,:,:]
a6=a[6,6,:,:]
a7=a[7,7,:,:]
a0=a1
a1=a2
a2=a3
a3=a4
a4=a5
a5=a6
a6=a7
a7=a0
a=[a0,
    a1,
    a2,
    a3,
    a4,
    a5,
    a6,
    a7]

a = torch.tensor(a[0][1])
ConvAutoencoder().conv3.weight=a

 
print("conv1 filters: ",a.data.size())


#%%
#test phase        

with torch.no_grad():                                                                     #3
    for data in test_loader:    
         data = torch.tensor(data[0][1])    
         
         output = model(data)     

    plt.imshow(output[0,0])</code></pre>

ptrblck: <p>As the error message explains, you have to assign <code>nn.Parameter</code> objects to the internal parameters, not tensors.<br>
Based on your code snippet you would have to wrap the <code>a</code> tensor into <code>nn.Parameter(a)</code> and assign it to the <code>weight</code>.<br>
Also note that you are creating a new model instance and are removing it directly in:</p>
<pre><code class="lang-python">ConvAutoencoder().conv3.weight=a
</code></pre>
<p>so you most likely want to assign a variable to the model creation.</p>
<p>Is this a double post from <a href="https://discuss.pytorch.org/t/i-want-to-shift-one-hidden-layer-in-test-phase-after-training-and-show-a-changes/119432">here</a> or what it the difference between both posts?</p>

students: <p>thanks…   <a href="https://discuss.pytorch.org/t/i-want-to-shift-one-hidden-layer-in-test-phase-after-training-and-show-a-changes/119432">here</a> another question about my project.</p>

students: <p>I dont know how do this:</p>
<blockquote>
<p>Based on your code snippet you would have to wrap the <code>a</code> tensor into <code>nn.Parameter(a)</code> and assign it to the <code>weight</code> .</p>
</blockquote>
<p>please guide me.</p>

students: <p>can you help me to write this section of my code?</p>

ptrblck: <p>Sure! Something like this should work to reassign a new parameter to a model:</p>
<pre><code class="lang-python">a = torch.randn(...)
model = ConvAutoencoder()
model.conv3.weight = nn.Parameter(a)
</code></pre>

students: <p>thanks a lot, but this error occured in my code:</p>
<pre><code class="lang-auto">a0=a[0,0,:,:]
a1=a[1,1,:,:]
a2=a[2,2,:,:]
a3=a[3,3,:,:]
a4=a[4,4,:,:]
a5=a[5,5,:,:]
a6=a[6,6,:,:]
a7=a[7,7,:,:]
a0=a1
a1=a2
a2=a3
a3=a4
a4=a5
a5=a6
a6=a7
a7=a0
a=[a0,
    a1,
    a2,
    a3,
    a4,
    a5,
    a6,
    a7]


model = ConvAutoencoder()
model.conv3.weight = nn.Parameter(a[0][1])

with torch.no_grad():                                                                     #3
    for data in test_loader:    
         data = torch.tensor(data[0])    
         
         output = model(data)     

    plt.imshow(output[0,0])

</code></pre>
<p>this is error section:</p>
<pre><code class="lang-auto">RuntimeError                              Traceback (most recent call last)
&lt;ipython-input-18-55487faab0c0&gt; in &lt;module&gt;()
      3          data = torch.tensor(data[0])
      4 
----&gt; 5          output = model(data)
      6 
      7     plt.imshow(output[0,0])

4 frames
/usr/local/lib/python3.7/dist-packages/torch/nn/modules/conv.py in _conv_forward(self, input, weight, bias)
    394                             _pair(0), self.dilation, self.groups)
    395         return F.conv2d(input, weight, bias, self.stride,
--&gt; 396                         self.padding, self.dilation, self.groups)
    397 
    398     def forward(self, input: Tensor) -&gt; Tensor:

RuntimeError: weight should have at least three dimensions</code></pre>

ptrblck: <p>In that case the shape of <code>a</code> is wrong and depending if you are using <code>nn.Conv1d</code> or <code>nn.Conv2d</code> the <code>weight</code> parameter should have the shape <code>[out_channels, in_channels, seq_len]</code> or <code>[out_channels, in_channels, height, width]</code>, respectively.</p>

students: <p>thanks, but I want to assign changed a to weight of conv2  (dimension is [8,8,3,3]) but i get this error. I dont know how to save type of weights of layer to  tensor and save dimension of weights [8,8,3,3] after change it.</p>
<pre><code class="lang-auto">a0=a[:,0,:,:]
a1=a[:,1,:,:]
a2=a[:,2,:,:]
a3=a[:,3,:,:]
a4=a[:,4,:,:]
a5=a[:,5,:,:]
a6=a[:,6,:,:]
a7=a[:,7,:,:]
a0=a1
a1=a2
a2=a3
a3=a4
a4=a5
a5=a6
a6=a7
a7=a0
a=[a0,
   a1,
   a2,
   a3,
   a4,
   a5,
   a6,
   a7]

print(a)
model = ConvAutoencoder()
model.conv3.weight = nn.Parameter(a)


</code></pre>
<p>this section is error:</p>
<pre><code class="lang-auto">---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
&lt;ipython-input-8-ff7cfd988db4&gt; in &lt;module&gt;()
     30 
     31 model = ConvAutoencoder()
---&gt; 32 model.conv3.weight = nn.Parameter(a)

/usr/local/lib/python3.7/dist-packages/torch/nn/parameter.py in __new__(cls, data, requires_grad)
     24         if data is None:
     25             data = torch.Tensor()
---&gt; 26         return torch.Tensor._make_subclass(cls, data, requires_grad)
     27 
     28     def __deepcopy__(self, memo):

TypeError: _make_subclass(): argument 'data' (position 2) must be Tensor, not list</code></pre>

students: <p>please guide me for this target…</p>

eqy: <p>You need to convert the list of tensors to a tensor (e.g., via cat:</p>
<pre><code class="lang-auto">a = torch.cat([a0, ..., a7], dim=1)
</code></pre>
<p>You can also use something like <code>torch.gather</code> to avoid manually unrolling all of the assignments.</p>

students: <aside class="quote no-group" data-username="eqy" data-post="11" data-topic="119462">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/eqy/48/46233_2.png" class="avatar"> eqy:</div>
<blockquote>
<p><code>a = torch.cat([a0, ..., a7], dim=1)</code></p>
</blockquote>
</aside>
<p>thanks but when I use this I get this error:</p>
<pre><code class="lang-auto">a0=a[:,0,:,:]
a1=a[:,1,:,:]
a2=a[:,2,:,:]
a3=a[:,3,:,:]
a4=a[:,4,:,:]
a5=a[:,5,:,:]
a6=a[:,6,:,:]
a7=a[:,7,:,:]
a0=a1
a1=a2
a2=a3
a3=a4
a4=a5
a5=a6
a6=a7
a7=a0
#a=[a0,
 #  a1,
 #  a2,
 #  a3,
 #  a4,
 #  a5,
 #  a6,
 #  a7]
#a = torch.FloatTensor(a)
#type(a)
#print(a)

a = torch.cat([a0, ..., a7], dim=1)

model = ConvAutoencoder()
model.conv3.weight = nn.Parameter(a)

print(model.conv3.weight)


</code></pre>
<p>error:</p>
<pre data-code-wrap="TypeError"><code class="lang-TypeError">&lt;ipython-input-8-526580d6fd64&gt; in &lt;module&gt;()
     28 #print(a)
     29 
---&gt; 30 a = torch.cat([a0, ..., a7], dim=1)
     31 
     32 model = ConvAutoencoder()

TypeError: expected Tensor as element 1 in argument 0, but got ellipsis</code></pre>

eqy: <p>Right, I do not mean to use <code>...</code> literally, but rather to replace it with the list of slices <code>[a0, a1, a2, a3, a4, a5, a6, a7]</code>.</p>

students: <p>thanks for your help, I could to transform a by size of [8,8,3,3], but I can not in phase test after changing one layer, to giving data for changed model and taking the images in output of test phase. please guide me…<br>
my code:</p>
<pre><code class="lang-auto">def test(model,test_loader):

    with torch.no_grad():
     for data in test_loader:
      output = model(data)
      return output 
      
for k in range(0, max_epochs, 9):
        plt.figure(figsize=(9, 2))
        imgs = output[k][0].detach().numpy()
        recon = output[k][0].detach().numpy()
        for i, item in enumerate(imgs):
         if i &gt;= 9: break
         plt.subplot(2, 9, i+1)
         plt.imshow(item[0])
        
        for i, item in enumerate(recon):
         if i &gt;= 9: break
         plt.subplot(2, 9, 9+i+1)
         plt.imshow(item[0])  

</code></pre>
<p>error of this section is:</p>
<pre><code class="lang-auto">TypeError                                 Traceback (most recent call last)
&lt;ipython-input-34-6393568d6986&gt; in &lt;module&gt;()
     20          if i &gt;= 9: break
     21          plt.subplot(2, 9, i+1)
---&gt; 22          plt.imshow(item[0])
     23 
     24         for i, item in enumerate(recon):

5 frames
/usr/local/lib/python3.7/dist-packages/matplotlib/image.py in set_data(self, A)
    697                 or self._A.ndim == 3 and self._A.shape[-1] in [3, 4]):
    698             raise TypeError("Invalid shape {} for image data"
--&gt; 699                             .format(self._A.shape))
    700 
    701         if self._A.ndim == 3:

TypeError: Invalid shape () for image data</code></pre>

students: <p>please guide me for this target…</p>

students: <p><a class="mention" href="/u/ptrblck">@ptrblck</a><br>
exuse me, can you respond to my question? thanks a lot…</p>