RuntimeError: Expected object of scalar type Float but got scalar type Double for argument #2 'weight' in call to _thnn_conv2d_forward

username: MJN
cooked: <p>I am trying to implement a small CNN with PyTorch .I have changed the model.named_parameters() with other numbers, nothing else is changed. Could someone please help me with this issue? with the following logic, I updated the new weights.</p>
<p>count=0</p>
<pre><code>for name,param in model.named_parameters():
    print(type(param.data))
    print("before:",param.data)
    #sizes is the sizes of the weights
    #un_params is my new wieghts list of all layers
    if(len(sizes[count])==1):
        param.data=torch.from_numpy(un_params[count][0])
    else:
        param.data=torch.as_tensor(un_params[count])
    print(type(param.data))
    print("after :",param.data)
    count+=1</code></pre>

username: ptrblck
cooked: <p>Numpy uses <code>float64</code> as the default data type. If you load or create an array without changing this <code>dtype</code>, <code>torch.from_numpy()</code> will respect the <code>dtype</code> and will also return a <code>DoubleTensor</code>.<br>
You should solve this issue by using:</p>
<pre><code class="lang-python">param = torch.from_numpy(un_params[count][0]).float()
</code></pre>
<p>PS: the usage of the <code>.data</code> attribute is discouraged as it might yield unexpected side effects.<br>
Instead you should wrap the assignment in a <code>with torch.no_grad()</code> block and directly assign new <code>nn.Parameter</code>s.</p>

username: MJN
cooked: <p>thank you, .float() solved the problem! I’ll look into how to wrap the assignment using 'with torch.no_grad() ’ but could you please elaborate it if possible? My goal is to only change the weight and bias values of all layers keeping the other attributes of the layers(like drop-out functionality etc) the same as before. Will using the 'with torch.no_grad() block ’ help in attaining my goal?</p>

username: ptrblck
cooked: <p>If you want to manually update the parameters, this dummy code snippet should work:</p>
<pre><code class="lang-python">with torch.no_grad():
    for param in model.parameters():
        param -= learning_rate * param.grad
</code></pre>
<p>I’m not sure how your use case works and how you are calculating the <code>un_params</code>.<br>
Could you explain it a bit?</p>

username: MJN
cooked: <p>my use case is to get to the global solution in the solution space, as gradient descent can get stuck in local optimum sometimes.I am applying GA(genetic algorithm) on the 1D array (i.e model.named_prameters() after flattening everything to a single array). After GA is done I unflatten this 1D array back and load them to model.named_parameters(). I am using the below code snippets to unflatten the 1d array and to load the unflattened arrays to model_parameters().</p>
<p>def unflatten(matrix,shapes):<br>
final_matrix=[]<br>
<span class="hashtag">#print</span>(“len matrix:”,len(matrix))<br>
index=0</p>
<pre><code>for shape in shapes:
    prod=1
    for num in shape:
        prod=prod*num
    final_matrix.append(np.reshape(matrix[index:index+prod],shape))
    #print("final length:",len(final_matrix))
    # print (prod,final_matrix,np.reshape(matrix[index:index+prod],shape))
    index+=prod
return final_matrix
</code></pre>
<p><span class="hashtag">#shapes</span> are list of shapes of weights and biases in every layer<br>
un_params = unflatten(1d_array, shapes)</p>
<p><span class="hashtag">#for</span> loading the new weights<br>
def cnn_after_unflatten(un_params,sizes,model):<br>
count=0</p>
<pre><code>for name,param in model.named_parameters():
    #print(param.data.dtype)
    #print("before:",param.data)
    #sizes is the sizes of the weights
    #un_params is my new wieghts list of all layers
    if(len(sizes[count])==1):
        param.data=torch.from_numpy(un_params[count][0]).to(torch.float32)
    else:
        param.data=torch.from_numpy(un_params[count]).to(torch.float32)
    #print(param.data.dtype)
    #print("after :",param.data)
    count+=1</code></pre>