RuntimeError: expected scalar type Double but found Float, stuck for a while!

username: Aya_Hk
cooked: <p>Hello,</p>
<p>I have pre-trained a neural net over multivariate time series and saved it, and when I load it and try to transfer it to another dataset, I get this error:</p>
<h3>Traceback (most recent call last):</h3>
<p>File “…\TL.py”, line 176, in <br>
transfer_L(net, data, learning_rate, True, output_size)<br>
File “…\TL.py”, line 138, in transfer_L<br>
_ , loss = train(input_tensor, output_tensor, net, optimizer)<br>
File “…\TL.py”, line 89, in train<br>
output = net(input_tensor)<br>
File “…\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.9_qbz5n2kfra8p0\LocalCache\local-packages\Python39\site-packages\torch\nn\modules\module.py”, line 727, in _call_impl<br>
result = self.forward(*input, **kwargs)<br>
File “…\TL.py”, line 76, in forward<br>
lstm_out, _ = self.lstm(input_tensor)<br>
File “…\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.9_qbz5n2kfra8p0\LocalCache\local-packages\Python39\site-packages\torch\nn\modules\module.py”, line 727, in _call_impl<br>
result = self.forward(*input, **kwargs)<br>
File “…\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.9_qbz5n2kfra8p0\LocalCache\local-packages\Python39\site-packages\torch\nn\modules\rnn.py”, line 581, in forward<br>
result = _VF.lstm(input, hx, self._flat_weights, self.bias, self.num_layers,<br>
RuntimeError: expected scalar type Double but found Float ###</p>
<p>The TL.py is used for the Transfer Learning, by fine-tuning only the last layer of my network, and here is the function def transfer_L(…) that applies the TL:</p>
<h3></h3>
<p>net = torch.load(model_path)<br>
input_size =len(households_train[0][0][0][0])<br>
output_size = input_size<br>
learning_rate = 0.0005<br>
data = households_train<br>
lastL = True<br>
if lastL:<br>
for param in model.parameters():<br>
param.requires_grad = False<br>
model.fc2 = nn.Linear(1000, output_size)<br>
params_to_update = []<br>
for name,param in model.named_parameters():<br>
if param.requires_grad == True:<br>
params_to_update.append(param)<br>
optimizer = torch.optim.Adam(params_to_update, lr=learning_rate)<br>
current_loss = 0<br>
all_losses = []<br>
plot_steps = 100<br>
n_iters = len(data)<br>
for i in range(n_iters):<br>
input_tensor , output_tensor = data[i][0] ,  data[i][1]<br>
_ , loss = train(input_tensor, output_tensor, net, optimizer)<br>
…</p>
<h3></h3>
<p>I have tried fixing it (based on previous topics from StackOverflow and Pytorch) by:</p>
<ul>
<li>
<p>Converting the input tensor to double with:<br>
input_tensor , output_tensor = input_tensor.type(torch.DoubleTensor) , output_tensor.type(torch.DoubleTensor)<br>
=&gt; does not solve the issue</p>
</li>
<li>
<p>Before converting the numpy array to torch.Tensor, doing: input = input.astype(np.double)</p>
</li>
<li>
<p>Converting the weights_to_upgrade to Double (because idk where the float problem comes from) by : params_to_update.append(param.double())<br>
=&gt; gave back : ValueError: can’t optimize a non-leaf Tensor</p>
</li>
</ul>
<p>Nothing worked, and I am stuck for a while now… Please, any help is much needed!</p>
<p>Thanks in advance,</p>
<p>Aya</p>

username: Dwight_Foster
cooked: <p>Have you tried changing the hidden state to a double?</p>

username: Aya_Hk
cooked: <p>I actually changed the input and output tensors to float, and now it works (which is odd given the RuntimeError statement)<br>
Thanks tho!</p>

username: Tushar_Goel
cooked: <p>I faced the same problem and converting it to float worked. Does anyone know why this happens?</p>

username: ptrblck
cooked: <aside class="quote no-group" data-username="Tushar_Goel" data-post="4" data-topic="115572">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/tushar_goel/48/30397_2.png" class="avatar"> Tushar_Goel:</div>
<blockquote>
<p>Does anyone know why this happens?</p>
</blockquote>
</aside>
<p>Since you’ve fixed the issue by transforming a tensor or model to <code>float()</code>, check its creation and narrow down why it was created as a <code>DoubleTensor</code> in the first place. Often this issue is caused by transforming numpy arrays to tensors, as the former uses <code>float64</code> by default.</p>