Gpytorch: RuntimeError: expected backend CPU and dtype Double but got backend CPU and dtype Float?

username: G0n_F
cooked: <p>Hello, all. This is my first time within the PyTorch ecosystem (having switched over because of GpyTorch’s seemingly excellent GP modelling capabilities), and I would appreciate help getting through my first model.</p>
<p>Following the first example in the <a href="https://buildmedia.readthedocs.org/media/pdf/gpytorch/latest/gpytorch.pdf" rel="nofollow noopener">docs</a>, I want to run a very simple model, but I encounter an error at the <code>loss = -mll(output, y_torch)</code> step:</p>
<pre><code class="lang-auto">## Generating data for regression
# First, regular sine wave + normal noise
x = np.linspace(0,40, num=300)
noise1 = np.random.normal(0,0.3,300)
y = np.sin(x) + noise1

# Second, an upward trending starting halfway, with its own normal noise
temp = x[150:]
noise2 = 0.004*temp**2 + np.random.normal(0,0.1,150)
y[150:] = y[150:] + noise2

x_torch = torch.from_numpy(x)
y_torch = torch.from_numpy(y)

# initialize likelihood and model
class ExactGPModel(gpytorch.models.ExactGP):
    def __init__(self, train_x, train_y, likelihood):
        super(ExactGPModel, self).__init__(train_x, train_y, likelihood)
        self.mean_module = gpytorch.means.ConstantMean()
        self.covar_module = gpytorch.kernels.ScaleKernel(gpytorch.kernels.RBFKernel())
        
    def forward(self, x):
        mean_x = self.mean_module(x)
        covar_x = self.covar_module(x)
        return gpytorch.distributions.MultivariateNormal(mean_x, covar_x)


likelihood = gpytorch.likelihoods.GaussianLikelihood()
model = ExactGPModel(x_torch, y_torch, likelihood)

# Find optimal model hyperparameters
model.train()
likelihood.train()

# Use the adam optimizer
optimizer = torch.optim.Adam([
    {'params': model.parameters()},  # Includes GaussianLikelihood parameters
], lr=0.1)

# "Loss" for GPs - the marginal log likelihood
mll = gpytorch.mlls.ExactMarginalLogLikelihood(likelihood, model)

training_iter = 50
for i in range(training_iter):
    
    # Zero gradients from previous iteration
    optimizer.zero_grad()
    
    # Output from model
    output = model(x_torch)
    
    # Calc loss and backprop gradients
    loss = -mll(output, y_torch)
    loss.backward()
    
    print('Iter %d/%d - Loss: %.3f   lengthscale: %.3f   noise: %.3f' % (
        i + 1, training_iter, loss.item(),
        model.covar_module.base_kernel.lengthscale.item(),
        model.likelihood.noise.item()
    ))
    
    optimizer.step()
</code></pre>
<p>At that line, I get a <code>RuntimeError: expected backend CPU and dtype Double but got backend CPU and dtype Float</code> error. Having looked around, this seems like a common problem, but always has to do with CPU/CUDA errors, with the answers being vague, and I just want to run my model first on my CPU.</p>
<p>I would appreciate any help on the matter.</p>

username: IPolaris
cooked: <p>maybe you can use model.cpu():<br>
mdel.cpu()<br>
likelihood.cpu()</p>
<h1>Find optimal model hyperparameters</h1>
<p>model.train()<br>
likelihood.train()</p>

username: PauloHubert
cooked: <p>Not sure why this happens, but the solution for me was to call .double() on the pytorch Tensors:</p>
<pre><code class="lang-auto">x_torch = torch.from_numpy(x).double()
y_torch = torch.from_numpy(y).double()
</code></pre>
<p>and also on the model, before calling train:</p>
<pre><code class="lang-auto">model.double()
model.train()
</code></pre>