Can you create a CVAE on the MNIST dataset in PyTorch without using Cuda?

PyTorch2603: <p>I have Windows 10 and an Intel GPU but am unable to use Cuda without an Nvidia GPU. I was trying to run a CVAE on the MNIST dataset with use_cuda=True but when model.cuda() is called I get the error: AssertionError: Torch not compiled with CUDA enabled If I set use_cuda to False I get the following error: optimizer = optim.Adam(model.parameters(), lr=1e-3) TypeError: ‘collections.OrderedDict’ object is not callable I wanted to know if there was a way to solve the above error(s), and if not, if it would be possible to still create CVAE on the MNIST dataset without using Cuda. I would appreciate any help to do this, and if anyone knew of any tutorials which create a CVAE on the MNIST Dataset in PyTorch without using Cuda. Thanks!</p>

AlphaBetaGamma96: <p>Can you post a minimal reproducible example of this error? (It helps with debugging <img src="https://discuss.pytorch.org/images/emoji/apple/slight_smile.png?v=12" title=":slight_smile:" class="emoji" alt=":slight_smile:" loading="lazy" width="20" height="20"> )</p>
<p>As you have an Intel GPU you can’t use CUDA. So, remove all references in your code to <code>.cuda()</code> etc.</p>
<p>The error you get for <code>optim.Adam(model.parameters(), lr=1e-3)</code> <code>TypeError: 'collections.OrderedDict' object is not callable</code>.</p>
<p>Can you check what both <code>model</code> and <code>model.parameters</code> (no brackets here) are? It seems that your <code>model.parameters</code> object is an <code>OrderedDict</code> itself whereas if I recalled correctly <code>model.parameters()</code> should return an <code>OrderedDict</code> object. So something is wrong with your model.</p>
<p>TL;DR - share a minimal reproducible error of this error so we can debug it.</p>

PyTorch2603: <p>Thanks for the response. I am unsure what you mean by a minimal reproducible example of this error, but this is the code I found online that I am trying to run without Cuda. I hope this helps give more context. I also tried changing model.parameters() to model.parameters but then get the error “ValueError: optimizer got an empty parameter list”</p>
<p>from <strong>future</strong> import print_function<br>
import matplotlib.pyplot as plt<br>
import matplotlib.gridspec as gridspec<br>
import torch<br>
import torch.utils.data<br>
from torch import nn, optim<br>
from collections import OrderedDict<br>
from torch.autograd import Variable<br>
from torch.nn import functional as F<br>
from torchvision import datasets, transforms<br>
from torchvision.utils import save_image<br>
%matplotlib inline</p>
<p>use_cuda = False<br>
batch_size = 32<br>
latent_size = 20 # z dim</p>
<p>kwargs = {‘num_workers’: 1, ‘pin_memory’: True} if use_cuda else {}<br>
train_loader = torch.utils.data.DataLoader(<br>
datasets.MNIST(‘…/data’, train=True, download=True,<br>
transform=transforms.ToTensor()),<br>
batch_size=batch_size, shuffle=True, **kwargs)<br>
test_loader = torch.utils.data.DataLoader(<br>
datasets.MNIST(‘…/data’, train=False,<br>
transform=transforms.ToTensor()),<br>
batch_size=batch_size, shuffle=True, **kwargs)<br>
def to_var(x):<br>
x = Variable(x)<br>
return x</p>
<p>def one_hot(labels, class_size):<br>
targets = torch.zeros(labels.size(0), class_size)<br>
for i, label in enumerate(labels):<br>
targets[i, label] = 1<br>
return to_var(targets)</p>
<p>def loss_function(recon_x, x, mu, logvar):<br>
BCE = F.binary_cross_entropy(recon_x, x.view(-1, 28*28))<br>
KLD = -0.5 * torch.sum(1 + logvar - mu.pow(2) - logvar.exp())<br>
return BCE + KLD<br>
class CVAE(nn.Module):<br>
def <strong>init</strong>(self, feature_size, latent_size, class_size):<br>
super(CVAE, self).<strong>init</strong>()<br>
self.feature_size = feature_size<br>
self.class_size = class_size</p>
<pre><code>    self.fc1  = nn.Linear(feature_size + class_size, 400)
    self.fc21 = nn.Linear(400, latent_size)
    self.fc22 = nn.Linear(400, latent_size)

    self.fc3 = nn.Linear(latent_size + class_size, 400)
    self.fc4 = nn.Linear(400, feature_size)

    self.relu = nn.ReLU()
    self.sigmoid = nn.Sigmoid()

def encode(self, x, c): 
    '''
    x: (bs, feature_size)
    c: (bs, class_size)
    '''
    inputs = torch.cat([x, c], 1) # (bs, feature_size+class_size)
    h1 = self.relu(self.fc1(inputs))
    z_mu = self.fc21(h1)
    z_var = self.fc22(h1)
    return z_mu, z_var
def reparametrize(self, mu, logvar):
    if self.training:
        std = logvar.mul(0.5).exp_()
        eps = Variable(std.data.new(std.size()).normal_())
        return eps.mul(std) + mu
    else:
        return mu

def decode(self, z, c): # P(x|z, c)
    '''
    z: (bs, latent_size)
    c: (bs, class_size)
    '''
    inputs = torch.cat([z, c], 1) # (bs, latent_size+class_size)
    h3 = self.relu(self.fc3(inputs))
    return self.sigmoid(self.fc4(h3))

def forward(self, x, c):
    mu, logvar = self.encode(x.view(-1, 28*28), c)
    z = self.reparametrize(mu, logvar)
    return self.decode(z, c), mu, logvar
</code></pre>
<p>def train(epoch):<br>
model.train()<br>
train_loss = 0<br>
for batch_idx, (data, labels) in enumerate(train_loader):<br>
data = to_var(data)<br>
labels = one_hot(labels, 10)<br>
recon_batch, mu, logvar = model(data, labels)<br>
optimizer.zero_grad()<br>
loss = loss_function(recon_batch, data, mu, logvar)<br>
loss.backward()<br>
train_loss += loss.data[0]<br>
optimizer.step()<br>
if batch_idx % 500 == 0:<br>
print(‘Train Epoch: {} [{}/{} ({:.0f}%)]\tLoss: {:.6f}’.format(<br>
epoch, batch_idx * len(data), len(train_loader.dataset),<br>
100. * batch_idx / len(train_loader),<br>
loss.data[0] / len(data)))</p>
<p>def test(epoch):<br>
model.eval()<br>
test_loss = 0<br>
for i, (data, labels) in enumerate(test_loader):<br>
data = to_var(data)<br>
labels = one_hot(labels, 10)<br>
recon_batch, mu, logvar = model(data, labels)<br>
test_loss += loss_function(recon_batch, data, mu, logvar).data[0]<br>
if i == 0:<br>
n = min(data.size(0), 8)<br>
comparison = torch.cat([data[:n],<br>
recon_batch.view(batch_size, 1, 28, 28)[:n]])<br>
save_image(comparison.data.cpu(),<br>
‘results/reconstruction_’ + str(epoch) + ‘.png’, nrow=n)</p>
<pre><code>test_loss /= len(test_loader.dataset)
print('====&gt; Test set loss: {:.4f}'.format(test_loss))
</code></pre>
<p>model = CVAE(28*28, latent_size, 10)<br>
optimizer = optim.Adam(model.parameters(), lr=1e-3)<br>
for epoch in range(1, 11):<br>
train(epoch)</p>

AlphaBetaGamma96: <aside class="quote no-group" data-username="PyTorch2603" data-post="3" data-topic="157377">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/p/3bc359/48.png" class="avatar"> PyTorch2603:</div>
<blockquote>
<p>super(CVAE, self).<strong>init</strong>()</p>
</blockquote>
</aside>
<p>This should be <code>super(CVAE, self).__init__()</code></p>
<p>Can you re-run it with this change and what errors you get?</p>

PyTorch2603: <aside class="quote no-group" data-username="AlphaBetaGamma96" data-post="4" data-topic="157377">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/a/3da27b/48.png" class="avatar"> AlphaBetaGamma96:</div>
<blockquote>
<p>super(CVAE, self).<strong>init</strong>()</p>
</blockquote>
</aside>
<p>I get the same error TypeError: ‘collections.OrderedDict object is not callable’ or “ValueError: optimizer got an empty parameter list” if model.parameters is used instead of model.parameters().</p>

AlphaBetaGamma96: <p>now replace <code>model.parameters</code> with <code>model.parameters()</code>. And keep <code>super(CVAE, self).__init__()</code> with the underscores.</p>

PyTorch2603: <p>I still get "TypeError: ‘collections.OrderedDict object is not callable’ after these edits.</p>

AlphaBetaGamma96: <p>On what line does this error appear? I need a bit more context on what line is causing the error.</p>
<p>Also, I see you’re using <code>Variable</code> and <code>.data</code>. I’d advise you don’t use them as <code>Varaible</code> is deprecated and <code>.data</code> messes with metadata in your tensors.</p>
<p>Secondly, can you edit your initial comment with the code and make sure it’s all indented properly? It’s pretty hard to read without the indentations! Make sure to wrap the entire code in three backticks ``` to properly highlight any code.</p>
<p>Also, can you change <code>optimizer.zero_grad()</code> and <code>optimizer.step()</code> to <code>optim.zero_grad()</code> and <code>optim.step()</code> respectively? You’re calling the base class instead of its instance.</p>

PyTorch2603: <p>optimizer = optim.Adam(model.parameters(), lr=1e-3)<br>
This line causes the error ‘collections.OrderedDict’ object is not callable’.<br>
I tried to fix the indentations in my initial comment but while they show up in the text box when I am typing they are not fixed in the actual message once I save the edit. I also changed optimizer.zero_grad and optimizer.step().<br>
What would you use as an alternative for Variable and .data?</p>

AlphaBetaGamma96: <aside class="quote no-group" data-username="PyTorch2603" data-post="3" data-topic="157377">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/p/3bc359/48.png" class="avatar"> PyTorch2603:</div>
<blockquote>
<pre><code class="lang-auto">    '''
    x: (bs, feature_size)
    c: (bs, class_size)
    '''
</code></pre>
</blockquote>
</aside>
<p>It’s because you have other ```'s within your comment. So it’s doing part of it within, and part of it without. Make sure there’s just one at the top and one at the bottom, and I don’t think tabs materialize so you’ll have to use spaces rather than tabs to get the right indentation.</p>
<aside class="quote no-group" data-username="PyTorch2603" data-post="9" data-topic="157377">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/p/3bc359/48.png" class="avatar"> PyTorch2603:</div>
<blockquote>
<p>What would you use as an alternative for Variable and .data?</p>
</blockquote>
</aside>
<p>Variable has been deprecated since 0.4 (IIRC) so you don’t need it at all. For <code>.data</code> you don’t need that either, if you’re trying to just keep track of a value (without needing to backprop) you can use <code>.detach()</code> instead.</p>
<p>When you updated <code>init</code> to <code>__init__</code>, did you do it for both <strong>init</strong> below? That could be the problem as you won’t initialize your <code>model</code> as an <code>nn.Module</code> object and hence <code>model.parameters()</code> won’t behave properly.</p>
<aside class="quote no-group" data-username="PyTorch2603" data-post="3" data-topic="157377">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/p/3bc359/48.png" class="avatar"> PyTorch2603:</div>
<blockquote>
<p>def <strong>init</strong>(self, feature_size, latent_size, class_size):<br>
super(CVAE, self).<strong>init</strong>()</p>
</blockquote>
</aside>
<p>should be</p>
<pre><code class="lang-auto">def __init__(self, feature_size, latent_size, class_size):
  super(CVAE, self).__init__()
</code></pre>