ValueError: Using a target size (torch.Size([32, 1])) that is different to the input size (torch.Size([7200, 1])) is deprecated. Please ensure they have the same size

sabrina00: <h2>
<a name="starting-training-loop-usrlocallibpython37dist-packagestorchutilsdatadataloaderpy477-userwarning-this-dataloader-will-create-4-worker-processes-in-total-our-suggested-max-number-of-worker-in-current-system-is-2-which-is-smaller-than-what-this-dataloader-is-going-to-create-please-be-aware-that-excessive-worker-creation-might-get-dataloader-running-slow-or-even-freeze-lower-the-worker-number-to-avoid-potential-slownessfreeze-if-necessary-cpuset_checked-usrlocallibpython37dist-packagesipykernel_launcherpy42-userwarning-implicit-dimension-choice-for-softmax-has-been-deprecated-change-the-call-to-include-dimx-as-an-argument-1" class="anchor" href="#starting-training-loop-usrlocallibpython37dist-packagestorchutilsdatadataloaderpy477-userwarning-this-dataloader-will-create-4-worker-processes-in-total-our-suggested-max-number-of-worker-in-current-system-is-2-which-is-smaller-than-what-this-dataloader-is-going-to-create-please-be-aware-that-excessive-worker-creation-might-get-dataloader-running-slow-or-even-freeze-lower-the-worker-number-to-avoid-potential-slownessfreeze-if-necessary-cpuset_checked-usrlocallibpython37dist-packagesipykernel_launcherpy42-userwarning-implicit-dimension-choice-for-softmax-has-been-deprecated-change-the-call-to-include-dimx-as-an-argument-1"></a>Starting Training Loop…<br>
/usr/local/lib/python3.7/dist-packages/torch/utils/data/dataloader.py:477: UserWarning: This DataLoader will create 4 worker processes in total. Our suggested max number of worker in current system is 2, which is smaller than what this DataLoader is going to create. Please be aware that excessive worker creation might get DataLoader running slow or even freeze, lower the worker number to avoid potential slowness/freeze if necessary.<br>
cpuset_checked))<br>
/usr/local/lib/python3.7/dist-packages/ipykernel_launcher.py:42: UserWarning: Implicit dimension choice for softmax has been deprecated. Change the call to include dim=X as an argument.</h2>
<p>ValueError                                Traceback (most recent call last)<br>
 in ()<br>
36         # train with real<br>
37         s_output, c_output = netD(real_image)<br>
—&gt; 38         s_errD_real = s_criterion(s_output, real_target)  # realfake<br>
39         c_errD_real = c_criterion(c_output, real_label)  # class<br>
40         errD_real = s_errD_real + c_errD_real</p>
<p>2 frames<br>
/usr/local/lib/python3.7/dist-packages/torch/nn/functional.py in binary_cross_entropy(input, target, weight, size_average, reduce, reduction)<br>
2753         raise ValueError(<br>
2754             "Using a target size ({}) that is different to the input size ({}) is deprecated. "<br>
 → 2755             “Please ensure they have the same size.”.format(target.size(), input.size())<br>
2756         )<br>
2757</p>
<p>ValueError: Using a target size (torch.Size([32, 1])) that is different to the input size (torch.Size([7200, 1])) is deprecated. Please ensure they have the same size.</p>
<p>I am unable to solve the above problem.Would anyone kindly help me to solve it?</p>

ptrblck: <p>The error is raised, since <code>s_output</code> and <code>real_target</code> have a different shape, which is not supported.<br>
I don’t know how the output is calculated or what the used batch size is supposed to be, so you would have to check where this size difference is coming from and what the expected shape would be.</p>
<p>PS: you can post code snippets by wrapping them into three backticks ```, which makes debugging easier. <img src="https://discuss.pytorch.org/images/emoji/apple/wink.png?v=9" title=":wink:" class="emoji" alt=":wink:"></p>

diogo_tavares: <p>Hi <a class="mention" href="/u/ptrblck">@ptrblck</a>  ,  i am having a similar problem i wonder if you could help me. It goes like this:<br>
My target is a tensor torch.Size([731022, 1]) but my model returns torch.Size([1462044, 2])<br>
<div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/5/4/5447ffcaa4b20e2dee3f1dc5ed0ceb4d5849f3ae.png" data-download-href="https://discuss.pytorch.org/uploads/default/5447ffcaa4b20e2dee3f1dc5ed0ceb4d5849f3ae" title="imagem"><img src="https://discuss.pytorch.org/uploads/default/original/3X/5/4/5447ffcaa4b20e2dee3f1dc5ed0ceb4d5849f3ae.png" alt="imagem" data-base62-sha1="c1AmDiua9Mqa5XvQcbvk5MYRnZQ" width="685" height="500" data-dominant-color="EFF0F0"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">imagem</span><span class="informations">996×727 40.5 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div><br>
And after i do this:<br>
////////////////////////////////////////////////////////////////////////////////////////////////////////////<br>
criterion = torch.nn.MSELoss()<br>
optimizer = torch.optim.Adam(lstm1.parameters(), lr=learning_rate)</p>
<p>for epoch in range(num_epochs):<br>
outputs = lstm1.forward(X_train_tensors_final) <span class="hashtag-raw">#forward</span> pass<br>
optimizer.zero_grad() <span class="hashtag-raw">#caluclate</span> the gradient, manually setting to 0<br>
loss = criterion(outputs, y_train_tensors)<br>
loss.backward() <span class="hashtag-raw">#calculates</span> the loss of the loss function<br>
optimizer.step() <span class="hashtag-raw">#improve</span> from loss, i.e backprop<br>
if epoch % 1 == 0:<br>
print(“Epoch: %d, loss: %1.5f” % (epoch, loss.item()))<br>
///////////////////////////////////////////////////////////////////////////////////////////////<br>
And the error i get is this:<br>
UserWarning: Using a target size (torch.Size([731022, 1])) that is different to the input size (torch.Size([1462044, 2])). This will likely lead to incorrect results due to broadcasting. Please ensure they have the same size.<br>
return F.mse_loss(input, target, reduction=self.reduction)</p>
<p>Now i know its returning the output, hn and cn but what i dont understand is why is it returning 1462044 values which is double the values of the input. I hope you can help me, thanks!</p>

ptrblck: <p>I guess the issue is created by flattening <code>hn</code>, which has the shape <code>[num_layers*num_directions, batch_size, hidden_size]</code> as described in the <a href="https://pytorch.org/docs/stable/generated/torch.nn.LSTM.html#torch.nn.LSTM">docs</a>, so you would need to check the shape before and after the flattening.<br>
You might have missed my last sentence, so your code also cannot be debugged, since you posted a picture <img src="https://discuss.pytorch.org/images/emoji/apple/wink.png?v=9" title=":wink:" class="emoji" alt=":wink:"></p>

diogo_tavares: <p>Hi <a class="mention" href="/u/ptrblck">@ptrblck</a> thank you for answering, after looking at the docs its possible thats the problem but it also specifies “If the LSTM is bidirectional, num_directions should be 2, else it should be 1.”. My end goal is to make it bidirectional but right now i am trying without it being bidirectional so num_directions should be 1 i don’t really get where this variable comes from or where it is being defined. Also i’m sorry about the code format i thought it would be easier with the picture so i’ll just put it all down so you can get the hole picture.<br>
/////////////////////////////////////////////<br>
import torch<br>
from torch import nn<br>
from torch.autograd import Variable<br>
import pandas as pd<br>
from sklearn.preprocessing import StandardScaler, MinMaxScaler</p>
<p><span class="hashtag">#loading</span> data<br>
training_data = pd.read_excel(r’rawDATA_2outputs.xlsx’)<br>
test_data = pd.read_excel(r’test_2outputs.xlsx’)<br>
<span class="hashtag">#shuffle</span> data<br>
training = training_data.sample(frac=1)<br>
test = test_data.sample(frac=1)</p>
<p><span class="hashtag">#data</span> preprocessing<br>
X = training_data.iloc[:, :-1]   <span class="hashtag">#training</span> data sem labels<br>
y = training_data.iloc[:, 42:43] <span class="hashtag">#labels</span> da training_data<br>
X_test = test_data.iloc[:, :-1]       <span class="hashtag">#test_data</span> sem labels<br>
y_test = test_data.iloc[:, 42:43]     <span class="hashtag">#labels</span> da test_data</p>
<p><span class="hashtag">#scaling</span> the data<br>
mm = MinMaxScaler()<br>
ss = StandardScaler()<br>
X_ss = ss.fit_transform(X)<br>
y_mm = mm.fit_transform(y)<br>
X_test_ss = ss.fit_transform(X_test)<br>
y_test_mm = ss.fit_transform(y_test)</p>
<p><span class="hashtag">#data</span> as tensors<br>
X_train_tensors = Variable(torch.Tensor(X_ss))<br>
y_train_tensors = Variable(torch.Tensor(y_mm))<br>
X_test_tensors = Variable(torch.Tensor(X_test_ss))<br>
y_test_tensors = Variable(torch.Tensor(y_test_mm))</p>
<p><span class="hashtag">#reshape</span><br>
X_train_tensors_final = torch.reshape(X_train_tensors,   (X_train_tensors.shape[0], 1, X_train_tensors.shape[1]))<br>
X_test_tensors_final = torch.reshape(X_test_tensors,  (X_test_tensors.shape[0], 1, X_test_tensors.shape[1]))</p>
<p><span class="hashtag">#build</span> the network<br>
class LSTM1(nn.Module):<br>
def <strong>init</strong>(self, num_classes, input_size, hidden_size, num_layers, seq_length):<br>
super(LSTM1, self).<strong>init</strong>()<br>
self.num_classes = num_classes  # number of classes<br>
self.num_layers = num_layers  # number of layers<br>
self.input_size = input_size  # input size<br>
self.hidden_size = hidden_size  # hidden state<br>
self.seq_length = seq_length  # sequence length</p>
<pre><code>    self.lstm = nn.LSTM(input_size=input_size, hidden_size=hidden_size,
                        num_layers=num_layers, batch_first=True)  # lstm
    self.fc_1 = nn.Linear(hidden_size, 128)  # fully connected 1
    self.fc = nn.Linear(128, num_classes)  # fully connected last layer

    self.relu = nn.ReLU()

def forward(self, x):
    h_0 = Variable(torch.zeros(self.num_layers, x.size(0), self.hidden_size))  # hidden state
    c_0 = Variable(torch.zeros(self.num_layers, x.size(0), self.hidden_size))  # internal state
    # Propagate input through LSTM
    output, (hn, cn) = self.lstm(x, (h_0, c_0))  # lstm with input, hidden, and internal state
    hn = hn.view(-1, self.hidden_size)  # reshaping the data for Dense layer next
    out = self.relu(hn)
    out = self.fc_1(out)  # first Dense
    out = self.relu(out)  # relu
    out = self.fc(out)  # Final Output
    return out
</code></pre>
<p><span class="hashtag">#hyperparameters</span><br>
num_epochs = 1<br>
learning_rate = 0.001<br>
input_size = 42      <span class="hashtag">#number</span> of features<br>
hidden_size = 128 <span class="hashtag">#number</span> of features in hidden state<br>
num_layers = 2     <span class="hashtag">#number</span> of stacked lstm layers<br>
num_classes = 2  <span class="hashtag">#number</span> of output classes</p>
<p><span class="hashtag">#aplicar</span> a network e definir criterio de erro e optimizador<br>
lstm1 = LSTM1(num_classes, input_size, hidden_size, num_layers, X_train_tensors_final.shape[1])<br>
criterion = torch.nn.MSELoss()<br>
optimizer = torch.optim.Adam(lstm1.parameters(), lr=learning_rate)</p>
<p><span class="hashtag">#train</span><br>
for epoch in range(num_epochs):<br>
outputs = lstm1.forward(X_train_tensors_final)  # forward pass<br>
optimizer.zero_grad()  # caluclate the gradient, manually setting to 0</p>
<pre><code># obtain the loss function
loss = criterion(outputs, y_train_tensors)

loss.backward()  # calculates the loss of the loss function

optimizer.step()  # improve from loss, i.e backprop
if epoch % 1 == 0:
    print("Epoch: %d, loss: %1.5f" % (epoch, loss.item())) 
</code></pre>
<p>//////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////<br>
I’ll also try to debug and see what happens, but anyway thank you for your time and help!</p>

ptrblck: <p>This code illustrates your usage and the wrong flattening:</p>
<pre><code class="lang-python"># setup
input_size = 42 
hidden_size = 128 
num_layers = 2

lstm = nn.LSTM(input_size=input_size, hidden_size=hidden_size,
                        num_layers=num_layers, batch_first=True)

batch_size, seq_len = 3, 4
x = torch.randn(batch_size, seq_len, input_size)
output, (hn, cn) = lstm(x)
print(output.shape) # [batch_size, seq_len, num_directions*hidden_size]
&gt; torch.Size([3, 4, 128])
print(hn.shape) # [num_layers*num_directions, batch_size, hidden_size]
&gt; torch.Size([2, 3, 128])
print(cn.shape) # [num_layers*num_directions, batch_size, hidden_size]
&gt; torch.Size([2, 3, 128])

# this is most likely not what you want!
hn = hn.view(-1, hidden_size)
print(hn.shape) # [num_layers*num_directions*batch_size, hidden_size]
&gt; torch.Size([6, 128])
</code></pre>
<p>I guess you might want to use the <code>output</code> (or its last step) instead of flattening the batch dimension into the layer/direction dim of the hidden state.</p>

sabrina00: <aside class="quote no-group quote-modified" data-username="ptrblck" data-post="2" data-topic="121877">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/ptrblck/48/1823_2.png" class="avatar"> ptrblck:</div>
<blockquote>
<pre><code class="lang-auto">dataroot = "/content"
workers = 4 
batch_size = 32
image_size = 299
nc = 3 #rgb
nz = 598
ngf = 299
ndf = 299
num_epochs = 2 
lr = 0.001
beta1 = 0.5
ngpu = 1
num_show = 2
n_class = 120

outf = '/content/drive/MyDrive/output_result'
if not os.path.exists(outf):
    os.mkdir(outf)



class Generator(nn.Module):

    def __init__(self, ngpu, nz=nz, ngf=ngf, nc=nc, n_class=n_class):

        super(Generator, self).__init__()
        self.ngpu = ngpu
        self.ReLU = nn.ReLU(True)
        self.Tanh = nn.Tanh()
        self.conv1 = nn.ConvTranspose2d(nz+n_class, ngf * 8, 4, 1, 0, bias=False)
        self.BatchNorm1 = nn.BatchNorm2d(ngf * 8)

        self.conv2 = nn.ConvTranspose2d(ngf * 8, ngf * 4, 4, 2, 1, bias=False)
        self.BatchNorm2 = nn.BatchNorm2d(ngf * 4)

        self.conv3 = nn.ConvTranspose2d(ngf * 4, ngf * 2, 4, 2, 1, bias=False)
        self.BatchNorm3 = nn.BatchNorm2d(ngf * 2)

        self.conv4 = nn.ConvTranspose2d(ngf * 2, ngf * 1, 4, 2, 1, bias=False)
        self.BatchNorm4 = nn.BatchNorm2d(ngf * 1)

        self.conv5 = nn.ConvTranspose2d(ngf * 1, nc, 4, 2, 1, bias=False)

        self.apply(weights_init)


    def forward(self, input):

        x = self.conv1(input)
        x = self.BatchNorm1(x)
        x = self.ReLU(x)

        x = self.conv2(x)
        x = self.BatchNorm2(x)
        x = self.ReLU(x)

        x = self.conv3(x)
        x = self.BatchNorm3(x)
        x = self.ReLU(x)

        x = self.conv4(x)
        x = self.BatchNorm4(x)
        x = self.ReLU(x)

        x = self.conv5(x)
        output = self.Tanh(x)
        return output

# Create the generator
netG = Generator(ngpu).to(device)
# Apply the weights_init function to randomly initialize all weights
netG.apply(weights_init)
# Print the model
print(netG)


class Discriminator(nn.Module):

    def __init__(self, ngpu, ndf=ndf, nc=nc, n_class=n_class):

        super(Discriminator, self).__init__()
        self.LeakyReLU = nn.LeakyReLU(0.2, inplace=True)
        self.conv1 = nn.Conv2d(nc, ndf, 4, 2, 1, bias=False)
        self.conv2 = nn.Conv2d(ndf, ndf * 2, 4, 2, 1, bias=False)
        self.BatchNorm2 = nn.BatchNorm2d(ndf * 2)
        self.conv3 = nn.Conv2d(ndf * 2, ndf * 4, 4, 2, 1, bias=False)
        self.BatchNorm3 = nn.BatchNorm2d(ndf * 4)
        self.conv4 = nn.Conv2d(ndf * 4, ndf * 8, 4, 2, 1, bias=False)
        self.BatchNorm4 = nn.BatchNorm2d(ndf * 8)
        self.conv5 = nn.Conv2d(ndf * 8, ndf * 1, 4, 1, 0, bias=False)
        self.disc_linear = nn.Linear(ndf * 1, 1)
        self.aux_linear = nn.Linear(ndf * 1, n_class)
        self.softmax = nn.Softmax()
        self.sigmoid = nn.Sigmoid()
        self.ndf = ndf
        self.apply(weights_init)

    def forward(self, input):

        x = self.conv1(input)
        x = self.LeakyReLU(x)

        x = self.conv2(x)
        x = self.BatchNorm2(x)
        x = self.LeakyReLU(x)

        x = self.conv3(x)
        x = self.BatchNorm3(x)
        x = self.LeakyReLU(x)

        x = self.conv4(x)
        x = self.BatchNorm4(x)
        x = self.LeakyReLU(x)

        x = self.conv5(x)
        x = x.view(-1, self.ndf * 1)
        c = self.aux_linear(x)
        c = self.softmax(c)
        s = self.disc_linear(x)
        s = self.sigmoid(s)
        return s,c


# Create the Discriminator
netD = Discriminator(ngpu).to(device)
# Apply the weights_init function to randomly initialize all weights
netD.apply(weights_init)
# Print the model
print(netD)

# Training Loop

# Lists to keep track of progress
G_losses = []
D_losses = []
fake_img_list=[]
iters = 0

print("Starting Training Loop...")
for epoch in range(num_epochs):
    for i, data in enumerate(dataloader,0):
        # prepare real image and label
        real_label = data[1].to(device) 
        real_image = data[0].to(device) 
        b_size = real_label.size(0)    
        
        # prepare fake image and label
        fake_label = torch.randint(n_class, (b_size,), dtype=torch.long, device=device) 
        noise = torch.randn(b_size, nz, 1, 1, device=device).squeeze(0)
        noise = concat_noise_label(noise, real_label, device) 
        fake_image = netG(noise) 
        
        
        # target
        real_target = torch.full((b_size,), r_label, device=device, dtype=torch.float) 
        fake_target = torch.full((b_size,), f_label, device=device, dtype=torch.float) 

        real_target = real_target.view(-1,1)
        fake_target = fake_target.view(-1,1)
        
        #-----------------------
        # Update Discriminator
        #-----------------------
        netD.zero_grad()
        
        # train with real
        s_output, c_output = netD(real_image) 
        s_errD_real = s_criterion(s_output, real_target)  
        c_errD_real = c_criterion(c_output, real_label)  
        errD_real = s_errD_real + c_errD_real
        errD_real.backward()
        D_x = s_output.data.mean()


        # train with fake
        s_output,c_output = netD(fake_image.detach())
        s_errD_fake = s_criterion(s_output, fake_target)  # realfake
        c_errD_fake = c_criterion(c_output, real_label)  # class
        errD_fake = s_errD_fake + c_errD_fake
        errD_fake.backward()
        D_G_z1 = s_output.data.mean()
        
        errD = s_errD_real + s_errD_fake
        optimizerD.step()        

        #-----------------------
        # Update Generator
        #-----------------------
        netG.zero_grad()
        
        s_output,c_output = netD(fake_image)
        s_errG = s_criterion(s_output, real_target)  # realfake
        c_errG = c_criterion(c_output, real_label)  # class
        errG = s_errG + c_errG
        errG.backward()
        D_G_z2 = s_output.data.mean()
        
        optimizerG.step()

        # Save Losses for plotting later
        G_losses.append(errG.item())
        D_losses.append(errD.item())

        
        iters += 1

    # scheduler.step(errD.item())

  
    
    print('[%d/%d][%d/%d]\nLoss_D: %.4f\tLoss_G: %.4f\nD(x): %.4f\tD(G(z)): %.4f / %.4f'
          % (epoch+1, num_epochs, i+1, len(dataloader),
             errD.item(), errG.item(), D_x, D_G_z1, D_G_z2))
    
    show_generated_img(num_show)
  
   
    # --------- save fake image  ----------
    fake_image = netG(fixed_noise_label) 
    vutils.save_image(fake_image.detach(), '{}/fake_samples_epoch_{:03d}.png'.format(outf, epoch + 1),
                    normalize=True, nrow=5)
    fake_img_list.append(fake_image) 
  

</code></pre>
</blockquote>
</aside>
<p>Thank you for your response.<br>
I’m working with acgan to generate image from noise.I am trying to save the images of mine as per the size of input image(299x299).For giving image_size =64 it works perfectly but when it’s changed the training part gives error.</p>

ptrblck: <p>Could you change this line of code:</p>
<pre><code class="lang-python">x = x.view(-1, self.ndf * 1)
</code></pre>
<p>to</p>
<pre><code class="lang-python">x = x.view(x.size(0), -1)
</code></pre>
<p>and check, if you would get a shape mismatch?<br>
The latter approach makes sure the batch dimension stays constant, while the latter one could push additional dimensions to the batch dim.</p>

sabrina00: <p>After changing</p>
<pre><code class="lang-auto">x = x.view(-1, self.ndf * 1)
</code></pre>
<p>to</p>
<pre><code class="lang-auto">x = x.view(x.size(0), -1)
</code></pre>
<p>it’s showing → “RuntimeError: mat1 dim 1 must match mat2 dim 0” error.</p>

ptrblck: <p>This new error points to the expected shape mismatch in the activation and the next layer (I assume it’s <code>self.aux_linear</code> and <code>self.disc_linear</code>).<br>
You can print the shape of <code>x</code> after the proper flattening and set the <code>in_features</code> of both linear layers to this value.</p>

sabrina00: <p><div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/2/f/2f7097a372222f2e2914fd3e770021cf98e118f1.png" data-download-href="https://discuss.pytorch.org/uploads/default/2f7097a372222f2e2914fd3e770021cf98e118f1" title="Screenshot (685)"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/2/f/2f7097a372222f2e2914fd3e770021cf98e118f1_2_690x388.png" alt="Screenshot (685)" data-base62-sha1="6LFFtX0Zo6Ywe0pXdO4SYLT7DB7" width="690" height="388" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/2/f/2f7097a372222f2e2914fd3e770021cf98e118f1_2_690x388.png, https://discuss.pytorch.org/uploads/default/optimized/3X/2/f/2f7097a372222f2e2914fd3e770021cf98e118f1_2_1035x582.png 1.5x, https://discuss.pytorch.org/uploads/default/optimized/3X/2/f/2f7097a372222f2e2914fd3e770021cf98e118f1_2_1380x776.png 2x" data-dominant-color="F3F4F4"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">Screenshot (685)</span><span class="informations">1920×1080 236 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>
<p>why it’s not printing the shape of anything else?</p>
<p>``` def onehot_encode(label, device, n_class=n_class):<br>
eye = torch.eye(n_class, device=device)<br>
return eye[label].view(-1, n_class, 1, 1)</p>
<p>def concat_image_label(image, label, device, n_class=n_class):<br>
B, C, H, W = image.shape<br>
oh_label = onehot_encode(label, device=device)<br>
oh_label = oh_label.expand(B, n_class, H, W)<br>
return torch.cat((image, oh_label), dim=1)</p>
<p>def concat_noise_label(noise, label, device):<br>
oh_label = onehot_encode(label, device=device)<br>
return torch.cat((noise, oh_label), dim=1) ```</p>
<p>as well is this causing any issue???</p>

ptrblck: <p>I don’t know why the print isn’t working, as I cannot see the error message in your screenshot.<br>
You could post an executable code snippet by wrapping in into three backticks ```, in case you get stuck.<br>
This would allow us to copy-paste the code and debug it easily, which is not possible with unformatted source code or screenshots.</p>

sabrina00: <aside class="quote no-group" data-username="inside quote">
<div class="title">
<div class="quote-controls"></div>
 inside quote:</div>
<blockquote>
<pre><code class="lang-auto"># from __future__ import print_function
# #%matplotlib inline
# import argparse
import os
import PIL
from PIL import Image
# import glob
# import xml.etree.ElementTree as ET
import random
import torch
import torch.nn as nn
# import torch.nn.parallel
# import torch.backends.cudnn as cudnn
import torch.optim as optim
import torch.utils.data
import torchvision.datasets as dset
import torchvision.transforms as transforms
import torchvision.transforms.functional as tf
import torchvision.utils as vutils
import numpy as np
import matplotlib.pyplot as plt
# import matplotlib.animation as animation
# import seaborn as sns
# from IPython.display import HTML
from torchvision.utils import save_image
# from torch.optim.lr_scheduler import StepLR, ReduceLROnPlateau, CosineAnnealingLR
# from tqdm import tqdm_notebook as tqdm
# from IPython.display import clear_output
# from scipy.stats import truncnorm
%matplotlib inline
# plt.rcParams['image.interpolation'] = 'nearest'


dataloader = torch.utils.data.DataLoader(dataset, batch_size=batch_size,shuffle=True, num_workers=workers)

# Always good to check if gpu support available or not
# Decide which device we want to run on
device = torch.device("cuda:0" if (torch.cuda.is_available() and ngpu &gt; 0) else "cpu")
print(device)


# visualization batch image
real_batch = iter(dataloader).next()
plt.figure(figsize=(25,15)) #img size 25x15(hxw)
plt.axis("off")
plt.title("Training Images")
image = np.transpose(vutils.make_grid(real_batch[0].to(device)[:4], normalize=True).cpu(),axes=(1,2,0)) #4 img show
plt.imshow(image)
# image = np.transpose(vutils.make_grid(real_batch[0].to(device), normalize=True).cpu(),axes=(1,2,0))
# plt.imshow(image)


def weights_init(m):
    classname = m.__class__.__name__
    if classname.find('Conv') != -1:
        nn.init.normal_(m.weight.data, 0.0, 0.02)
    elif classname.find('BatchNorm') != -1:
        nn.init.normal_(m.weight.data, 1.0, 0.02)
        nn.init.constant_(m.bias.data, 0)


class Generator(nn.Module):

    def __init__(self, ngpu, nz=nz, ngf=ngf, nc=nc, n_class=n_class):

        super(Generator, self).__init__()
        self.ngpu = ngpu
        self.ReLU = nn.ReLU(True)
        self.Tanh = nn.Tanh()
        self.conv1 = nn.ConvTranspose2d(nz+n_class, ngf * 8, 4, 1, 0, bias=False)
        self.BatchNorm1 = nn.BatchNorm2d(ngf * 8)

        self.conv2 = nn.ConvTranspose2d(ngf * 8, ngf * 4, 4, 2, 1, bias=False)
        self.BatchNorm2 = nn.BatchNorm2d(ngf * 4)

        self.conv3 = nn.ConvTranspose2d(ngf * 4, ngf * 2, 4, 2, 1, bias=False)
        self.BatchNorm3 = nn.BatchNorm2d(ngf * 2)

        self.conv4 = nn.ConvTranspose2d(ngf * 2, ngf * 1, 4, 2, 1, bias=False)
        self.BatchNorm4 = nn.BatchNorm2d(ngf * 1)

        self.conv5 = nn.ConvTranspose2d(ngf * 1, nc, 4, 2, 1, bias=False)

        self.apply(weights_init)


    def forward(self, input):

        x = self.conv1(input)
        x = self.BatchNorm1(x)
        x = self.ReLU(x)

        x = self.conv2(x)
        x = self.BatchNorm2(x)
        x = self.ReLU(x)

        x = self.conv3(x)
        x = self.BatchNorm3(x)
        x = self.ReLU(x)

        x = self.conv4(x)
        x = self.BatchNorm4(x)
        x = self.ReLU(x)

        x = self.conv5(x)
        output = self.Tanh(x)
        return output

# Create the generator
netG = Generator(ngpu).to(device)
# Apply the weights_init function to randomly initialize all weights
netG.apply(weights_init)
# Print the model
print(netG)


class Discriminator(nn.Module):

    def __init__(self, ngpu, ndf=ndf, nc=nc, n_class=n_class):

        super(Discriminator, self).__init__()
        self.LeakyReLU = nn.LeakyReLU(0.2, inplace=True)
        self.conv1 = nn.Conv2d(nc, ndf, 4, 2, 1, bias=False)
        self.conv2 = nn.Conv2d(ndf, ndf * 2, 4, 2, 1, bias=False)
        self.BatchNorm2 = nn.BatchNorm2d(ndf * 2)
        self.conv3 = nn.Conv2d(ndf * 2, ndf * 4, 4, 2, 1, bias=False)
        self.BatchNorm3 = nn.BatchNorm2d(ndf * 4)
        self.conv4 = nn.Conv2d(ndf * 4, ndf * 8, 4, 2, 1, bias=False)
        self.BatchNorm4 = nn.BatchNorm2d(ndf * 8)
        self.conv5 = nn.Conv2d(ndf * 8, ndf * 1, 4, 1, 0, bias=False)
        self.disc_linear = nn.Linear(ndf * 1, 1)
        self.aux_linear = nn.Linear(ndf * 1, n_class)
        self.softmax = nn.Softmax()
        self.sigmoid = nn.Sigmoid()
        self.ndf = ndf
        self.apply(weights_init)

    def forward(self, input):

        x = self.conv1(input)
        x = self.LeakyReLU(x)

        x = self.conv2(x)
        x = self.BatchNorm2(x)
        x = self.LeakyReLU(x)

        x = self.conv3(x)
        x = self.BatchNorm3(x)
        x = self.LeakyReLU(x)

        x = self.conv4(x)
        x = self.BatchNorm4(x)
        x = self.LeakyReLU(x)

        x = self.conv5(x)
        # x = x.view(-1, self.ndf * 1)
        x = x.view(x.size(0), -1) #It can handle any batch size
        c = self.aux_linear(x)
        c = self.softmax(c)
        s = self.disc_linear(x)
        s = self.sigmoid(s)
        return s,c

# Create the Discriminator
netD = Discriminator(ngpu).to(device)
# Apply the weights_init function to randomly initialize all weights
netD.apply(weights_init)
# Print the model
print(netD)

# Setup Adam optimizers for both G and D
optimizerD = optim.Adam(netD.parameters(), lr=lr, betas=(beta1, 0.999))
optimizerG = optim.Adam(netG.parameters(), lr=lr, betas=(beta1, 0.999))

def onehot_encode(label, device, n_class=n_class):  
    eye = torch.eye(n_class, device=device) 
    return eye[label].view(-1, n_class, 1, 1)   
 
def concat_image_label(image, label, device, n_class=n_class):
    B, C, H, W = image.shape   
    oh_label = onehot_encode(label, device=device)
    oh_label = oh_label.expand(B, n_class, H, W)
    return torch.cat((image, oh_label), dim=1)
 
def concat_noise_label(noise, label, device):
    oh_label = onehot_encode(label, device=device)
    return torch.cat((noise, oh_label), dim=1)

def show_generated_img(num_show):
    gen_images = []
    for _ in range(num_show):
        noise = torch.randn(1, nz, 1, 1, device=device) #4
        covid_label = torch.randint(0, n_class, (1, ), device=device)
        gen_image = concat_noise_label(noise, covid_label, device)
        gen_image = netG(gen_image).to("cpu").clone().detach().squeeze(0)
        # gen_image = gen_image.numpy().transpose(0, 2, 3, 1)
        gen_image = gen_image.numpy().transpose(1, 2, 0)
        gen_images.append(gen_image)
        
    fig = plt.figure(figsize=(10, 5))
    # fig = plt.figure(figsize=(299,299))
    for i, gen_image in enumerate(gen_images):
        ax = fig.add_subplot(1, num_show, i + 1, xticks=[], yticks=[])
        plt.imshow(gen_image + 1 / 2)
        print(gen_image.shape)
        # plt.imshow(gen_image + 1 / 2, cmap="gray_r")
    plt.show()
    
# Training Loop

# Lists to keep track of progress
G_losses = []
D_losses = []
fake_img_list=[]
iters = 0

print("Starting Training Loop...")
for epoch in range(num_epochs):
    for i, data in enumerate(dataloader,0):
        # prepare real image and label
        real_label = data[1].to(device) 
        real_image = data[0].to(device) 
        b_size = real_label.size(0)    
        
        # prepare fake image and label
        fake_label = torch.randint(n_class, (b_size,), dtype=torch.long, device=device) 
        noise = torch.randn(b_size, nz, 1, 1, device=device).squeeze(0)
        noise = concat_noise_label(noise, real_label, device) 
        fake_image = netG(noise) 
        
        
        # target
        real_target = torch.full((b_size,), r_label, device=device, dtype=torch.float) 
        fake_target = torch.full((b_size,), f_label, device=device, dtype=torch.float) 

        real_target = real_target.view(-1,1)
        fake_target = fake_target.view(-1,1)
        
        #-----------------------
        # Update Discriminator
        #-----------------------
        netD.zero_grad()
        
        # train with real
        s_output, c_output = netD(real_image) 
        s_errD_real = s_criterion(s_output, real_target)  
        c_errD_real = c_criterion(c_output, real_label)  
        errD_real = s_errD_real + c_errD_real
        errD_real.backward()
        D_x = s_output.data.mean()


        # train with fake
        s_output,c_output = netD(fake_image.detach())
        s_errD_fake = s_criterion(s_output, fake_target)  # realfake
        c_errD_fake = c_criterion(c_output, real_label)  # class
        errD_fake = s_errD_fake + c_errD_fake
        errD_fake.backward()
        D_G_z1 = s_output.data.mean()
        
        errD = s_errD_real + s_errD_fake
        optimizerD.step()        

        #-----------------------
        # Update Generator
        #-----------------------
        netG.zero_grad()
        
        s_output,c_output = netD(fake_image)
        s_errG = s_criterion(s_output, real_target)  # realfake
        c_errG = c_criterion(c_output, real_label)  # class
        errG = s_errG + c_errG
        errG.backward()
        D_G_z2 = s_output.data.mean()
        
        optimizerG.step()

        # Save Losses for plotting later
        G_losses.append(errG.item())
        D_losses.append(errD.item())

        
        iters += 1

    # scheduler.step(errD.item())

  
    
    print('[%d/%d][%d/%d]\nLoss_D: %.4f\tLoss_G: %.4f\nD(x): %.4f\tD(G(z)): %.4f / %.4f'
          % (epoch+1, num_epochs, i+1, len(dataloader),
             errD.item(), errG.item(), D_x, D_G_z1, D_G_z2))
    
    show_generated_img(num_show)
  
   
    # --------- save fake image  ----------
    fake_image = netG(fixed_noise_label) 
    vutils.save_image(fake_image.detach(), '{}/fake_samples_epoch_{:03d}.png'.format(outf, epoch + 1),
                    normalize=True, nrow=5)
    fake_img_list.append(fake_image) 
  

</code></pre>
<p>I have provided you the full code as well the drive link to access dataset.would you kindly check it?i am new in this platform as well in data science so i am having too many trouble to deal with these.sorry for taking your times and thank you so much for your every response.</p>
</blockquote>
</aside>

ptrblck: <p>Thanks for the code.<br>
The printing works as expected and shows that the <code>x</code> activation has a shape of <code>torch.Size([32, 67275])</code> before being passed to <code>self.aux_linear</code> and <code>self.disc_linear</code>.<br>
However, changing the <code>in_features</code> to this value won’t solve the issue, since you are using different input shapes for <code>netD</code>. While <code>real_image</code> has a shape of <code>[batch_size, 3, 299, 299]</code>, <code>fake_image</code> is <code>[batch_size, 3, 64, 64]</code> (which would work for <code>in_features=299</code> as was originally set), so you would have to resize the inputs or use an adaptive pooling layer internally and make sure the activation has the same size.</p>

sabrina00: <p>Thank you for your continuous support <img src="https://discuss.pytorch.org/images/emoji/apple/blush.png?v=9" title=":blush:" class="emoji" alt=":blush:"></p>

sabrina00: <p><div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/b/a/ba60fab191cedab44a3921d29811ec67c8d60611.png" data-download-href="https://discuss.pytorch.org/uploads/default/ba60fab191cedab44a3921d29811ec67c8d60611" title="Screenshot (699)"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/b/a/ba60fab191cedab44a3921d29811ec67c8d60611_2_690x388.png" alt="Screenshot (699)" data-base62-sha1="qAMAn1FwTNqy5dx6O5LUcCO28Uh" width="690" height="388" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/b/a/ba60fab191cedab44a3921d29811ec67c8d60611_2_690x388.png, https://discuss.pytorch.org/uploads/default/optimized/3X/b/a/ba60fab191cedab44a3921d29811ec67c8d60611_2_1035x582.png 1.5x, https://discuss.pytorch.org/uploads/default/optimized/3X/b/a/ba60fab191cedab44a3921d29811ec67c8d60611_2_1380x776.png 2x" data-dominant-color="F6F7F7"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">Screenshot (699)</span><span class="informations">1920×1080 264 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div><br>
I am have updated for my genrator and discriminator and as per the output you may see that both works properly but now i am getting " RuntimeError: CUDA out of memory. Tried to allocate 2.00 GiB (GPU 0; 14.76 GiB total capacity; 12.20 GiB already allocated; 523.75 MiB free; 13.21 GiB reserved in total by PyTorch) " error.It’s mentioned that i have taken image_size=256,ndf and ngf=256,nz=128,n_classes=120…I have reduced batch_size 32 to 8 but reducing batch size haven’t give any benefit.What this error is for?</p>

ptrblck: <p>The error shows that your are running out of memory, so you would need to either reduce the batch size further or try to save memory by e.g. lowering the memory footprint in the overall training using smaller models or by trading compute for memory via <code>torch.utils.checkpoint</code>.</p>

sabrina00: <p>I used batch size 8.Would I reduce more?</p>

diogo_tavares: <p>Hi <a class="mention" href="/u/ptrblck">@ptrblck</a> i wonder if you could clarify me on something. I’m running this bidirectional LSTM which i feed with dataloaders and loss is reducing but the accuracy remains the same. I feel like i’m doing something wrong on the preds variable when i call the softmax function. The code is as below.</p>
<p>class BiLSTM(nn.Module):</p>
<pre><code>def __init__(self, input_dim, hidden_dim, layer_dim, output_dim):
    super().__init__()
    self.input_dim = input_dim
    self.hidden_dim = hidden_dim
    self.layer_dim = layer_dim
    self.lstm = nn.LSTM(input_dim, hidden_dim, layer_dim, dropout=0.2, bidirectional=True, batch_first=True) 
    self.fc_1 =  nn.Linear(hidden_dim*2, 128) #fully connected 1  
    self.fc = nn.Linear(hidden_dim, output_dim) #fully connected last layer
    self.relu = nn.ReLU()  #activation function
       
def forward(self,x):
    h_0 = torch.zeros(self.layer_dim*2, x.size(0), self.hidden_dim).requires_grad_() #hidden state
    c_0 = torch.zeros(self.layer_dim*2, x.size(0), self.hidden_dim).requires_grad_() #internal state
    # Propagate input through LSTM
    output, (hn, cn) = self.lstm(x, (h_0, c_0)) #lstm with input, hidden, and internal state
    output = output[:,-1,:]
    out = self.relu(output)
    out = self.fc_1(out) #first Dense
    out = self.relu(out) #relu
    out = self.fc(out) #Final Output
    #sigmoid
    out = torch.sigmoid(out)
    #out = F.log_softmax(out, dim=1).argmax(dim=1)
    
    return out
</code></pre>
<p>input_dim = 42<br>
hidden_dim = 128<br>
layer_dim = 2<br>
output_dim = 1</p>
<p>lr = 0.001<br>
n_epochs = 100<br>
iterations_per_epoch = len(trn_ds)<br>
best_acc = 0<br>
patience, trials = 10, 0</p>
<p>model = BiLSTM(input_dim, hidden_dim, layer_dim, output_dim)<br>
model = model.train()<br>
criterion = nn.BCELoss()<br>
opt = torch.optim.Adam(model.parameters(), lr=lr)</p>
<p>print(‘Start model training’)</p>
<p>for epoch in range(1, n_epochs + 1):</p>
<pre><code>for i, (x_batch, y_batch) in enumerate(trn_dl):
    y_batch = y_batch.unsqueeze(1)
    y_batch = y_batch.float()
    out = model(x_batch)
    loss = criterion(out.float(), y_batch)
    #loss.requires_grad = True
    opt.zero_grad()
    loss.backward()
    opt.step()

model.eval()
correct, total = 0, 0
for x_val, y_val in val_dl:
    x_val, y_val = [t.cpu() for t in (x_val, y_val)]
    out = model(x_val)
    preds = F.log_softmax(out, dim=1).argmax(dim=1)
    total += y_val.size(0)
    correct += (preds == y_val).sum().item()

acc = correct / total
</code></pre>
<p><span class="hashtag">#if</span> epoch % 5 == 0:<br>
print(f’Epoch: {epoch:3d}. Loss: {loss.item():.4f}. Acc.: {acc:2.2%}’)</p>
<pre><code>if acc &gt; best_acc:
    trials = 0
    best_acc = acc
    torch.save(model.state_dict(), 'bestSimpleLSTM.pth')
    print(f'Epoch {epoch} best model saved with accuracy: {best_acc:2.2%}')
else:
    trials += 1
    if trials &gt;= patience:
        print(f'Early stopping on epoch {epoch}')
        break
</code></pre>
<p>Thank you.</p>

ptrblck: <p><code>torch.argmax(output, dim=1)</code> is usually used on outputs containing the logits (or probabilities) with <code>nb_classes</code> in <code>dim1</code> (i.e. in the shape <code>[batch_size, nb_classes]</code>), which would be the case for a multi-class classification using <code>nn.CrossEntropyLoss</code>.<br>
In your case you are using a single output unit, apply <code>sigmoid</code> on it, and use <code>nn.BCELoss</code> as the criterion, which points towards a binary classification (small advice: remove the <code>sigmoid</code> and use <code>nn.BCEWithLogitsLoss</code> for more numerical stability).<br>
In that case you should use a threshold to get the predicated class labels, e.g. via:</p>
<pre><code class="lang-python">preds = out &gt; 0.0 # if out contains logits
preds = torch.sigmoid(out) &gt; 0.5 # since out now contains probabilities
</code></pre>