Mismatching size of the input training tensor and target

username: pedram1
cooked: <p>Hello everyone</p>
<p>I am trying to train my 1d conv autoencoder. In part of training the NN,I faced with the below error:</p>
<p>//////////////////////////////////////////////////////////////////////////////////////////////////<br>
“Using a target size (torch.Size([2048, 1, 94])) that is different to the input size (torch.Size([2048, 1, 93])). This will likely lead to incorrect results due to broadcasting. Please ensure they have the same size.<br>
return F.mse_loss(input, target, reduction=self.reduction)”<br>
//////////////////////////////////////////////////////////////////////////////////</p>
<p>I am  quite sure that my input tensor has the right size. because just before running the training loop, when ever I checked the size of that it gave me the right answer.<br>
My input size is (2040,1,94) which 2040 is batch number in every singel training, 1 is the input channel size and 94 is length of my signal.</p>
<p>below is my training codes:</p>
<p><span class="hashtag">#training</span> loop<br>
​<br>
​<br>
for epoch in range(num_epochs):<br>
for data in dataloader:<br>
img = data</p>
<pre><code>    # ===================forward=====================
    output = model(img)
    loss = criterion(output, img)
    # ===================backward====================
    optimizer.zero_grad()
    loss.backward()
    optimizer.step()
</code></pre>
<p>​</p>

username: albanD
cooked: <p>If you change your code to the following, what does it print?</p>
<pre><code class="lang-auto">    output = model(img)
    print("out", output.size())
    print("img", img.size())
    loss = criterion(output,  img)
</code></pre>

username: pedram1
cooked: <aside class="quote no-group quote-modified" data-username="albanD" data-post="2" data-topic="58527">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/alband/48/215_2.png" class="avatar"> albanD:</div>
<blockquote>
<p>print(“out”, output.size()) print(“img”, img.size())</p>
</blockquote>
</aside>
<p>HI<br>
Thank you for your reply,<br>
I did that. below you could see the results:</p>
<p>out torch.Size([2048, 1, 93])<br>
img torch.Size([2048, 1, 94])</p>

username: alex.veuthey
cooked: <p>That looks like your model does not output the correct size… It could happen in many places, I’d recommend printing the shape of internal variables in the <code>forward</code> method of your network, so that you can pinpoint the exact location that introduces the error and debug that.</p>
<p>It would also help if you can share the code of your architecture! But it would be understandable if you can’t <img src="https://discuss.pytorch.org/images/emoji/apple/slight_smile.png?v=9" title=":slight_smile:" class="emoji" alt=":slight_smile:"></p>

username: pedram1
cooked: <p>Hi<br>
Thank you for your reply.<br>
My codes are not really something important <img src="https://discuss.pytorch.org/images/emoji/apple/slight_smile.png?v=9" title=":slight_smile:" class="emoji" alt=":slight_smile:"><br>
Below you could be able to see them all:<br>
//////////////////////////////////////////////////////////////////////////////////<br>
<span class="hashtag">#Autoencoder</span> and Autodecoder conv layer…<br>
class Autoencoder(nn.Module):</p>
<p>def <strong>init</strong>(self):<br>
super(Autoencoder,self).<strong>init</strong>()</p>
<pre><code>    self.encoder = nn.Sequential(
        nn.Conv1d(1,5,kernel_size=5, stride=2),
        nn.ReLU(True),
        nn.Conv1d(5,10,kernel_size=5, stride = 2),
        nn.ReLU(True),
        nn.Conv1d(10,20,kernel_size=5, stride = 2),
        nn.ReLU(True))
    self.decoder = nn.Sequential(          
        nn.ConvTranspose1d(20,10,kernel_size=5, stride = 2),
        nn.ReLU(True),
        nn.ConvTranspose1d(10,5,kernel_size=5, stride = 2),
        nn.ReLU(True),
        nn.ConvTranspose1d(5,1,kernel_size=5, stride = 2),
        nn.Sigmoid())
</code></pre>
<p>def forward(self,x):<br>
x = self.encoder(x)<br>
x = self.decoder(x)<br>
return x<br>
//////////////////////////////////////////////////////////////////<br>
num_epochs = 32<br>
batch_size = 2048<br>
learning_rate = 1e-3<br>
////////////////////////////////////////////////////////////<br>
model = Autoencoder().cpu()<br>
criterion = nn.MSELoss()<br>
optimizer = torch.optim.Adam(model.parameters(), lr=learning_rate,<br>
weight_decay=1e-5)<br>
dataloader = DataLoader(data_pixel2, batch_size=batch_size, shuffle=True)<br>
///////////////////////////////////////////////////////////////////////////////</p>
<p><span class="hashtag">#training</span> loop<br>
​<br>
​<br>
for epoch in range(num_epochs):<br>
for data in dataloader:<br>
img = data</p>
<pre><code>    # ===================forward=====================
    output = model(img)
    print("out", output.size())
    print("img", img.size())
    loss = criterion(output, img)
    # ===================backward====================
    optimizer.zero_grad()
    loss.backward()
    optimizer.step()
# ===================log========================
print('epoch [{}/{}], loss:{:.4f}'
      .format(epoch+1, num_epochs, loss.data[0]))
if epoch % 10 == 0:
    pic = to_img(output.cpu().data)
    save_image(pic, './dc_img/image_{}.png'.format(epoch))
</code></pre>
<p>///////////////////////////////////////////////</p>
<p>Regards,</p>

username: alex.veuthey
cooked: <p>I think I know where the error comes from…</p>
<p>Your convolutions have a stride of 2. This means roughly (I hope I’m right) that each conv layer divides your input size by 2 (and removes a few elements on the borders due to no padding by default in the conv layer).</p>
<p>By printing the shape of the output of each layer, which you can do for the encoder with the following (the same with the decoder):</p>
<pre><code class="lang-python">for module in self.encoder:
    x = module(x)
    print(x.shape)
</code></pre>
<p>You will see that the shape follows the following sequence: <code>[94, 45, 21, 9]</code> in the encoder, but the decoder gives <code>[9, 21, 45, 93]</code>.</p>
<p>Now, I’m not familiar with the autoencoder architecture, specifically with how you need to setup your input sizes and kernel sizes for correct downsizing and upsizing… So you’ll need to check some documentation on that, I’m afraid this is the furthest I can help you.</p>
<p>I’ve just tried quickly changing the kernel size of the outermost convolutions (so the first conv in the encoder and the last conv transpose of the decoder) to 6 instead of 5, and this resulted in an output size of 94, but I’m not sure this is what you want or if it’s even valid…</p>
<p>P.S: if you format your code properly (by inserting it in a code block enclosed in two groups of three backticks `), it would be much easier to read it and help you! Copy-pasting the code directly failed, because the <code>__init__</code> was changed to <strong>init</strong> due to markdown’s formatting shortcuts.</p>

username: pedram1
cooked: <p>HI<br>
Thank you for your clear explanation. My problem actually solved, however when I run the training loop on my network loss of my NN was about  1088392064.000000 …</p>
<p>Do you have any idea ???</p>

username: alex.veuthey
cooked: <p>Is that the training loss? Does it start at this point or does it increase and stay there? Does it explode after that point?</p>
<p>That can be many things, but try reducing the learning rate first, and see how that goes.</p>

username: pedram1
cooked: <p>Yes it is training loss as far as I understand.<br>
Below you can see my codes in order to calculate the loss:<br>
<span class="hashtag">#training</span> loop</p>
<p>for epoch in range(num_epochs):<br>
for data in dataloader:<br>
img = data<br>
img = Variable(img)</p>
<pre><code>output = model (img)
loss = criterion (output,img)

optimizer.zero_grad()
loss.backward()
optimizer.step()

if epoch%10 == 1:
  print ('\r Train Epoch : {}/{} \tLoss : {:.6f}'.format (epoch+1,num_epochs,loss))
</code></pre>
<p>I also wrote you down the loss of the last epoch :<br>
Train Epoch : 72/80 	Loss : 1088451712.000000<br>
Train Epoch : 72/80 	Loss : 1088443520.000000<br>
Train Epoch : 72/80 	Loss : 1088414208.000000<br>
Train Epoch : 72/80 	Loss : 1088391808.000000<br>
Train Epoch : 72/80 	Loss : 1088355584.000000<br>
Train Epoch : 72/80 	Loss : 1088334464.000000</p>

username: alex.veuthey
cooked: <p>I see.</p>
<p>Well the MSELoss can be quite large due to its nature, but this is still very high… It also depends on the nature of your problem (so your <code>X</code> and <code>y</code> values).</p>
<p>Also, does it start at the same values in the first epoch? This might indicate that your network is not learning anything.</p>

username: pedram1
cooked: <p>Hi Alex<br>
Sorry for late replying back.<br>
m.mmmm you are right.<br>
Actually I realized that I did not normalized my data. Do you have any idea that how I can do it by using data loader module ? I tried to use below code, however I don’t have good feeling about that. I believe that there might be a way of normalizing in pytorch rather than do it by for loops<br>
//////////////////////////////////////////////////////////////////////////////////////////////<br>
<span class="hashtag">#again</span> reshape the data file<br>
new_data = np.empty((256,256,94))<br>
for i in range (256) :<br>
for j in range (256) :<br>
for k in range (94) :</p>
<pre><code>  new_data[i,j,k] = (data[k,i,j]-(np.mean(data[:,i,j])))/((np.amax(data[:,i,j]))-(np.amin([data[:,i,j]])))
</code></pre>
<p>////////////////////////////////////////////////////////////////////////////////////////</p>
<p>Thank you</p>

username: alex.veuthey
cooked: <p>Indeed, this is far from efficient, and good thinking for the normalization, it slipped my mind!</p>
<p>I’m not sure if this is the right way for your kind of data (I’ve only worked on images), but in general, what you want to do is compute your statistics in advance, on the whole training set (do not include validation or test sets, as this would introduce some bias), and then normalize input data in the dataset (not the data loader!) as it gets called, using your previously computed stats.</p>