Chamge latent space dimenasion

marlonufrgs: <p>Hi every one.</p>
<p>I am trying to create an AE and insert a neuron after encode part. I can easily do it on the forward part by concatenating a new neuron in the encoded data and fitting it into decoder. However, the backward didn’r allow me to do it, given mismatching dismensions.</p>
<blockquote>
<h1>Creating a PyTorch class</h1>
<p>class AE(torch.nn.Module):<br>
def <strong>init</strong>(self):<br>
super().<strong>init</strong>()<br>
self.encoder = torch.nn.Sequential(<br>
torch.nn.Linear(9351, 128),<br>
torch.nn.Linear(128, 9)<br>
)</p>
<pre><code>    self.decoder = torch.nn.Sequential(
        torch.nn.Linear(10, 18),
        torch.nn.Linear(18, 9351),
        torch.nn.Sigmoid()
    )

def forward(self, x ,x2):
    encoded = self.encoder(x)
    encoded.data = torch.FloatTensor(np.concatenate((encoded.data, x2),axis = 0)       )
    decoded = self.decoder(encoded)
    return decoded
</code></pre>
<p>model = AE()</p>
<h1>Validation using MSE Loss function</h1>
<p>loss_function = torch.nn.MSELoss()</p>
<h1>Using an Adam Optimizer with lr = 0.1</h1>
<p>optimizer = torch.optim.Adam(model.parameters(), lr = 1e-1, weight_decay = 1e-8)</p>
<p>epochs = 10<br>
outputs = <span class="chcklst-box fa fa-square-o fa-fw"></span><br>
losses = <span class="chcklst-box fa fa-square-o fa-fw"></span><br>
torch.autograd.set_detect_anomaly(True)</p>
<p>for epoch in range(epochs):<br>
for i,image in enumerate(loader):<br>
x2 = [temp[i]]<br>
reconstructed = model(image,x2)<br>
optimizer.zero_grad()<br>
optimizer.step()<br>
loss = loss_function(reconstructed, image)<br>
loss.backward(retain_graph=True,inputs = reconstructed)<br>
losses.append(loss)</p>
</blockquote>
<p>RuntimeError: Function UnsqueezeBackward0 returned an invalid gradient at index 0 - got [10] but expected shape compatible with [9]</p>
<p>any help will be very appreciated!</p>

ptrblck: <p>Manipulating the internal <code>.data</code> attribute is deprecated and could raise these kind of errors:</p>
<pre><code class="lang-python">encoded.data = torch.FloatTensor(np.concatenate((encoded.data, x2),axis = 0))
</code></pre>
<p>as it would skip Autograd checks.<br>
Could you explain why you want to assign this new tensor to the activation instead of creating a new tensor?</p>

marlonufrgs: <p>Our data is a signal (function of temperature), and we would like to have a neuron that keeps a fixed information (temperature). So we thought to add it after the encoder part. However, any better idea is welcome.</p>

ptrblck: <p>Use <code>torch.cat</code> to create the tensor and remove the usage of the <code>.data</code> attribute.</p>

marlonufrgs: <p>Thank you, i managed to concatenate using cat in the forward, but i still have the problem in the backward <img src="https://discuss.pytorch.org/images/emoji/apple/frowning.png?v=12" title=":frowning:" class="emoji" alt=":frowning:" loading="lazy" width="20" height="20"><br>
RuntimeError: Function UnsqueezeBackward0 returned an invalid gradient at index 0 - got [10] but expected shape compatible with [9]</p>

ptrblck: <p>I can reproduce the issue using your code, but it works for me after applying my suggested fixes:</p>
<pre><code class="lang-python">encoded = torch.cat((encoded, x2), dim=1)
</code></pre>
<p>so I guess you are still using your old code?</p>

marlonufrgs: <p>Indeed. I was using:</p>
<pre><code class="lang-auto">encoded.data = torch.cat((encoded, x2), 0)
</code></pre>
<p>It worked fine now! Thank you.</p>