Linear Variational Autoencoder - Not Learning? Help!

username: grsilva
cooked: <p>Hello, PyTorch community.</p>
<p>For the past few weeks I have been struggling real hard to make a piece of code work, something I supposedly thought to be a trivial task. Here it is:</p>
<p>I. The task.</p>
<ol>
<li>Download some Bitcoin data and pre-process it. (Transform closing prices into returns).</li>
<li>Slice the BTC data set in windows of length 100.</li>
<li>Define a VAE architecture, so I could use the decoder to simulate time series.</li>
</ol>
<p>II. The problem.</p>
<p>First I started with LSTM and GRU layers. Somehow, I wasn´t sure if my model was learning something at all since it seemed the results were flattening, no matter how much I tweaked the hyperparameters. Then before going for other more sophisticated layers (Conv1d), I decided just to go for Linear layers (the code I present below), just to test if it is able to work.</p>
<p>Somehow, the output of the decoder part does indeed look somehow  a time series. However, the scale is different from the input (similar to: <a href="https://discuss.pytorch.org/t/the-network-output-matches-the-target-curve-but-with-a-different-scale/30576/15" class="inline-onebox">The network output matches the target curve but with a different scale - #15 by vmirly1</a>).</p>
<p>I tried: Increasing the number of epochs, adding more layers (linear transformations), changing learning rate, etc, etc. The results are the same. It looks like a time series, but in a very different scale.</p>
<p>Definitely, I am new to PyTorch, I was wondering if someone could please shed a light in this matter because cause literally I don´t know what else to try. I am uploading my Notebook. I would really appreciate if someone could take a look!<br>
<div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/c/c/cc1af7de020dc61a426111207aa2a48e1a72cf21.png" data-download-href="https://discuss.pytorch.org/uploads/default/cc1af7de020dc61a426111207aa2a48e1a72cf21" title="Train_Vali_Loss"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/c/c/cc1af7de020dc61a426111207aa2a48e1a72cf21_2_690x378.png" alt="Train_Vali_Loss" data-base62-sha1="t7BblZz9rU1C6o7h7rd03hIbmfv" width="690" height="378" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/c/c/cc1af7de020dc61a426111207aa2a48e1a72cf21_2_690x378.png, https://discuss.pytorch.org/uploads/default/optimized/3X/c/c/cc1af7de020dc61a426111207aa2a48e1a72cf21_2_1035x567.png 1.5x, https://discuss.pytorch.org/uploads/default/original/3X/c/c/cc1af7de020dc61a426111207aa2a48e1a72cf21.png 2x" data-dominant-color="FDFDFD"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">Train_Vali_Loss</span><span class="informations">1203×660 36 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div><br>
<img src="https://discuss.pytorch.org/uploads/default/original/3X/7/6/76170e3bc61d8d82ca8ca958d18d05d62f671d3b.png" alt="Output_BTC" data-base62-sha1="gQFJKvXJHJVPJHPRItKJ59GJIrF" width="615" height="342"><br>
<img src="https://discuss.pytorch.org/uploads/default/original/3X/7/0/70c6d9be96616817796cf8baa6fe8bccc9b41bcf.png" alt="Norm BTC" data-base62-sha1="g5FvKm99Ztsw0MFTuUTXBz6FZdZ" width="614" height="347"></p>
<p>Colab Link: <a href="https://colab.research.google.com/drive/1ieJPbLrHnscJCP-BrPvtYd6F4PkHDyxu?usp=sharing" class="inline-onebox" rel="noopener nofollow ugc">Google Colab</a></p>
<p>I thank you for your time!</p>