How to save the hidden layer output after each epoch?

Aaron_Wang1: <p>Dear sir or madam,</p>
<p>I have a question about how to get the output of the hidden layer after each epoch because when I tried to do it under the callback, the last result will be covered by the newer one, hence I can only get the final result of the training process. Could you give me some idea about it?</p>

ptrblck: <p>I guess you are storing the output in e.g. a <code>list</code> and might replace it in each iteration.<br>
In this case, either <code>append</code> the new output to the <code>list</code> or use a new <code>key</code> in a <code>dict</code>.<br>
Let me know if these approaches would work or where you are stuck.</p>

Aaron_Wang1: <p>Hi ptrblck,</p>
<p>I was trying to save the Hidden layer output in the callback, but after that, I would like to calculate the gradient of the hidden layer output with respect to t, could you give me some idea to do this?</p>
<p>Thank you so much for your help!</p>
<p>Best,<br>
Aaron</p>

Aaron_Wang1: <p><div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/c/b/cb0678cc13ceb565f56b036b1a46045e2916fdd0.png" data-download-href="https://discuss.pytorch.org/uploads/default/cb0678cc13ceb565f56b036b1a46045e2916fdd0" title="image"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/c/b/cb0678cc13ceb565f56b036b1a46045e2916fdd0_2_690x300.png" alt="image" data-base62-sha1="sY2N80tWf6ARRGPOEfg18t5QfoQ" width="690" height="300" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/c/b/cb0678cc13ceb565f56b036b1a46045e2916fdd0_2_690x300.png, https://discuss.pytorch.org/uploads/default/optimized/3X/c/b/cb0678cc13ceb565f56b036b1a46045e2916fdd0_2_1035x450.png 1.5x, https://discuss.pytorch.org/uploads/default/optimized/3X/c/b/cb0678cc13ceb565f56b036b1a46045e2916fdd0_2_1380x600.png 2x" data-dominant-color="EFF0F0"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">image</span><span class="informations">3106×1354 295 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div><br>
This is the Callback I wrote, and I can only get the final result instead of the weights during the process.<br>
<div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/8/d/8d02debc6ab10ced09aee9c6292d91af74e549bf.png" data-download-href="https://discuss.pytorch.org/uploads/default/8d02debc6ab10ced09aee9c6292d91af74e549bf" title="image"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/8/d/8d02debc6ab10ced09aee9c6292d91af74e549bf_2_665x500.png" alt="image" data-base62-sha1="k7rthXsltOBgRiVXx5YceVszRN5" width="665" height="500" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/8/d/8d02debc6ab10ced09aee9c6292d91af74e549bf_2_665x500.png, https://discuss.pytorch.org/uploads/default/optimized/3X/8/d/8d02debc6ab10ced09aee9c6292d91af74e549bf_2_997x750.png 1.5x, https://discuss.pytorch.org/uploads/default/optimized/3X/8/d/8d02debc6ab10ced09aee9c6292d91af74e549bf_2_1330x1000.png 2x" data-dominant-color="E7E7E7"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">image</span><span class="informations">2182×1640 501 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>

Aaron_Wang1: <p>Hi ptrblck,</p>
<p>When I was trying to save the weights, the result only showed the final result of the updated weights. And below is my code and my result.<br>
<div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/5/6/564d03de3af371af2914e4e04680a6c71654d836.png" data-download-href="https://discuss.pytorch.org/uploads/default/564d03de3af371af2914e4e04680a6c71654d836" title="image"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/5/6/564d03de3af371af2914e4e04680a6c71654d836_2_530x500.png" alt="image" data-base62-sha1="cjs47TxwuRsjhlIJrJqX4SoP1v8" width="530" height="500" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/5/6/564d03de3af371af2914e4e04680a6c71654d836_2_530x500.png, https://discuss.pytorch.org/uploads/default/optimized/3X/5/6/564d03de3af371af2914e4e04680a6c71654d836_2_795x750.png 1.5x, https://discuss.pytorch.org/uploads/default/optimized/3X/5/6/564d03de3af371af2914e4e04680a6c71654d836_2_1060x1000.png 2x" data-dominant-color="E2E3E3"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">image</span><span class="informations">1792×1688 502 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div><br>
<div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/4/1/418ea0cb186b45f0314e5400e0c9329b080230b9.png" data-download-href="https://discuss.pytorch.org/uploads/default/418ea0cb186b45f0314e5400e0c9329b080230b9" title="image"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/4/1/418ea0cb186b45f0314e5400e0c9329b080230b9_2_690x359.png" alt="image" data-base62-sha1="9lWC7zcDuntH3WfbWhy4vdZn3Lj" width="690" height="359" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/4/1/418ea0cb186b45f0314e5400e0c9329b080230b9_2_690x359.png, https://discuss.pytorch.org/uploads/default/optimized/3X/4/1/418ea0cb186b45f0314e5400e0c9329b080230b9_2_1035x538.png 1.5x, https://discuss.pytorch.org/uploads/default/optimized/3X/4/1/418ea0cb186b45f0314e5400e0c9329b080230b9_2_1380x718.png 2x" data-dominant-color="EEEFEF"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">image</span><span class="informations">2612×1362 275 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>