3 networks and one loss

username: slavavs
cooked: <p>I want to connect 3 networks on the basis of one Loss.<br>
The first network is the main one.<br>
The second network takes data from the main input and places it (copies) into the additional input.<br>
The third network removes irrelevant data.<br>
This is necessary so that the network can remember important data for a long time.<br>
Who knows how such a network can be implemented?<br>
<div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/2X/4/4ec9711c5a3c028ff6029ffd93de47032adb86f6.png" data-download-href="https://discuss.pytorch.org/uploads/default/4ec9711c5a3c028ff6029ffd93de47032adb86f6" title="rnn8"><img src="https://discuss.pytorch.org/uploads/default/optimized/2X/4/4ec9711c5a3c028ff6029ffd93de47032adb86f6_2_690x487.png" alt="rnn8" data-base62-sha1="beYPrV926qCcYGfNljQkUEdbd5k" width="690" height="487" srcset="https://discuss.pytorch.org/uploads/default/optimized/2X/4/4ec9711c5a3c028ff6029ffd93de47032adb86f6_2_690x487.png, https://discuss.pytorch.org/uploads/default/optimized/2X/4/4ec9711c5a3c028ff6029ffd93de47032adb86f6_2_1035x730.png 1.5x, https://discuss.pytorch.org/uploads/default/optimized/2X/4/4ec9711c5a3c028ff6029ffd93de47032adb86f6_2_1380x974.png 2x" data-dominant-color="F4F6F6"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">rnn8</span><span class="informations">1626×1148 65.6 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>

username: alex.veuthey
cooked: <p>How do you go from having 50 outputs in the second network and having 10 additional inputs? I don’t really understand this aspect, and the use case in general. Could you give more details?</p>

username: slavavs
cooked: <p>I have 50 main inputs and 10 additional inputs. There are 60 inputs in total. 10 additional inputs are initialized with zeros. The first primary network uses all 60 inputs. The second network selects from 50 inputs and adds them to the additional inputs section. The third network removes from the section of additional inputs by writing zeros into them.</p>

username: slavavs
cooked: <p>Maybe it will be so clear<br>
<div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/2X/5/58072ac33f005153f5b86d00fd4980829425600f.png" data-download-href="https://discuss.pytorch.org/uploads/default/58072ac33f005153f5b86d00fd4980829425600f" title="rnn9"><img src="https://discuss.pytorch.org/uploads/default/original/2X/5/58072ac33f005153f5b86d00fd4980829425600f.png" alt="rnn9" data-base62-sha1="cyJn6IyFpK3JKrGqIUHgzRuXDOL" width="554" height="500" data-dominant-color="F1F0F0"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">rnn9</span><span class="informations">1142×1029 17.4 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>

username: slavavs
cooked: <p>I do not know how to make an independent area (fc2.2) in the fc2 layer.<br>
The first neuron is the main neuron that makes loss.<br>
The second neuron decides from which input to add data to fc2.2<br>
The third neuron decides what data to delete.<br>
The fc2.2 region is independent and the data in it depends on the neuron 2, 3.<br>
Neurons 1, 2, 3 are independent (this is not one layer), but they should see the loss function.<br>
<div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/2X/3/3c2907739217d8c4ca19f317dd0b74aa28edde72.png" data-download-href="https://discuss.pytorch.org/uploads/default/3c2907739217d8c4ca19f317dd0b74aa28edde72" title="rnn10"><img src="https://discuss.pytorch.org/uploads/default/original/2X/3/3c2907739217d8c4ca19f317dd0b74aa28edde72.png" alt="rnn10" data-base62-sha1="8Acygw0LdxqUFFfrp3gVHsWPe1Q" width="648" height="500" data-dominant-color="F3F4F3"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">rnn10</span><span class="informations">1163×896 20.2 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>