Best Network for 1-D data

lrpy_trainer: <p>Hello,<br>
I am trying to train a production network, where the agent has to decide on 3 actions:</p>
<p>0-continue production mode<br>
1-exchange filter<br>
2-harvest production</p>
<p>I used a Linear network to process the state signal containing of:<br>
x - Product Mass - from 0 to 4<br>
y - side product (currently not important) from 1 to 3<br>
z - Filter level - from 1 to 0.2<br>
d - Time after production start - bounded to 9 days<br>
t - overall time - max 183 days</p>
<p>So basically you have to continue production for some days, until you reach the perfect harvest point, where you get the reward for the accumlated product mass.<br>
Starting state is : (0,1,1,0,0). After each action we have a state transition.<br>
When you harvest, you come back to initial state (0,1,z,0,t) and need to start a new production. When you harvest you get reward for the product and you have to use your filter, which will result in a decay.<br>
When you continue production, you can grow your product for one day.<br>
If the filter is below the 0.2 treshold, you should exchange the filter. Which will reset z to 1.</p>
<p>Should I just use the vector input? Or would a convolution make more sene? Why?<br>
I try to train the perfect production mode over the time horizon of 183 days.</p>
<p>I tried PPO and rainbow Reinforcement Learning implementations. Using Linear input with the size of the vector shape.</p>
<p>While the PPO learns to harvest at the best time, he did not manage to learn to exchange the filters efficiently. The rainbow implementations finds a local optimum after some time, which is way beow of the baseline.</p>
<p>My rainbow-implementation gets stucked on a local minimum. I am not sure, if it will find out of this. There should be reward in the range of 4,0e9</p>
<p><div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/2X/6/631ab74e8268cc253e8f34c2bc12cfda1b092ab4.png" data-download-href="https://discuss.pytorch.org/uploads/default/631ab74e8268cc253e8f34c2bc12cfda1b092ab4" title="grafik"><img src="https://discuss.pytorch.org/uploads/default/optimized/2X/6/631ab74e8268cc253e8f34c2bc12cfda1b092ab4_2_690x299.png" alt="grafik" data-base62-sha1="e8Ivwp43J82nMO0iJLQ8S9RsVVO" width="690" height="299" srcset="https://discuss.pytorch.org/uploads/default/optimized/2X/6/631ab74e8268cc253e8f34c2bc12cfda1b092ab4_2_690x299.png, https://discuss.pytorch.org/uploads/default/original/2X/6/631ab74e8268cc253e8f34c2bc12cfda1b092ab4.png 1.5x, https://discuss.pytorch.org/uploads/default/original/2X/6/631ab74e8268cc253e8f34c2bc12cfda1b092ab4.png 2x" data-dominant-color="EAF1F5"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">grafik</span><span class="informations">767×333 25.8 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>
<p>Any suggestions how to tackle the problem? I would appreciate every help <img src="https://discuss.pytorch.org/images/emoji/apple/slight_smile.png?v=12" title=":slight_smile:" class="emoji" alt=":slight_smile:" loading="lazy" width="20" height="20"></p>