Heap size increase constantly when inference with new thread

username: bsting
cooked: <p>Heap size increases constantly when i tried to run the extract function in a new thread with the same instance of torch jit script module. I noticed that new heap allocated after the forward call. If commented out the forward call, no heap would be allocated. I am using CPU only with pytorch version 1.2.0. No extra heap allocated if running on single thread.<br>
Am i doing something wrong?</p>
<p>A new thread is created with the code below</p>
<blockquote>
<p>do{</p>
<blockquote>
<pre><code>std::thread th1(extract);
th1.join();
std::cin.get();
</code></pre>
</blockquote>
<p>} while (true);</p>
</blockquote>
<p><div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/9/7/9719fcb5b9f38c6801f40a641290ed6e6fe9542e.png" data-download-href="https://discuss.pytorch.org/uploads/default/9719fcb5b9f38c6801f40a641290ed6e6fe9542e" title="image"><img src="https://discuss.pytorch.org/uploads/default/original/3X/9/7/9719fcb5b9f38c6801f40a641290ed6e6fe9542e.png" alt="image" data-base62-sha1="lyHM42HjGfgFwOjPKmLearpuiOa" width="690" height="254" data-dominant-color="F9F9F9"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">image</span><span class="informations">1099×405 21.3 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>
<p><div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/1/7/17d75120620717d45caf4dde38cf5560c0d2e65b.png" data-download-href="https://discuss.pytorch.org/uploads/default/17d75120620717d45caf4dde38cf5560c0d2e65b" title="image"><img src="https://discuss.pytorch.org/uploads/default/original/3X/1/7/17d75120620717d45caf4dde38cf5560c0d2e65b.png" alt="image" data-base62-sha1="3oUimpgOspEaroHEDb6E2kGRnt1" width="690" height="256" data-dominant-color="E8ECF0"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">image</span><span class="informations">1229×457 44.2 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>

username: TomB
cooked: <p>(No experience loading models in C++ yet, so could be way off, but in case it takes a while for more knowledgeable people to reply)<br>
Have you tried ensuring your module is in eval mode. Looks like in C++ you can do <code>module.eval();</code> like in python. That behaviour would be consistent with forward creating a graph for backwards that never gets destroyed.<br>
It might also be that putting your module in eval mode in Python before exporting will carry over to C++, but not sure of that, think I saw something to that effect though.</p>

username: bsting
cooked: <p>Yes. module.eval() has been call prior to forward. Besides that, torch::NoGradGuard guard; is used and requires_grad(false) has also set to tensor</p>

username: bsting
cooked: <p>Found similar problem but seems like there is no answer yet <a href="https://github.com/pytorch/pytorch/issues/24237" rel="nofollow noopener">https://github.com/pytorch/pytorch/issues/24237</a></p>