Nsight profile shows many pthread_cond_wait during the first 20 iterations

username: GabbyChan
cooked: <p>Hi ! I have tried to profile the inference runtime of my model by looping the inference for 100 iterations (100 test samples).</p>
<p>However, I have come to notice the multiple <code>pthread_cond_wait</code> that shows during the first 20 (out of 100) iterations. My questions are:</p>
<ul>
<li>
<p>What could have caused the problem? I used only Pytorch and Pytorch Geometry package.</p>
</li>
<li>
<p>After 20 iterations, the <code>pthread_cond_wait</code> has disappear, should I trust the runtime after those 20 iterations?</p>
</li>
</ul>
<p>Here I also provide the following captured screen to show my problem.</p>
<p><div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/8/b/8b813417305418c76d625f93155502e5c203664c.png" data-download-href="https://discuss.pytorch.org/uploads/default/8b813417305418c76d625f93155502e5c203664c" title="Screenshot from 2021-05-02 23-10-27"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/8/b/8b813417305418c76d625f93155502e5c203664c_2_690x388.png" alt="Screenshot from 2021-05-02 23-10-27" data-base62-sha1="jU7bvkk7hPOyszxWUhDgQI89BJq" width="690" height="388" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/8/b/8b813417305418c76d625f93155502e5c203664c_2_690x388.png, https://discuss.pytorch.org/uploads/default/optimized/3X/8/b/8b813417305418c76d625f93155502e5c203664c_2_1035x582.png 1.5x, https://discuss.pytorch.org/uploads/default/optimized/3X/8/b/8b813417305418c76d625f93155502e5c203664c_2_1380x776.png 2x" data-dominant-color="D5D6D5"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">Screenshot from 2021-05-02 23-10-27</span><span class="informations">1920×1080 254 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>
<p>(Zoom in)<br>
<div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/1/d/1dad905351e76efbff13c3017ef6d95df8feb9f3.png" data-download-href="https://discuss.pytorch.org/uploads/default/1dad905351e76efbff13c3017ef6d95df8feb9f3" title="Screenshot from 2021-05-02 23-10-45"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/1/d/1dad905351e76efbff13c3017ef6d95df8feb9f3_2_690x388.png" alt="Screenshot from 2021-05-02 23-10-45" data-base62-sha1="4exHJcGvEdltbS7J3ch2BapmbVp" width="690" height="388" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/1/d/1dad905351e76efbff13c3017ef6d95df8feb9f3_2_690x388.png, https://discuss.pytorch.org/uploads/default/optimized/3X/1/d/1dad905351e76efbff13c3017ef6d95df8feb9f3_2_1035x582.png 1.5x, https://discuss.pytorch.org/uploads/default/optimized/3X/1/d/1dad905351e76efbff13c3017ef6d95df8feb9f3_2_1380x776.png 2x" data-dominant-color="D0D4D4"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">Screenshot from 2021-05-02 23-10-45</span><span class="informations">1920×1080 253 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>

username: PiotrDabkowski
cooked: <p>It is a TorchScript model? I found that the model is really slow during first ~10 iterations due to some optimisation passes. For my sequence model this was unacceptable as I was passing multiple different lengths, where each new length seemed to be causing optimisation to run.</p>
<p>Can you try:</p>
<pre><code class="lang-python">with torch.jit.optimized_execution(False):
  # call your model here.
</code></pre>

username: GabbyChan
cooked: <aside class="quote no-group" data-username="PiotrDabkowski" data-post="2" data-topic="120046">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/piotrdabkowski/48/2257_2.png" class="avatar"> PiotrDabkowski:</div>
<blockquote>
<p><code>with torch.jit.optimized_execution(False):</code></p>
</blockquote>
</aside>
<p>That solves the problem! Thank you !!! …</p>
<p>However, what does it means for <code>torch.jit.optimized_execution(False)</code> ?</p>