How to check if PyTorch is using BLAS?

username: HMEIatJHU
cooked: <p>In Numpy, there is a simple way to check which BLAS is being used by</p>
<pre><code class="lang-python">numpy.show_config()
</code></pre>
<p>Is there a similar way in PyTorch?</p>
<p>I am experiencing abnormally-slow performance of PyTorch-CPU on a remote server.<br>
And I suspect PyTorch is not using BLAS.<br>
So I am looking for ways to check:</p>
<ol>
<li>if PyTorch is using BLAS;</li>
<li>which BLAS</li>
</ol>
<p>Thanks in advance!</p>

username: SimonW
cooked: <p>The binaries are all built with MKL, as you can verify by printing <code>torch.__config__.show()</code>.</p>
<p>However, also be sure to check <code>torch.__config__.parallel_info()</code> as it could be that the number of threads is not properly set.</p>

username: HMEIatJHU
cooked: <p>Thanks very much! On torch-1.0.1.post2, it shows</p>
<pre><code class="lang-bash">AttributeError: module 'torch' has no attribute '__config__'
</code></pre>

username: SimonW
cooked: <p>Oh it could be a new thing added in 1.1.0.</p>

username: HMEIatJHU
cooked: <p>Thanks. Just updated to ‘1.1.0’ and it shows a lot of information.</p>
<p>Question-1: do they literally mean the libs that PyTorch are linked to and will be using in runtime?</p>
<p>Reason for question: I observe &gt;100 times slowdown on a remote server (where I am not an admin) than my person laptop, so I suspect it is not really using the resources it should and need a way to check that.</p>
<p>Question-2:</p>
<pre><code class="lang-bash">AttributeError: module 'torch.__config__' has no attribute 'parallel_info'?
</code></pre>

username: SimonW
cooked: <p>Ah my bad, I am using a nightly build. That function was probably added later. You can still check the number of threads using usual posix functionalities.</p>

username: HMEIatJHU
cooked: <p>Thanks all the same! I used torch.get_num_threads() to see # threads being used, and I found that was the cause of abnormal slow-down. When I set it back to OMP_NUM_THREADS=# cores, it came back to normal.</p>
<p>However, I was wondering:</p>
<ol>
<li>What is the best practice for setting OMP_NUM_THREADS? # physical cores or logical cores? or neither?</li>
<li>Where is the best place to set it? I have two options:<br>
2.1) set env vars OMP_NUM_THREADS when submitting jobs to cluster<br>
2.2) set via torch.set_num_threads()<br>
Not sure which way is better and why. Can you help?</li>
</ol>

username: SimonW
cooked: <aside class="quote no-group" data-username="HMEIatJHU" data-post="7" data-topic="50068">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/hmeiatjhu/48/13892_2.png" class="avatar"> HMEIatJHU:</div>
<blockquote>
<p>What is the best practice for setting OMP_NUM_THREADS? # physical cores or logical cores? or neither?</p>
</blockquote>
</aside>
<p>I usually try with different values between those two numbers. But I am not really an expert…</p>
<aside class="quote no-group" data-username="HMEIatJHU" data-post="7" data-topic="50068">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/hmeiatjhu/48/13892_2.png" class="avatar"> HMEIatJHU:</div>
<blockquote>
<p>Where is the best place to set it? I have two options:<br>
2.1) set env vars OMP_NUM_THREADS when submitting jobs to cluster<br>
2.2) set via torch.set_num_threads()<br>
Not sure which way is better and why. Can you</p>
</blockquote>
</aside>
<p>Use <code>torch.set_num_threads</code>. PyTorch uses OMP, MKL, and a native thread pool (as well as TBB maybe). The function takes care of all of them. Not sure if the env flag will set all of them.</p>

username: HMEIatJHU
cooked: <aside class="quote no-group quote-modified" data-username="SimonW" data-post="8" data-topic="50068" data-full="true">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/simonw/48/1702_2.png" class="avatar"> SimonW:</div>
<blockquote>
<aside class="quote no-group" data-username="HMEIatJHU" data-post="7" data-topic="50068">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/hmeiatjhu/48/13892_2.png" class="avatar"> HMEIatJHU:</div>
<blockquote>
<p>What is the best practice for setting OMP_NUM_THREADS? # physical cores or logical cores? or neither?</p>
</blockquote>
</aside>
<p>I usually try with different values between those two numbers. But I am not really an expert…</p>
</blockquote>
</aside>
<p>I tried many options and found smaller values tend to give good performance in my case.</p>
<aside class="quote no-group quote-modified" data-username="SimonW" data-post="8" data-topic="50068" data-full="true">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/simonw/48/1702_2.png" class="avatar"> SimonW:</div>
<blockquote>
<aside class="quote no-group" data-username="HMEIatJHU" data-post="7" data-topic="50068">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/hmeiatjhu/48/13892_2.png" class="avatar"> HMEIatJHU:</div>
<blockquote>
<p>Where is the best place to set it? I have two options:<br>
2.1) set env vars OMP_NUM_THREADS when submitting jobs to cluster<br>
2.2) set via torch.set_num_threads()<br>
Not sure which way is better and why. Can you</p>
</blockquote>
</aside>
<p>Use <code>torch.set_num_threads</code>. PyTorch uses OMP, MKL, and a native thread pool (as well as TBB maybe). The function takes care of all of them. Not sure if the env flag will set all of them.</p>
</blockquote>
</aside>
<p>Thanks for the advice!</p>