Pytorch seems very slow on CPU

ecolss: <p>I implemented a word2vec (skip-gram with ns) using pytorch, but it’s running much much slower than the <code>gensim</code> version of word2vec.<br>
<code>gensim</code> took like 2min to finish the training, whereas the pytorch version seems will take half a day though.</p>
<p>The setup of pytorch is on Macbook pro with only CPU, and I can see the CPU of the pytorch word2vec is 100% (should be only 1 core utilized).</p>
<p>Whereas <code>gensim</code> uses multiple processes to run the training, it’s using <code>numpy</code> and <code>cython</code> functions (release GIL).</p>
<p>So I wonder how could I optimize the pytorch implementation further to match the performance of <code>gensim</code> version?</p>

albanD: <p>Hi,</p>
<p>How did you install pytorch? If you install the provided binaries and don’t force it to run single threaded with <code>torch.set_num_threads()</code> or <code>OMP_NUM_THREADS</code> env variable, then it should use all available cores.</p>

ecolss: <p>Hey <a class="mention" href="/u/alband">@albanD</a>, thanks for your quick reply.</p>
<p>I install it via <code>conda install pytorch torchvision -c pytorch</code>.</p>
<p>And I didn’t set the threads by  <code>torch.set_num_threads()</code> or <code>OMP_NUM_THREADS</code>, so it should be the default value or setting.</p>
<p>BTW, <code>set_num_threads()</code> doesn’t work for me in any way, it’s always 1</p>
<pre><code class="lang-auto">In [2]: th.get_num_threads()
Out[2]: 1

In [3]: th.set_num_threads(2)

In [4]: th.get_num_threads()
Out[4]: 1
</code></pre>

albanD: <p>That would mean that your version of torch is compiled without multithreading support.<br>
Which OS are you using? Do you see the same behavior if you install from pip?</p>

ecolss: <p>Hey <a class="mention" href="/u/alband">@albanD</a><br>
macOS Catalina, version 10.15.6.</p>
<p>Yes, same behavior with pip, thought conda would be different, but conda turns out to be the same.</p>
<p>So how could I get multi-threading support?</p>

ecolss: <p>Hi <a class="mention" href="/u/ptrblck">@ptrblck</a>, could you please tell me how to get the multi-threading support for pytroch?</p>
<p>I’ve searched for a while, looks like I need to manually build and install from source?</p>

albanD: <p>Hi,</p>
<p>Sorry it took a bit of time to track it down.<br>
We always had that limitation for mac binaries.<br>
There has been some work towards this so you might want to try the nightly builds: <a href="https://github.com/pytorch/pytorch/issues/43036">https://github.com/pytorch/pytorch/issues/43036</a></p>

ecolss: <p>I forgot to leave a few notes here, just for the reference of other people (who might have the same doubts).</p>
<p>On macOS, here is the config of pytorch,</p>
<pre><code class="lang-auto">In [1]: print(th.__config__.parallel_info())
ATen/Parallel:
	at::get_num_threads() : 8
	at::get_num_interop_threads() : 8
OpenMP not found
Intel(R) Math Kernel Library Version 2019.0.4 Product Build 20190411 for Intel(R) 64 architecture applications
	mkl_get_max_threads() : 1
Intel(R) MKL-DNN v1.7.0 (Git Hash 7aed236906b1f7a05c0917e5257a1af05e9ff683)
std::thread::hardware_concurrency() : 16
Environment variables:
	OMP_NUM_THREADS : [not set]
	MKL_NUM_THREADS : [not set]
ATen parallel backend: native thread pool
</code></pre>
<p>OpenMP is not found for macOS pytorch, and by default <code>get_num_threads=8</code>.<br>
By setting <code>MKL_NUM_THREADS=16</code>, the <code>get_num_threads</code> can be changed as well, since macOS pytorch is using MKL.</p>
<p>So, by setting <code>MKL_NUM_THREADS</code>, more CPU could be utilized.</p>

sanyomn: <p>Did you find any solutions to your problem. In my case pyTorch implementation of word2vec training takes at least 5 hours whereas gensim word2vec training takes only a couple of seconds using the Brown dataset. I’m new to PyTorch but I found this much difference quite surprising. I searched a lot but could not find a satisfactory answer yet.</p>