Cpu cores not working

jmaronas: <p>Hello.</p>
<p>I am running some experiments on pytorch with a titan xp. The problem is that pytorch only uses one core of CPU, even if I set n_workers=10 for example in a data loader. I installed pytorch with pip install and the version is 0.3.1.</p>
<p>lscpu gives:</p>
<p>Arquitectura:          <strong>x86_64</strong><br>
modo(s) de operación de las CPUs:32-bit, 64-bit<br>
Orden de bytes:        Little Endian<br>
CPU(s):                8<br>
On-line CPU(s) list:   0-7<br>
Hilo(s) de procesamiento por núcleo:2<br>
Núcleo(s) por «socket»:4<br>
Socket(s):             1<br>
Modo(s) NUMA:          1<br>
ID de fabricante:      GenuineIntel<br>
Familia de CPU:        6<br>
Modelo:                60<br>
Model name:            <strong>Intel® Core™ i7-4790 CPU @ 3.60GHz</strong></p>

string111: <p>Have you tried: <code>torch.set_num_threads(num_cores)</code>?</p>

jmaronas: <p>yes, I actually realized about the problem because one of the parameters of my code is how many cores to use and I use that function to set them. However when running top on terminal the process only get up to 100% cpu usage.</p>

string111: <p>Check <a href="https://discuss.pytorch.org/t/does-pytorch-support-parallelism-across-multiple-cpus/1631">this</a> out, I am not sure if PyTorch supports multiple CPU computation yet.</p>

albanD: <p>Hi,</p>
<p>It is possible that even with 10 workers for dataloading you only use one core. That depends what processing you’re doing in the dataloader? If you don’t do any fancy preprocessing, then the workers just need to load the data which barely take any cpu.<br>
If you’re performing heavy cpu computation (with large matrices) and you see a single core used, then you can use <a href="https://pytorch.org/docs/stable/torch.html?highlight=get_num_threads#torch.get_num_threads"><code>torch.get_num_threads()</code></a> to check how many threads are available for compute in pytorch.</p>
<p>In your particular case, why do you believe cpu usage should be greater? What is the GPU usage?</p>

jmaronas: <p>Well I put as example the n_workers but I refer to any parallelism that can be performed. In this case I do not use data preprocessing however computation is really heavy.</p>
<p>I made a checked training a convolutional variatonal autoencoder over mnist. I hoped that pytorch uses cpu parallelism at other level different than just dataloading.  Is it that true? I do not now if at C level there are loop unrolls or stuff like that.</p>
<p>torch.get_num_threads() give me numbers greater than one.</p>

albanD: <p>If you use the GPU, then no computations that really needs to be done by the cpu. You will see one core being used to run the python code + queue up cuda kernels but that’s it. Since python can use at most one thread at a time and cuda kernels launch are directed by python, only one core is used.</p>

jmaronas: <p>Well i cannot agree with you because sometimes the same code executed on other machine uses more than one core, Top output:</p>
<p>PID USUARIO   PR  NI    VIRT    RES    SHR S  <strong>%CPU</strong> %MEM     HORA+ ORDEN<br>
22854 jmarona+  20   0 30,861g 1,379g 261656 R <strong>279,0</strong>  8,9   2:32.98 python</p>
<p>Lscpu output:<br>
Arquitectura:          <strong>x86_64</strong><br>
modo(s) de operación de las CPUs:32-bit, 64-bit<br>
Orden de bytes:        Little Endian<br>
CPU(s):                4<br>
On-line CPU(s) list:   0-3<br>
Hilo(s) de procesamiento por núcleo:2<br>
Núcleo(s) por «socket»:2<br>
Socket(s):             1<br>
Modo(s) NUMA:          1<br>
ID de fabricante:      GenuineIntel<br>
Familia de CPU:        6<br>
Modelo:                158<br>
Model name:            <strong>Intel® Core™ i3-7100 CPU @ 3.90GHz</strong></p>
<p><strong>And in other machine:</strong><br>
lscpu output:<br>
Architecture:          <strong>x86_64</strong><br>
CPU op-mode(s):        32-bit, 64-bit<br>
Byte Order:            Little Endian<br>
CPU(s):                4<br>
On-line CPU(s) list:   0-3<br>
Thread(s) per core:    2<br>
Core(s) per socket:    2<br>
Socket(s):             1<br>
NUMA node(s):          1<br>
Vendor ID:             GenuineIntel<br>
CPU family:            6<br>
Model:                 94<br>
Model name:            <strong>Intel® Core™ i3-6100 CPU @ 3.70GHz</strong></p>
<p>top output:<br>
PID USER      PR  NI    VIRT    RES    SHR S  <strong>%CPU</strong> %MEM     TIME+ COMMAND<br>
24971 jmarona+  20   0 16,284g 1,468g 298328 R <strong>278,1</strong>  9,6 834:10.23 python</p>
<p>In this case n_workers is set to 0</p>

albanD: <p>Does the following code uses multiple cores?</p>
<pre><code class="lang-auto">import torch

a = torch.rand(100, 1000, 1000)
b = torch.rand(100, 1000, 1000)

while True:
    c = torch.bmm(a, b)
</code></pre>
<p>If so, try upgrading to 0.4 via pip to check that it is not a build problem from the old version.</p>

John_Deterious: <p>No. I just tried it.<br>
But, sometimes I see that 100% of CPU is utilized, with other codes less than 2%, how do you explain that?</p>

sina_mr: <p>Hi,</p>
<p>I have the same problem on a Debian server. Finally, I realized that the number of workers should be zero for the best performance, which contradicts the official guideline. Similar situation:<br>
<aside class="onebox githubissue">
  <header class="source">
      <a href="https://github.com/pytorch/pytorch/issues/12831" target="_blank" rel="nofollow noopener">github.com/pytorch/pytorch</a>
  </header>
  <article class="onebox-body">
    <a href="https://github.com/taohu88" rel="nofollow noopener">
<img src="https://avatars1.githubusercontent.com/u/547665?v=2&amp;s=96" class="thumbnail onebox-avatar" width="420" height="420">
</a>

<h4><a href="https://github.com/pytorch/pytorch/issues/12831" target="_blank" rel="nofollow noopener">Issue: In windows, DataLoader with num_workers &gt; 0 is extremely slow (50 times slower)</a></h4>

<div class="date" style="margin-top:10px;">
	<div class="user" style="margin-top:10px;">
	opened by <a href="https://github.com/taohu88" target="_blank" rel="nofollow noopener">taohu88</a>
	on <a href="https://github.com/pytorch/pytorch/issues/12831" target="_blank" rel="nofollow noopener">2018-10-18</a>
	</div>
	<div class="user">
	</div>
</div>

<pre class="content" style="white-space: pre-wrap;">🐛 Bug
In windows, DataLoader with num_workers &gt; 0 is extremely slow (pytorch=0.41)
To Reproduce
Step 1: create two loader, one with num_workers and...</pre>

<div class="labels">
 	<span style="display:inline-block;margin-top:2px;background-color: #B8B8B8;padding: 2px;border-radius: 4px;color: #fff;margin-left: 3px;">high priority</span>
 	<span style="display:inline-block;margin-top:2px;background-color: #B8B8B8;padding: 2px;border-radius: 4px;color: #fff;margin-left: 3px;">module: multiprocessing</span>
 	<span style="display:inline-block;margin-top:2px;background-color: #B8B8B8;padding: 2px;border-radius: 4px;color: #fff;margin-left: 3px;">module: windows</span>
 	<span style="display:inline-block;margin-top:2px;background-color: #B8B8B8;padding: 2px;border-radius: 4px;color: #fff;margin-left: 3px;">triaged</span>
</div>

  </article>
  <div class="onebox-metadata">
    
    
  </div>
  <div style="clear: both"></div>
</aside>
<br>
<aside class="onebox whitelistedgeneric">
  <header class="source">
      <img src="https://forums.fast.ai/images/default-favicon.ico" class="site-icon" width="32" height="32">
      <a href="https://forums.fast.ai/t/very-slow-dataloader/36886/6" target="_blank" title="05:21PM - 01 July 2019" rel="nofollow noopener">Deep Learning Course Forums – 1 Jul 19</a>
  </header>
  <article class="onebox-body">
    <img src="https://forums.fast.ai/letter_avatar_proxy/v3/letter/b/bc79bd/200.png" class="thumbnail onebox-avatar" width="200" height="200">

<h3><a href="https://forums.fast.ai/t/very-slow-dataloader/36886/6" target="_blank" rel="nofollow noopener">Very slow dataloader</a></h3>

<p>This explains a lot!  I assumed the speed I was getting in windows was normal until I tried fastai in Ubuntu on my system on the exact same notebook, and it ran / trained ~3x faster.  On my hardware, your fix took an unet / resnet101 128x128x3 BS = 8...</p>


  </article>
  <div class="onebox-metadata">
    
    
  </div>
  <div style="clear: both"></div>
</aside>
</p>
<p>Best,<br>
Sina</p>

khalil: <p>I met the same problem,I compile the source code with openmp and I found I can only use one core.Do you solve this problem?</p>