How to change the default device of GPU? device_ids[0]

lynnea718: <p>How to change the default device of GPU?<br>
for some reason ,I can not use the <code>device_ids[0]</code> of GPU,<br>
I change the following code:（in <code>data_parallel.py</code>）</p>
<pre><code class="lang-auto">if output_device is None:
output_device =device_ids[0]
</code></pre>
<p>to</p>
<pre><code class="lang-auto">if output_device is None:
output_device =device_ids[1]
</code></pre>
<p>but it still seem to used the <code>device_ids[0]</code><br>
all tensors must be on <code>devices[0]</code>？<br>
How to change it？</p>

fmassa: <p>I think the best way is to directly specify with the <code>CUDA_VISIBLE_DEVICES</code> environment variable.<br>
Something like</p>
<pre><code class="lang-auto">CUDA_VISIBLE_DEVICES=1,2 python myscript.py
</code></pre>
<p>so your script will only see GPUs number 1 and 2, and won’t touch the other GPUs</p>

lynnea718: <p>i try add<br>
CUDA_VISIBLE_DEVICES=1,2<br>
in <a href="http://main.py" rel="nofollow noopener">main.py</a><br>
but it does not work</p>
<p>We have 8 gpu. I used the devices[1,2] ,the rest is used by others</p>

colesbury: <p><code>CUDA_VISIBLE_DEVICES</code> is an environment variable. You set it on the command line, not in the Python script.</p>

cold_wind: <p>So in the Python script, How to choose the GPU devices. Especially, I want to change GPU devices in the training processing</p>

uzeful: <p>I also want to know how to choose the GPU device in the python script.</p>

Jing: <blockquote>
<p>torch.cuda keeps track of currently selected GPU, and all CUDA tensors you allocate will be created on it. The selected device can be changed with a torch.cuda.device context manager.</p>
</blockquote>
<p>ex:</p>
<pre><code class="lang-auto">with torch.cuda.device(1):
    w = torch.FloatTensor(2,3).cuda()
    # w was placed in  device_1 by default.
</code></pre>
<p>Or you can specify gpu.id via .cuda() directly.</p>
<pre><code class="lang-auto">w = torch.FloatTensor(2,3).cuda(2)
# w was placed in device_2

</code></pre>
<p>See more on: <a href="http://pytorch.org/docs/master/notes/cuda.html" class="inline-onebox" rel="noopener nofollow ugc">CUDA semantics — PyTorch master documentation</a></p>

uzeful: <p>Thanks for your reply, I will do some experiments to verify these functions. Besides, I found out some other useful functions at <a href="https://discuss.pytorch.org/t/how-to-specify-gpu-usage/945">How to specify GPU usage?</a>.</p>

jdhao: <p>I have tried to set <code>CUDA_VISIBLE_DEVICES</code> in shell, then I run a simple script to test if the setting has taken effect, unfortunately, it seems does not work.</p>
<pre><code class="lang-auto">CUDA_VISIBLE_DEVICES=3; python test.py
</code></pre>
<p>the script <code>test.py</code> is</p>
<pre><code class="lang-auto">import torch
print(torch.cuda.current_device())
</code></pre>
<p>the above script still shows that current device is 0.</p>
<p>I find that <code>torch.cuda.set_device(device_num)</code> works fine in setting the desired GPU to use.</p>

SimonW: <p>two things you did wrong:</p>
<ol>
<li>there shouldn’t be semicolon. with the semicolon, they are on two different lines, and python won’t see it.</li>
<li>even with the correct command <code>CUDA_VISIBLE_DEVICES=3 python test.py</code>, you won’t see <code>torch.cuda.current_device() = 3</code>, because it completely changes what devices pytorch can see. So in pytorch land device#0 is actually your device#3 of the system. You can verify that from nvidia-smi.</li>
</ol>

jdhao: <p>Thanks for the information. I thought that PyTorch would print the actual GPU id even if we use <code>CUDA_VISIBLE_DEVICES</code> to set available GPU.</p>

SimonW: <p>That controls what devices CUDA exposes and PyTorch can’t do nothing in this regards.</p>

hszhao: <p>Hi, you can specify used gpu in python script as following:</p>
<p>import os<br>
from argparse import ArgumentParser</p>
<p>parser = ArgumentParser(description=‘Example’)<br>
parser.add_argument(’–gpu’, type=int, default=[0,1], nargs=’+’, help=‘used gpu’)</p>
<p>args = parser.parse_args()<br>
os.environ[“CUDA_VISIBLE_DEVICES”] = ‘,’.join(str(x) for x in args.gpu)</p>

jianzhong: <p>it cannot work for me, it always use the first(ie, 0) gpu</p>

kaiyuyue: <p>Thanks, this is the easiest way to solve this problem.</p>

SimonW: <p>It shouldn’t happen. That is a CUDA flag. Once set, PyTorch will never have access to the excluded device(s).</p>

jianzhong: <p>I change the place of that and it worked,thanks for your reply!</p>
<p>获取 Outlook for iOS<a href="https://aka.ms/o0ukef" rel="nofollow noopener">https://aka.ms/o0ukef</a></p>

moonlightlane: <p>or <code>torch.cuda.set_device(device_id)</code></p>

maplewizard: <p>This is a very useful solution, especially you are going to run with someone else’s code without specifying the cuda id.</p>

Sun_ShiJie: <aside class="quote no-group" data-username="kaiyuyue" data-post="16" data-topic="1041">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/kaiyuyue/48/6052_2.png" class="avatar"> kaiyuyue:</div>
<blockquote>
<p>s is the easi</p>
</blockquote>
</aside>
<p>According to the tutorial, it’s better to use environment variable <img src="https://discuss.pytorch.org/images/emoji/apple/slight_smile.png?v=12" title=":slight_smile:" class="emoji" alt=":slight_smile:" loading="lazy" width="20" height="20"></p>
<pre><code class="lang-auto">torch.cuda.set_device(device)

Sets the current device.

Usage of this function is discouraged in favor of device. In most cases it’s better to use CUDA_VISIBLE_DEVICES environmental variable.

Parameters:	device (torch.device or int) – selected device. This function is a no-op if this argument is negative.
</code></pre>