Which device is model / tensor stored on?

arogozhnikov: <p>Hi,<br>
I have such a simple method in my model</p>
<pre><code class="lang-auto">    def get_normal(self, std):
        if &lt;here I need to know which device is used&gt; :
            eps = torch.cuda.FloatTensor(std.size()).normal_()
        else:
            eps = torch.FloatTensor(std.size()).normal_()
        return Variable(eps).mul(std)
</code></pre>
<p>To work efficiently, it needs to know which device is currently used (CPU or GPU).</p>
<p>I was looking for something like <code>model.is_cuda</code> - but different tensors can be placed on different devices, so probably there is something like <code>std.device_context</code>, but I haven’t found such method either.</p>
<p>What is recommended to handle such situations?</p>

arogozhnikov: <p>So far I am using std.is_cuda (which seems to be ok solution if there is one GPU device), but better options are welcome.</p>

alexis-jacq: <p>The common way is to start your code with:</p>
<pre><code class="lang-auto">use_cuda = torch.cuda.is_available()
</code></pre>
<p>Then, each time you create a new instance of any tensor/variable/module, just do:</p>
<pre><code class="lang-auto">if use_cuda:
    my_obect.cuda()
</code></pre>
<p>That way you make sure that everything is stored or not on GPU or CPU (by default, without calling .cuda() it will be on CPU)</p>

arogozhnikov: <blockquote>
<p>The common way is to start your code with:</p>
</blockquote>
<p>Yup, I’ve noted this (this code is based on pytorch examples).</p>
<p>However, there may be different possible situations to handle and I want the model code to be quite isolated from the environment (e.g. place it in a separate python module), and this doesn’t look like a general solution.</p>

arogozhnikov: <p>Just came up with a better idea: <code>tensor.new(sizes).normal_(0, 1)</code> seems to be the right way to get gaussian noise on the right device.</p>

david-leon: <p>This is a feature wanted for quite a long time. I don’t see any reason why pytorch hasn’t provided API simply as .device() to return the device a model/variable/tensor resides on</p>

vishwakftw: <p>I guess you could use this:</p>
<pre><code>cuda_check = my_tensor.is_cuda
if cuda_check:
    get_cuda_device = my_tensor.get_device()</code></pre>

david-leon: <p>Which version are you using. For 0.2.1 release, I don’t find <code>.get_device()</code> API</p>

vishwakftw: <p>You can only use the <code>get_device()</code> API if the Tensor is a CUDA Tensor.</p>

david-leon: <p>OK, I get it, thanks a lot</p>

aliutkus: <p>I would also appreciate it to have a .device() function I could call, that would work for models, tensors, etc</p>

jet: <p>It would be convenient in many cases if we have a method <code>device()</code> that returns the exact device that the model/tensors are located, then</p>
<pre><code class="lang-python">some_tensor.to(some_model.device())
</code></pre>
<p>would be an elegant solution for many functions (which accept a model as input and perform some inference on the model).</p>

ptrblck: <p>You could try</p>
<pre><code class="lang-python">a = torch.randn(10).to('cuda:0')
b = torch.randn(10).to(a.device)
</code></pre>

drcege: <p>Is there a way to get the device of a module (a torchvision model or criterion) ?</p>

ptrblck: <p>Yes, although it might be a bit misleading in some special cases.<br>
In case your model is stored on just one GPU, you could simply print the device of one parameter, e.g.:</p>
<pre><code class="lang-python">print(next(model.parameters()).device)
</code></pre>
<p>However, you could also use model sharding and split the model among a few GPUs.<br>
In that case you could have to check all parameters for their device.</p>

mgpoirot: <p>I am am using copy.deepcopy to instantiate multiple instances of the same model (including same initialization parameters).<br>
I would like to reaffirm if deepcopy copies this storage property, so I’d still have to check somehow.</p>