How to check if Model is on cuda

username: napsternxg
cooked: <p>When I have an object of a class which inherits from <code>nn.Module</code> is there anyway I can check if I the object is on cuda or not. We can do this for tensors by calling <code>var_name.is_cuda</code> however no such variable is available for modules.</p>

username: smth
cooked: <p>As replied on the github issues, an easy way is:</p>
<pre><code class="lang-python">next(model.parameters()).is_cuda # returns a boolean
</code></pre>

username: mikolchon
cooked: <p>Hi, is this still the preferred way in <code>0.4</code>? Thanks.</p>

username: thadar
cooked: <aside class="quote no-group" data-username="smth" data-post="2" data-topic="180">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/smth/48/13_2.png" class="avatar"> smth:</div>
<blockquote>
<p>next(model.parameters()).is_cuda</p>
</blockquote>
</aside>
<p>I tested it right now, and it works even in pytorch v0.4</p>

username: justusschock
cooked: <p>But why is it even necessary? If a model is on cuda and you call model.cuda() it should be a no-op and if the model is on cpu and you call model.cpu() it should also be a no-op.</p>

username: mikolchon
cooked: <p>Yes, it will work because it is verifying the type of the model weights, but I was wondering if there’s a new attribute similar to <code>model.device</code> as is the case for the new tensors in <code>0.4</code>.</p>

username: rasbt
cooked: <blockquote>
<p>But why is it even necessary? If a model is on cuda and you call model.cuda() it should be a no-op and if the model is on cpu and you call model.cpu() it should also be a no-op.</p>
</blockquote>
<p>It’s necessary if you want to make the code compatible to machines that don’t support cuda. E.g. if you do a <code>model.cuda()</code> or a <code>sometensor.cuda()</code>, you will get a <code>RuntimeError</code>.</p>
<p>Personally, I develop and debug 99% of the code on macOS, and then sync it over to a headless cluster, which is why this pattern is useful to me, for example.</p>
<blockquote>
<p>if there’s a new attribute similar to model.device as is the case for the new tensors in 0.4.</p>
</blockquote>
<p>Yes, e.g., you can now specify the device 1 time at the top of your script, e.g.,</p>
<pre><code>device = torch.device("cuda:0" if torch.cuda.is_available() else "cpu") 
</code></pre>
<p>and then for the model, you can use</p>
<pre><code>model = model.to(device)
</code></pre>
<p>The same applies also to tensors, e.g,.</p>
<pre><code>for features, targets in data_loader:
    features = features.to(device)
    targets = targets.to(device)
</code></pre>

username: chiragraman
cooked: <p>It’s an old conversation but I just had a similar question and found this in the doc notes on best practices within <a href="https://pytorch.org/docs/stable/notes/cuda.html?highlight=model%20device" rel="noopener nofollow ugc">CUDA semantics</a>, so thought of leaving it here in case it helps others:</p>
<blockquote>
<p>This is the recommended practice when creating modules in which new tensors need to be created internally during the forward pass.</p>
</blockquote>
<p>where ‘this’ refers to using one of the <code>new_*</code> methods for creating tensors that preserve the device context.</p>

username: Martin_Ennemoser
cooked: <p>You can get the device by:</p>
<pre><code class="lang-auto">next(network.parameters()).device
</code></pre>

username: Brando_Miranda
cooked: <p>why doesn’t:</p>
<pre><code class="lang-auto">self.device
</code></pre>
<p>work if my current object is a <code>nn.module</code>?</p>
<pre><code class="lang-auto">(Pdb) self.device
*** torch.nn.modules.module.ModuleAttributeError: 'MyNN' object has no attribute 'device'
</code></pre>

username: ptrblck
cooked: <p><code>.device</code> is a tensor attribute as described in the <a href="https://pytorch.org/docs/stable/tensors.html#torch.Tensor.device">docs</a> and is not set for the <code>nn.Module</code>, since modules can have parameters and buffers on different and multiple devices.</p>

username: Brando_Miranda
cooked: <aside class="quote no-group" data-username="Martin_Ennemoser" data-post="9" data-topic="180">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/martin_ennemoser/48/24226_2.png" class="avatar"> Martin_Ennemoser:</div>
<blockquote>
<p><code>next(network.parameters()).device</code></p>
</blockquote>
</aside>
<p>so is <code>next(network.parameters()).device</code> simply getting the device name for the first parameter of the nn module?</p>

username: ptrblck
cooked: <p>Yes, that’s correct.</p>