My code can't run on two GPUs

Roqyah_bdeen: <p>I use a self-attention layer in my network but Unfortunately I got cuda out of memory error even though I reduced the batch_size to one only<br>
So, I tried to run the code on two GPUs but I got this error</p>
<pre><code class="lang-auto">RuntimeError: Expected tensor for argument #1 'input' to have the same device as tensor for argument #2 'weight'; but device 1 does not equal 0 (while checking arguments for cudnn_convolution)

</code></pre>
<p>I moved the model and the input image to cuda0 to be on similar gpu but each time I move one item to cuda0 I found a new item should be on the same gpu too <img src="https://discuss.pytorch.org/images/emoji/apple/frowning.png?v=12" title=":frowning:" class="emoji" alt=":frowning:"><br>
how could I resolve this problem please? any help <img src="https://discuss.pytorch.org/images/emoji/apple/confused.png?v=12" title=":confused:" class="emoji" alt=":confused:"></p>

JuanFMontesinos: <p>Could you paste an script? How are you running in “two gpus”?</p>

Roqyah_bdeen: <p>first I prepare devices</p>
<pre><code class="lang-auto">def _prepare_device(self, n_gpu_use):
        """
        setup GPU device if available, move model into configured device
        """
        n_gpu = torch.cuda.device_count()
        if n_gpu_use &gt; 0 and n_gpu == 0:
            self._log.warning("Warning: There\'s no GPU available on this machine,"
                              "training will be performed on CPU.")
            n_gpu_use = 0
        if n_gpu_use &gt; n_gpu:
            self._log.warning(
                "Warning: The number of GPU\'s configured to use is {}, "
                "but only {} are available.".format(n_gpu_use, n_gpu))
            n_gpu_use = n_gpu
        device = torch.device('cuda:0,1' if n_gpu_use &gt; 0 else 'cpu')
        list_ids = list(range(n_gpu_use))
        return device, list_ids
</code></pre>
<p>next initiate model</p>
<pre><code class="lang-auto">  def _init_model(self, model):
        model = model.to(self.device)
        #print("model dvc",self.device)
        if self.cfg.pretrained_model:
            self._log.info("=&gt; using pre-trained weights {}.".format(
                self.cfg.pretrained_model))
            epoch, weights = load_checkpoint(self.cfg.pretrained_model)
            model.load_state_dict(weights)
        else:
            self._log.info("=&gt; Train from scratch.")
            model.init_weights()
        model = torch.nn.DataParallel(model, device_ids=self.device_ids)
        return model
</code></pre>

Roqyah_bdeen: <pre><code class="lang-auto">device_ids=[0,1]
self.device=cuda:0
</code></pre>

JuanFMontesinos: <aside class="quote no-group" data-username="Roqyah_bdeen" data-post="3" data-topic="142570">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/roqyah_bdeen/48/39618_2.png" class="avatar"> Roqyah_bdeen:</div>
<blockquote>
<p><code>model = torch.nn.DataParallel(model, device_ids=self.device_ids)</code></p>
</blockquote>
</aside>
<p>I would say you have some hard-coded device inside the model.<br>
Basically your input is in GPU 1 (this is done automatically by DataParallel<br>
If your layer is in GPU0 it probably means you have some line with a fixed device.</p>
<p>Could you try your script with a simple<br>
<code>model = torch.nn.Linear(1,2)</code><br>
and then a silly input like<br>
<code>model(torch.rand(4,1))</code></p>

Roqyah_bdeen: <p>First thanks for your reply<br>
I used <code>model = torch.nn.Linear(1,2)</code> instead of <code>model = torch.nn.DataParallel(model, device_ids=self.device_ids)</code> and got this error</p>
<pre><code class="lang-auto">Traceback (most recent call last):
  File "train.py", line 43, in &lt;module&gt;
    basic_train.main(cfg, _log)
  File "/home/ubuntu/Rokia/UnRigidFlow-master-2-smooth -attention/basic_train.py", line 49, in main
    train_loader, valid_loader, model, loss, _log, cfg.save_root, cfg.train)
  File "/home/ubuntu/Rokia/UnRigidFlow-master-2-smooth -attention/trainer/kitti_flow_trainer.py", line 13, in __init__
    train_loader, valid_loader, model, loss_func, _log, save_root, config)
  File "/home/ubuntu/Rokia/UnRigidFlow-master-2-smooth -attention/trainer/base_trainer.py", line 28, in __init__
    self.optimizer = self._create_optimizer()
  File "/home/ubuntu/Rokia/UnRigidFlow-master-2-smooth -attention/trainer/base_trainer.py", line 70, in _create_optimizer
    {'params': bias_parameters(self.model.module),
  File "/home/ubuntu/anaconda3/envs/py37/lib/python3.7/site-packages/torch/nn/modules/module.py", line 539, in __getattr__
    type(self).__name__, name))
AttributeError: 'Linear' object has no attribute 'module'
</code></pre>
<p>is this what you have asked me to do?</p>