How is it possible to move a model wrapped in DataParallel to CPU?

Shisho_Sama: <p>Hello everyone.<br>
How can I convert a model thats been trained on multiple GPUs (wrapped in DataParallel) to <code>cpu</code>?<br>
I tried  <code>model = model.module.to('cpu')</code> without any luck.<br>
How should I be going about this?<br>
Notes:<br>
pn1:  I’m trying to save this as a jit traced model thats why I’m trying t o convert it to cpu before hand<br>
pn2:  I know I can simply use model.module in torch.jit.trace and then map the weights to cpu when loading  the jit model. however, I want to know why this is failing here!</p>

ptrblck: <p>What kind of error are you seeing with the posted line of code?</p>

Shisho_Sama: <p>It gives me :</p>
<blockquote>
<p>RuntimeError: module must have its parameters and buffers on device cuda:0 (device_ids[0]) but found one of them on device cpu</p>
</blockquote>
<p>basically I did :</p>
<pre><code class="lang-auto">model.eval()
model_cpu = model.module.to('cpu')
model_traced = torch.jit.trace(model_cpu, dummy_input)
</code></pre>

ptrblck: <p>Are you using <code>to()</code>, <code>cpu()</code> or <code>cuda()</code> operations inside your model?<br>
If not, could you post an executable code snippet to reproduce this issue?</p>

Shisho_Sama: <p>Yes  the model is moved into <code>cuda</code> mode! using <code>.to()</code> and its is defined like this :</p>
<pre><code class="lang-auto">        model = nn.DataParallel(model)
        metric_fc = ArcMarginModel(args)
        metric_fc = nn.DataParallel(metric_fc)

        if args.optimizer == 'sgd':
            optimizer = InsightFaceOptimizer(
                torch.optim.SGD([{'params': model.parameters()}, {'params': metric_fc.parameters()}],
                                lr=args.lr, momentum=args.mom, weight_decay=args.weight_decay))
        else:
            optimizer = InsightFaceOptimizer(
                torch.optim.Adam([{'params': model.parameters()}, {'params': metric_fc.parameters()}],
                                 lr=args.lr, weight_decay=args.weight_decay))

    else:
        checkpoint = torch.load(checkpoint)
        start_epoch = checkpoint['epoch'] + 1
        epochs_since_improvement = checkpoint['epochs_since_improvement']
        model = checkpoint['model']
        metric_fc = checkpoint['metric_fc']
        optimizer = checkpoint['optimizer']

    logger = get_logger()

    # Move to GPU, if available
    model = model.to(device)
    metric_fc = metric_fc.to(device)
</code></pre>
<p>and by the way this is run pytorch <code>1.5.1</code></p>

ptrblck: <p>The code snippet is unfortunately not executable so that I cannot debug it. <img src="https://discuss.pytorch.org/images/emoji/apple/confused.png?v=9" title=":confused:" class="emoji" alt=":confused:"></p>

Shisho_Sama: <p>Ah I missed the executable part! this is taken from this <a href="https://github.com/foamliu/InsightFace-v2" rel="noopener nofollow ugc">repo</a><br>
I’ll try to see if I can create a minimal repreducible example.</p>

Shisho_Sama: <p>Creating an MRE! indeed do wonders!<br>
OK, it turns out there was no problem to begin with! the error was comming from the other section of the script that expected a cuda while I was giving it a model in cpu mode and thus the error.<br>
silly me! completely missed that!<br>
Thanks a lot for your generous  help <a class="mention" href="/u/ptrblck">@ptrblck</a> really apprecaite it.</p>