Custom Loss function : Clarification

nisharaichur: <p>I have a custom loss function defined like this:</p>
<pre><code class="lang-auto">class Quaternion_Multiplicative_Error(torch.nn.Module):
    def __init__(self):
        print("QME optimized")
        super(Quaternion_Multiplicative_Error, self).__init__()
        self.conj = torch.tensor([1,-1,-1,-1], requires_grad=False)
       
    def qme(self, pred, true):
        true = torch.mul(true, self.conj)
        pro = self.hamilton_product(pred, true)
        img_part = pro[1:]
        norm = np.linalg.norm(img_part, ord=1)
        return 2 * norm

    def forward(self, pred, true):
        batch_size = pred.shape[0]
        return sum(self.qme(x, y) for x, y in zip(pred, true))/batch_size

</code></pre>
<p>I need to use this custom loss function in another main class, which looks something like this:</p>
<pre><code class="lang-auto">class FusionCriterion_LearnParms(torch.nn.Module):
    def __init__(self, loss_pos="L1Loss", loss_ori="QMELoss", alpha=0.0, beta=-3.0):
        super(FusionCriterion_LearnParms, self).__init__()
        self.loss_pos = self.select_loss(loss_pos)
        self.loss_ori = self.select_loss(loss_ori)
        self.alpha = torch.nn.Parameter(torch.tensor([alpha], dtype=torch.double), requires_grad=True)
        self.beta = torch.nn.Parameter(torch.tensor([beta], dtype=torch.double), requires_grad=True)
        
    def select_loss(self, loss):
        if loss == "L1Loss":
            return torch.nn.L1Loss()    
        elif loss == "MSELoss":
            return torch.nn.MSELoss()
        else:
            return Quaternion_Multiplicative_Error()

    def forward(self, predicted, actual):
        position_loss = (torch.exp(-self.alpha) * self.loss_pos(predicted[:, :3], actual[:, :3])) + self.alpha
        orientation_loss = (torch.exp(-self.beta) * self.loss_ori(predicted[:, 3:]), actual[:, 3:]) + self.beta
        total_loss =   position_loss + orientation_loss
        return total_loss
</code></pre>
<p>I get an error</p>
<pre><code class="lang-auto">File "fusion.py", line 62, in qme
    true = torch.mul(true, self.conj)
RuntimeError: expected device cuda:0 but got device cpu
</code></pre>
<blockquote>
<p>I have all the Classes above(nn.Module) moved to the torch.device(“cuda”)<br>
I was hoping all its member functions will also be moved to “cuda”</p>
</blockquote>

ilkarman: <aside class="quote no-group" data-username="nisharaichur" data-post="1" data-topic="102276">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/nisharaichur/48/31342_2.png" class="avatar"> nisharaichur:</div>
<blockquote>
<p><code>self.conj = torch.tensor([1,-1,-1,-1], requires_grad=False)</code></p>
</blockquote>
</aside>
<p>Instead of: <code>self.conj = torch.tensor([1,-1,-1,-1], requires_grad=False)</code> perhaps:</p>
<pre><code class="lang-auto">self.register_buffer('conj ', torch.tensor([1,-1,-1,-1]))
</code></pre>
<p>So that it is transferred to gpu also?</p>

nisharaichur: <p>So  this means, we can have  this as a constant and also this parameter will not be optimized right.</p>
<p>Yes it works <img src="https://discuss.pytorch.org/images/emoji/apple/slight_smile.png?v=9" title=":slight_smile:" class="emoji" alt=":slight_smile:"></p>
<p>Thanks</p>