Custom tweedie loss throwing an error in pytorch

username: Jordan_Howell
cooked: <p>Hello,</p>
<p>I’m having trouble implementing a GLM where the y follows a Tweedie distribution using the stars models package.   Is there. A way to do this in pytorch?  I’ve searched and haven’t found any literature or posts on it.</p>
<p><strong>UPDATE</strong></p>
<p>I’ve tried to define a custom loss function as such:</p>
<pre><code class="lang-auto">def tweedieloss(predicted, observed):
    '''
    Custom loss fuction designed to minimize the deviance using stochastic gradient descent
    '''
    p = torch.tensor([1.5])

    QLL = predicted**-p(((predicted*observed)/(torch.tensor([1])-p)) - ((predicted**2)/(torch.tensor([2])-p)))
    QLL.cuda()
    return -torch.abs(QLL)def tweedieloss(predicted, observed):
    '''
    Custom loss fuction designed to minimize the deviance using stochastic gradient descent
    '''
    p = torch.tensor([1.5])

    QLL = predicted**-p(((predicted*observed)/(torch.tensor([1])-p)) - ((predicted**2)/(torch.tensor([2])-p)))
    QLL.cuda()
    return -torch.abs(QLL)

</code></pre>
<p>I’m still not sure if this is exactly correct however, I do know it’s giving me an error.</p>
<p>I’m using it in the following model:</p>
<pre><code class="lang-auto"># Create the linear regression model
model = nn.Linear(x.shape[1], 1)
# Loss and optimizer
criterion = tweedieloss
optimizer = torch.optim.SGD(model.parameters(), lr=0.1)

# PyTorch uses float32 by default
# Numpy creates float64 by default
inputs = torch.from_numpy(X.astype(np.float32))
targets = torch.from_numpy(Y.astype(np.float32))

# Train the model
n_epochs = 20
losses = []
for it in range(n_epochs):
    # zero the parameter gradients
    optimizer.zero_grad()

    # Forward pass
    outputs = model(inputs)
    loss = criterion(outputs, targets, p = torch.tensor([1.5]))

    # keep the loss so we can plot it later
    losses.append(loss.item())

    # Backward and optimize
    loss.backward()
    optimizer.step()

    print(f'Epoch {it+1}/{n_epochs}, Loss: {loss.item():.4f}')
```

The error I'm getting is the following:

```
---------------------------------------------------------------------------
RuntimeError                              Traceback (most recent call last)
&lt;ipython-input-182-87301b9d4374&gt; in &lt;module&gt;
      9     # Forward pass
     10     outputs = model(inputs)
---&gt; 11     loss = criterion(outputs, targets)
     12 
     13     # keep the loss so we can plot it later

&lt;ipython-input-180-288c1e1a6f6d&gt; in tweedieloss(predicted, observed)
      5     p = torch.tensor([1.5])
      6 
----&gt; 7     QLL = predicted**-p(((predicted*observed)/(torch.tensor([1])-p)) - ((predicted**2)/(torch.tensor([2])-p)))
      8     QLL.cuda()
      9     return -torch.abs(QLL)

RuntimeError: expected device cuda:0 but got device cpu
```

i'm not sure what in that custom loss function I should send to cuda.  When I try to run everything with the CPU, I get the following:
```
---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
&lt;ipython-input-188-87301b9d4374&gt; in &lt;module&gt;
      9     # Forward pass
     10     outputs = model(inputs)
---&gt; 11     loss = criterion(outputs, targets)
     12 
     13     # keep the loss so we can plot it later

&lt;ipython-input-183-43f44140ed7b&gt; in tweedieloss(predicted, observed)
      5     p = torch.tensor([1.5])
      6 
----&gt; 7     QLL = predicted**-p(((predicted*observed)/(torch.tensor([1])-p)) - ((predicted**2)/(torch.tensor([2])-p)))
      8     return -torch.abs(QLL)

TypeError: 'Tensor' object is not callable
```

I'm not sure how to implement this custom loss function.</code></pre>

username: babababadukeduke
cooked: <aside class="quote no-group" data-username="Jordan_Howell" data-post="1" data-topic="76349">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/jordan_howell/48/16421_2.png" class="avatar"> Jordan_Howell:</div>
<blockquote>
<p>torch.tensor([1])</p>
</blockquote>
</aside>
<p>Why do you need to use torch.tensor([1])? Can’t you simply pass 1</p>

username: Jordan_Howell
cooked: <p>Because I was desperate to find an answer and thought that may help.  When I take that out, I get the following:</p>
<pre><code class="lang-auto">---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
&lt;ipython-input-322-16213b95ac4b&gt; in &lt;module&gt;
     12 #     print(loss)
     13     # keep the loss so we can plot it later
---&gt; 14     losses.append(loss.item())
     15 
     16     # Backward and optimize

ValueError: only one element tensors can be converted to Python scalars
</code></pre>
<p>So I took the append out and printed the loss but it’s producing all NaNs.</p>

username: babababadukeduke
cooked: <p>This error is happening because item() can only be called on a Tensor containing single element for example [1]. If you call item() on [1, 1] it will give this error. Also in PyTorch custom loss functions are suppose to return a scale value. For example below is a simple implementation of mean squared loss function</p>
<aside class="quote no-group" data-username="ptrblck" data-post="2" data-topic="29387">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/ptrblck/48/1823_2.png" class="avatar"><a href="https://discuss.pytorch.org/t/custom-loss-functions/29387/2">Custom loss functions</a></div>
<blockquote>
<p>def my_loss(output, target): loss = torch.mean((output - target)**2) return loss</p>
</blockquote>
</aside>

username: Jordan_Howell
cooked: <p>Ahhh!  Well that got it to run on cpu.  When i run it on GPU, I get the following:</p>
<pre><code class="lang-auto">---------------------------------------------------------------------------
RuntimeError                              Traceback (most recent call last)
&lt;ipython-input-355-5f0610597548&gt; in &lt;module&gt;
      9     # Forward pass
     10     outputs = model(inputs)
---&gt; 11     loss = tweedieloss(outputs, targets)
     12     print(loss)
     13     # keep the loss so we can plot it later

&lt;ipython-input-345-982fe7ade6d7&gt; in tweedieloss(predicted, observed)
     23     p = torch.tensor(1.5)
     24 
---&gt; 25     QLL = torch.pow(predicted, (-p))*(((predicted*observed)/(1-p)) - ((torch.pow(predicted, 2))/(2-p)))
     26 
     27     return torch.mean(-torch.abs(QLL))

RuntimeError: iter.device(arg).is_cuda() INTERNAL ASSERT FAILED at C:/w/1/s/tmp_conda_3.7_100118/conda/conda-bld/pytorch_1579082551706/work/aten/src\ATen/native/cuda/Loops.cuh:197, please report a bug to PyTorch. 
</code></pre>
<p>Not sure how I’m supposed to report the bug.</p>

username: babababadukeduke
cooked: <p>Did you check if both your outputs and targets are in the same CUDA device?</p>

username: Jordan_Howell
cooked: <p>I (and when I say I, I mean a friend) figured this out late yesterday. I had to change the function and we’re not sure why changing the simple math enabled it to run.  But in case anyone wants a custom function that maximizes the Tweedie QLL, the below works:</p>
<pre><code class="lang-auto">def QLL(predicted, observed):
    p = torch.tensor(1.5)
    QLL = QLL = torch.pow(predicted, (-p))*(((predicted*observed)/(1-p)) - ((torch.pow(predicted, 2))/(2-p)))

    return QLL
        
def tweedieloss(predicted, observed, n):
    '''
    Custom loss fuction designed to minimize the deviance using stochastic gradient descent
    tweedie deviance from McCullagh 1983

    '''
    d = -2*QLL(predicted, observed)
#     loss = (weight*d)/1


    return torch.mean(d)
</code></pre>