Error when custom loss function

username: Schroter
cooked: <p>Hi All,<br>
I am trying to use a custom loss function when training. Please see the code below.</p>
<pre><code class="lang-auto">#Custom function
def pfbeta_fast(labels, predictions, beta=1.3):

    pTP = np.sum(labels * predictions)
    pFP = np.sum((1-labels) * predictions)
    num_positives = np.sum(labels)  #  = pTP+pFN

    pPrecision = pTP/(pTP+pFP)
    pRecall = pTP/num_positives

    beta_squared = beta**2

    if (pPrecision &gt; 0 and pRecall &gt; 0):
        pF1 = (1+beta_squared) * pPrecision * pRecall/(beta_squared*pPrecision + pRecall)
        return pF1
    else:
        return 0

model_conv = torchvision.models.regnet_y_32gf(weights = 'RegNet_Y_32GF_Weights.IMAGENET1K_SWAG_E2E_V1')
#model_conv = torchvision.models.efficientnet_b7(weights = 'EfficientNet_B7_Weights.IMAGENET1K_V1')

for param in model_conv.parameters():
    param.requires_grad = False

# Parameters of newly constructed modules have requires_grad=True by default
num_ftrs = model_conv.fc.in_features
model_conv.fc = nn.Linear(num_ftrs, 2)

model_conv = model_conv.to(device)

criterion = pfbeta_fast #set model criterion 

# Observe that only parameters of final layer are being optimized as
# opposed to before.
optimizer_conv = optim.SGD(model_conv.fc.parameters(), lr=0.001, momentum=0.9)

# Decay LR by a factor of 0.1 every 7 epochs
exp_lr_scheduler = lr_scheduler.StepLR(optimizer_conv, step_size=7, gamma=0.1)

model_conv = train_model(model_conv, criterion, optimizer_conv,
                         exp_lr_scheduler, num_epochs=1)
</code></pre>
<p>For the above code I get the error below:</p>
<pre><code class="lang-auto">Epoch 0/0
----------
---------------------------------------------------------------------------
RuntimeError                              Traceback (most recent call last)
/tmp/ipykernel_23/3648721085.py in &lt;module&gt;
      1 model_conv = train_model(model_conv, criterion, optimizer_conv,
----&gt; 2                          exp_lr_scheduler, num_epochs=1)

/tmp/ipykernel_23/3892717004.py in train_model(model, criterion, optimizer, scheduler, num_epochs)
     32                     outputs = model(inputs)
     33                     _, preds = torch.max(outputs, 1)
---&gt; 34                     loss = criterion(outputs, labels)
     35 
     36                     # backward + optimize only if in training phase

/tmp/ipykernel_23/3685174594.py in pfbeta_fast(labels, predictions, beta)
      1 def pfbeta_fast(labels, predictions, beta=1.3):
      2 
----&gt; 3     pTP = np.sum(labels * predictions)
      4     pFP = np.sum((1-labels) * predictions)
      5     num_positives = np.sum(labels)  #  = pTP+pFN

RuntimeError: The size of tensor a (2) must match the size of tensor b (16) at non-singleton dimension 1
</code></pre>
<p>Would anyone be able to help me in this regards? Also is there a specific procedure that needed to be followed as in convert the loss function to pytorch commands for usage? Please</p>
<p>Thanks &amp; Best Regards<br>
AMJS</p>

username: ptrblck
cooked: <p>Check the shape of <code>labels</code> and <code>predictions</code> in <code>pTP = np.sum(labels * predictions)</code> and make sure you can multiply these numpy arrays.<br>
Based on your code it seems your model output and the targets are expected to match in their shape, which isn’t the case.</p>

username: Schroter
cooked: <p>Hi, Thanks for the reply. But please find the code below.<br>
Train Function:</p>
<pre><code class="lang-auto">def train_model(model, criterion, optimizer, scheduler, num_epochs=25):
    since = time.time()

    best_model_wts = copy.deepcopy(model.state_dict())
    best_acc = 0.0

    for epoch in range(num_epochs):
        print(f'Epoch {epoch}/{num_epochs - 1}')
        print('-' * 10)

        # Each epoch has a training and validation phase
        for phase in ['train', 'val']:
            if phase == 'train':
                model.train()  # Set model to training mode
            else:
                model.eval()   # Set model to evaluate mode

            running_loss = 0.0
            running_corrects = 0

            # Iterate over data.
            for inputs, labels in dataloaders[phase]:
                inputs = inputs.to(device)
                labels = labels.to(device)

                # zero the parameter gradients
                optimizer.zero_grad()

                # forward
                # track history if only in train
                with torch.set_grad_enabled(phase == 'train'):
                    outputs = model(inputs)
                    _, preds = torch.max(outputs, 1)
                    loss = criterion(labels,preds) #The previous code was: criterion(outputs,labels) 

                    # backward + optimize only if in training phase
                    if phase == 'train':
                        loss.backward()
                        optimizer.step()

                # statistics
                running_loss += loss.item() * inputs.size(0)
                running_corrects += torch.sum(preds == labels.data)
            if phase == 'train':
                scheduler.step()

            epoch_loss = running_loss / dataset_sizes[phase]
            epoch_acc = running_corrects.double() / dataset_sizes[phase]

            print(f'{phase} Loss: {epoch_loss:.4f} Acc: {epoch_acc:.4f}')

            # deep copy the model
            if phase == 'val' and epoch_acc &gt; best_acc:
                best_acc = epoch_acc
                best_model_wts = copy.deepcopy(model.state_dict())

        print()

    time_elapsed = time.time() - since
    print(f'Training complete in {time_elapsed // 60:.0f}m {time_elapsed % 60:.0f}s')
    print(f'Best val Acc: {best_acc:4f}')

    # load best model weights
    model.load_state_dict(best_model_wts)
    return model
</code></pre>
<p>Custom Loss function converted to pytorch tensors:</p>
<pre><code class="lang-auto">def pfbeta_fast(labels, predictions, beta=1.3):
    labels = torch.Tensor(labels)
    predictions = torch.Tensor(predictions)
    
    pTP = torch.sum(labels * predictions)
    pFP = torch.sum((1-labels) * predictions)
    num_positives = torch.sum(labels)  #  = pTP+pFN

    pPrecision = pTP/(pTP+pFP)
    pRecall = pTP/num_positives

    beta_squared = beta**2

    if (pPrecision &gt; 0 and pRecall &gt; 0):
        pF1 = (1+beta_squared) * pPrecision * pRecall/(beta_squared*pPrecision + pRecall)
        return pF1
    else:
        return 0
</code></pre>
<p>I still get another error as below:</p>
<pre><code class="lang-auto">/opt/conda/lib/python3.7/site-packages/ipykernel_launcher.py:2: UserWarning: To copy construct from a tensor, it is recommended to use sourceTensor.clone().detach() or sourceTensor.clone().detach().requires_grad_(True), rather than torch.tensor(sourceTensor).
  
/opt/conda/lib/python3.7/site-packages/ipykernel_launcher.py:3: UserWarning: To copy construct from a tensor, it is recommended to use sourceTensor.clone().detach() or sourceTensor.clone().detach().requires_grad_(True), rather than torch.tensor(sourceTensor).
  This is separate from the ipykernel package so we can avoid doing imports until
---------------------------------------------------------------------------
RuntimeError                              Traceback (most recent call last)
/tmp/ipykernel_27/1892088165.py in &lt;module&gt;
----&gt; 1 model_conv = train_model(model_conv, criterion, optimizer_conv, exp_lr_scheduler, num_epochs=1)

/tmp/ipykernel_27/3619938709.py in train_model(model, criterion, optimizer, scheduler, num_epochs)
     37                     # backward + optimize only if in training phase
     38                     if phase == 'train':
---&gt; 39                         loss.backward()
     40                         optimizer.step()
     41 

/opt/conda/lib/python3.7/site-packages/torch/_tensor.py in backward(self, gradient, retain_graph, create_graph, inputs)
    486             )
    487         torch.autograd.backward(
--&gt; 488             self, gradient, retain_graph, create_graph, inputs=inputs
    489         )
    490 

/opt/conda/lib/python3.7/site-packages/torch/autograd/__init__.py in backward(tensors, grad_tensors, retain_graph, create_graph, grad_variables, inputs)
    197     Variable._execution_engine.run_backward(  # Calls into the C++ engine to run the backward pass
    198         tensors, grad_tensors_, retain_graph, create_graph, inputs,
--&gt; 199         allow_unreachable=True, accumulate_grad=True)  # Calls into the C++ engine to run the backward pass
    200 
    201 def grad(

RuntimeError: element 0 of tensors does not require grad and does not have a grad_fn
</code></pre>
<p>Would anyone be able to help please in this regards<br>
Thanks &amp; Best Regards<br>
AMJS</p>

username: ptrblck
cooked: <p>Rewrapping tensors will detach them from the computation graph:</p>
<pre><code class="lang-python">    labels = torch.Tensor(labels)
    predictions = torch.Tensor(predictions)
</code></pre>
<p>and you would need to use the tensors instead.</p>

username: Schroter
cooked: <aside class="quote no-group" data-username="Schroter" data-post="3" data-topic="174084">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/schroter/48/57964_2.png" class="avatar"> Schroter:</div>
<blockquote>
<pre><code class="lang-auto">outputs = model(inputs)
_, preds = torch.max(outputs, 1)
loss = criterion(labels,preds) #The previous code was: criterion(outputs,labels) 
</code></pre>
</blockquote>
</aside>
<p>Hi, thanks for the reply. In the above code snippet in the last line the original <a href="https://pytorch.org/tutorials/beginner/transfer_learning_tutorial.html" rel="noopener nofollow ugc">code</a> was:<br>
<code>loss = criterion(outputs,labels)</code> as mentioned in my comment. The original code worked well for inbuilt loss function <code>nn.CrossEntropyLoss()</code></p>
<p>Furthermore, when looking at the <code>outputs</code> variable, it is of the form of:</p>
<pre><code class="lang-auto">tensor([[-0.1908,  0.4115],
        [-1.0019, -0.1685],
        [-1.1265, -0.3025],
        [-0.5925, -0.6610],
        [-0.4076, -0.4897],
        [-0.6450, -0.2863],
        [ 0.1632,  0.4944],
        [-1.0743,  0.1003],
        [ 0.6172,  0.5104],
        [-0.2296, -0.0551],
        [-1.3165,  0.3386],
        [ 0.2705,  0.1200],
        [-1.3767, -0.6496],
        [-0.5603,  1.0609],
        [-0.0109,  0.5767],
        [-1.1081,  0.8886]], grad_fn=&lt;AddmmBackward0&gt;)
</code></pre>
<p>From what I understand, the first value of the first list element is the raw model output, and the second value is the prediction value, and the 16 elements in the list represent the batch size. If so, how is the model able to predict without going through the loss function even once? (Since its a pre-trained model?) If I am incorrect then what are the values of the <code>outputs</code> variable?</p>
<p>According to my knowledge I adjusted the loss function and used the original code. Since, I did not want to adjust the <code>train</code> function to accommodate a different <code>loss</code> functions.<br>
Please find the code of the edited cost function below:</p>
<pre><code class="lang-auto">def pfbeta_torch(predictions, labels , beta=1.3):
 '''#labels = torch.tensor(labels.clone().detach().requires_grad(True),dtype=torch.float64)
    #predictions = torch.tensor(predictions.clone().detach().requires_grad(True),dtype=torch.float64)'''
    predictions = torch.tensor([1 if x &gt; 0.5 else 0 for x in predictions[:,1]])
    pTP = torch.sum(labels * predictions)
    pFP = torch.sum((1-labels) * predictions)
    num_positives = torch.sum(labels)  #  = pTP+pFN

    pPrecision = pTP/(pTP+pFP)
    pRecall = pTP/num_positives

    beta_squared = beta**2

    if (pPrecision &gt; 0 and pRecall &gt; 0):
        pF1 = (1+beta_squared) * pPrecision * pRecall/(beta_squared*pPrecision + pRecall)
        return pF1
    else:
        return 0
</code></pre>
<p>Now I get a different error please have a look below:</p>
<pre><code class="lang-auto">---------------------------------------------------------------------------
RuntimeError                              Traceback (most recent call last)
/tmp/ipykernel_27/1892088165.py in &lt;module&gt;
----&gt; 1 model_conv = train_model(model_conv, criterion, optimizer_conv, exp_lr_scheduler, num_epochs=1)

/tmp/ipykernel_27/3271713126.py in train_model(model, criterion, optimizer, scheduler, num_epochs)
     37                     # backward + optimize only if in training phase
     38                     if phase == 'train':
---&gt; 39                         loss.backward()
     40                         optimizer.step()
     41 

/opt/conda/lib/python3.7/site-packages/torch/_tensor.py in backward(self, gradient, retain_graph, create_graph, inputs)
    486             )
    487         torch.autograd.backward(
--&gt; 488             self, gradient, retain_graph, create_graph, inputs=inputs
    489         )
    490 

/opt/conda/lib/python3.7/site-packages/torch/autograd/__init__.py in backward(tensors, grad_tensors, retain_graph, create_graph, grad_variables, inputs)
    197     Variable._execution_engine.run_backward(  # Calls into the C++ engine to run the backward pass
    198         tensors, grad_tensors_, retain_graph, create_graph, inputs,
--&gt; 199         allow_unreachable=True, accumulate_grad=True)  # Calls into the C++ engine to run the backward pass
    200 
    201 def grad(

RuntimeError: element 0 of tensors does not require grad and does not have a grad_fn
</code></pre>
<p>Afterwards I changed the loss function code as below and re-ran it.</p>
<pre><code class="lang-auto">def pfbeta_torch(predictions, labels , beta=1.3):
    #labels = torch.tensor(labels.clone().detach().requires_grad(True),dtype=torch.float64)
    #predictions = torch.tensor(predictions.clone().detach().requires_grad(True),dtype=torch.float64)
    predictions = [1 if x &gt; 0.5 else 0 for x in predictions[:,1]] #Get second element and recode as 1 if value &gt; than .5 else 0
    predictions = torch.tensor(predictions, requires_grad=True) #Loss function changed here
    pTP = torch.sum(labels * predictions)
    pFP = torch.sum((1-labels) * predictions)
    num_positives = torch.sum(labels)  #  = pTP+pFN

    pPrecision = pTP/(pTP+pFP)
    pRecall = pTP/num_positives

    beta_squared = beta**2

    if (pPrecision &gt; 0 and pRecall &gt; 0):
        pF1 = (1+beta_squared) * pPrecision * pRecall/(beta_squared*pPrecision + pRecall)
        return pF1
    else:
        return 0
</code></pre>
<p>However, I get another error:</p>
<pre><code class="lang-auto">---------------------------------------------------------------------------
RuntimeError                              Traceback (most recent call last)
/tmp/ipykernel_27/1892088165.py in &lt;module&gt;
----&gt; 1 model_conv = train_model(model_conv, criterion, optimizer_conv, exp_lr_scheduler, num_epochs=1)

/tmp/ipykernel_27/3271713126.py in train_model(model, criterion, optimizer, scheduler, num_epochs)
     33                     _, preds = torch.max(outputs, 1)
     34                     print(outputs,labels)
---&gt; 35                     loss = criterion(outputs,labels) #Changed here to criterion(labels,preds)
     36 
     37                     # backward + optimize only if in training phase

/tmp/ipykernel_27/3035900216.py in pfbeta_torch(predictions, labels, beta)
      3     #predictions = torch.tensor(predictions.clone().detach().requires_grad(True),dtype=torch.float64)
      4     predictions = [1 if x &gt; 0.5 else 0 for x in predictions[:,1]] #Get second element and recode as 1 if value &gt; than .5 else 0
----&gt; 5     predictions = torch.tensor(predictions,requires_grad=True)
      6     pTP = torch.sum(labels * predictions)
      7     pFP = torch.sum((1-labels) * predictions)

RuntimeError: Only Tensors of floating point and complex dtype can require gradients

</code></pre>
<p>Would you be able to help me in this matter please.<br>
Thanks &amp; Best Regards<br>
AMJS</p>

username: Schroter
cooked: <aside class="quote no-group" data-username="ptrblck" data-post="4" data-topic="174084">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/ptrblck/48/1823_2.png" class="avatar"> ptrblck:</div>
<blockquote>
<pre><code class="lang-auto">labels = torch.Tensor(labels)
    predictions = torch.Tensor(predictions)
</code></pre>
</blockquote>
</aside>
<p>Hi, I am not sure that I understand what you said. Could you please elaborate considering my additional info as well<br>
Thanks &amp; Best Regards<br>
AMJS</p>

username: ptrblck
cooked: <p>You are still recreating tensors in your updated code, now with the <code>requires_grad=True</code> attribute, which is wrong as it will still detach the tensor from the computation graph.<br>
Besides that you are trying to recreate the tensor containing integer values, which is also invalid since only floating point values can require gradients.<br>
This threshold operation:</p>
<pre><code class="lang-python">predictions = [1 if x &gt; 0.5 else 0 for x in predictions[:,1]]
</code></pre>
<p>creates the integer values and detaches <code>prediction</code> from the computation graph since the comparison is not differentiable.</p>

username: Schroter
cooked: <p>Hi, Thanks for the reply. This is the new <code>loss</code> function which I have edited:</p>
<pre><code class="lang-auto">def pfbeta_torch(predictions, labels, beta=1.3):
    #labels = torch.tensor(labels.clone().detach(), dtype=torch.float64, requires_grad=True)
    predictions = torch.tensor(predictions.clone(), dtype=torch.float64, requires_grad=True)
    pTP = torch.sum(labels * predictions)
    pFP = torch.sum((1 - labels) * predictions)
    num_positives = torch.sum(labels)  #  = pTP+pFN

    pPrecision = pTP / (pTP + pFP)
    pRecall = pTP / num_positives

    beta_squared = beta ** 2
    # x=0
    if (pPrecision &gt; 0 and pRecall &gt; 0):
        pF1 = (1 + beta_squared) * pPrecision * pRecall / (beta_squared * pPrecision + pRecall)
        return pF1
    else:
        return torch.tensor(0, dtype=torch.float64, requires_grad=True)
</code></pre>
<p>The inputs for the function from the train loop:</p>
<pre><code class="lang-auto">preds:  tensor([0, 0, 1, 1, 1, 1, 1, 1, 1, 0, 1, 1, 1, 1, 1, 0])
labels:  tensor([0, 1, 0, 0, 1, 1, 1, 0, 0, 1, 1, 0, 0, 1, 1, 1])
</code></pre>
<p>Nevertheless I still get an warning as below:</p>
<pre><code class="lang-auto">/opt/conda/lib/python3.7/site-packages/ipykernel_launcher.py:3: UserWarning: To copy construct from a tensor, it is recommended to use sourceTensor.clone().detach() or sourceTensor.clone().detach().requires_grad_(True), rather than torch.tensor(sourceTensor).
  This is separate from the ipykernel package so we can avoid doing imports until
</code></pre>
<p>Would you be able help me in this matter please.</p>
<p>Thanks &amp; Best Regards<br>
AMJS</p>

username: thecho7
cooked: <p>To solve the following warning,</p>
<pre><code class="lang-auto">/opt/conda/lib/python3.7/site-packages/ipykernel_launcher.py:3: UserWarning: To copy construct from a tensor, it is recommended to use sourceTensor.clone().detach() or sourceTensor.clone().detach().requires_grad_(True), rather than torch.tensor(sourceTensor).
</code></pre>
<p>Modify</p>
<pre><code class="lang-auto">predictions = torch.tensor(predictions.clone(), dtype=torch.float64, requires_grad=True)
</code></pre>
<p>to</p>
<pre><code class="lang-auto">predictions = predictions.clone().detach().requires_grad_(True)
</code></pre>
<p><code>clone</code>: copy a tensor (with gradient if the source has it)<br>
<code>detach</code>: remove the computational graph from the tensor<br>
<code>requires_grad_(True)</code>: in-place operation that assign the target tensor to have a graph</p>

username: Schroter
cooked: <p>Hi,<br>
Thanks for the reply. However, it gives me this error:<br>
<code>RuntimeError: only Tensors of floating point dtype can require gradients</code>.</p>
<p>Thanks &amp; Best Regards<br>
AMJS</p>

username: thecho7
cooked: <pre><code class="lang-auto">predictions = predictions.clone().detach().requires_grad_(True).to(torch.float64)
</code></pre>

username: ptrblck
cooked: <p>You are again recreating the <code>predictions</code> tensor:</p>
<pre><code class="lang-python">predictions = torch.tensor(predictions.clone(), dtype=torch.float64, requires_grad=True)
</code></pre>
<p>which will detach it from the computation graph and raise the warning. Just remove this line of code.</p>
<p><a class="mention" href="/u/thecho7">@thecho7</a>’s suggestion will get rid of the error but will explicitly detach the tensor. If you want to use <code>pfbeta_torch</code> as a loss function and want eventually call <code>.backward()</code> on the output tensor, this is also wrong.</p>

username: Schroter
cooked: <p>Hi,<br>
Thanks for the reply. After commenting out the line in question I ran the code again. Only to get a another error which is below;</p>
<pre><code class="lang-auto">---------------------------------------------------------------------------
RuntimeError                              Traceback (most recent call last)
/tmp/ipykernel_23/2530295950.py in &lt;module&gt;
----&gt; 1 model_conv = train_model(model_conv, criterion, optimizer_conv, exp_lr_scheduler, num_epochs=100)

/tmp/ipykernel_23/2815325787.py in train_model(model, criterion, optimizer, scheduler, num_epochs)
     39                     # backward + optimize only if in training phase
     40                     if phase == 'train':
---&gt; 41                         loss.backward()
     42                         optimizer.step()
     43 

/opt/conda/lib/python3.7/site-packages/torch/_tensor.py in backward(self, gradient, retain_graph, create_graph, inputs)
    486             )
    487         torch.autograd.backward(
--&gt; 488             self, gradient, retain_graph, create_graph, inputs=inputs
    489         )
    490 

/opt/conda/lib/python3.7/site-packages/torch/autograd/__init__.py in backward(tensors, grad_tensors, retain_graph, create_graph, grad_variables, inputs)
    197     Variable._execution_engine.run_backward(  # Calls into the C++ engine to run the backward pass
    198         tensors, grad_tensors_, retain_graph, create_graph, inputs,
--&gt; 199         allow_unreachable=True, accumulate_grad=True)  # Calls into the C++ engine to run the backward pass
    200 
    201 def grad(

RuntimeError: element 0 of tensors does not require grad and does not have a grad_fn

</code></pre>
<p>Would you be able to help me in this regards<br>
Thanks &amp; Best Regards<br>
AMJS</p>

username: ptrblck
cooked: <p>I guess you are still using non-differentiable operations, such as the threshold operation, as already mentioned:</p>
<aside class="quote no-group" data-username="ptrblck" data-post="8" data-topic="174084">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/ptrblck/48/1823_2.png" class="avatar"> ptrblck:</div>
<blockquote>
<p>This threshold operation:</p>
<pre><code class="lang-auto">predictions = [1 if x &gt; 0.5 else 0 for x in predictions[:,1]]
</code></pre>
<p>creates the integer values and detaches <code>prediction</code> from the computation graph since the comparison is not differentiable.</p>
</blockquote>
</aside>
<p>If not, check if you are still somewhere re-wrapping a tensor or if any operation creates an integer type output tensor or generally removes the <code>.grad_fn</code> attribute of a tensor.</p>

username: Schroter
cooked: <aside class="quote no-group quote-modified" data-username="Schroter" data-post="9" data-topic="174084">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/schroter/48/57964_2.png" class="avatar"> Schroter:</div>
<blockquote>
<pre><code class="lang-auto">def pfbeta_torch(predictions, labels, beta=1.3):
    #labels = torch.tensor(labels.clone().detach(), dtype=torch.float64, requires_grad=True)
    predictions = torch.tensor(predictions.clone(), dtype=torch.float64, requires_grad=True)
    pTP = torch.sum(labels * predictions)
    pFP = torch.sum((1 - labels) * predictions)
    num_positives = torch.sum(labels)  #  = pTP+pFN

    pPrecision = pTP / (pTP + pFP)
    pRecall = pTP / num_positives

    beta_squared = beta ** 2
    # x=0
    if (pPrecision &gt; 0 and pRecall &gt; 0):
        pF1 = (1 + beta_squared) * pPrecision * pRecall / (beta_squared * pPrecision + pRecall)
        return pF1
    else:
        return torch.tensor(0, dtype=torch.float64, requires_grad=True)
</code></pre>
<p>The inputs for the function from the train loop  ():</p>
<pre><code class="lang-auto">preds:  tensor([0, 0, 1, 1, 1, 1, 1, 1, 1, 0, 1, 1, 1, 1, 1, 0])
labels:  tensor([0, 1, 0, 0, 1, 1, 1, 0, 0, 1, 1, 0, 0, 1, 1, 1])
</code></pre>
<p>Nevertheless I still get an warning as below:</p>
<pre><code class="lang-auto">/opt/conda/lib/python3.7/site-packages/ipykernel_launcher.py:3: UserWarning: To copy construct from a tensor, it is recommended to use sourceTensor.clone().detach() or sourceTensor.clone().detach().requires_grad_(True), rather than torch.tensor(sourceTensor).
  This is separate from the ipykernel package so we can avoid doing imports until
</code></pre>
<p>Would you be able help me in this matter please.</p>
</blockquote>
</aside>
<p>Hi, As above I have changes my <code>loss</code> function since. Also, would you be able to explain why the tensor list <code>preds</code> do not have AutoGrad <code>(requires_grad=True)</code> attached to it for back propagation purpose please. This <a href="https://pytorch.org/docs/stable/generated/torch.nn.L1Loss.html" rel="noopener nofollow ugc">example</a> from <a href="https://pytorch.org/docs/stable/generated/torch.nn.L1Loss.html" rel="noopener nofollow ugc">pytorch</a> clearly shows that its needed. Not sure why it is not generated in my in my example which I have borrowed from <a href="https://pytorch.org/tutorials/beginner/transfer_learning_tutorial.html" rel="noopener nofollow ugc">this</a> pytorch tutorial.</p>
<p>Please let me know your feedback on this matter.<br>
Thanks &amp; Best Regards<br>
AMJS</p>