Data not balanced

username: mathwseg
cooked: <p>I have an issue in the size of the dataset which is relatively small in some classes as its distribution is not balanced well enough. Although I used augmentation and got the accuracy 93 % but the confusion matrix is  very bad in some classes (got zero in precision and recall) . How can i solve this problem. i can drop the classes which causes this issue but i need to try solutions to handle this classes please can someone help me</p>
<p>the loss curve like<br>
<div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/d/6/d657e80bcc250c583e5f136f133a110e250f7f2c.png" data-download-href="https://discuss.pytorch.org/uploads/default/d657e80bcc250c583e5f136f133a110e250f7f2c" title="image"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/d/6/d657e80bcc250c583e5f136f133a110e250f7f2c_2_690x355.png" alt="image" data-base62-sha1="uAaw4D7YPscj6xUNIybQ3Vva4AA" width="690" height="355" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/d/6/d657e80bcc250c583e5f136f133a110e250f7f2c_2_690x355.png, https://discuss.pytorch.org/uploads/default/optimized/3X/d/6/d657e80bcc250c583e5f136f133a110e250f7f2c_2_1035x532.png 1.5x, https://discuss.pytorch.org/uploads/default/original/3X/d/6/d657e80bcc250c583e5f136f133a110e250f7f2c.png 2x" data-dominant-color="FBFAF9"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">image</span><span class="informations">1063×547 66 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>
<p>Accuracy is 93 % from the curve but got average accuracy 0.89 from confusion<br>
average precision is 0.29<br>
average recall is 0.23<br>
average f-score is 0.24</p>

username: soulitzer
cooked: <p>What does the distribution in your classes look like?</p>

username: J_Johnson
cooked: <p>Here is how you can create a weighted vector for scaling the loss of various classes to their representation in the training dataset:</p>
<aside class="quote quote-modified" data-post="2" data-topic="194579">
  <div class="title">
    <div class="quote-controls"></div>
    <img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/j_johnson/48/55494_2.png" class="avatar">
    <a href="https://discuss.pytorch.org/t/how-do-i-get-the-accuracy-of-an-unbalanced-dataset-or-segmentation-task/194579/2">How do I get the accuracy of an unbalanced dataset or segmentation task?</a> <a class="badge-category__wrapper " href="/c/faq/30"><span data-category-id="30" style="--category-badge-color: #0088CC; --category-badge-text-color: #FFFFFF;" data-drop-close="true" class="badge-category " title="The FAQ category contains commonly-asked questions and their answers. Please refer to this section before you post your query."><span class="badge-category__name">FAQ</span></span></a>
  </div>
  <blockquote>
    Here is a function for calculating the weight vector used by CrossEntropyLoss based on a set of targets with integers. 
def scaled_weights(targets):
    '''
    Gets class weights
    :param targets: pass in the matrix of size (batch, classes) containing correct targets or
        labels as one_hot
    :return: returns a vector for weight rescaling for use in cross entropy loss function
    '''

    N, C = targets.size(0), targets.size(1)

    # get vector of total for each class
    totals = ta…
  </blockquote>
</aside>

<p>That gets passed into PyTorch <a href="https://pytorch.org/docs/stable/generated/torch.nn.CrossEntropyLoss.html" rel="noopener nofollow ugc">Cross Entropy Loss</a> <code>weight</code> argument.</p>

username: mathwseg
cooked: <aside class="quote no-group" data-username="soulitzer" data-post="2" data-topic="214375" data-full="true">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/s/839c29/48.png" class="avatar"> soulitzer:</div>
<blockquote>
<p>What does the distribution in your classes look like?</p>
</blockquote>
</aside>
<p><div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/e/9/e9bf0160c7e1d2f7e19cbfe6507ebd910b06d856.png" data-download-href="https://discuss.pytorch.org/uploads/default/e9bf0160c7e1d2f7e19cbfe6507ebd910b06d856" title="image"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/e/9/e9bf0160c7e1d2f7e19cbfe6507ebd910b06d856_2_690x342.png" alt="image" data-base62-sha1="xlOtCvG19Nj3uSeTpGNVnAYxSFo" width="690" height="342" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/e/9/e9bf0160c7e1d2f7e19cbfe6507ebd910b06d856_2_690x342.png, https://discuss.pytorch.org/uploads/default/optimized/3X/e/9/e9bf0160c7e1d2f7e19cbfe6507ebd910b06d856_2_1035x513.png 1.5x, https://discuss.pytorch.org/uploads/default/original/3X/e/9/e9bf0160c7e1d2f7e19cbfe6507ebd910b06d856.png 2x" data-dominant-color="E4F3F9"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">image</span><span class="informations">1226×609 20.3 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>

username: mathwseg
cooked: <p>i will try it now . Thanks</p>

username: mathwseg
cooked: <p>the curve after using scaled weights<br>
<div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/3/f/3fbcfd6aed32d046ef5dd3502ac82691515e4313.png" data-download-href="https://discuss.pytorch.org/uploads/default/3fbcfd6aed32d046ef5dd3502ac82691515e4313" title="image"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/3/f/3fbcfd6aed32d046ef5dd3502ac82691515e4313_2_690x356.png" alt="image" data-base62-sha1="95QZmftbtlXx333RCBsRcOK77p1" width="690" height="356" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/3/f/3fbcfd6aed32d046ef5dd3502ac82691515e4313_2_690x356.png, https://discuss.pytorch.org/uploads/default/optimized/3X/3/f/3fbcfd6aed32d046ef5dd3502ac82691515e4313_2_1035x534.png 1.5x, https://discuss.pytorch.org/uploads/default/original/3X/3/f/3fbcfd6aed32d046ef5dd3502ac82691515e4313.png 2x" data-dominant-color="FCFBFA"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">image</span><span class="informations">1060×548 49.8 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div><br>
accuracy become 93.37 but sill the confusion is bad<br>
i’m using this code to load the .pth file after trained the model</p>
<pre><code class="lang-auto">import torch
import numpy as np
from sklearn.metrics import confusion_matrix
import seaborn as sns
import matplotlib.pyplot as plt

# Load the trained model
checkpoint_path = 'checkpoint.pth'
checkpoint = torch.load(checkpoint_path)
model.load_state_dict(checkpoint['model_state_dict'])
model.to(device)
model.eval()

# Evaluate on the validation or test set
y_true = []
y_pred = []

with torch.no_grad():
    for inputs, labels in tqdm(val_loader): 
        inputs, labels = inputs.to(device), labels.to(device)
        outputs = model(inputs)
        _, preds = torch.max(outputs, 1)

        y_true.extend(labels.cpu().numpy())
        y_pred.extend(preds.cpu().numpy())

# Compute confusion matrix
conf_matrix = confusion_matrix(y_true, y_pred)

# Plot confusion matrix
plt.figure(figsize=(10, 8))
sns.heatmap(conf_matrix, annot=True, fmt='d', cmap='Blues', xticklabels=classes, yticklabels=classes)
plt.xlabel('Predicted Labels')
plt.ylabel('True Labels')
plt.title('Confusion Matrix')
plt.show()
</code></pre>