Classifier predicting wrong label on single sentence during inference time

Soumya_Jain: <p>I designed a classifier for the customer review dataset. I am getting high training as well as test accuracy. But when I am using the pre-trained model for classifying a single sentence (checked a sentence from both training and test set), the model is predicting the wrong answer with high probability.<br>
This is my code-</p>
<pre><code class="lang-auto">def inference(text):
  doc_idx = [fields[0][1].vocab.stoi[token] for token in text]
  doc_idx_tensor = torch.tensor([doc_idx]).cuda()
  len_tensor = torch.tensor([len(doc_idx)])
  with torch.no_grad():
    model = torch.load(PATH)
    model.eval()
    prediction = model(doc_idx_tensor, len_tensor)
    label = torch.argmax(prediction)
    print(prediction , label)
  return
</code></pre>
<p>I don’t know what to do. Any help will be highly appreciated. Thanks in advance.</p>
<p>Here is an example -<br>
text - ‘the product is not good’</p>
<pre><code class="lang-auto">output - [[-9.544 17.824]] tensor(1, device='cuda:0')
</code></pre>

asvskartheek: <p><a class="mention" href="/u/soumya_jain">@Soumya_Jain</a> can you check if during the training your label encoding is such that 1 is negative and 0 is positive? Since the training and test accuracy are high, then I’m guessing this might be the issue.</p>

Soumya_Jain: <p><a class="mention" href="/u/asvskartheek">@asvskartheek</a>, Thanks for the reply.<br>
I checked it. During training, my label encoding is 0 for negative and 1 for positive only.</p>

asvskartheek: <p>Is this the only example for which there is a misclassification? This can be the case because, at a fundamental level, ML models are probabilistic; even a 99% test accuracy model can fail over a trivial data point. I would suggest you test this over other examples (try something like 20-30 if manual); if it is working well on those examples, it is acceptable for a model to make this mistake. If the majority of the cases are misclassification, then I guess you have to try different architectures to design a better model.</p>

KarthikR: <p>Just curious, what is the pre-trained model you are using? I would like to see the last layer (which can confirm your ‘high’ values are perhaps what is expected and your approach to take argmax is right).<br>
It could also be you may need to fine-tune the ‘head’ of the model, to classify better?</p>

Soumya_Jain: <p><a class="mention" href="/u/asvskartheek">@asvskartheek</a><br>
No, this is happening for almost all examples.</p>

Soumya_Jain: <p><a class="mention" href="/u/karthikr">@KarthikR</a></p>
<p>This is how I am comparing true and predicted labels-</p>
<pre><code class="lang-auto">num_corrects = (torch.max(predictions_batch, 1)[1].view(target.size()).data == target.data).float().sum()
</code></pre>
<p>This is the last layer-</p>
<pre><code class="lang-auto"> h = self.fc(self.dropout(h))
</code></pre>
<p>I am saving the model with the best validation accuracy and then using that model for classifying single sentences.</p>

KarthikR: <p>ok, what is the pre-trained model and are you adding layers to the end of the model? Can you share the model architecture?</p>