Repository: flairNLP/flair

Issue:
TextClassifier : reported metrics after training always report precision=1.0
**Describe the bug**
The reported metrics after training always report precision=1.0.

**To Reproduce**

Training code:
```python
from torch.optim.adam import Adam

from flair.data import Corpus
from flair.datasets import TREC_6
from flair.embeddings import TransformerDocumentEmbeddings
from flair.models import TextClassifier
from flair.trainers import ModelTrainer

# 1. get the corpus
corpus: Corpus = TREC_6()

# 2. create the label dictionary
label_dict = corpus.make_label_dictionary()

# 3. initialize transformer document embeddings (many models are available)
document_embeddings = TransformerDocumentEmbeddings('distilbert-base-uncased', fine_tune=True)

# 4. create the text classifier
classifier = TextClassifier(document_embeddings, label_dictionary=label_dict)

# 5. initialize the text classifier trainer with Adam optimizer
trainer = ModelTrainer(classifier, corpus, optimizer=Adam)

# 6. start the training
trainer.train('/tmp/taggers/trec',
              learning_rate=3e-5, # use very small learning rate
              mini_batch_size=16,
              mini_batch_chunk_size=4, # optionally set this if transformer is too much for your machine
              max_epochs=5, # terminate after 5 epochs
              )
 ```

Example of produced report:
```text
2020-07-09 09:50:21,395 Testing using best model ...
2020-07-09 09:50:21,395 loading file /tmp/taggers/trec/best-model.pt
2020-07-09 09:50:27,486         0.964
2020-07-09 09:50:27,487 
Results:
- F-score (micro) 0.9823
- F-score (macro) 0.9745
- Accuracy 0.964

By class:
              precision    recall  f1-score   support

        DESC     1.0000    0.9931    0.9965       145
        ENTY     1.0000    0.8750    0.9333        96
        ABBR     1.0000    0.8889    0.9412         9
         HUM     1.0000    0.9851    0.9925        67
         NUM     1.0000    0.9915    0.9957       117
         LOC     1.0000    0.9762    0.9880        84

   micro avg     1.0000    0.9653    0.9823       518
   macro avg     1.0000    0.9516    0.9745       518
weighted avg     1.0000    0.9653    0.9818       518
 samples avg     1.0000    0.9820    0.9880       518

2020-07-09 09:50:27,487 ----------------------------------------------------------------------------------------------------
```


**Expected behavior**
Reports correct metrics.

**Screenshots**
N/A

**Environment (please complete the following information):**
 - OS [e.g. iOS, Linux]: CentOS
 - Version: flair 0.5.1, scikit-learn==0.23.1

**Additional context**
Same problem with other datasets.


Discussion:

Resolving patch:
diff --git a/flair/data.py b/flair/data.py
--- a/flair/data.py
+++ b/flair/data.py
@@ -218,6 +218,10 @@ def set_label(self, label_type: str, value: str, score: float = 1.):
 
         return self
 
+    def remove_labels(self, label_type: str):
+        if label_type in self.annotation_layers.keys():
+            del self.annotation_layers[label_type]
+
     def get_labels(self, label_type: str = None):
         if label_type is None:
             return self.labels
diff --git a/flair/models/text_classification_model.py b/flair/models/text_classification_model.py
--- a/flair/models/text_classification_model.py
+++ b/flair/models/text_classification_model.py
@@ -114,6 +114,7 @@ def _get_state_dict(self):
             "state_dict": self.state_dict(),
             "document_embeddings": self.document_embeddings,
             "label_dictionary": self.label_dictionary,
+            "label_type": self.label_type,
             "multi_label": self.multi_label,
             "beta": self.beta,
             "weight_dict": self.weight_dict,
@@ -176,7 +177,7 @@ def predict(
         'gpu' to store embeddings in GPU memory.
         """
         if label_name == None:
-            label_name = self.label_type if self.label_type is not None else 'class'
+            label_name = self.label_type if self.label_type is not None else 'label'
 
         with torch.no_grad():
             if not sentences:
@@ -268,6 +269,12 @@ def evaluate(
 
                 batch_count += 1
 
+                # remove previously predicted labels
+                [sentence.remove_labels('predicted') for sentence in batch]
+
+                # get the gold labels
+                true_values_for_batch = [sentence.get_labels(self.label_type) for sentence in batch]
+
                 # predict for batch
                 loss = self.predict(batch,
                                     embedding_storage_mode=embedding_storage_mode,
@@ -279,7 +286,7 @@ def evaluate(
 
                 sentences_for_batch = [sent.to_plain_string() for sent in batch]
 
-                true_values_for_batch = [sentence.get_labels(self.label_type) for sentence in batch]
+                # get the predicted labels
                 predictions = [sentence.get_labels('predicted') for sentence in batch]
 
                 for sentence, prediction, true_value in zip(


Test patch: