Repository: deepset-ai/haystack

Issue:
`TransformersDocumentClassifier`: inconsistent output between ordinary and zero-shot classification
**Describe the bug**
When using `TransformersDocumentClassifier`, the output structure is different between ordinary and zero-shot classification.
For example, for the same document, we have these different outputs for `doc.meta['classification']`

*Classification*:
`{'label': 'joy', 'score': 0.9433773756027222}`

*Zero-shot classification*:
```
{'sequence': "The soundtrack album for the second season of HBO series ''Game of Thrones'', titled '''''Game of Thrones: Season 2''''', was published on June 19, 2012. The instrumental music by Ramin Djawadi was performed by the Czech Film Orchestra and Choir and recorded at the Rudolfinum concert hall in Prague.",
 'labels': ['music', 'history', 'natural language processing'],
 'scores': [0.953018069267273, 0.023982945829629898, 0.02299901656806469],
 'label': 'music'}
```

**Expected behavior**
I think that the output structure should always be the same.

**To Reproduce**

```python
from haystack import Document
from haystack.nodes import TransformersDocumentClassifier

doc = Document.from_dict(
    {'content': "The soundtrack album for the second season of HBO series ''Game of Thrones'', titled '''''Game of Thrones: Season 2''''', was published on June 19, 2012. The instrumental music by Ramin Djawadi was performed by the Czech Film Orchestra and Choir and recorded at the Rudolfinum concert hall in Prague.",
 'content_type': 'text',
 'meta': {'name': '25_Game_of_Thrones__Season_2__soundtrack_.txt'
  }})

doc_classifier = TransformersDocumentClassifier(model_name_or_path="bhadresh-savani/distilbert-base-uncased-emotion")
doc_classifier_zero_shot = TransformersDocumentClassifier(
    model_name_or_path="cross-encoder/nli-distilroberta-base",
    task="zero-shot-classification",
    labels=["music", "natural language processing", "history"]
)

print(doc_classifier.predict([doc])[0].meta['classification'])
print(doc_classifier_zero_shot.predict([doc])[0].meta['classification'])
```

**FAQ Check**
- [X] Have you had a look at [our new FAQ page](https://haystack.deepset.ai/overview/faq)?

**System:**
 - Haystack version (commit or version number): 1.8.1rc0

**How do we want to tackle this issue?
Can the output structure for ordinary classification** (similar to that reported in [docstrings/documentation](https://github.com/deepset-ai/haystack/blob/356537c88332cca0e1d6123a0f0402f2b409e6e5/haystack/nodes/document_classifier/transformers.py#L23-L24)) **be considered the correct one?**


Discussion:
@ZanSara any thoughts on this?
If you agree with my approach (adapt the output of the zero-shot classification to the ordinary output), I can take charge of this issue.
Hey @anakin87! Actually I believe the zero-shot output to be more informative! We can still generalize by adapting the regular classifier's output to the zero-shot one. Something like:

Regular: 
```
{
  'labels': ['music'],
  'scores': [0.953018069267273],
}
```
and zero-shot: 
```
{
  'labels': ['music', 'history', 'natural language processing'],
  'scores': [0.953018069267273, 0.023982945829629898, 0.02299901656806469],
}
```
What do you think?  (We should update the docstring in this case ofc).

If you think there's a risk of making this dict huge and noisy due to the presence of thousands of labels, we can also add a filtering param, like a top_k or a threshold, to the class and reduce the labels list that way.
I'm a bit unsure 🤔 

I agree that the zero-shot output conveys more information.
Nonetheless, for filtering purposes, the regular output seems more suitable (see [docs](https://haystack.deepset.ai/tutorials/doc-class-index#querying-the-data)).

I'm also thinking of how to adapt this possible output to fix #3019...

@ZanSara If you come out with a conclusive idea,  please let me know...
After all I think this is a case where both options are good enough. It won't be an issue to stick with your idea, but I'll leave the last word to @masci to make sure.
Another option for `doc.meta['classification']` could be like the following:
```
{
  'label': 'music',
  'details': {
                 'music': 0.953018069267273,
                 'history': 0.023982945829629898,
                 'natural language processing': 0.02299901656806469
                 }
}
```

The `label` field may allow straight-forward filtering/routing,
while `details` would contain complete information on the classification.
I like this last approach! Let's use that :+1:

Resolving patch:
diff --git a/haystack/nodes/document_classifier/transformers.py b/haystack/nodes/document_classifier/transformers.py
--- a/haystack/nodes/document_classifier/transformers.py
+++ b/haystack/nodes/document_classifier/transformers.py
@@ -21,7 +21,7 @@ class TransformersDocumentClassifier(BaseDocumentClassifier):
     While the underlying model can vary (BERT, Roberta, DistilBERT ...), the interface remains the same.
     This node classifies documents and adds the output from the classification step to the document's meta data.
     The meta field of the document is a dictionary with the following format:
-    ``'meta': {'name': '450_Baelor.txt', 'classification': {'label': 'neutral', 'probability' = 0.9997646, ...} }``
+    ``'meta': {'name': '450_Baelor.txt', 'classification': {'label': 'love', 'score': 0.960899, 'details': {'love': 0.960899, 'joy': 0.032584, ...}}}``
 
     Classification is run on document's content field by default. If you want it to run on another field,
     set the `classification_field` to one of document's meta fields.
@@ -70,7 +70,7 @@ def __init__(
         model_version: Optional[str] = None,
         tokenizer: Optional[str] = None,
         use_gpu: bool = True,
-        return_all_scores: bool = False,
+        top_k: Optional[int] = 1,
         task: str = "text-classification",
         labels: Optional[List[str]] = None,
         batch_size: int = 16,
@@ -98,7 +98,7 @@ def __init__(
         :param model_version: The version of model to use from the HuggingFace model hub. Can be tag name, branch name, or commit hash.
         :param tokenizer: Name of the tokenizer (usually the same as model)
         :param use_gpu: Whether to use GPU (if available).
-        :param return_all_scores:  Whether to return all prediction scores or just the one of the predicted class. Only used for task 'text-classification'.
+        :param top_k: The number of top predictions to return. The default is 1. Enter None to return all the predictions. Only used for task 'text-classification'.
         :param task: 'text-classification' or 'zero-shot-classification'
         :param labels: Only used for task 'zero-shot-classification'. List of string defining class labels, e.g.,
         ["positive", "negative"] otherwise None. Given a LABEL, the sequence fed to the model is "<cls> sequence to
@@ -150,10 +150,10 @@ def __init__(
                 tokenizer=tokenizer,
                 device=resolved_devices[0],
                 revision=model_version,
-                return_all_scores=return_all_scores,
+                top_k=top_k,
                 use_auth_token=use_auth_token,
             )
-        self.return_all_scores = return_all_scores
+        self.top_k = top_k
         self.labels = labels
         self.task = task
         self.batch_size = batch_size
@@ -177,22 +177,31 @@ def predict(self, documents: List[Document], batch_size: Optional[int] = None) -
             for doc in documents
         ]
         batches = self.get_batches(texts, batch_size=batch_size)
-        batched_predictions = []
-        pb = tqdm(total=len(texts), disable=not self.progress_bar, desc="Generating questions")
+        predictions = []
+        pb = tqdm(total=len(texts), disable=not self.progress_bar, desc="Classifying documents")
         for batch in batches:
             if self.task == "zero-shot-classification":
                 batched_prediction = self.model(batch, candidate_labels=self.labels, truncation=True)
             elif self.task == "text-classification":
-                batched_prediction = self.model(batch, return_all_scores=self.return_all_scores, truncation=True)
-            batched_predictions.append(batched_prediction)
+                batched_prediction = self.model(batch, top_k=self.top_k, truncation=True)
+            predictions.extend(batched_prediction)
             pb.update(len(batch))
         pb.close()
-        predictions = [pred for batched_prediction in batched_predictions for pred in batched_prediction]
 
         for prediction, doc in zip(predictions, documents):
             if self.task == "zero-shot-classification":
-                prediction["label"] = prediction["labels"][0]
-            doc.meta["classification"] = prediction
+                formatted_prediction = {
+                    "label": prediction["labels"][0],
+                    "score": prediction["scores"][0],
+                    "details": {label: score for label, score in zip(prediction["labels"], prediction["scores"])},
+                }
+            elif self.task == "text-classification":
+                formatted_prediction = {
+                    "label": prediction[0]["label"],
+                    "score": prediction[0]["score"],
+                    "details": {el["label"]: el["score"] for el in prediction},
+                }
+            doc.meta["classification"] = formatted_prediction
 
         return documents
 


Test patch:
diff --git a/test/conftest.py b/test/conftest.py
--- a/test/conftest.py
+++ b/test/conftest.py
@@ -621,7 +621,7 @@ def ranker():
 @pytest.fixture
 def document_classifier():
     return TransformersDocumentClassifier(
-        model_name_or_path="bhadresh-savani/distilbert-base-uncased-emotion", use_gpu=False
+        model_name_or_path="bhadresh-savani/distilbert-base-uncased-emotion", use_gpu=False, top_k=2
     )
 
 
diff --git a/test/document_stores/test_faiss_and_milvus.py b/test/document_stores/test_faiss_and_milvus.py
--- a/test/document_stores/test_faiss_and_milvus.py
+++ b/test/document_stores/test_faiss_and_milvus.py
@@ -16,7 +16,7 @@
 from haystack.pipelines import Pipeline
 from haystack.nodes.retriever.dense import EmbeddingRetriever
 
-from ..conftest import document_classifier, ensure_ids_are_correct_uuids, SAMPLES_PATH, MockDenseRetriever
+from ..conftest import ensure_ids_are_correct_uuids, SAMPLES_PATH, MockDenseRetriever
 
 
 DOCUMENTS = [
diff --git a/test/nodes/test_document_classifier.py b/test/nodes/test_document_classifier.py
--- a/test/nodes/test_document_classifier.py
+++ b/test/nodes/test_document_classifier.py
@@ -22,6 +22,16 @@ def test_document_classifier(document_classifier):
         assert doc.to_dict()["meta"]["classification"]["label"] == expected_labels[i]
 
 
+@pytest.mark.integration
+def test_document_classifier_details(document_classifier):
+
+    docs = [Document(content="""That's good. I like it."""), Document(content="""That's bad. I don't like it.""")]
+    results = document_classifier.predict(documents=docs)
+    for doc in results:
+        assert "details" in doc.meta["classification"]
+        assert len(doc.meta["classification"]["details"]) == 2  # top_k = 2
+
+
 @pytest.mark.integration
 def test_document_classifier_batch_single_doc_list(document_classifier):
     docs = [
@@ -65,6 +75,16 @@ def test_zero_shot_document_classifier(zero_shot_document_classifier):
         assert doc.to_dict()["meta"]["classification"]["label"] == expected_labels[i]
 
 
+@pytest.mark.integration
+def test_zero_shot_document_classifier_details(zero_shot_document_classifier):
+
+    docs = [Document(content="""That's good. I like it."""), Document(content="""That's bad. I don't like it.""")]
+    results = zero_shot_document_classifier.predict(documents=docs)
+    for doc in results:
+        assert "details" in doc.meta["classification"]
+        assert len(doc.meta["classification"]["details"]) == 2  # n_labels = 2
+
+
 @pytest.mark.integration
 def test_document_classifier_batch_size(batched_document_classifier):
     assert isinstance(batched_document_classifier, BaseDocumentClassifier)