Repository: huggingface/transformers

Issue:
NER pipeline doesn't work for a list of sequences
## Environment info
<!-- You can run the command `transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->

- `transformers` version: transformers==4.3.2
- Platform: Linux Ubuntu 20.04
- Python version:  3.6
- PyTorch version (GPU?): torch==1.7.0+cu101
- Tensorflow version (GPU?): tensorflow==2.4.1
- Using GPU in script?: No
- Using distributed or parallel set-up in script?: No

### Who can help
<!-- Your issue will be replied to more quickly if you can figure out the right person to tag with @
 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

Models:

- albert, bert, xlm: @LysandreJik
- blenderbot, bart, marian, pegasus, encoderdecoder,  t5: @patrickvonplaten, @patil-suraj
- longformer, reformer, transfoxl, xlnet: @patrickvonplaten
- fsmt: @stas00
- funnel: @sgugger
- gpt2: @patrickvonplaten, @LysandreJik
- rag: @patrickvonplaten, @lhoestq
- tensorflow: @jplu

Library:

- benchmarks: @patrickvonplaten
- deepspeed: @stas00
- ray/raytune: @richardliaw, @amogkam
- text generation: @patrickvonplaten
- tokenizers: @n1t0, @LysandreJik
- trainer: @sgugger
- pipelines: @LysandreJik

Documentation: @sgugger

HF projects:

- nlp datasets: [different repo](https://github.com/huggingface/nlp)
- rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)

Examples:

- maintained examples (not research project or legacy): @sgugger, @patil-suraj
- research_projects/bert-loses-patience: @JetRunner
- research_projects/distillation: @VictorSanh

 -->

Library:

- pipelines: @LysandreJik

Documentation: @sgugger

## Information

Model I am using (Bert, XLNet ...):

The problem arises when using:
* [x] the official example scripts: (give details below)
* [ ] my own modified scripts: (give details below)

The tasks I am working on is:
* [x] an official GLUE/SQUaD task: (give the name)
* [ ] my own task or dataset: (give details below)

## To reproduce

Steps to reproduce the behavior:

1. i used the steps  [here](https://huggingface.co/transformers/task_summary.html#named-entity-recognition) to use pipelines for NER task with a little change, so my script is as follow:
```
from transformers import pipeline
nlp = pipeline("ner")
sequence = [
    "Hugging Face Inc. is a company based in New York City.",
    "Hugging Face Inc. is a company based in New York City. Its headquarters are in DUMBO, therefore very close to the Manhattan Bridge which is visible from the window."
]
print(nlp(sequence))
```

<!-- If you have code snippets, error messages, stack traces please provide them here as well.
     Important! Use code tags to correctly format your code. See https://help.github.com/en/github/writing-on-github/creating-and-highlighting-code-blocks#syntax-highlighting
     Do not use screenshots, as they are hard to read and (more importantly) don't allow others to copy-and-paste your code.-->

## Expected behavior
 i expected to get a list like this:
```
[
    [
        {'word': 'Hu', 'score': 0.999578595161438, 'entity': 'I-ORG', 'index': 1, 'start': 0, 'end': 2}
        {'word': '##gging', 'score': 0.9909763932228088, 'entity': 'I-ORG', 'index': 2, 'start': 2, 'end': 7}
        {'word': 'Face', 'score': 0.9982224702835083, 'entity': 'I-ORG', 'index': 3, 'start': 8, 'end': 12}
        {'word': 'Inc', 'score': 0.9994880557060242, 'entity': 'I-ORG', 'index': 4, 'start': 13, 'end': 16}
        {'word': 'New', 'score': 0.9994344711303711, 'entity': 'I-LOC', 'index': 11, 'start': 40, 'end': 43}
        {'word': 'York', 'score': 0.9993196129798889, 'entity': 'I-LOC', 'index': 12, 'start': 44, 'end': 48}
        {'word': 'City', 'score': 0.9993793964385986, 'entity': 'I-LOC', 'index': 13, 'start': 49, 'end': 53}
    ],
    [
        {'word': 'Hu', 'score': 0.9995632767677307, 'entity': 'I-ORG'},
        {'word': '##gging', 'score': 0.9915938973426819, 'entity': 'I-ORG'},
        {'word': 'Face', 'score': 0.9982671737670898, 'entity': 'I-ORG'},
        {'word': 'Inc', 'score': 0.9994403719902039, 'entity': 'I-ORG'},
        {'word': 'New', 'score': 0.9994346499443054, 'entity': 'I-LOC'},
        {'word': 'York', 'score': 0.9993270635604858, 'entity': 'I-LOC'},
        {'word': 'City', 'score': 0.9993864893913269, 'entity': 'I-LOC'},
        {'word': 'D', 'score': 0.9825621843338013, 'entity': 'I-LOC'},
        {'word': '##UM', 'score': 0.936983048915863, 'entity': 'I-LOC'},
        {'word': '##BO', 'score': 0.8987102508544922, 'entity': 'I-LOC'},
        {'word': 'Manhattan', 'score': 0.9758241176605225, 'entity': 'I-LOC'},
        {'word': 'Bridge', 'score': 0.990249514579773, 'entity': 'I-LOC'}
    ]
]
```
but i got this error

```
ValueError                                Traceback (most recent call last)
/usr/local/lib/python3.6/dist-packages/transformers/tokenization_utils_base.py in convert_to_tensors(self, tensor_type, prepend_batch_axis)
    770                 if not is_tensor(value):
--> 771                     tensor = as_tensor(value)
    772 

ValueError: expected sequence of length 16 at dim 1 (got 38)

During handling of the above exception, another exception occurred:

ValueError                                Traceback (most recent call last)
6 frames
/usr/local/lib/python3.6/dist-packages/transformers/tokenization_utils_base.py in convert_to_tensors(self, tensor_type, prepend_batch_axis)
    786                     )
    787                 raise ValueError(
--> 788                     "Unable to create tensor, you should probably activate truncation and/or padding "
    789                     "with 'padding=True' 'truncation=True' to have batched tensors with the same length."
    790                 )

ValueError: Unable to create tensor, you should probably activate truncation and/or padding with 'padding=True' 'truncation=True' to have batched tensors with the same length.

```

i know the problem is from tokenizer and i should use tokenizer with some arguments like this:

```
tokenizer(
    sequence,
    return_tensors="pt",
    truncation=True,
    padding=True,
    max_length=512,
)
```

but it's not clear from the documentation how can we define these argument("truncation=True", "padding=True", "max_length=512") when using pipelines for NER task

<!-- A clear and concise description of what you would expect to happen. -->



Discussion:
@Narsil, do you want to take a look at this?

Resolving patch:
diff --git a/src/transformers/pipelines/token_classification.py b/src/transformers/pipelines/token_classification.py
--- a/src/transformers/pipelines/token_classification.py
+++ b/src/transformers/pipelines/token_classification.py
@@ -28,11 +28,14 @@ class TokenClassificationArgumentHandler(ArgumentHandler):
     Handles arguments for token classification.
     """
 
-    def __call__(self, *args, **kwargs):
+    def __call__(self, inputs: Union[str, List[str]], **kwargs):
 
-        if args is not None and len(args) > 0:
-            inputs = list(args)
+        if inputs is not None and isinstance(inputs, (list, tuple)) and len(inputs) > 0:
+            inputs = list(inputs)
             batch_size = len(inputs)
+        elif isinstance(inputs, str):
+            inputs = [inputs]
+            batch_size = 1
         else:
             raise ValueError("At least one input is required.")
 
@@ -137,11 +140,11 @@ def __call__(self, inputs: Union[str, List[str]], **kwargs):
               Only exists if the offsets are available within the tokenizer
         """
 
-        inputs, offset_mappings = self._args_parser(inputs, **kwargs)
+        _inputs, offset_mappings = self._args_parser(inputs, **kwargs)
 
         answers = []
 
-        for i, sentence in enumerate(inputs):
+        for i, sentence in enumerate(_inputs):
 
             # Manage correct placement of the tensors
             with self.device_placement():


Test patch:
diff --git a/tests/test_pipelines_ner.py b/tests/test_pipelines_ner.py
--- a/tests/test_pipelines_ner.py
+++ b/tests/test_pipelines_ner.py
@@ -14,14 +14,17 @@
 
 import unittest
 
-from transformers import AutoTokenizer, pipeline
+from transformers import AutoTokenizer, is_torch_available, pipeline
 from transformers.pipelines import Pipeline, TokenClassificationArgumentHandler
 from transformers.testing_utils import require_tf, require_torch, slow
 
 from .test_pipelines_common import CustomInputPipelineCommonMixin
 
 
-VALID_INPUTS = ["A simple string", ["list of strings"]]
+if is_torch_available():
+    import numpy as np
+
+VALID_INPUTS = ["A simple string", ["list of strings", "A simple string that is quite a bit longer"]]
 
 
 class NerPipelineTests(CustomInputPipelineCommonMixin, unittest.TestCase):
@@ -334,17 +337,26 @@ def test_pt_defaults(self):
     @require_torch
     def test_simple(self):
         nlp = pipeline(task="ner", model="dslim/bert-base-NER", grouped_entities=True)
-        output = nlp("Hello Sarah Jessica Parker who Jessica lives in New York")
+        sentence = "Hello Sarah Jessica Parker who Jessica lives in New York"
+        sentence2 = "This is a simple test"
+        output = nlp(sentence)
 
         def simplify(output):
-            for i in range(len(output)):
-                output[i]["score"] = round(output[i]["score"], 3)
-            return output
+            if isinstance(output, (list, tuple)):
+                return [simplify(item) for item in output]
+            elif isinstance(output, dict):
+                return {simplify(k): simplify(v) for k, v in output.items()}
+            elif isinstance(output, (str, int, np.int64)):
+                return output
+            elif isinstance(output, float):
+                return round(output, 3)
+            else:
+                raise Exception(f"Cannot handle {type(output)}")
 
-        output = simplify(output)
+        output_ = simplify(output)
 
         self.assertEqual(
-            output,
+            output_,
             [
                 {
                     "entity_group": "PER",
@@ -358,6 +370,21 @@ def simplify(output):
             ],
         )
 
+        output = nlp([sentence, sentence2])
+        output_ = simplify(output)
+
+        self.assertEqual(
+            output_,
+            [
+                [
+                    {"entity_group": "PER", "score": 0.996, "word": "Sarah Jessica Parker", "start": 6, "end": 26},
+                    {"entity_group": "PER", "score": 0.977, "word": "Jessica", "start": 31, "end": 38},
+                    {"entity_group": "LOC", "score": 0.999, "word": "New York", "start": 48, "end": 56},
+                ],
+                [],
+            ],
+        )
+
     @require_torch
     def test_pt_small_ignore_subwords_available_for_fast_tokenizers(self):
         for model_name in self.small_models:
@@ -386,7 +413,7 @@ def test_simple(self):
         self.assertEqual(inputs, [string])
         self.assertEqual(offset_mapping, None)
 
-        inputs, offset_mapping = self.args_parser(string, string)
+        inputs, offset_mapping = self.args_parser([string, string])
         self.assertEqual(inputs, [string, string])
         self.assertEqual(offset_mapping, None)
 
@@ -394,25 +421,35 @@ def test_simple(self):
         self.assertEqual(inputs, [string])
         self.assertEqual(offset_mapping, [[(0, 1), (1, 2)]])
 
-        inputs, offset_mapping = self.args_parser(string, string, offset_mapping=[[(0, 1), (1, 2)], [(0, 2), (2, 3)]])
+        inputs, offset_mapping = self.args_parser(
+            [string, string], offset_mapping=[[(0, 1), (1, 2)], [(0, 2), (2, 3)]]
+        )
         self.assertEqual(inputs, [string, string])
         self.assertEqual(offset_mapping, [[(0, 1), (1, 2)], [(0, 2), (2, 3)]])
 
     def test_errors(self):
         string = "This is a simple input"
 
-        # 2 sentences, 1 offset_mapping
-        with self.assertRaises(ValueError):
+        # 2 sentences, 1 offset_mapping, args
+        with self.assertRaises(TypeError):
             self.args_parser(string, string, offset_mapping=[[(0, 1), (1, 2)]])
 
-        # 2 sentences, 1 offset_mapping
-        with self.assertRaises(ValueError):
+        # 2 sentences, 1 offset_mapping, args
+        with self.assertRaises(TypeError):
             self.args_parser(string, string, offset_mapping=[(0, 1), (1, 2)])
 
+        # 2 sentences, 1 offset_mapping, input_list
+        with self.assertRaises(ValueError):
+            self.args_parser([string, string], offset_mapping=[[(0, 1), (1, 2)]])
+
+        # 2 sentences, 1 offset_mapping, input_list
+        with self.assertRaises(ValueError):
+            self.args_parser([string, string], offset_mapping=[(0, 1), (1, 2)])
+
         # 1 sentences, 2 offset_mapping
         with self.assertRaises(ValueError):
             self.args_parser(string, offset_mapping=[[(0, 1), (1, 2)], [(0, 2), (2, 3)]])
 
         # 0 sentences, 1 offset_mapping
-        with self.assertRaises(ValueError):
+        with self.assertRaises(TypeError):
             self.args_parser(offset_mapping=[[(0, 1), (1, 2)]])