Problem with PreTrainedTokenizerFast and return_offsets_mapping
Author: mary-design-testing
Pull request: no
Labels: name: wontfix

name: Core: Pipeline

name: Fast Tokenizers

# 🐛 Bug

## Information

Model I am using (Bert, XLNet ...): Bert, bert-large-uncased-whole-word-masking-finetuned-squad

Language I am using the model on (English, Chinese ...): English

The problem arises when using:
* [ ] the official example scripts: 
* [X] my own modified scripts:

The tasks I am working on is:
* [X] an official GLUE/SQUaD task: SQUaD
* [ ] my own task or dataset: (give details below)

## To reproduce

Script to reproduce the behavior:

```python
# transformers v2.5.1 (https://github.com/huggingface/transformers/releases)
from transformers import AutoTokenizer, AutoModelForQuestionAnswering
import torch

model_name = "bert-large-uncased-whole-word-masking-finetuned-squad"
tokeniser = AutoTokenizer.from_pretrained(model_name, use_fast=True)
inputs = tokeniser.encode_plus("Who is Bert?", "Bert is a puppet by Jim Henson", add_special_tokens=True, return_tensors="pt", return_offsets_mapping=True)
```

This script produces the following wrong output:

```
Traceback (most recent call last):
  File "C:\Program Files\JetBrains\PyCharm 2018.3.5\helpers\pydev\pydevd.py", line 1741, in <module>
    main()
  File "C:\Program Files\JetBrains\PyCharm 2018.3.5\helpers\pydev\pydevd.py", line 1735, in main
    globals = debugger.run(setup['file'], None, None, is_module)
  File "C:\Program Files\JetBrains\PyCharm 2018.3.5\helpers\pydev\pydevd.py", line 1135, in run
    pydev_imports.execfile(file, globals, locals)  # execute the script
  File "C:\Program Files\JetBrains\PyCharm 2018.3.5\helpers\pydev\_pydev_imps\_pydev_execfile.py", line 18, in execfile
    exec(compile(contents+"\n", file, 'exec'), glob, loc)
  File "D:/OneDrive/RC/Segovia/DEMO-11/Adam11/_QUESTION_ANSWERER.py", line 8, in <module>
    inputs = tokeniser.encode_plus("Who is Bert?", "Bert is a puppet by Jim Henson", add_special_tokens=True, return_tensors="pt", return_offsets_mapping=True)
  File "C:\Users\mary\.conda\envs\adam11\lib\site-packages\transformers\tokenization_utils.py", line 1889, in encode_plus
    **kwargs,
  File "C:\Users\mary\.conda\envs\adam11\lib\site-packages\transformers\tokenization_utils.py", line 1843, in batch_encode_plus
    stack = torch.stack(stack, dim=0)
TypeError: expected Tensor as element 0 in argument 0, but got list
```

## Expected behavior

According to the documentation that is available in file "tokenization_utils.py", the behaviour should be as follows: 

>return_offsets_mapping: 
>(optional) Set to True to return (char_start, char_end) for each token (default False).
>If using Python's tokenizer, this method will raise NotImplementedError. This one is only available on Rust-based tokenizers inheriting from PreTrainedTokenizerFast.

## Environment info

- `transformers` version: 2.5.1
- Platform: Windows-10-10.0.18362-SP0
- Python version: 3.7.6
- PyTorch version (GPU?): 1.4.0+cpu (False)
- Tensorflow version (GPU?): not installed (NA)
- Using GPU in script?: No
- Using distributed or parallel set-up in script?: No

## My Patch

I have isolated the problem in file `tokenization_utils.py`.  Take a look at method `batch_encode_plus`.  It has the following statements:

```python
        # Sanitize the output to have dict[list] from list[dict]
        sanitized = {}
        for key in tokens[0].keys():
            stack = [e for item in tokens for e in item[key]]
            if return_tensors == "tf":
                stack = tf.stack(stack, axis=0)
            elif return_tensors == "pt":
                stack = torch.stack(stack, dim=0)
            elif not return_tensors and len(stack) == 1:
                stack = stack[0]

            sanitized[key] = stack
```
The problem is that `stack` may be a list with a toch.Tensor, but also a list of tuples with start/end offsets when `return_offsets_mapping=True`.  In such cases, `tf.stack` or `torch.stack` will fail because they expect a list of tensors as argument, not a list of tuples.

I have patched my local transformers installation as follows, and it seems to work well:

```python
            if return_tensors and len(stack) == 1 and isinstance(stack[0], torch.Tensor):
                if return_tensors == "tf":
                    stack = tf.stack(stack, axis=0)
                elif return_tensors == "pt":
                    stack = torch.stack(stack, dim=0)
            elif not return_tensors and len(stack) == 1:
                stack = stack[0]
```

Thanks!

Comments:
I am running into the same issue. Any progress on getting this into a release?

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.