FastTokenizer not returning batch_size for offset_mapping for short texts
Author: xbelda
Pull request: no
Labels: name: wontfix

## Environment info
- `transformers` version: 3.0.2
- Platform: Linux-5.4.0-42-generic-x86_64-with-glibc2.10
- Python version: 3.8.2
- PyTorch version (GPU?): 1.6.0 (True)
- Tensorflow version (GPU?): not installed (NA)
- Using GPU in script?: No
- Using distributed or parallel set-up in script?: No

# Who can help
 tokenizers: @mfuntowicz

## Information
When working with `padding` and `truncation` on short texts (smaller than `max_len`),
the FastTokenizer will return the batch_size dimension if `return_tensors=None`. 
However, when `return_tensors="pt"` or `return_tensors="np"` are enabled (I haven't tested it on Tensorflow), they **won't return the batch dimension**.

## To reproduce

Loading fast tokenizer:
```python3
from transformers import AutoTokenizer
tokenizer = AutoTokenizer.from_pretrained("bert-base-uncased", use_fast=True)
```

Behavior on "short" texts without `return_tensors`:
```python3

out = tokenizer("test text",
                padding='max_length',
                truncation=True,
                return_overflowing_tokens=True,
                return_offsets_mapping=True,
                )

# Convert to tensor outside the tokenizer
print(torch.tensor(out["offset_mapping"]).shape)
>>> torch.Size([1, 512, 2])
```

Behavior with `return_tensors`:
```python3

out = tokenizer("test text",
                padding='max_length',
                truncation=True,
                return_overflowing_tokens=True,
                return_offsets_mapping=True,
                return_tensors="pt"  # Similarly with "np"
                )

print(out["offset_mapping"].shape)
>>> torch.Size([512, 2])
```

## Expected behavior
```python3

out = tokenizer("test text",
                padding='max_length',
                truncation=True,
                return_overflowing_tokens=True,
                return_offsets_mapping=True,
                return_tensors="pt"
                )

print(out["offset_mapping"].shape)
>>> torch.Size([1, 512, 2])
```


Comments:
After inspecting the code, it looks that the cause can be found inside `tokenization_utils_base.py`.

Then, in the method `convert_to_tensors` form `BatchEncoding` there are the following lines:
```python3
# Do the tensor conversion in batch
for key, value in self.items():
    try:
        if prepend_batch_axis:
            value = [value]

        tensor = as_tensor(value)

        # at-least2d
        if tensor.ndim > 2:
            tensor = tensor.squeeze(0)
        elif tensor.ndim < 2:
            tensor = tensor[None, :]
```

In this case, right before the squeeze, the `offset_mapping` tensor is of shape [1, 512, 2], which becomes [512, 2] after being squeezed.

This would explain why it doesn't fail with longer sequences, since squeezing a tensor of shape [n, 512, 2] (n>1) leaves the tensor unaltered.

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.