Extra indicators for BPE for Unicode Characters 
Author: lolipopshock
Pull request: no

## Environment info
<!-- You can run the command `transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->

- `transformers` version: 4.2.2
- Platform: Linux-4.15.0-45-generic-x86_64-with-glibc2.10
- Python version: 3.8.3
- PyTorch version (GPU?): 1.6.0 (True)
- Tensorflow version (GPU?): not installed (NA)
- Using GPU in script?: No
- Using distributed or parallel set-up in script?: No

### Who can help
<!-- Your issue will be replied to more quickly if you can figure out the right person to tag with @
 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

 tokenizers: @mfuntowicz
 -->

## Information

Model I am using (Bert, XLNet ...):

The problem arises when using:
* my own modified scripts: (give details below)

The tasks I am working on is:
* my own task or dataset: (give details below)

## To reproduce

Steps to reproduce the behavior:

1. The code
    ```
    from transformers import AutoTokenizer
    tokenizer = AutoTokenizer.from_pretrained("bert-base-uncased")
    words = ['(cid:3)', '하셨습니까', '하다']
    tokenizer.batch_encode_plus(
        [words],
        max_length=512,
        truncation=True,
        padding=True,
        is_split_into_words=True,
        return_offsets_mapping=True,
        return_special_tokens_mask=True,
        return_tensors="pt",
    )
    ```
2. The `offset_mapping` in the output is 
    ```python
    tensor([[[0, 0],        # [CLS]
             [0, 1],        # for '('
             [1, 4],        # for 'cid'
             [4, 5],        # for ':'
             [5, 6],        # for '3'
             [6, 7],        # for ')'
             [0, 5],        # for '하셨습니까'
             [0, 1],        # for '하'
             [0, 1],        # for '하'
             [1, 2],        # for '다'
             [1, 2],        # for '다'
             [0, 0]]])
    ```
3. As you could find, it generates four tokens for `하다`. The output is correct according to Byte pair encoding. However, it generates duplicated `[0,1]` and `[1,2]`s, which changes the structure of the outputs (for regular tokens, it can only have one `[0,x]`, which can be used to project the encoded tokens back to their original positions). Therefore, we need extra indicators for positions where Byte-pair encoding is used. 

<!-- If you have code snippets, error messages, stack traces please provide them here as well.
     Important! Use code tags to correctly format your code. See https://help.github.com/en/github/writing-on-github/creating-and-highlighting-code-blocks#syntax-highlighting
     Do not use screenshots, as they are hard to read and (more importantly) don't allow others to copy-and-paste your code.-->

## Expected behavior

<!-- A clear and concise description of what you would expect to happen. -->
1. An additional output showing the mapping for input_ids -> original_token_ids . In this case, it should be something like: 
    ```
    [0, 1, 1, 1, 1, 1, 2, 3, 3, 3, 3 0]
    ```
Therefore, we could use this map to figure out byte code embedding is used for the 3rd token. 

Updated - @n1t0 

Comments:
Pinging @n1t0 

Thanks! I think it's something related to the Byte-Level Subwords trick (https://arxiv.org/pdf/1909.03341.pdf)? 

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.