Add token_type_ids to Esm tokenizer
Author: lhallee
Pull request: yes

# What does this PR do?
Enables EsmTokenizer to correctly return token_type_ids. Before, ignored special tokens. create_token_type_ids_from_sequences was adapted from BertTokenizer with eos instead of sep per Esm special tokens.

This
```
tokenizer = EsmTokenizer.from_pretrained('facebook/esm2_t6_8M_UR50D')
seq = 'PROTEIN'
len(seq) # 7

tokens = tokenizer(seq, seq, return_token_type_ids=True)

len(tokens.input_ids), len(tokens.token_type_ids) # 17, 14 
```
To this
`len(tokens.input_ids), len(tokens.token_type_ids) # 17, 17 `

Fixes [#28656](https://github.com/huggingface/transformers/issues/28656)


## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [X ] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?
@ArthurZucker 

Comments:
cc @Rocketknight1 

Hi @lhallee - I appreciate the work but I'm not sure we can accept this! ESM was not trained with `token_type_ids` and will not accept them by default, so even if we get the tokenizer to return them then the ESM model code will also have to be altered. In general at Hugging Face we aim to reproduce models precisely, and adding an extra input that the original paper/checkpoints/etc. do not support and do not have weights for doesn't really fit that philosophy.

I completely agree that there are valid use-cases for adding `token_type_ids` to protein models, though, but I think the right place to do that is with a custom code model, rather than by modifying the built-in ESM class. There's a guide [here](https://huggingface.co/docs/transformers/main/en/custom_models) - you can use the ESM code as a base and make the changes you want to add token type IDs, and then train and share the model as a custom code checkpoint. This is the approach used by e.g. the [Nucleotide Transformer v2 models](https://huggingface.co/InstaDeepAI/nucleotide-transformer-v2-50m-multi-species).

Could we just set return token type ids as false by default? It won't break the code, return token token ids is already an option for the ESM tokenizer because it inherits the base, but it's just currently wrong if you do use it.

Here's the modified version (using the same suggested code) not returning any token_type_ids when not prompted to do so.

```
tokenizer = EsmTokenizer.from_pretrained('facebook/esm2_t6_8M_UR50D')
tokens = tokenizer(seq, seq)
{'input_ids': [0, 14, 10, 28, 11, 9, 12, 17, 2, 14, 10, 28, 11, 9, 12, 17, 2], 'attention_mask': [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]}
```

while being correct when needed

```
tokens = tokenizer(seq, seq, return_token_type_ids=True)
[print(len(v)) for k, v in tokens.items()]
17
17
17
{'input_ids': [0, 14, 10, 28, 11, 9, 12, 17, 2, 14, 10, 28, 11, 9, 12, 17, 2], 'token_type_ids': [0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 1, 1, 1], 'attention_mask': [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]}

```

Hi @lhallee - I understand that it won't break the code, but we still prefer to stick to the principle of faithfully reproducing the original model. Unfortunately, we can't accept this one!

On a personal level, though, I'm definitely interested to see if people can add auxiliary information to DNA/protein models using inputs like `token_type_ids` - if you'd like any help or advice with making a custom code model, I'm happy to offer advice or support!

Hi @Rocketknight1 - I don't really understand the logic as one can call return_token_type_ids=True in the current EsmTokenizer and it will output token_type_ids, just wrong ones that ignore special tokens. So, seems like there should either be a warning "Hey, Esm doesn't use these just so you know" or removal of this capability if the EsmTokenizer is going to be completely "faithful."

However, I respect your decision and commitment to faithful representations. Thanks!

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.