A bug in the padding of input examples in the NER fine-tuning example
Author: AMR-KELEG
Pull request: no
Labels: name: wontfix

# 🐛 Bug

## Information

Model I am using (Bert, XLNet ...): Roberta

Language I am using the model on (English, Chinese ...): English

The problem arises when using:
* [X] the official example scripts: (give details below)
* [ ] my own modified scripts: (give details below)

The tasks I am working on is:
* [ ] an official GLUE/SQUaD task: (give the name)
* [X] my own task or dataset: (give details below)

## To reproduce

Steps to reproduce the behavior:

1. TODO

<!-- If you have code snippets, error messages, stack traces please provide them here as well.
     Important! Use code tags to correctly format your code. See https://help.github.com/en/github/writing-on-github/creating-and-highlighting-code-blocks#syntax-highlighting
     Do not use screenshots, as they are hard to read and (more importantly) don't allow others to copy-and-paste your code.-->

## Expected behavior

https://github.com/huggingface/transformers/blob/c59b1e682d6ebaf7295c63418d4570228904e690/examples/ner/utils_ner.py#L123
This line is supposed to return 3 for Roberta models but it's just returning 2 causing the length of the input_ids to be more than the max_seq_len.
This might be the reason for that: https://github.com/huggingface/transformers/blob/master/src/transformers/tokenization_roberta.py#L288
TODO: Share the notebook.

## Environment info
<!-- You can run the command `transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->
     
- `transformers` version: 2.8.0
- Platform: Linux-4.19.104+-x86_64-with-Ubuntu-18.04-bionic
- Python version: 3.6.9
- PyTorch version (GPU?): 1.4.0 (True)
- Tensorflow version (GPU?): 2.2.0-rc2 (True)
- Using GPU in script?: <fill in>
- Using distributed or parallel set-up in script?: <fill in>


Comments:
This PR https://github.com/huggingface/transformers/pull/3803 might be related to the bug but my initial thought is that it will not fix it.

Hello! Why do you think it should return 3 for RoBERTa models?

It should return 2, single sequences for RoBERTa are built like: 

`
 tok_0 ... tok_n 
`, 

with only two special tokens added.

For sequence pairs however, 4 tokens are added: 

`
 tok_0 ... tok_n 

 tok_(n + 1) ... tok2n 
`

> Hello! Why do you think it should return 3 for RoBERTa models?
> 
> It should return 2, single sequences for RoBERTa are built like:
> 
> `
 tok_0 ... tok_n 
`,
> 
> with only two special tokens added.
> 
> For sequence pairs however, 4 tokens are added:
> 
> `
 tok_0 ... tok_n 

 tok_(n + 1) ... tok2n 
`

Well, this line suggested this:
https://github.com/huggingface/transformers/blob/c59b1e682d6ebaf7295c63418d4570228904e690/examples/ner/utils_ner.py#L122

Additionally, the current code produced lists of length > `max_seq_length` so for sure there is a problem there.

I had an issue with the running the NER model. In this commit https://github.com/huggingface/transformers/commit/96ab75b8dd48a9384a74ba4307a4ebfb197343cd `num_added_tokens` got changed into `num_special_tokens_to_add`.  Just changing the name of the variable in the `utils_ner.py` fixed the issue for me. However, I had an issue with variable name not being found. Let me know if this fixes you problem.

> I had an issue with the running the NER model. In this commit [96ab75b](https://github.com/huggingface/transformers/commit/96ab75b8dd48a9384a74ba4307a4ebfb197343cd) `num_added_tokens` got changed into `num_special_tokens_to_add`. Just changing the name of the variable in the `utils_ner.py` fixed the issue for me. However, I had an issue with variable name not being found. Let me know if this fixes you problem.

Hi @TarasPriadka 
Yes, the edit you have suggested solved the problem.
I have found that you have already reported the issue before (https://github.com/huggingface/transformers/issues/3686).
Don't you think that we should open a simple Pull Request to fix this problem?

@AMR-KELEG I think it got fixed just now with this huggingface/transformers#3800 PR

@TarasPriadka, @AMR-KELEG 

I had a similar issue using `preprocess.py` on an NER dataset.

```
Traceback (most recent call last):
  File "preprocess.py", line 12, in <module>
    max_len -= tokenizer.num_special_tokens_to_add()
AttributeError: 'BertTokenizer' object has no attribute 'num_special_tokens_to_add'
```

I think the PyPi file hasn't been updated, so `pip install transformers` won't have the files you need.  I built from source and the errors went away.  If you try building from source, I think your problem might go away too. 

> @TarasPriadka, @AMR-KELEG
> 
> I had a similar issue using `preprocess.py` on an NER dataset.
> 
> ```
> Traceback (most recent call last):
>   File "preprocess.py", line 12, in <module>
>     max_len -= tokenizer.num_special_tokens_to_add()
> AttributeError: 'BertTokenizer' object has no attribute 'num_special_tokens_to_add'
> ```
> 
> I think the PyPi file hasn't been updated, so `pip install transformers` won't have the files you need. I built from source and the errors went away. If you try building from source, I think your problem might go away too.

Well, I was using the source version but as said before, seems like the bug was there and got fixed in later commits.

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.