Wrong subword aggregation when using aggregation_strategy
Author: Guidofaassen
Pull request: no

## Environment info
- `transformers` version: 4.7.0.dev0
- Platform: Windows
- Python version: 3.9.4
- PyTorch version (GPU?): No
- Tensorflow version (GPU?): No
- Using GPU in script?: No
- Using distributed or parallel set-up in script?: No

### Who can help
@Narsil 
@francescorubbo 
@elk-cloner 

## Information

xlm-roberta-large-finetuned-conll02-dutch

The problem arises when using aggregation_strategy.


## To reproduce

Steps to reproduce the behavior:

Given this code:
```sentence = "Groenlinks praat over Schiphol."
nlp = pipeline('ner', model='xlm-roberta-large-finetuned-conll02-dutch')
nlp(sentence)
```

I get the following result:

```
[{'entity': 'B-ORG',
  'score': 0.9769433,
  'index': 1,
  'word': '▁Groen',
  'start': 0,
  'end': 5},
 {'entity': 'I-ORG',
  'score': 0.9935022,
  'index': 2,
  'word': 'link',
  'start': 5,
  'end': 9},
 {'entity': 'B-LOC',
  'score': 0.9999288,
  'index': 6,
  'word': '▁Schi',
  'start': 22,
  'end': 26},
 {'entity': 'I-LOC',
  'score': 0.99987257,
  'index': 8,
  'word': 'hol',
  'start': 27,
  'end': 30}]
```

We received subwords, where I would prefer to have real words. I found that `aggregation_strategy` was added in the latest release (master branch, 4.7.0.dev0). In an attempt to fix this, I tried this:

```
sentence = "Groenlinks praat over Schiphol."
nlp = pipeline('ner', model='xlm-roberta-large-finetuned-conll02-dutch', aggregation_strategy="max")
nlp(sentence)
```

Which yields:

```
[{'entity_group': 'ORG',
  'score': 0.98522276,
  'word': 'Groenlink',
  'start': 0,
  'end': 9},
 {'entity_group': 'LOC',
  'score': 0.99987257,
  'word': 'hol',
  'start': 27,
  'end': 30}]
```

## Expected behavior

This is different than expected, as subwords are merged in the wrong way. `Groenlink` and `hol` were both not part of the original sentence. I would expect this:

```
[{'entity_group': 'ORG',
  'score': 0.98522276,
  'word': 'Groenlinks',
  'start': 0,
  'end': 9},
 {'entity_group': 'LOC',
  'score': 0.99987257,
  'word': 'Schiphol',
  'start': 27,
  'end': 30}]
```

Do you have any clues how to fix this?


Comments:
I suspect we might have an issue recognizing subwords when using the BPE tokenizer.
Using a BERT based model works as expected:
```
>> nlp = transformers.pipeline('ner', model='wietsedv/bert-base-dutch-cased-finetuned-conll2002-ner', aggregation_strategy='first')
>> nlp("Groenlinks praat over Schiphol.")
[{'entity_group': 'org',
  'score': 0.99999315,
  'word': 'Groenlinks',
  'start': 0,
  'end': 10},
 {'entity_group': 'loc',
  'score': 0.9999975,
  'word': 'Schiphol',
  'start': 22,
  'end': 30}]
```

I'll take a closer look at the logic when using BPE sub-tokens and report back.

@LysandreJik @sgugger The problem is how we decide whether or not a token is a subword [here](https://github.com/huggingface/transformers/blob/master/src/transformers/pipelines/token_classification.py#L273),
where we compare the token length with the corresponding span in the original text.
For WordPiece this works because `Groenlinks` is tokenized as `['Groen', '##link', '##s']`, so the last two tokens are tagged as subwords. However BPE tokenizes as `['_Groen', 'link', 's']`, so we incorrectly tag `_Groen` as subword and the other two tokens as words.

Similar to https://github.com/huggingface/transformers/issues/11794

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.

Any update on this issue?

@francescorubbo thanks for investigating this. So the `gather_pre_entities` function of the NER pipeline needs an update to also work with BPE tokenizers. 

cc @Narsil 

Do you mind opening a PR to support BPE tokenizers?

I think the problem is tricky, I don't think it is properly linked to BPE, but more for tokenizers that are word aware vs not.

Right now, we use tokenizers that use `continuing_subword_prefix` to determine if a token is a subword.
I don't think there is a "correct" way to do that with byte level BPE like gpt2 (roberta) as they don't posess the notion of "word".

As mentioned in a previous issue, if we can find a good heuristic that would be great, but byte BPE can :
- have space as a prefix or suffix 
- use a different char than ' '  for space (_ for spm, `G for gpt2)
- Potentially contain spaces (hence different words) within a single token (although I don't think I've seen it done for major tokenizers)

So classifying subwords for these tokenizers is always going to be tricky. We could however disable "word"-based strategies for tokenizers that do no provide "continuing_subword_prefix". It would be more correct (but less usable for sure)

I result is still incorrect after the new changes @Narsil.

Given the nightly build: '4.10.0.dev0'

With given code

```
nlp = pipeline('ner', model='xlm-roberta-large-finetuned-conll02-dutch', grouped_entities=True)
sentence = "Groenlinks praat over Schiphol."
nlp(sentence)
```

yields

```
[{'entity_group': 'ORG',
  'score': 0.98522276,
  'word': 'Groenlink',
  'start': 0,
  'end': 9},
 {'entity_group': 'LOC',
  'score': 0.9999288,
  'word': 'Schi',
  'start': 22,
  'end': 26},
 {'entity_group': 'LOC',
  'score': 0.99987257,
  'word': 'hol',
  'start': 27,
  'end': 30}]
```

The subwords are still not merged correctly as the found entities do not exist in the original text. I also tried setting `aggregation_strategy=AggregationStrategy.SIMPLE`, but that did not help either. Am I doing something wrong?