XLM-RoBERTa NER extraction breaks/splitting the words !
Author: dummynov1
Pull request: no

I have been using the huggingface xlm-roberta-large-finetuned-conll03-english model NER pipeline for extracting Names, Location and Organization Entities.

But i'm facing an issue now and then with certain entity extraction from short sentences where a word is broken down into sub-word tokens with different entity types. Code used as below 

```
from transformers import pipeline, AutoTokenizer, AutoModelForTokenClassification
  
tokenizer = AutoTokenizer.from_pretrained("xlm-roberta-large-finetuned-conll03-english")
model = AutoModelForTokenClassification.from_pretrained("xlm-roberta-large-finetuned-conll03-english")

ner_model = pipeline("ner", model = model, tokenizer = tokenizer, grouped_entities = True)

text = "Brennan Nov2018"
ner_model(text)
```


output:
`
[
  {
    "entity_group": "PER",
    "score": 0.6225427985191345,
    "word": "Brenn",
    "start": 0,
    "end": 5
  },
  {
    "entity_group": "LOC",
    "score": 0.759472668170929,
    "word": "an",
    "start": 5,
    "end": 7
  }
]
`

![image](https://user-images.githubusercontent.com/83328005/124965311-9dcdbe00-e03f-11eb-927f-904ee641d564.png)

Even though i'm using `grouped_entities = True` , i'm still getting some words broken down into 2 different entity groups.

Is there a way to prevent this to happen and to return only complete words as entity ?

 - PyTorch Version : 1.7.1
 - transformers : 4.6.0
 - Python : 3.8.5

Comments:
Cc @Narsil 

Hi @dummynov1 ,

You are using `grouped_entities` which will only to attempt to fuse *valid* entities (B-PER, I-PER, I-PER). Any break in that structure won't get merged and you might break words up.

We recently added other aggregation strategies ( https://huggingface.co/transformers/main_classes/pipelines.html?highlight=aggregation_strategy#transformers.TokenClassificationPipeline ) but they only work for word aware tokenizers (which is not the case of roberta).

Your issue is not isolated, so I actually looked into it, and I think I figured a better heuristic that you could end up using: https://github.com/huggingface/transformers/pull/12611

> Hi @dummynov1 ,
> 
> You are using `grouped_entities` which will only to attempt to fuse _valid_ entities (B-PER, I-PER, I-PER). Any break in that structure won't get merged and you might break words up.
> 
> We recently added other aggregation strategies ( https://huggingface.co/transformers/main_classes/pipelines.html?highlight=aggregation_strategy#transformers.TokenClassificationPipeline ) but they only work for word aware tokenizers (which is not the case of roberta).
> 
> Your issue is not isolated, so I actually looked into it, and I think I figured a better heuristic that you could end up using: #12611

Could you elaborate, what needs to be done to fix this.? Should i use the aggregation strategies, but i'm using transformers 4.6.0 (have to use this version only, due to other dependencies).

You won't be able to fix it correctly in a super reliable way. Simply because `xlm` doesn't know what a "word" is.
**The only real fix you can do is make the model better by more finetuning, with more data probably. (To get correct tags on all your tokens)**

That being with the proposed PR you will be able to have a bit of a better heuristic that might be good enough for you:
you will be able to write:

```python
from transformers import pipeline, AutoTokenizer, AutoModelForTokenClassification
  
tokenizer = AutoTokenizer.from_pretrained("xlm-roberta-large-finetuned-conll03-english")
model = AutoModelForTokenClassification.from_pretrained("xlm-roberta-large-finetuned-conll03-english")

ner_model = pipeline("ner", model = model, tokenizer = tokenizer, aggregation_strategy = "max")

text = "Brennan Nov2018"
ner_model(text)
```

Because `xlm` doesn't know words, everything non space will be treated as a word

- "Brenn" "ann" will be fused as intended
- "Some", "one", ","  ("Someone," ) too unfortunately.
- Any punctuation within string really. Any formatting within `yaml`, `json`, `markdown` etc..

@Narsil Could you advise if there is a model on HuggingFace hub that is "word-aware"? I am not sure if I understand it properly, but in my mind, none of the BERT models are actually "word-aware".

I struggled with this problem earlier last year, and did a lot of search online without a solution. I ended up with an ugly patch downstream to absorb this problem. So thanks for making some improvements to the pipelines.

Hi @ninjalu,

Do you mind explaining a little more what your issue is ?
Without context it's a bit hard to guide you correctly.

Tokenizers "word-aware" are the ones with `continuing_subword_prefix` set (`tokenizer.backend_tokenizer.model.continuing_subword_prefix` variable, if it exists). But most likely you shouldn't choose a tokenizer based purely on this, but probably first on considerations like what data it was trained on and the leveraging you can use in the underlying model (if you're doing fine-tuning for instance, it's better to pick a good model for your target data/langage than starting the whole model+tokenizer from scratch)