Allow the creation of "entity groups" for NerPipeline #3548
Author: enzoampil
Pull request: yes

### This pull request applies the entity group transformation by setting the parameter: group=True.

This was done by reflecting the transformation inside NerPipeline. This is similar to a previously closed [PR](https://github.com/huggingface/transformers/pull/3607), which I closed because I accidentlly deleted my fork (apologies for my clumsiness).

Details of what I want to be able to do can be found in issue #3548.

cc @julien-c @mfuntowicz @petulla

Sample code:
```
# Install branch
# Make sure to restart runtime after installing if using Google Colab
!pip install -e git+git://github.com/enzoampil/transformers.git@add_index_to_ner_pipeline#egg=transformers

# Grouped NER
from transformers import pipeline
nlp = pipeline('ner', grouped_entities=True)
nlp("Enzo works at the Australian National University (AUN)")

# [{'entity_group': 'I-PER', 'score': 0.9968132972717285, 'word': 'Enzo'},
# {'entity_group': 'I-ORG', 'score': 0.9970400333404541, 'word': 'Australian National University'},
# {'entity_group': 'I-ORG', 'score': 0.9831967651844025, 'word': 'AUN'}]

# Ungrouped NER
nlp = pipeline('ner', grouped_entities=False)
nlp("Enzo works at the Australian National University (AUN)")

# [{'entity': 'I-PER', 'index': 1, 'score': 0.9983270168304443, 'word': 'En'},
# {'entity': 'I-PER', 'index': 2, 'score': 0.9952995777130127, 'word': '##zo'},
# {'entity': 'I-ORG', 'index': 6, 'score': 0.9984350204467773, 'word': 'Australian'},
# {'entity': 'I-ORG','index': 7, 'score': 0.9967807531356812, 'word': 'National'},
# {'entity': 'I-ORG', 'index': 8  'score': 0.9959043264389038, 'word': 'University'},
# {'entity': 'I-ORG', 'index': 10, 'score': 0.9900023937225342, 'word': 'AU'},
# {'entity': 'I-ORG', 'index': 11, 'score': 0.9763911366462708, 'word': '##N'}]
```

Tutorial on how to do Entity Grouping w/ NerPipeline [here](https://colab.research.google.com/drive/1CVLP0n3Q5t5qiWpode7jyhUNZpmLg0mS)

I'm very keen to get feedback for the above, so please let me know if I should change anything, or perform additional steps to bring its quality to an acceptable level.

Comments:
Was hoping this could get added to the next release! Would be very useful for me.

_Note for @LysandreJik and myself_: We need to rebase the PR before merging, I've made some changes to the way batching is handled on pipelines, I just want to be sure we're not missing anything through the unittests.

Otherwise LGTM 👍 

@mfuntowicz I've changed the `group` parameter to `grouped_entities` and have added corresponding unit tests for the "grouped" `NerPipeline` for both pytorch and tensorflow: `test_ner_grouped` and `test_tf_ner_grouped`.

Please let me know if that parameter name is acceptable or if you want me to change it!

Also, I noticed the tests failing but for tests outside the scope of this PR.

If you can rebase the PR against master, we can check the unittests (some failures are related to some recent changes we made) and then merge into master 👍 

@mfuntowicz Rebased and got the tests to pass :smile:

Perfect, thanks for your contribution @enzoampil :)

Very welcome @mfuntowicz ! Looking forward to contributing more 🙂 

Reminder that we also want to return char offsets here too 

@julien-c good point, can work on this in a separate PR over the weekend :)

@enzoampil That was more of an internal reminder for e.g. @mfuntowicz as this will be mostly built-in when we turn the flip on fast tokenizers

thank you @enzoampil ! this PR helped a lot for a downstream NER task. 

Would someone from hugging face (maybe @mfuntowicz) be able to update a README or document to reflect this change so that other folks can avoid some unnecessary work?

Yes, more doc would be awesome

For the cases of NER using BIO tagging. This algorithm of grouping entities will separate `B-Label` from `I-Label` which is not correct.

Could you provide an example where it fails so that we may adapt the algorithm? Thank you.

@LysandreJik 

`token_classifier = pipeline("ner", model=model,aggregation_strategy='simple', tokenizer=tokenizer,grouped_entities=True)`

```
{'entity_group': 'B_name', 'score': 0.96226656, 'word': 'Pratik', 'start': 1141, 'end': 1149}
{'entity_group': 'I_name', 'score': 0.9272271, 'word': 'kr', 'start': 1150, 'end': 1157}
{'entity_group': 'L_name', 'score': 0.7290683, 'word': 'kumar', 'start': 1158, 'end': 1163}
```

Ideally, it should be grouped to just `name` ? How to achieve this?