Add Sliding Window to TokenClassificationPipeline
Author: boyleconnor
Pull request: no

# 🚀 Feature request

The current [implementation of TokenClassificationPipeline](https://github.com/huggingface/transformers/blob/3977b58437b8ce1ea1da6e31747d888efec2419b/src/transformers/pipelines/token_classification.py#L82) cannot do anything with text input beyond the length (measured in tokens) allowed by the model's number of positional embeddings. I think it would be useful to many users to have a setting in the pipeline that allows for a "sliding window" approach that can take documents longer than the `.config.max_position_embeddings` of the model in use.

The modified version of `TokenClassificationPipeline` could be instantiated with the optional new params `window_size` (defaults to `model.max_position_embeddings - 2` to accomodate the `[CLS]` and `[SEP]` special tokens) and `stride` (defaults to `window_size / 2`, the user can disable the sliding window by setting `stride = 0`) like so:

```
model, tokenizer = # Some model, some tokenizer
pipeline = TokenClassificationPipeline(model=model, tokenizer=tokenizer, aggregation_strategy='first', stride=254)
```

## Motivation

I run tasks that involve doing named entity recognition on longer documents. I have to use my own modified subclass of `TokenClassificationPipeline` in order to do so, since the existing class does not support processing texts longer than the maximum number of position embeddings for the underlying models.

## Your contribution

I will submit a PR adding sliding window functionality to `TokenClassificationPipeline`. I have previously created a sliding window implementation of Transformers' NER Pipeline [here](https://github.com/nlpsandbox/phi-annotator-huggingface/blob/634e1736837b50a190f42c2fc741b262e88bf3ac/server/openapi_server/huggingface.py#L15), and will re-work this to work with TensorFlow and meet the code standards of HuggingFace/Transformers.


Comments:
Hi @cascadianblue, 

This could be an interesting option to add. 
IMHO:
- This should be opt-in, since at the boundaries, you might still get issues if the entities are across your split. FYI, for question answering we use a stride argument to have an overlap on the chunks and give opportunity to the model to see each token with some context, even at the boundaries. Should we do that here, we should be prepared to handle resolution logic. I can see the stride but not resolution mechanism in your code.
- This would be slightly tricky to do as a simple pipeline (because of the `batch_size`), maybe something like this will be necessary: https://github.com/huggingface/transformers/pull/14225
- I can see in your code hardcoded CLS ,and SEP tokens, this unfortunately cannot be allowed since this is very model specific. `self.tokenizer("some text")` should always be use to add those automatically
- A PR should obviously add tests for that option and importantly with at least 1 test showcasing the boundary handling. Like "Welcome to New York city" with a model max_length 3 for instance, should still be able to yield "New York" (if it was able to on a simple test ofc)

Feel free to start a PR and publish it even if not refined to get help/support or if unsure about anything.

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.

Is there any continued work on this (@cascadianblue / @Narsil)?

@cgpeltier I haven't worked on this since making the feature request. You're welcome to copy my code with attribution, if it works for your case.

Are there any plans to add this feature? 

@boyleconnor  and I have implemented a new class called [`SlidingWindowTokenClassificationPipeline`](https://github.com/connor-qingxia/transformers/blob/71d6f938485c45fc6d494cebf501065f04242c0c/src/transformers/pipelines/token_classification.py#L496) which inherits from `TokenClassificationPipeline`. 

The class essentially operates as @boyleconnor has described in this issue. First the input text is separated into list of tokens using the sliding window method with given window length and stride. And on the output side, we aggregated the duplicated tokens by averaging their output logits and picked the highest one to be the chosen NER. 

However, `TokenClassficationPipeline` just updated a new feature while we are working on the implementation of this new class. The updated version of `TokenClassificationPipeline` allows another argument `stride` so that when the input text is longer than the `model_max_length`, this would allow the pipeline to separate input text into several chunks using the sliding window method. When it comes to output,  the current code would first generate the NER output and then use the aggregate function defined [here](https://github.com/huggingface/transformers/blob/d04ec99bec8a0b432fc03ed60cea9a1a20ebaf3c/src/transformers/pipelines/token_classification.py#L331) to compare the output scores of the duplicated token and select the NER with higher score. 

Both implementation follows more or less the same spirit but differ slightly in how the final NER gets chosen. So I am wondering if it is still valuable for us to create a PR that adds `SlidingWindowTokenClassificationPipeline` to the script in spite of the current `TokenClassificationPipeline` can already perform somewhat similar functionalities. @Narsil 

It depends.

This is an advanced feature, if you can provide a simple PR that adds support for the way you handle this, it would be a nice addition.

Also you will find https://github.com/huggingface/transformers/pull/21771 which contains more details about choices, and more importantly scripts to compare various implementations.