Allow `TextClassificationPipeline` to handle input longer than `model_max_length` tokens
Author: boyleconnor
Pull request: no
Labels: name: Feature request

### Problem

Running a `TextClassificationPipeline` on a text with more tokens than its model's maximum position embeddings (e.g. 512 for BERT) like so:

```python
from transformers import pipeline
classifier = pipeline('sentiment-analysis')
classifier("Hello, world! " * 1000)
```

will lead to this error:

```
RuntimeError: The size of tensor a (4002) must match the size of tensor b (512) at non-singleton dimension 1
```

Note: the numbers (`4002` and `512`, above) will vary depending on the max length of the model in use and the length (in tokens) of the text that triggered the error.

(_**If you found this issue through web-searching for the above error or some other means, look at the linked PR for an implemented code fix to this problem, and consider giving a thumbs-up to this comment if you think it should be merged into the main codebase**_)

### Feature request

We should add "chunking"/"sliding window" functionality to `TextClassificationPipeline`, allowing it to process documents longer than the `model_max_length` of its `.model`. Specifically, this would run an instance of the model on each of several "sliding window" views of each input sequence, then take the mean, similar to (but somewhat simpler than) how [`TokenClassificationPipeline`](https://github.com/huggingface/transformers/blob/ad78d9597b224443e9fe65a94acc8c0bc48cd039/src/transformers/pipelines/token_classification.py#L96) does so in part by subclassing from `ChunkPipeline`.

### Motivation

It would be nice to easily do, e.g., sentiment analysis on documents longer than the `model_max_length` of the given model/tokenizer. I have in the past tried to do this in a time-sensitive context and was unable to do so.

### Your contribution

I have already opened a draft PR: #24312. I would be happy to finish the missing parts (e.g. documentation) if someone on the Huggingface team (I believe @Narsil is the appropriate person to tag) can confirm that they would accept this feature as I plan to implement it.

Comments:
I don't have a lot of time to review this atm.

@amyeroberts Do you know someone that could ?

Overall I'm hesitant to think it's a good idea. Maintenance is much higher for those `ChunkPipeline` and splitting a document into bits is relatively easy to do outside of the pipeline.
The merge strategies are also not entirely obvious to me.
That being said, it's definitely very convenient if implemented directly in the pipeline.

@boyleconnor Thanks for opening this feature request and for opening an example PR! As @Narsil mentions, there's a maintenance cost to adding this and the set of people who could review this are all pretty busy. 

What I suggest is leaving the PR as an example for anyone who might wish to see how to implement this. If this issue gets a lot of attention (we'll measure with 👍  on the feature description) then we can revisit. 



@amyeroberts, is there any particular threshold for when HF would consider adding this feature? As I write this comment, there are 13 👍's on the feature description.

I should also note that I posted about this feature request on social media, and so most of these 👍's are from NLP professionals & academics whom I personally know.

WDYT @Rocketknight1 @ArthurZucker?

I think the solution is valid and the PR is well-implemented, but my intuition is not to add it!

My reasoning is:
- Chunking inputs in the pipeline makes the pipeline more awkward, and will make it harder to add other features.
- Models are getting longer and longer context lengths, including the addition of things like alibi/rope position embeddings that let them scale up to arbitrarily long contexts. That means this feature will be less useful over time.

Thus, I feel this feature is a workaround for a very temporary limitation that some models have, but with more modern models the workaround shouldn't be needed, and so we shouldn't hardcode it into our pipelines in 2023. Instead, we should consider replacing some of the default models for tasks like `sentiment-analysis` with newer models that have better performance and that  support longer inputs.

Hey @boyleconnor, sorry for the very slow follow-up, we're catching up on a backlog of older issues.

Your PR (#24312) was genuinely well-implemented, but as @Narsil, @amyeroberts and @Rocketknight1 discussed in the thread, we're going to leave this as won't-implement rather than bake it into the pipeline.
The reasons still hold: adding chunking plus a merge strategy (mean/max/first) increases the maintenance surface of `TextClassificationPipeline`, it's straightforward to do outside the pipeline, and modern default models increasingly ship with long-context support that removes the underlying limitation. Signal has stayed thin (15 👍 over three years), so we'd rather not lock a chunking contract into the pipeline's public behavior.
Your PR will remain available as a reference for anyone who wants to wire this up themselves. Thanks again for the thorough write-up and the contribution!