Splitting texts longer that `tokenizer.max_length` into blocks of same size 
Author: hebecked
Pull request: no

## Environment info
`transformers-cli env` raises an ModuleNotFoundError, though I don't think it is relevant for my problem.

- `transformers` version: 4.0.0
- Platform: Arch Linux x86_64
- Python version: 3.9.1
- CPU only

### Who can help
It's a probably trivial tokenizer problem: @mfuntowicz
using a pretrained bert: @LysandreJik

## Information

I'm running successfully (exemplary for several models): 
`
tokenizer = AutoTokenizer.from_pretrained("oliverguhr/german-sentiment-bert")
model = AutoModelForSequenceClassification.from_pretrained("oliverguhr/german-sentiment-bert")
inputs = tokenizer(text, return_tensors = "pt")
proOrCon = self.model(**inputs)
`

Now I have several `text`s that produce more than 512 tokens. I tried to split the `inputs` manually by copying and modifying as well as creating a dict in the same format, but apparently, the class object stores additional information that are required and not easily accessible.
I Also tried built-in functions from the Tokenizer:
`inputs = tokenizer(text, return_tensors = "pt", max_length=512, stride=0, return_overflowing_tokens=True, truncation=True, padding=True)
mapping = inputs.pop('overflow_to_sample_mapping')
`
But I don't get how to use the mapping for the next iteration, It's just a tensor with as many entries as tokens, counting up from 0.

I've looked at the documentation (@sgugger) here https://huggingface.co/transformers/internal/tokenization_utils.html and here https://huggingface.co/transformers/main_classes/tokenizer.html#transformers.PreTrainedTokenizer but the output format does not exactly match my results, since I don't get the overflowing tokens, just the mapping. I tried to look at the flair library as well, since it already implements something similar for transformers but their approach seems to be for another data format too. ( https://github.com/flairNLP/flair/blob/4d1bfec296ae8000268f8bbf62d71042e3714ace/flair/embeddings/token.py#L949 )

Can someone tell me what I am doing wrong? I just want to split the tokens in sizes that a bert model (512) can handle (blocks or sliding-window, I will have to test what works best). I didn't think it would be that hard, but I spent already a whole day on this.


Comments:
I think this notebook could help you: https://github.com/huggingface/notebooks/blob/master/examples/language_modeling.ipynb you should check out the `def tokenizer(...)`, `def group_texts(...)` functions. I think they should help at what you want to achieve. 

Thank you for the fast response @patrickvonplaten.
I reviewed your link only to find out it was an input problem on my side that I did not see before. Sorry to bother you for that.
Just in case anyone comes across a similar issue here is the solution I found to be working for me.

```
class german_bert_sentiment: 
	"""
	Sentiment analyzer module based on a range of sources including twitter, facebook, product reviews
	https://huggingface.co/oliverguhr/german-sentiment-bert?text = Du+Arsch%21
	"""

	def __init__(self, truncate=False):
		self.tokenizer = AutoTokenizer.from_pretrained("oliverguhr/german-sentiment-bert")
		self.model = AutoModelForSequenceClassification.from_pretrained("oliverguhr/german-sentiment-bert")
		self.truncate=truncate
		self.max_length=512

	def analyze(self, text):
		averages=[]
		errors=[]
		inputs = self.tokenizer(text, return_tensors = "pt")#, max_length=512, stride=0, return_overflowing_tokens=True, truncation=True, padding=True)
		length=len(inputs['input_ids'][0])
		while length>0:
			if length>self.max_length:
				next_inputs={k: (i[0][self.max_length:]).reshape(1,len(i[0][self.max_length:])) for k, i in inputs.items()}
				inputs={k: (i[0][:self.max_length]).reshape(1,len(i[0][:self.max_length])) for k, i in inputs.items()}
			else:
				next_inputs=False
			proOrCon = self.model(**inputs)
			weights = proOrCon[0].detach().numpy()[0]
			weights[2], weights[1] = weights[1], weights[2]
			weights = softmax(weights)
			average=np.average(np.linspace(1, -1, 3), weights = weights)
			averages.append(average)
			errors.append(
				np.sqrt(np.average(np.array(np.linspace(1, -1, 3)-average)**2, weights = weights))
				)
			#from IPython import embed; embed()
			if self.truncate:
				break
			if next_inputs:
				inputs=next_inputs
			else:
				break
			length=len(inputs['input_ids'][0])
		average = np.average(averages, weights = 1./np.array(errors)**2)
		error = np.sqrt(1./np.sum(1./np.array(errors)**2))
		return [average, error]
```