How to use Longformer for summarization
Author: chetanambi
Pull request: no

Hi - Do you have a sample code on how to use Longformer for summarization tasks. 

Comments:
Perhaps this will help: https://huggingface.co/patrickvonplaten/longformer2roberta-cnn_dailymail-fp16

`LongformerEncoderDecoder` will be added to the lib once #9278 is merged! It can be used for summarization or any other seq2seq task.

@patil-suraj, @christianversloot I tried this model [longformer2roberta](https://huggingface.co/patrickvonplaten/longformer2roberta-cnn_dailymail-fp16). It is actually giving better summary than Pegasus (reddit-tifu). I will be waiting for LongformerEncoderDecoder to be added to the library.

I just have one question - how many input tokens longformer2roberta model supports? I believe it's 2048. Could you please confirm,

`longformer2roberta` should support 4096 tokens.

And LED is now on master!

@patil-suraj  Awesome! I am trying LED but getting below error. Could you please take a look?

```
from transformers import LEDForConditionalGeneration, LEDTokenizer
model_name = 'allenai/led-base-16384'
tokenizer = LEDTokenizer.from_pretrained(model_name) 
model = LEDTokenizer.from_pretrained(model_name)
batch = tokenizer.prepare_seq2seq_batch(article, truncation=True, padding='longest', return_tensors="pt").to(torch_device)
translated = model.generate(**batch)
tgt_text = tokenizer.batch_decode(translated, skip_special_tokens=True)
```

---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
<ipython-input-33-d4827d7b5770> in <module>()
      5 model = LEDTokenizer.from_pretrained(model_name)
      6 batch = tokenizer.prepare_seq2seq_batch(src_text, truncation=True, padding='longest', return_tensors="pt").to(torch_device)
----> 7 translated = model.generate(**batch)
      8 tgt_text = tokenizer.batch_decode(translated, skip_special_tokens=True)

AttributeError: 'LEDTokenizer' object has no attribute 'generate'

```python
model = LEDTokenizer.from_pretrained(model_name)
```
here you are assigning tokenizer as the model, it should be 
```python
model = LEDForConditionalGeneration.from_pretrained(model_name)
```

@patil-suraj Thank you! My bad. I have corrected the typo. Below code seems to be working fine.

```
from transformers import LEDForConditionalGeneration, LEDTokenizer
model_name = 'allenai/led-base-16384'
tokenizer = LEDTokenizer.from_pretrained(model_name) 
model = LEDForConditionalGeneration.from_pretrained(model_name)
input_ids = tokenizer(src_text, return_tensors="pt").input_ids
output_ids = model.generate(input_ids)
output = tokenizer.decode(output_ids[0], skip_special_tokens=True)
```

I am getting very short summary of just 20 tokens from the above code. So I was looking for the default values for below parameters for LED model. I could not find it in [config.json](https://huggingface.co/allenai/led-base-16384/blob/main/config.json) file. For the **longformer2roberta** model I found these values in [config.json](https://huggingface.co/patrickvonplaten/longformer2roberta-cnn_dailymail-fp16/blob/main/config.json). Could you please let me know where can I find these values for LED model.

- num_beams
- no_repeat_ngram_size,
- early_stopping,
- length_penalty,
- min_length,
- max_length

@patil-suraj Any inputs/suggestions here?

Hi, 
I have a question about the `LEDForConditionalGeneration` forward args. 
The `decoder_input_ids`  has a comment that `decoder_input_ids (torch.LongTensor of shape (batch_size, target_sequence_length), optional) – Provide for translation and summarization training. By default, the model will create this tensor by shifting the input_ids to the right, following the paper.`. 
Form the forward method in  `LEDForConditionalGeneration`, i can see that when not  assigning the `decoder_input_ids` in the forward method of  `LEDForConditionalGeneration` object , the `decoder_input_ids` will be generated by [shifting  the `labels`  value one token to right in the  forward method](https://github.com/huggingface/transformers/blob/17b6e0d474b797cdddf5225b0f51bf0e928091b9/src/transformers/models/led/modeling_led.py#L2337). 

So my question is  if i want to explictly pass the `decoder_input_ids` to the forward method, do i need to explictly shift it one token as the [code](https://github.com/huggingface/transformers/blob/17b6e0d474b797cdddf5225b0f51bf0e928091b9/src/transformers/models/led/modeling_led.py#L2337) shows before the forward pass?


This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.