metadata
language: en
datasets:
- cnn_dailymail
license: apache-2.0
DistilLED Large CNN 16384
distil-led-large-cnn-16384 was initialized from sshleifer/distilbart-cnn-12-6, in a fashion similar to allenai/led-large-16384.
To be able to process 16K tokens, sshleifer/distilbart-cnn-12-6's position embedding matrix was simply copied 16 times.
This checkpoint should be loaded into LEDForConditionalGeneration.from_pretrained. See the LED documentation for more information.