Update README.md
Browse files
README.md
CHANGED
|
@@ -5,16 +5,14 @@ language: en
|
|
| 5 |
|
| 6 |
# BART (large-sized model)
|
| 7 |
|
| 8 |
-
BART model pre-trained on English language. It was introduced in the paper [BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension](https://arxiv.org/abs/1910.13461) by Lewis et al. and first released in [this repository](https://github.com/pytorch/fairseq/tree/master/examples/bart).
|
| 9 |
-
|
| 10 |
-
Disclaimer: The team releasing BART did not write a model card for this model so this model card has been written by the Hugging Face team.
|
| 11 |
-
|
| 12 |
## Model description
|
| 13 |
|
| 14 |
BART is a transformer encoder-decoder (seq2seq) model with a bidirectional (BERT-like) encoder and an autoregressive (GPT-like) decoder. BART is pre-trained by (1) corrupting text with an arbitrary noising function, and (2) learning a model to reconstruct the original text.
|
| 15 |
|
| 16 |
BART is particularly effective when fine-tuned for text generation (e.g. summarization, translation) but also works well for comprehension tasks (e.g. text classification, question answering).
|
| 17 |
|
|
|
|
|
|
|
| 18 |
## Intended uses & limitations
|
| 19 |
|
| 20 |
There have been quite a few issues related to finetuning BART for text generation, and this repo implements solution discussed in [#15559](https://github.com/huggingface/transformers/issues/15559).
|
|
|
|
| 5 |
|
| 6 |
# BART (large-sized model)
|
| 7 |
|
|
|
|
|
|
|
|
|
|
|
|
|
| 8 |
## Model description
|
| 9 |
|
| 10 |
BART is a transformer encoder-decoder (seq2seq) model with a bidirectional (BERT-like) encoder and an autoregressive (GPT-like) decoder. BART is pre-trained by (1) corrupting text with an arbitrary noising function, and (2) learning a model to reconstruct the original text.
|
| 11 |
|
| 12 |
BART is particularly effective when fine-tuned for text generation (e.g. summarization, translation) but also works well for comprehension tasks (e.g. text classification, question answering).
|
| 13 |
|
| 14 |
+
Weights shared here are effectively from facebook/bart-large but with added noise for BOS embedding to assist the finetuning.
|
| 15 |
+
|
| 16 |
## Intended uses & limitations
|
| 17 |
|
| 18 |
There have been quite a few issues related to finetuning BART for text generation, and this repo implements solution discussed in [#15559](https://github.com/huggingface/transformers/issues/15559).
|