vedu commited on
Commit
ea7b3a2
1 Parent(s): 72745a0

update readme

Browse files
Files changed (1) hide show
  1. README.md +5 -2
README.md CHANGED
@@ -17,6 +17,9 @@ BART is particularly effective when fine-tuned for text generation (e.g. summari
17
 
18
  ## Intended uses & limitations
19
 
 
 
 
20
  You can use the raw model for text infilling. However, the model is mostly meant to be fine-tuned on a supervised dataset. See the [model hub](https://huggingface.co/models?search=bart) to look for fine-tuned versions on a task that interests you.
21
 
22
  ### How to use
@@ -26,8 +29,8 @@ Here is how to use this model in PyTorch:
26
  ```python
27
  from transformers import BartTokenizer, BartModel
28
 
29
- tokenizer = BartTokenizer.from_pretrained('facebook/bart-large')
30
- model = BartModel.from_pretrained('facebook/bart-large')
31
 
32
  inputs = tokenizer("Hello, my dog is cute", return_tensors="pt")
33
  outputs = model(**inputs)
 
17
 
18
  ## Intended uses & limitations
19
 
20
+ There have been quite a few issues related to finetuning BART for text generation, and this repo implements solution discussed in [#15559](https://github.com/huggingface/transformers/issues/15559).
21
+ Particularly adding some noise to pre-trained model's BOS embedding. This seems to solve the problem of endless BOS generation for a finetuned BART model.
22
+
23
  You can use the raw model for text infilling. However, the model is mostly meant to be fine-tuned on a supervised dataset. See the [model hub](https://huggingface.co/models?search=bart) to look for fine-tuned versions on a task that interests you.
24
 
25
  ### How to use
 
29
  ```python
30
  from transformers import BartTokenizer, BartModel
31
 
32
+ tokenizer = BartTokenizer.from_pretrained('vedu/bart-large-perturbed')
33
+ model = BartModel.from_pretrained('vedu/bart-large-perturbed')
34
 
35
  inputs = tokenizer("Hello, my dog is cute", return_tensors="pt")
36
  outputs = model(**inputs)