patrickvonplaten commited on
Commit
f56866a
1 Parent(s): d291741

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +4 -3
README.md CHANGED
@@ -13,6 +13,7 @@ license: apache-2.0
13
  The model was pre-trained using T5's denoising objective on [C4](https://huggingface.co/datasets/c4), subsequently additionally pre-trained using [REALM](https://arxiv.org/pdf/2002.08909.pdf)'s salient span masking objective on [Wikipedia](https://huggingface.co/datasets/wikipedia), and finally fine-tuned on [Natural Questions (NQ)](https://huggingface.co/datasets/natural_questions).
14
 
15
  **Note**: The model was fine-tuned on 90% of the train splits of [Natural Questions (NQ)](https://huggingface.co/datasets/natural_questions) for 20k steps.
 
16
  Other community Checkpoints: [here](https://huggingface.co/models?search=ssm)
17
 
18
  Paper: [How Much Knowledge Can You Pack
@@ -26,15 +27,15 @@ The model can be used as follows for **closed book question answering**:
26
  ```python
27
  from transformers import AutoModelForSeq2SeqLM, AutoTokenizer
28
 
29
- t5_qa_model = AutoModelForSeq2SeqLM.from_pretrained("google/t5-large-ssm-nq")
30
- t5_tok = AutoTokenizer.from_pretrained("google/t5-large-ssm-nq")
31
 
32
  input_ids = t5_tok("When was Franklin D. Roosevelt born?", return_tensors="pt").input_ids
33
  gen_output = t5_qa_model.generate(input_ids)[0]
34
 
35
  print(t5_tok.decode(gen_output, skip_special_tokens=True))
36
 
37
- # should give "December 26, 1892" => close, but not correct.
38
  ```
39
 
40
  ## Abstract
13
  The model was pre-trained using T5's denoising objective on [C4](https://huggingface.co/datasets/c4), subsequently additionally pre-trained using [REALM](https://arxiv.org/pdf/2002.08909.pdf)'s salient span masking objective on [Wikipedia](https://huggingface.co/datasets/wikipedia), and finally fine-tuned on [Natural Questions (NQ)](https://huggingface.co/datasets/natural_questions).
14
 
15
  **Note**: The model was fine-tuned on 90% of the train splits of [Natural Questions (NQ)](https://huggingface.co/datasets/natural_questions) for 20k steps.
16
+
17
  Other community Checkpoints: [here](https://huggingface.co/models?search=ssm)
18
 
19
  Paper: [How Much Knowledge Can You Pack
27
  ```python
28
  from transformers import AutoModelForSeq2SeqLM, AutoTokenizer
29
 
30
+ t5_qa_model = AutoModelForSeq2SeqLM.from_pretrained("google/t5-large-ssm-nqo")
31
+ t5_tok = AutoTokenizer.from_pretrained("google/t5-large-ssm-nqo")
32
 
33
  input_ids = t5_tok("When was Franklin D. Roosevelt born?", return_tensors="pt").input_ids
34
  gen_output = t5_qa_model.generate(input_ids)[0]
35
 
36
  print(t5_tok.decode(gen_output, skip_special_tokens=True))
37
 
38
+ # should give "On February 13, 1904" => not correct sadly.
39
  ```
40
 
41
  ## Abstract