patrickvonplaten commited on
Commit
b4ed1b6
1 Parent(s): 3700573

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +14 -4
README.md CHANGED
@@ -22,6 +22,18 @@ Paper: [How Much Knowledge Can You Pack
22
  Into the Parameters of a Language Model?](https://arxiv.org/abs/1910.10683.pdf)
23
 
24
  Authors: *Adam Roberts, Colin Raffel, Noam Shazeer*
 
 
 
 
 
 
 
 
 
 
 
 
25
  ## Usage
26
 
27
  The model can be used as follows for **closed book question answering**:
@@ -29,15 +41,13 @@ The model can be used as follows for **closed book question answering**:
29
  ```python
30
  from transformers import AutoModelForSeq2SeqLM, AutoTokenizer
31
 
32
- t5_qa_model = AutoModelForSeq2SeqLM.from_pretrained("google/t5-xl-ssm-nq")
33
- t5_tok = AutoTokenizer.from_pretrained("google/t5-xl-ssm-nq")
34
 
35
  input_ids = t5_tok("When was Franklin D. Roosevelt born?", return_tensors="pt").input_ids
36
  gen_output = t5_qa_model.generate(input_ids)[0]
37
 
38
  print(t5_tok.decode(gen_output, skip_special_tokens=True))
39
-
40
- # should give "December 26, 1892" => close, but not correct.
41
  ```
42
 
43
  ## Abstract
 
22
  Into the Parameters of a Language Model?](https://arxiv.org/abs/1910.10683.pdf)
23
 
24
  Authors: *Adam Roberts, Colin Raffel, Noam Shazeer*
25
+
26
+ ## Results on Natural Questions - Open Test
27
+
28
+ |Id | link | Exact Match |
29
+ |---|---|---|
30
+ |T5-small|https://huggingface.co/google/t5-small-ssm-nq|25.5|
31
+ |T5-large|https://huggingface.co/google/t5-large-ssm-nq|30.4|
32
+ |**T5-xl**|**https://huggingface.co/google/t5-xl-ssm-nq**|**35.6**|
33
+ |T5-xxl|https://huggingface.co/google/t5-xxl-ssm-nq|37.9|
34
+ |T5-3b|https://huggingface.co/google/t5-3b-ssm-nq|33.2|
35
+ |T5-11b|https://huggingface.co/google/t5-11b-ssm-nq|36.6|
36
+
37
  ## Usage
38
 
39
  The model can be used as follows for **closed book question answering**:
 
41
  ```python
42
  from transformers import AutoModelForSeq2SeqLM, AutoTokenizer
43
 
44
+ t5_qa_model = AutoModelForSeq2SeqLM.from_pretrained("google/t5-large-ssm-nq")
45
+ t5_tok = AutoTokenizer.from_pretrained("google/t5-large-ssm-nq")
46
 
47
  input_ids = t5_tok("When was Franklin D. Roosevelt born?", return_tensors="pt").input_ids
48
  gen_output = t5_qa_model.generate(input_ids)[0]
49
 
50
  print(t5_tok.decode(gen_output, skip_special_tokens=True))
 
 
51
  ```
52
 
53
  ## Abstract