patrickvonplaten commited on
Commit
92a0cac
1 Parent(s): 8ea9a13

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +16 -4
README.md CHANGED
@@ -5,6 +5,8 @@ datasets:
5
  - wikipedia
6
  - natural_questions
7
 
 
 
8
  license: apache-2.0
9
  ---
10
 
@@ -20,6 +22,18 @@ Paper: [How Much Knowledge Can You Pack
20
  Into the Parameters of a Language Model?](https://arxiv.org/abs/1910.10683.pdf)
21
 
22
  Authors: *Adam Roberts, Colin Raffel, Noam Shazeer*
 
 
 
 
 
 
 
 
 
 
 
 
23
  ## Usage
24
 
25
  The model can be used as follows for **closed book question answering**:
@@ -27,15 +41,13 @@ The model can be used as follows for **closed book question answering**:
27
  ```python
28
  from transformers import AutoModelForSeq2SeqLM, AutoTokenizer
29
 
30
- t5_qa_model = AutoModelForSeq2SeqLM.from_pretrained("google/t5-small-ssm-nq")
31
- t5_tok = AutoTokenizer.from_pretrained("google/t5-small-ssm-nq")
32
 
33
  input_ids = t5_tok("When was Franklin D. Roosevelt born?", return_tensors="pt").input_ids
34
  gen_output = t5_qa_model.generate(input_ids)[0]
35
 
36
  print(t5_tok.decode(gen_output, skip_special_tokens=True))
37
-
38
- # should give "1917" => not correct sadly.
39
  ```
40
 
41
  ## Abstract
 
5
  - wikipedia
6
  - natural_questions
7
 
8
+ pipeline_tag: text2text-generation
9
+
10
  license: apache-2.0
11
  ---
12
 
 
22
  Into the Parameters of a Language Model?](https://arxiv.org/abs/1910.10683.pdf)
23
 
24
  Authors: *Adam Roberts, Colin Raffel, Noam Shazeer*
25
+
26
+ ## Results on Natural Questions - Open Test
27
+
28
+ |Id | link | Exact Match |
29
+ |---|---|---|
30
+ |**T5-small**|**https://huggingface.co/google/t5-small-ssm-nq**|**25.5**|
31
+ |T5-large|https://huggingface.co/google/t5-large-ssm-nq|30.4|
32
+ |T5-xl|https://huggingface.co/google/t5-xl-ssm-nq|35.6|
33
+ |T5-xxl|https://huggingface.co/google/t5-xxl-ssm-nq|37.9|
34
+ |T5-3b|https://huggingface.co/google/t5-3b-ssm-nq|33.2|
35
+ |T5-11b|https://huggingface.co/google/t5-11b-ssm-nq|36.6|
36
+
37
  ## Usage
38
 
39
  The model can be used as follows for **closed book question answering**:
 
41
  ```python
42
  from transformers import AutoModelForSeq2SeqLM, AutoTokenizer
43
 
44
+ t5_qa_model = AutoModelForSeq2SeqLM.from_pretrained("google/t5-large-ssm-nq")
45
+ t5_tok = AutoTokenizer.from_pretrained("google/t5-large-ssm-nq")
46
 
47
  input_ids = t5_tok("When was Franklin D. Roosevelt born?", return_tensors="pt").input_ids
48
  gen_output = t5_qa_model.generate(input_ids)[0]
49
 
50
  print(t5_tok.decode(gen_output, skip_special_tokens=True))
 
 
51
  ```
52
 
53
  ## Abstract