pszemraj commited on
Commit
9a0c01e
1 Parent(s): 8123a13
Files changed (1) hide show
  1. README.md +18 -2
README.md CHANGED
@@ -99,5 +99,21 @@ print(result[0]['summary_text'])
99
 
100
  # Results
101
 
102
- - note that while the dataset has three subsets (chapter, book, paragraph) - see the [paper](https://arxiv.org/abs/2105.08209). the scores below are run in aggregate
103
- - full eval results soon
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
99
 
100
  # Results
101
 
102
+ - note that while the dataset has three subsets (chapter, book, paragraph) - see the [paper](https://arxiv.org/abs/2105.08209). the scores below are run in aggregate. The paper has some benchmark scores listed, which this model competes with.
103
+ - note that eval generations are run & computed at a length of 128 tokens.
104
+
105
+
106
+ ```
107
+ {
108
+ "eval_gen_len": 126.7581,
109
+ "eval_loss": 3.732882499694824,
110
+ "eval_rouge1": 31.1471,
111
+ "eval_rouge2": 4.9621,
112
+ "eval_rougeL": 16.867,
113
+ "eval_rougeLsum": 28.0703,
114
+ "eval_runtime": 6083.71,
115
+ "eval_samples_per_second": 0.244,
116
+ "eval_steps_per_second": 0.244
117
+ }
118
+
119
+ ```