add score
Browse files
README.md
CHANGED
@@ -99,5 +99,21 @@ print(result[0]['summary_text'])
|
|
99 |
|
100 |
# Results
|
101 |
|
102 |
-
- note that while the dataset has three subsets (chapter, book, paragraph) - see the [paper](https://arxiv.org/abs/2105.08209). the scores below are run in aggregate
|
103 |
-
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
99 |
|
100 |
# Results
|
101 |
|
102 |
+
- note that while the dataset has three subsets (chapter, book, paragraph) - see the [paper](https://arxiv.org/abs/2105.08209). the scores below are run in aggregate. The paper has some benchmark scores listed, which this model competes with.
|
103 |
+
- note that eval generations are run & computed at a length of 128 tokens.
|
104 |
+
|
105 |
+
|
106 |
+
```
|
107 |
+
{
|
108 |
+
"eval_gen_len": 126.7581,
|
109 |
+
"eval_loss": 3.732882499694824,
|
110 |
+
"eval_rouge1": 31.1471,
|
111 |
+
"eval_rouge2": 4.9621,
|
112 |
+
"eval_rougeL": 16.867,
|
113 |
+
"eval_rougeLsum": 28.0703,
|
114 |
+
"eval_runtime": 6083.71,
|
115 |
+
"eval_samples_per_second": 0.244,
|
116 |
+
"eval_steps_per_second": 0.244
|
117 |
+
}
|
118 |
+
|
119 |
+
```
|