Update README.md
Browse files
README.md
CHANGED
@@ -9,8 +9,8 @@ tags:
|
|
9 |
-
|
10 |
license:
|
11 |
datasets:
|
12 |
-
- XSUM
|
13 |
-
-
|
14 |
metrics:
|
15 |
- Rouge
|
16 |
-
|
@@ -20,7 +20,7 @@ metrics:
|
|
20 |
|
21 |
## Model description
|
22 |
|
23 |
-
Pegasus XSUM model finetuned to Gigaword Summarization task
|
24 |
|
25 |
## Intended uses & limitations
|
26 |
Produces short summaries with the coherence of the XSUM Model
|
@@ -42,11 +42,24 @@ Initialized with pegasus-XSUM
|
|
42 |
Trained for 11500 iterations on Gigaword corpus using OOB seq2seq (from hugging face using the default parameters)
|
43 |
|
44 |
## Eval results
|
45 |
-
Evaluated on Gigaword
|
46 |
-
|
47 |
-
|
48 |
-
|
49 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
50 |
|
51 |
### BibTeX entry and citation info
|
52 |
|
|
|
9 |
-
|
10 |
license:
|
11 |
datasets:
|
12 |
+
- XSUM
|
13 |
+
- Gigaword
|
14 |
metrics:
|
15 |
- Rouge
|
16 |
-
|
|
|
20 |
|
21 |
## Model description
|
22 |
|
23 |
+
Pegasus XSUM model finetuned to Gigaword Summarization task, significantly better performance than pegasus gigaword, but still doesn't match model paper performance.
|
24 |
|
25 |
## Intended uses & limitations
|
26 |
Produces short summaries with the coherence of the XSUM Model
|
|
|
42 |
Trained for 11500 iterations on Gigaword corpus using OOB seq2seq (from hugging face using the default parameters)
|
43 |
|
44 |
## Eval results
|
45 |
+
Evaluated on Gigaword test set (from hugging face using the default parameters)
|
46 |
+
run_summarization.py --model_name_or_path pegasus-xsum/checkpoint-11500/ --do_predict --dataset_name gigaword --dataset_config "3.0.0" --source_prefix "summarize: " --output_dir pegasus-xsum --per_device_train_batch_size=8 --per_device_eval_batch_size=8 --overwrite_output_dir --predict_with_generate
|
47 |
+
|
48 |
+
| Metric | Score |
|
49 |
+
| ----------- | ----------- |
|
50 |
+
| eval_rouge1 | 34.1958 |
|
51 |
+
| eval_rouge2 | 15.4033 |
|
52 |
+
| eval_rougeL | 31.4488 |
|
53 |
+
|
54 |
+
|
55 |
+
run_summarization.py --model_name_or_path google/pegasus-gigaword --do_predict --dataset_name gigaword --dataset_config "3.0.0" --source_prefix "summarize: " --output_dir pegasus-xsum --per_device_train_batch_size=8 --per_device_eval_batch_size=8 --overwrite_output_dir --predict_with_generate
|
56 |
+
|
57 |
+
| Metric | Score |
|
58 |
+
| ----------- | ----------- |
|
59 |
+
| eval_rouge1 | 20.8111 |
|
60 |
+
| eval_rouge2 | 8.766 |
|
61 |
+
| eval_rougeL | 18.4431 |
|
62 |
+
|
63 |
|
64 |
### BibTeX entry and citation info
|
65 |
|