rohitsroch commited on
Commit
2e8eecc
1 Parent(s): 7f58acc

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +60 -22
README.md CHANGED
@@ -1,48 +1,86 @@
1
  ---
 
 
 
2
  tags:
3
- - generated_from_trainer
4
- model-index:
5
  - name: hybrid_utt-clusterrank_bart-base_samsum_sum
6
- results: []
 
 
 
 
 
7
  ---
8
 
9
- <!-- This model card has been generated automatically according to the information the Trainer had access to. You
10
- should probably proofread and complete it, then remove this comment. -->
11
 
12
- # hybrid_utt-clusterrank_bart-base_samsum_sum
 
13
 
14
- This model is a fine-tuned version of [best-models/bertDHighlighter/bart-base](https://huggingface.co/best-models/bertDHighlighter/bart-base) on an unknown dataset.
15
 
16
- ## Model description
17
 
18
- More information needed
19
 
20
- ## Intended uses & limitations
21
 
22
- More information needed
23
 
24
- ## Training and evaluation data
25
 
26
  More information needed
27
 
28
- ## Training procedure
29
-
30
  ### Training hyperparameters
31
 
32
  The following hyperparameters were used during training:
33
- - learning_rate: 0.0005
34
- - train_batch_size: 4
35
- - eval_batch_size: 4
36
  - seed: 42
37
  - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
38
  - lr_scheduler_type: linear
39
  - lr_scheduler_warmup_ratio: 0.1
40
- - num_epochs: 6.0
41
  - label_smoothing_factor: 0.1
42
 
 
 
 
 
 
 
 
 
 
 
 
 
43
  ### Framework versions
44
 
45
- - Transformers 4.17.0
46
- - Pytorch 1.6.0
47
- - Datasets 1.15.0
48
- - Tokenizers 0.12.1
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
  ---
2
+ language:
3
+ - en
4
+ license: apache-2.0
5
  tags:
6
+ - dialogue-summarization
7
+ model_index:
8
  - name: hybrid_utt-clusterrank_bart-base_samsum_sum
9
+ results:
10
+ - task:
11
+ name: Summarization
12
+ type: summarization
13
+ datasets:
14
+ - samsum
15
  ---
16
 
17
+ ## Paper
 
18
 
19
+ ## [Domain Adapted Abstractive Summarization of Dialogue using Transfer Learning](https://dl.acm.org/doi/10.1145/3508546.3508640)
20
+ Authors: *Rohit Sroch*
21
 
22
+ ## Abstract
23
 
24
+ Recently, the abstractive dialogue summarization task has been gaining a lot of attention from researchers. Also, unlike news articles and documents with well-structured text, dialogue differs in the sense that it often comes from two or more interlocutors, exchanging information with each other and having an inherent hierarchical structure based on the sequence of utterances by different speakers. This paper proposes a simple but effective hybrid approach that consists of two modules and uses transfer learning by leveraging pretrained language models (PLMs) to generate an abstractive summary. The first module highlights important utterances, capturing the utterance level relationship by adapting an auto-encoding model like BERT based on the unsupervised or supervised method. And then, the second module generates a concise abstractive summary by adapting encoder-decoder models like T5, BART, and PEGASUS. Experiment results on benchmark datasets show that our approach achieves a state-of-the-art performance by adapting to dialogue scenarios and can also be helpful in low-resource settings for domain adaptation.
25
 
26
+ *Rohit Sroch. 2021. Domain Adapted Abstractive Summarization of Dialogue using Transfer Learning. In 2021 4th International Conference on Algorithms, Computing and Artificial Intelligence (ACAI'21). Association for Computing Machinery, New York, NY, USA, Article 94, 1–6. https://doi.org/10.1145/3508546.3508640*
27
 
28
+ # hybrid_utt-clusterrank_bart-base_samsum_sum
29
 
30
+ This model is a fine-tuned version of [facebook/bart-base](https://huggingface.co/facebook/bart-base) on SAMSum dataset for dialogue summarization task.
31
 
32
+ ## Model description
33
 
34
  More information needed
35
 
 
 
36
  ### Training hyperparameters
37
 
38
  The following hyperparameters were used during training:
39
+ - learning_rate: 5e-5
40
+ - train_batch_size: 8
41
+ - eval_batch_size: 8
42
  - seed: 42
43
  - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
44
  - lr_scheduler_type: linear
45
  - lr_scheduler_warmup_ratio: 0.1
46
+ - num_epochs: 10.0
47
  - label_smoothing_factor: 0.1
48
 
49
+ ### Results on Test Set
50
+
51
+ - predict_gen_len = 23.9048
52
+ - predict_rouge1 = **47.355**
53
+ - predict_rouge2 = **22.4593**
54
+ - predict_rougeL = **38.694**
55
+ - predict_rougeLsum = **42.98**
56
+ - predict_samples = 819
57
+ - predict_samples_per_second = 9.279
58
+ - predict_steps_per_second = 2.322
59
+
60
+
61
  ### Framework versions
62
 
63
+ - Transformers>=4.8.0
64
+ - Pytorch>=1.6.0
65
+ - Datasets>=1.10.2
66
+ - Tokenizers>=0.10.3
67
+
68
+ If you use this model, please cite the following paper:
69
+
70
+ ```
71
+ @inproceedings{10.1145/3508546.3508640,
72
+ author = {Sroch, Rohit},
73
+ title = {Domain Adapted Abstractive Summarization of Dialogue Using Transfer Learning},
74
+ year = {2021},
75
+ isbn = {9781450385053},
76
+ publisher = {Association for Computing Machinery},
77
+ address = {New York, NY, USA},
78
+ url = {https://doi.org/10.1145/3508546.3508640},
79
+ doi = {10.1145/3508546.3508640},
80
+ articleno = {94},
81
+ numpages = {6},
82
+ keywords = {encoder-decoder, T5, abstractive summary, PEGASUS, BART, dialogue summarization, PLMs, BERT},
83
+ location = {Sanya, China},
84
+ series = {ACAI'21}
85
+ }
86
+ ```