1 ---
2 language:
3 - en
4 thumbnail:
5 tags:
6 - pytorch
7 - google/pegasus-reddit_tifu
8 - summarization
9 - samsum
10 license:
11 datasets:
12 - samsum
13 metrics:
14 - rouge
15 ---
16
17 # Samsum Pegasus (Reddit/TIFU) for conversational summaries
18
19 ## Model description
20
21 Pegasus (Reddit/TIFU) for conversational summaries trained on the samsum dataset!
22
23 ## Training data
24
25 The data is the [samsum](https://huggingface.co/datasets/samsum) dataset for conversional summaries.
26
27 The initial weigths were from the [google/pegasus-reddit_tifu](https://huggingface.co/google/pegasus-reddit_tifu). The hypothesis being that it would help the convergence on the samsum dataset to have weights trained on a larger summarization dataset first like the Reddit TIFU using casual language.
28
29 ## Training procedure
30
31 Used the _example/seq2seq/run_summarization.py_ script from the transformers source _4.5.0dev0_.
32
33 n_epochs: 3,\
34 batch_size: 8, \
35 max_source_length: 256,\
36 max_target_length: 128
37
38 ## Eval results
39
40 eval_gen_len: 35.9939,\
41 eval_loss: 1.4284523725509644,\
42 eval_rouge1: 46.5613,\
43 eval_rouge2: 23.6137,\
44 eval_rougeL: 37.2397,\
45 eval_rougeLsum: 42.7126,\
46 eval_samples_per_second: 4.302
47
48 ## Example
49
50 from transformers import PegasusForConditionalGeneration, PegasusTokenizer
51
52 model_name = "jpcorb20/pegasus-large-reddit_tifu-samsum-256"
53
54 tokenizer = PegasusTokenizer.from_pretrained(model_name)
55 model = PegasusForConditionalGeneration.from_pretrained(model_name)
56
57 src_text = """Carter: Hey Alexis, I just wanted to let you know that I had a really nice time with you tonight.\r\nAlexis: Thanks Carter. Yeah, I really enjoyed myself as well.\r\nCarter: If you are up for it, I would really like to see you again soon.\r\nAlexis: Thanks Carter, I'm flattered. But I have a really busy week coming up.\r\nCarter: Yeah, no worries. I totally understand. But if you ever want to go grab dinner again, just let me know.\r\nAlexis: Yeah of course. Thanks again for tonight. Carter: Sure. Have a great night.\r\n"""
58
59 token_params = dict(max_length=256, truncation=True, padding='longest', return_tensors="pt")
60 batch = tokenizer(src_text, **token_params)
61
62 translated = model.generate(**batch)
63
64 decode_params = dict(num_beams=5, min_length=16, max_length=128, length_penalty=2)
65 tgt_text = tokenizer.batch_decode(translated, skip_special_tokens=True, **decode_params)
66
67 print(tgt_text)