julien-c HF staff commited on
Commit
c81c6f1
1 Parent(s): 6bdc91a

Migrate model card from transformers-repo

Browse files

Read announcement at https://discuss.huggingface.co/t/announcement-all-model-cards-will-be-migrated-to-hf-co-model-repos/2755
Original file history: https://github.com/huggingface/transformers/commits/master/model_cards/microsoft/prophetnet-large-uncased-cnndm/README.md

Files changed (1) hide show
  1. README.md +38 -0
README.md ADDED
@@ -0,0 +1,38 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ language: en
3
+ datasets:
4
+ - cnn_dailymail
5
+ ---
6
+
7
+ ## prophetnet-large-uncased-cnndm
8
+ Fine-tuned weights(converted from [original fairseq version repo](https://github.com/microsoft/ProphetNet)) for [ProphetNet](https://arxiv.org/abs/2001.04063) on summarization task CNN/DailyMail.
9
+ ProphetNet is a new pre-trained language model for sequence-to-sequence learning with a novel self-supervised objective called future n-gram prediction.
10
+ ProphetNet is able to predict more future tokens with a n-stream decoder. The original implementation is Fairseq version at [github repo](https://github.com/microsoft/ProphetNet).
11
+
12
+ ### Usage
13
+ ```
14
+ from transformers import ProphetNetTokenizer, ProphetNetForConditionalGeneration, ProphetNetConfig
15
+
16
+ model = ProphetNetForConditionalGeneration.from_pretrained('microsoft/prophetnet-large-uncased-cnndm')
17
+ tokenizer = ProphetNetTokenizer.from_pretrained('microsoft/prophetnet-large-uncased-cnndm')
18
+
19
+ ARTICLE_TO_SUMMARIZE = "USTC was founded in Beijing by the Chinese Academy of Sciences (CAS) in September 1958. The Director of CAS, Mr. Guo Moruo was appointed the first president of USTC. USTC's founding mission was to develop a high-level science and technology workforce, as deemed critical for development of China's economy, defense, and science and technology education. The establishment was hailed as \"A Major Event in the History of Chinese Education and Science.\" CAS has supported USTC by combining most of its institutes with the departments of the university. USTC is listed in the top 16 national key universities, becoming the youngest national key university.".lower()
20
+ inputs = tokenizer([ARTICLE_TO_SUMMARIZE], max_length=100, return_tensors='pt')
21
+
22
+ # Generate Summary
23
+ summary_ids = model.generate(inputs['input_ids'], num_beams=4, max_length=512, early_stopping=True)
24
+ tokenizer.batch_decode(summary_ids, skip_special_tokens=True)
25
+
26
+ # should give: 'ustc was founded in beijing by the chinese academy of sciences in 1958. [X_SEP] ustc\'s mission was to develop a high - level science and technology workforce. [X_SEP] the establishment was hailed as " a major event in the history of chinese education and science "'
27
+ ```
28
+
29
+ Here, [X_SEP] is used as a special token to seperate sentences.
30
+ ### Citation
31
+ ```bibtex
32
+ @article{yan2020prophetnet,
33
+ title={Prophetnet: Predicting future n-gram for sequence-to-sequence pre-training},
34
+ author={Yan, Yu and Qi, Weizhen and Gong, Yeyun and Liu, Dayiheng and Duan, Nan and Chen, Jiusheng and Zhang, Ruofei and Zhou, Ming},
35
+ journal={arXiv preprint arXiv:2001.04063},
36
+ year={2020}
37
+ }
38
+ ```