julien-c HF staff commited on
Commit
635f3da
1 Parent(s): abd26c5

Migrate model card from transformers-repo

Browse files

Read announcement at https://discuss.huggingface.co/t/announcement-all-model-cards-will-be-migrated-to-hf-co-model-repos/2755
Original file history: https://github.com/huggingface/transformers/commits/master/model_cards/microsoft/xprophetnet-large-wiki100-cased-xglue-qg/README.md

Files changed (1) hide show
  1. README.md +31 -0
README.md ADDED
@@ -0,0 +1,31 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ## xprophetnet-large-wiki100-cased-xglue-ntg
2
+ Cross-lingual version [ProphetNet](https://arxiv.org/abs/2001.04063), pretrained on [wiki100 xGLUE dataset](https://arxiv.org/abs/2004.01401) and finetuned on xGLUE cross-lingual Question Generation task.
3
+ ProphetNet is a new pre-trained language model for sequence-to-sequence learning with a novel self-supervised objective called future n-gram prediction.
4
+ ProphetNet is able to predict more future tokens with a n-stream decoder. The original implementation is Fairseq version at [github repo](https://github.com/microsoft/ProphetNet).
5
+
6
+ xProphetNet is also served as the baseline model for xGLUE cross-lingual natural language generation tasks.
7
+ For xGLUE corss-lingual NLG tasks, xProphetNet is finetuned with English data, but inference with both English and other zero-shot language data.
8
+ ### Usage
9
+ A quick usage is like:
10
+ ```
11
+ from transformers import ProphetNetTokenizer, ProphetNetForConditionalGeneration, ProphetNetConfig
12
+
13
+ model = ProphetNetForConditionalGeneration.from_pretrained('microsoft/xprophetnet-large-wiki100-cased-xglue-qg')
14
+ tokenizer = ProphetNetTokenizer.from_pretrained('microsoft/xprophetnet-large-wiki100-cased-xglue-qg')
15
+
16
+ EN_SENTENCE = "Google left China in 2010"
17
+ ZH_SENTENCE = "Google在2010年离开中国"
18
+ inputs = tokenizer([EN_SENTENCE, ZH_SENTENCE], padding=True, max_length=256, return_tensors='pt')
19
+
20
+ summary_ids = model.generate(inputs['input_ids'], num_beams=4, max_length=100, early_stopping=True)
21
+ print([tokenizer.decode(g) for g in summary_ids])
22
+ ```
23
+ ### Citation
24
+ ```bibtex
25
+ @article{yan2020prophetnet,
26
+ title={Prophetnet: Predicting future n-gram for sequence-to-sequence pre-training},
27
+ author={Yan, Yu and Qi, Weizhen and Gong, Yeyun and Liu, Dayiheng and Duan, Nan and Chen, Jiusheng and Zhang, Ruofei and Zhou, Ming},
28
+ journal={arXiv preprint arXiv:2001.04063},
29
+ year={2020}
30
+ }
31
+ ```