nielsr
HF staff
commited on
Commit
74e4aa2
1 Parent(s): c841166

Add model card

Browse files
Files changed (1) hide show
  1. README.md +68 -0
README.md ADDED
@@ -0,0 +1,68 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ tags:
3
+ - exbert
4
+ language: en
5
+ license: mit
6
+ datasets:
7
+ - bookcorpus
8
+ - wikipedia
9
+ ---
10
+
11
+ # XLNet (base-sized model)
12
+
13
+ XLNet model pre-trained on English language. It was introduced in the paper [XLNet: Generalized Autoregressive Pretraining for Language Understanding](https://arxiv.org/abs/1906.08237) by Yang et al. and first released in [this repository](https://github.com/zihangdai/xlnet/).
14
+
15
+ Disclaimer: The team releasing XLNet did not write a model card for this model so this model card has been written by the Hugging Face team.
16
+
17
+ ## Model description
18
+
19
+ XLNet is a new unsupervised language representation learning method based on a novel generalized permutation language modeling objective. Additionally, XLNet employs Transformer-XL as the backbone model, exhibiting excellent performance for language tasks involving long context. Overall, XLNet achieves state-of-the-art (SOTA) results on various downstream language tasks including question answering, natural language inference, sentiment analysis, and document ranking.
20
+
21
+ ## Intended uses & limitations
22
+
23
+ You can use the raw model for masked language modeling, but it's mostly intended to be fine-tuned on a downstream task. See the [model hub](https://huggingface.co/models?search=xlm-roberta) to look for fine-tuned versions on a task that interests you.
24
+
25
+ Note that this model is primarily aimed at being fine-tuned on tasks that use the whole sentence (potentially masked) to make decisions, such as sequence classification, token classification or question answering. For tasks such as text generation, you should look at models like GPT2.
26
+
27
+ ## Usage
28
+
29
+ Here is how to use this model to get the features of a given text in PyTorch:
30
+
31
+ ```python
32
+ from transformers import XLNetTokenizer, XLNetModel
33
+
34
+ tokenizer = XLNetTokenizer.from_pretrained('xlnet-base-cased')
35
+ model = XLNetModel.from_pretrained('xlnet-base-cased')
36
+
37
+ inputs = tokenizer("Hello, my dog is cute", return_tensors="pt")
38
+ outputs = model(**inputs)
39
+
40
+ last_hidden_states = outputs.last_hidden_state
41
+ ```
42
+
43
+ ### BibTeX entry and citation info
44
+
45
+ ```bibtex
46
+ @article{DBLP:journals/corr/abs-1906-08237,
47
+ author = {Zhilin Yang and
48
+ Zihang Dai and
49
+ Yiming Yang and
50
+ Jaime G. Carbonell and
51
+ Ruslan Salakhutdinov and
52
+ Quoc V. Le},
53
+ title = {XLNet: Generalized Autoregressive Pretraining for Language Understanding},
54
+ journal = {CoRR},
55
+ volume = {abs/1906.08237},
56
+ year = {2019},
57
+ url = {http://arxiv.org/abs/1906.08237},
58
+ eprinttype = {arXiv},
59
+ eprint = {1906.08237},
60
+ timestamp = {Mon, 24 Jun 2019 17:28:45 +0200},
61
+ biburl = {https://dblp.org/rec/journals/corr/abs-1906-08237.bib},
62
+ bibsource = {dblp computer science bibliography, https://dblp.org}
63
+ }
64
+ ```
65
+
66
+ <a href="https://huggingface.co/exbert/?model=xlm-roberta-base">
67
+ <img width="300px" src="https://cdn-media.huggingface.co/exbert/button.png">
68
+ </a>