lucasbiagettia commited on
Commit
492221c
1 Parent(s): 6d4ed9b

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +90 -0
README.md CHANGED
@@ -1,3 +1,93 @@
1
  ---
2
  license: apache-2.0
3
  ---
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
  ---
2
  license: apache-2.0
3
  ---
4
+ ---
5
+ language:
6
+ - es
7
+ license: apache-2.0
8
+ tags:
9
+ - "borges"
10
+ - "spanish"
11
+ - "text-generation"
12
+ datasets:
13
+ - "borges_works"
14
+ widget:
15
+ - text: "El modelo del lenguaje GPT es capaz de"
16
+ - text: "Las obras de Borges son una fuente rica de conocimiento y creatividad"
17
+ ---
18
+
19
+ # GPT2-borges (gpt2-borges)
20
+
21
+ ## Table of Contents
22
+ <details>
23
+ <summary>Click to expand</summary>
24
+
25
+ - [Overview](#overview)
26
+ - [Model description](#model-description)
27
+ - [Intended uses and limitations](#intended-uses-and-limitations)
28
+ - [How to Use](#how-to-use)
29
+ - [Training](#training)
30
+ - [Training data](#training-data)
31
+ - [Training procedure](#training-procedure)
32
+ - [Additional information](#additional-information)
33
+ - [Author](#author)
34
+ - [Contact information](#contact-information)
35
+ - [Copyright](#copyright)
36
+ - [Licensing information](#licensing-information)
37
+ - [Funding](#funding)
38
+ - [Citation Information](#citation-information)
39
+ - [Disclaimer](#disclaimer)
40
+
41
+ </details>
42
+
43
+ ## Overview
44
+
45
+ - **Architecture:** gpt2-base
46
+ - **Language:** Spanish
47
+ - **Task:** text-generation
48
+ - **Data:** Borges Works
49
+
50
+ ## Model description
51
+
52
+ **GPT2-borges** is a transformer-based model for the Spanish language. It is based on the PlanTL-GOB-ES/gpt2-base-bne model and has been pre-trained using a curated dataset consisting of the complete works of Jorge Luis Borges, a renowned Argentine writer.
53
+
54
+ ## Intended uses and limitations
55
+ You can use the raw model for text generation or fine-tune it to a downstream task.
56
+
57
+ ## How to Use
58
+ Here is how to use this model:
59
+
60
+ You can use this model directly with a pipeline for text generation. Since the generation relies on some randomness, we set a seed for reproducibility:
61
+
62
+ ```python
63
+ from transformers import AutoTokenizer, AutoModelForCausalLM
64
+ model_name = 'lucasbiagettia/gpt2-base-borges'
65
+ tokenizer = AutoTokenizer.from_pretrained(model_name)
66
+ model = AutoModelForCausalLM.from_pretrained(model_name)
67
+ ```
68
+
69
+ ```python
70
+ input_text = "La arena me recuerda el profundo dolor de la nostalgia"
71
+
72
+ input_ids = self.tokenizer.encode(input_text, return_tensors="pt")
73
+ attention_mask = torch.ones(input_ids.shape, dtype=torch.long)
74
+
75
+
76
+ generated_text = self.model.generate(
77
+ input_ids=input_ids,
78
+ attention_mask=attention_mask,
79
+ max_new_tokens = 100,
80
+ num_return_sequences=1,
81
+ no_repeat_ngram_size=6,
82
+ top_k=35,
83
+ top_p=0.95,
84
+ temperature=0.8,
85
+ pad_token_id=50256,
86
+ do_sample=True,
87
+ )
88
+ ```
89
+
90
+ ## Training
91
+ was trained with the following dataset:
92
+
93
+ https://github.com/lucasbiagettia/borges_plain_text_dataset