Text Generation
Transformers
PyTorch
Safetensors
English
gpt2
text-generation-inference
Inference Endpoints
lgaalves commited on
Commit
c814f93
1 Parent(s): 6bf0a81

Create README.md

Browse files
Files changed (1) hide show
  1. README.md +71 -0
README.md ADDED
@@ -0,0 +1,71 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: mit
3
+ datasets:
4
+ - garage-bAInd/Open-Platypus
5
+ - databricks/databricks-dolly-15k
6
+ - timdettmers/openassistant-guanaco
7
+ language:
8
+ - en
9
+ pipeline_tag: text-generation
10
+ ---
11
+
12
+ # gpt2_guanaco-dolly-platypus
13
+
14
+ **gpt2_guanaco-dolly-platypus** is an instruction fine-tuned model based on the GPT-2 transformer architecture.
15
+
16
+
17
+ ### Benchmark Metrics
18
+
19
+ | Metric | gpt2_guanaco-dolly-platypus | GPT-2 (base) |
20
+ |-----------------------|-------|-------|
21
+ | Avg. | - | 29.9 |
22
+ | ARC (25-shot) | - | 21.84 |
23
+ | HellaSwag (10-shot) | - | 31.6 |
24
+ | MMLU (5-shot) | - | 25.86 |
25
+ | TruthfulQA (0-shot) | - | 40.67 |
26
+
27
+
28
+ We use state-of-the-art [Language Model Evaluation Harness](https://github.com/EleutherAI/lm-evaluation-harness) to run the benchmark tests above, using the same version as the HuggingFace LLM Leaderboard. Please see below for detailed instructions on reproducing benchmark results.
29
+
30
+ ### Model Details
31
+
32
+ * **Trained by**: Luiz G A Alves
33
+ * **Model type:** **gpt2_guanaco-dolly-platypus** is an auto-regressive language model based on the GPT-2 transformer architecture.
34
+ * **Language(s)**: English
35
+
36
+ ### How to use:
37
+
38
+ ```python
39
+ # Use a pipeline as a high-level helper
40
+ >>> from transformers import pipeline
41
+ >>> pipe = pipeline("text-generation", model="lgaalves/gpt2_guanaco-dolly-platypus")
42
+ >>> question = "What is a large language model?"
43
+ >>> answer = pipe(question)
44
+ >>> print(answer[0]['generated_text'])
45
+ ```
46
+
47
+ or, you can load the model direclty using:
48
+
49
+ ```python
50
+ # Load model directly
51
+ from transformers import AutoTokenizer, AutoModelForCausalLM
52
+
53
+ tokenizer = AutoTokenizer.from_pretrained("lgaalves/gpt2_open-platypus")
54
+ model = AutoModelForCausalLM.from_pretrained("lgaalves/gpt2_open-platypus")
55
+ ```
56
+
57
+ ### Training Dataset
58
+
59
+ `lgaalves/gpt2_guanaco-dolly-platypus` was trained using 3 datasets:
60
+ - [timdettmers/openassistant-guanaco](https://huggingface.co/datasets/timdettmers/openassistant-guanaco)
61
+ - [databricks/databricks-dolly-15k](https://huggingface.co/datasets/databricks/databricks-dolly-15k)
62
+ - [garage-bAInd/Open-Platypus](https://huggingface.co/datasets/garage-bAInd/Open-Platypus)
63
+
64
+ ### Training Procedure
65
+
66
+ `lgaalves/gpt2_guanaco-dolly-platypus` was instruction fine-tuned using LoRA on 1 T4 GPU on Google Colab. It took about 1 hour to train it.
67
+
68
+
69
+ # Intended uses, limitations & biases
70
+
71
+ You can use the raw model for text generation or fine-tune it to a downstream task. The model was not extensively tested and may produce false information. It contains a lot of unfiltered content from the internet, which is far from neutral.