TheBloke commited on
Commit
e756f8e
1 Parent(s): b670bf2

Updating model files

Browse files
Files changed (1) hide show
  1. README.md +25 -3
README.md CHANGED
@@ -4,6 +4,17 @@ datasets:
4
  - c-s-ale/alpaca-gpt4-data
5
  pipeline_tag: text2text-generation
6
  ---
 
 
 
 
 
 
 
 
 
 
 
7
 
8
  ## GPT4-Alpaca-LoRA_MLP-65B GPTQ
9
 
@@ -60,15 +71,26 @@ It was created with `--act-order` to increase quantisation quality, but without
60
  python llama.py gpt4-alpaca-lora_mlp-65B/HF/ c4 --wbits 4 --true-sequential --act-order --save_safetensors gpt4-alpaca-lora_mlp-65B/gptq/gpt4-alpaca-lora_mlp-65B-GPTQ-4bit.safetensors
61
  ```
62
 
 
 
 
 
 
 
 
 
 
 
 
63
  # Original model card
64
 
65
- This repo provides the training checkpoint of LLaMA on the alpaca_data_gpt4 dataset via LoRA [MLP] on 8xA100(80G).
66
 
67
  He et al. 2022 gave an insight that FFN can better utilize modification at larger capacities.
68
 
69
  The codes is provided by [tloen/alpaca-lora: Instruct-tune LLaMA on consumer hardware (github.com)](https://github.com/tloen/alpaca-lora).
70
 
71
- We modify the running scripts to
72
  ```bash
73
  torchrun --nproc_per_node=8 finetune.py \
74
  --base_model '/cache1/chtan/large_models/llama-hf/llama-65b' \
@@ -92,7 +114,7 @@ torchrun --nproc_per_node=8 finetune.py \
92
 
93
  **Instruction**: Tell me about alpacas.
94
 
95
- **gpt4-alpaca-lora_mlp-65b**:
96
 
97
  Alpacas are small, domesticated mammals that are closely related to llamas. They are native to the Andes Mountains of South America, primarily in Peru, Bolivia, and Chile. These animals have been domesticated for thousands of years and were used by the Incas for their fleece, meat, and as pack animals.
98
 
 
4
  - c-s-ale/alpaca-gpt4-data
5
  pipeline_tag: text2text-generation
6
  ---
7
+ <div style="width: 100%;">
8
+ <img src="https://i.imgur.com/EBdldam.jpg" alt="TheBlokeAI" style="width: 100%; min-width: 400px; display: block; margin: auto;">
9
+ </div>
10
+ <div style="display: flex; justify-content: space-between; width: 100%;">
11
+ <div style="display: flex; flex-direction: column; align-items: flex-start;">
12
+ <p><a href="https://discord.gg/UBgz4VXf">Chat & support: my new Discord server</a></p>
13
+ </div>
14
+ <div style="display: flex; flex-direction: column; align-items: flex-end;">
15
+ <p><a href="https://www.patreon.com/TheBlokeAI">Want to contribute? Patreon coming soon!</a></p>
16
+ </div>
17
+ </div>
18
 
19
  ## GPT4-Alpaca-LoRA_MLP-65B GPTQ
20
 
 
71
  python llama.py gpt4-alpaca-lora_mlp-65B/HF/ c4 --wbits 4 --true-sequential --act-order --save_safetensors gpt4-alpaca-lora_mlp-65B/gptq/gpt4-alpaca-lora_mlp-65B-GPTQ-4bit.safetensors
72
  ```
73
 
74
+ ## Want to support my work?
75
+
76
+ I've had a lot of people ask if they can contribute. I love providing models and helping people, but it is starting to rack up pretty big cloud computing bills.
77
+
78
+ So if you're able and willing to contribute, it'd be most gratefully received and will help me to keep providing models, and work on various AI projects.
79
+
80
+ Donaters will get priority support on any and all AI/LLM/model questions, and I'll gladly quantise any model you'd like to try.
81
+
82
+ * Patreon: coming soon! (just awaiting approval)
83
+ * Ko-Fi: https://ko-fi.com/TheBlokeAI
84
+ * Discord: https://discord.gg/UBgz4VXf
85
  # Original model card
86
 
87
+ This repo provides the training checkpoint of LLaMA on the alpaca_data_gpt4 dataset via LoRA [MLP] on 8xA100(80G).
88
 
89
  He et al. 2022 gave an insight that FFN can better utilize modification at larger capacities.
90
 
91
  The codes is provided by [tloen/alpaca-lora: Instruct-tune LLaMA on consumer hardware (github.com)](https://github.com/tloen/alpaca-lora).
92
 
93
+ We modify the running scripts to
94
  ```bash
95
  torchrun --nproc_per_node=8 finetune.py \
96
  --base_model '/cache1/chtan/large_models/llama-hf/llama-65b' \
 
114
 
115
  **Instruction**: Tell me about alpacas.
116
 
117
+ **gpt4-alpaca-lora_mlp-65b**:
118
 
119
  Alpacas are small, domesticated mammals that are closely related to llamas. They are native to the Andes Mountains of South America, primarily in Peru, Bolivia, and Chile. These animals have been domesticated for thousands of years and were used by the Incas for their fleece, meat, and as pack animals.
120