PEFT
Safetensors
Transformers
English
text-generation-inference
unsloth
llama
trl
Not-For-All-Audiences
mpasila commited on
Commit
89b79d6
1 Parent(s): f9c7b10

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +16 -3
README.md CHANGED
@@ -1,7 +1,7 @@
1
  ---
2
  language:
3
  - en
4
- license: apache-2.0
5
  tags:
6
  - text-generation-inference
7
  - transformers
@@ -9,14 +9,27 @@ tags:
9
  - llama
10
  - trl
11
  base_model: unsloth/llama-3-8b-Instruct-bnb-4bit
 
 
 
 
12
  ---
 
 
 
 
 
 
 
 
 
13
 
14
  # Uploaded model
15
 
16
  - **Developed by:** mpasila
17
- - **License:** apache-2.0
18
  - **Finetuned from model :** unsloth/llama-3-8b-Instruct-bnb-4bit
19
 
20
  This llama model was trained 2x faster with [Unsloth](https://github.com/unslothai/unsloth) and Huggingface's TRL library.
21
 
22
- [<img src="https://raw.githubusercontent.com/unslothai/unsloth/main/images/unsloth%20made%20with%20love.png" width="200"/>](https://github.com/unslothai/unsloth)
 
1
  ---
2
  language:
3
  - en
4
+ license: llama3
5
  tags:
6
  - text-generation-inference
7
  - transformers
 
9
  - llama
10
  - trl
11
  base_model: unsloth/llama-3-8b-Instruct-bnb-4bit
12
+ datasets:
13
+ - mpasila/LimaRP-PIPPA-Mix-8K-Context
14
+ - grimulkan/LimaRP-augmented
15
+ - KaraKaraWitch/PIPPA-ShareGPT-formatted
16
  ---
17
+ LoRA trained in 4-bit with 8k context using [meta-llama/Meta-Llama-3-8B-Instruct](https://huggingface.co/meta-llama/Meta-Llama-3-8B-Instruct/) as the base model for 1 epoch.
18
+
19
+ Dataset used is [mpasila/LimaRP-PIPPA-Mix-8K-Context](https://huggingface.co/datasets/mpasila/LimaRP-PIPPA-Mix-8K-Context) which was made using [grimulkan/LimaRP-augmented](https://huggingface.co/datasets/grimulkan/LimaRP-augmented) and [KaraKaraWitch/PIPPA-ShareGPT-formatted](https://huggingface.co/datasets/KaraKaraWitch/PIPPA-ShareGPT-formatted).
20
+
21
+ This has been trained on the instruct model and not the base model. The model trained with the base model using the same dataset is here: [mpasila/Llama-3-LiPPA-LoRA-8B](https://huggingface.co/mpasila/Llama-3-LiPPA-LoRA-8B)
22
+
23
+ ### Prompt format: Llama 3 Instruct
24
+
25
+ Unsloth changed assistant to gpt and user to human.
26
 
27
  # Uploaded model
28
 
29
  - **Developed by:** mpasila
30
+ - **License:** Llama 3 Community License
31
  - **Finetuned from model :** unsloth/llama-3-8b-Instruct-bnb-4bit
32
 
33
  This llama model was trained 2x faster with [Unsloth](https://github.com/unslothai/unsloth) and Huggingface's TRL library.
34
 
35
+ [<img src="https://raw.githubusercontent.com/unslothai/unsloth/main/images/unsloth%20made%20with%20love.png" width="200"/>](https://github.com/unslothai/unsloth)