PEFT
Safetensors
Transformers
English
text-generation-inference
unsloth
llama
trl
Not-For-All-Audiences
mpasila commited on
Commit
6214aeb
1 Parent(s): cdc0e8c

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +12 -1
README.md CHANGED
@@ -8,8 +8,19 @@ tags:
8
  - unsloth
9
  - llama
10
  - trl
 
11
  base_model: unsloth/llama-3-8b-bnb-4bit
 
 
 
12
  ---
 
 
 
 
 
 
 
13
 
14
  # Uploaded model
15
 
@@ -19,4 +30,4 @@ base_model: unsloth/llama-3-8b-bnb-4bit
19
 
20
  This llama model was trained 2x faster with [Unsloth](https://github.com/unslothai/unsloth) and Huggingface's TRL library.
21
 
22
- [<img src="https://raw.githubusercontent.com/unslothai/unsloth/main/images/unsloth%20made%20with%20love.png" width="200"/>](https://github.com/unslothai/unsloth)
 
8
  - unsloth
9
  - llama
10
  - trl
11
+ - not-for-all-audiences
12
  base_model: unsloth/llama-3-8b-bnb-4bit
13
+ datasets:
14
+ - grimulkan/LimaRP-augmented
15
+ - mpasila/LimaRP-augmented-8k-context
16
  ---
17
+ This was made using the Llama 3 Instruct prompt formatting so that it should be easier to be merged with other models using that format.
18
+
19
+ LoRA trained in 4-bit with 8k context using [meta-llama/Meta-Llama-3-8B](https://huggingface.co/meta-llama/Meta-Llama-3-8B/) as the base model for 1 epoch.
20
+
21
+ Dataset used is [a modified](https://huggingface.co/datasets/mpasila/LimaRP-augmented-8k-context) version of [grimulkan/LimaRP-augmented](https://huggingface.co/datasets/grimulkan/LimaRP-augmented).
22
+
23
+ ### Prompt format: Llama 3 Instruct
24
 
25
  # Uploaded model
26
 
 
30
 
31
  This llama model was trained 2x faster with [Unsloth](https://github.com/unslothai/unsloth) and Huggingface's TRL library.
32
 
33
+ [<img src="https://raw.githubusercontent.com/unslothai/unsloth/main/images/unsloth%20made%20with%20love.png" width="200"/>](https://github.com/unslothai/unsloth)