Update README.md
Browse files
README.md
CHANGED
@@ -9,7 +9,7 @@ Oh boy, here we go. Low-rank (`r=16, alpha=32`) LoRA on top of [WizardLM-2-8x22B
|
|
9 |
|
10 |
## Why A LoRA?
|
11 |
|
12 |
-
The choice was fully intentional. I briefly considered a FFT but for this particular use-case a LoRA seemed a better fit
|
13 |
|
14 |
## Prompting
|
15 |
|
|
|
9 |
|
10 |
## Why A LoRA?
|
11 |
|
12 |
+
The choice was fully intentional. I briefly considered a FFT but for this particular use-case a LoRA seemed a better fit. `WizardLM-2-8x22B` is smart by itself but its used vocabulary leaves much to be desired when it comes to RP. By training a low-rank LoRA on top of it to teach it some of Claude's writing style, we remedy that.
|
13 |
|
14 |
## Prompting
|
15 |
|