reeducator
commited on
Commit
•
94c60f8
1
Parent(s):
95554e4
Update readme
Browse files
README.md
CHANGED
@@ -7,10 +7,8 @@ language:
|
|
7 |
## General
|
8 |
Bluemoon roleplay finetune of LLaMA 13B (2 roleplayers only).
|
9 |
|
10 |
-
*Note.* This is an intermediate version which has not been trained for sufficiently long to reach a satisfactory final loss value. The repository will be updated later with a model trained over additional epochs.
|
11 |
-
|
12 |
## Models
|
13 |
-
Two models are provided, labeled (1) `4k-epoch6` and (2) `epoch3
|
14 |
|
15 |
*GGML 4-bit for llama.cpp*<br/>
|
16 |
|
|
|
7 |
## General
|
8 |
Bluemoon roleplay finetune of LLaMA 13B (2 roleplayers only).
|
9 |
|
|
|
|
|
10 |
## Models
|
11 |
+
Two models are provided, labeled (1) `4k-epoch6` and (2) `epoch3` (other branch). In case of the (1), the training is extended over more epochs to reduce the high training loss observed in (2). This release also tests a longer 4k context token size achieved with AliBi.
|
12 |
|
13 |
*GGML 4-bit for llama.cpp*<br/>
|
14 |
|