Update README.md
Browse files
README.md
CHANGED
@@ -21,8 +21,9 @@ license: other
|
|
21 |
|
22 |
These files are fp16 model files for [Allen AI's Tulu 30B](https://huggingface.co/allenai/tulu-30b) merged with [Kaio Ken's SuperHOT 30B 8K LoRA](https://huggingface.co/kaiokendev/superhot-30b-8k-no-rlhf-test) to produce a model capable of 8K context.
|
23 |
|
24 |
-
[Kaio Ken's SuperHOT 30B LoRA](https://huggingface.co/kaiokendev/superhot-30b-8k-no-rlhf-test) is merged on to the base model to produce a model capable of 8K context, via
|
25 |
|
|
|
26 |
|
27 |
## Repositories available
|
28 |
|
|
|
21 |
|
22 |
These files are fp16 model files for [Allen AI's Tulu 30B](https://huggingface.co/allenai/tulu-30b) merged with [Kaio Ken's SuperHOT 30B 8K LoRA](https://huggingface.co/kaiokendev/superhot-30b-8k-no-rlhf-test) to produce a model capable of 8K context.
|
23 |
|
24 |
+
[Kaio Ken's SuperHOT 30B LoRA](https://huggingface.co/kaiokendev/superhot-30b-8k-no-rlhf-test) is merged on to the base model to produce a model capable of 8K context, via the provided monkey patch (`llama_rope_scaled_monkey_patch.py`)
|
25 |
|
26 |
+
Alternatively, `config.json` can be modified to allow the monkey patch to load via trust_remote_code=True. I plan to update this repo shortly to include that method.
|
27 |
|
28 |
## Repositories available
|
29 |
|