nxnhjrjtbjfzhrovwl
commited on
Commit
•
f3b69d1
1
Parent(s):
79e1f81
Update README.md
Browse files
README.md
CHANGED
@@ -9,11 +9,12 @@ This repository contains the unquantized merge of [limarp-llama2 lora](https://h
|
|
9 |
You can quantize the f16 ggml to the quantization of your choice by following the below steps:
|
10 |
|
11 |
1. Download and extract the [llama.cpp binaries](https://github.com/ggerganov/llama.cpp/releases/download/master-41c6741/llama-master-41c6741-bin-win-avx2-x64.zip) ([or compile it yourself if you're on Linux](https://github.com/ggerganov/llama.cpp#build))
|
12 |
-
2.
|
|
|
13 |
```bash
|
14 |
quantize.exe limarp-llama2-13b.ggmlv3.f16.bin limarp-llama2-13b.ggmlv3.q4_0.bin q4_0
|
15 |
```
|
16 |
-
|
17 |
|
18 |
|
19 |
The below are the contents of the original model card:
|
|
|
9 |
You can quantize the f16 ggml to the quantization of your choice by following the below steps:
|
10 |
|
11 |
1. Download and extract the [llama.cpp binaries](https://github.com/ggerganov/llama.cpp/releases/download/master-41c6741/llama-master-41c6741-bin-win-avx2-x64.zip) ([or compile it yourself if you're on Linux](https://github.com/ggerganov/llama.cpp#build))
|
12 |
+
2. Move the "quantize" executable to the same folder where you downloaded the f16 ggml model.
|
13 |
+
3. Open a command prompt window in that same folder and write the following command, making the changes that you see fit.
|
14 |
```bash
|
15 |
quantize.exe limarp-llama2-13b.ggmlv3.f16.bin limarp-llama2-13b.ggmlv3.q4_0.bin q4_0
|
16 |
```
|
17 |
+
4. Press enter to run the command and the quantized model will be generated in the folder.
|
18 |
|
19 |
|
20 |
The below are the contents of the original model card:
|