nxnhjrjtbjfzhrovwl commited on
Commit
f3b69d1
1 Parent(s): 79e1f81

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +3 -2
README.md CHANGED
@@ -9,11 +9,12 @@ This repository contains the unquantized merge of [limarp-llama2 lora](https://h
9
  You can quantize the f16 ggml to the quantization of your choice by following the below steps:
10
 
11
  1. Download and extract the [llama.cpp binaries](https://github.com/ggerganov/llama.cpp/releases/download/master-41c6741/llama-master-41c6741-bin-win-avx2-x64.zip) ([or compile it yourself if you're on Linux](https://github.com/ggerganov/llama.cpp#build))
12
- 2. Open a command prompt window in the folder where you extracted the binaries and write the following command, making the changes that you see fit.
 
13
  ```bash
14
  quantize.exe limarp-llama2-13b.ggmlv3.f16.bin limarp-llama2-13b.ggmlv3.q4_0.bin q4_0
15
  ```
16
- 3. Press enter to run the command and the quantized model will be generated in the current folder.
17
 
18
 
19
  The below are the contents of the original model card:
 
9
  You can quantize the f16 ggml to the quantization of your choice by following the below steps:
10
 
11
  1. Download and extract the [llama.cpp binaries](https://github.com/ggerganov/llama.cpp/releases/download/master-41c6741/llama-master-41c6741-bin-win-avx2-x64.zip) ([or compile it yourself if you're on Linux](https://github.com/ggerganov/llama.cpp#build))
12
+ 2. Move the "quantize" executable to the same folder where you downloaded the f16 ggml model.
13
+ 3. Open a command prompt window in that same folder and write the following command, making the changes that you see fit.
14
  ```bash
15
  quantize.exe limarp-llama2-13b.ggmlv3.f16.bin limarp-llama2-13b.ggmlv3.q4_0.bin q4_0
16
  ```
17
+ 4. Press enter to run the command and the quantized model will be generated in the folder.
18
 
19
 
20
  The below are the contents of the original model card: