Update README.md
Browse files
README.md
CHANGED
@@ -44,16 +44,18 @@ It is strongly recommended to use the text-generation-webui one-click-installers
|
|
44 |
4. The model will start downloading. Once it's finished it will say "Done"
|
45 |
5. In the top left, click the refresh icon next to **Model**.
|
46 |
6. In the **Model** dropdown, choose the model you just downloaded: `Merak-7B-v3-Mini-Orca-Indo-GPTQ`
|
47 |
-
7.
|
48 |
-
8.
|
49 |
-
|
50 |
-
|
|
|
51 |
|
52 |
## How to use this GPTQ model from Python code
|
53 |
|
54 |
First make sure you have [AutoGPTQ](https://github.com/PanQiWei/AutoGPTQ) installed:
|
55 |
|
56 |
`GITHUB_ACTIONS=true pip install auto-gptq`
|
|
|
57 |
|
58 |
Then try the following example code:
|
59 |
|
@@ -62,7 +64,7 @@ from transformers import AutoTokenizer, pipeline, logging
|
|
62 |
from auto_gptq import AutoGPTQForCausalLM, BaseQuantizeConfig
|
63 |
|
64 |
model_name_or_path = "asyafiqe/Merak-7B-v3-Mini-Orca-Indo-GPTQ"
|
65 |
-
model_basename = "
|
66 |
|
67 |
use_triton = False
|
68 |
|
@@ -77,7 +79,7 @@ model = AutoGPTQForCausalLM.from_quantized(model_name_or_path,
|
|
77 |
quantize_config=None)
|
78 |
|
79 |
|
80 |
-
prompt = "
|
81 |
system_message = "Anda adalah asisten AI. Anda akan diberi tugas. Anda harus menghasilkan jawaban yang rinci dan panjang.\n"
|
82 |
prompt_template=f'''SYSTEM: {system_message}
|
83 |
USER: {prompt}
|
|
|
44 |
4. The model will start downloading. Once it's finished it will say "Done"
|
45 |
5. In the top left, click the refresh icon next to **Model**.
|
46 |
6. In the **Model** dropdown, choose the model you just downloaded: `Merak-7B-v3-Mini-Orca-Indo-GPTQ`
|
47 |
+
7. In the **Model Loader** dropdown, choose ExLlamav2_HF as the model loader.
|
48 |
+
8. Click load.
|
49 |
+
9. Click the **Default** tab
|
50 |
+
10. Copy prompt format mentioned above to the input box.
|
51 |
+
11. Enter a prompt and click generate! Click continue to get longer response.
|
52 |
|
53 |
## How to use this GPTQ model from Python code
|
54 |
|
55 |
First make sure you have [AutoGPTQ](https://github.com/PanQiWei/AutoGPTQ) installed:
|
56 |
|
57 |
`GITHUB_ACTIONS=true pip install auto-gptq`
|
58 |
+
pip install sentencepiece
|
59 |
|
60 |
Then try the following example code:
|
61 |
|
|
|
64 |
from auto_gptq import AutoGPTQForCausalLM, BaseQuantizeConfig
|
65 |
|
66 |
model_name_or_path = "asyafiqe/Merak-7B-v3-Mini-Orca-Indo-GPTQ"
|
67 |
+
model_basename = "Merak-7B-v3-Mini-Orca-Indo-GPTQ"
|
68 |
|
69 |
use_triton = False
|
70 |
|
|
|
79 |
quantize_config=None)
|
80 |
|
81 |
|
82 |
+
prompt = "Buat rencana untuk menghemat listrik di rumah"
|
83 |
system_message = "Anda adalah asisten AI. Anda akan diberi tugas. Anda harus menghasilkan jawaban yang rinci dan panjang.\n"
|
84 |
prompt_template=f'''SYSTEM: {system_message}
|
85 |
USER: {prompt}
|