asyafiqe commited on
Commit
557a4fc
1 Parent(s): bb9a583

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +8 -6
README.md CHANGED
@@ -44,16 +44,18 @@ It is strongly recommended to use the text-generation-webui one-click-installers
44
  4. The model will start downloading. Once it's finished it will say "Done"
45
  5. In the top left, click the refresh icon next to **Model**.
46
  6. In the **Model** dropdown, choose the model you just downloaded: `Merak-7B-v3-Mini-Orca-Indo-GPTQ`
47
- 7. The model will automatically load, and is now ready for use!
48
- 8. If you want any custom settings, set them and then click **Save settings for this model** followed by **Reload the Model** in the top right.
49
- * Note that you do not need to set GPTQ parameters any more. These are set automatically from the file `quantize_config.json`.
50
- 9. Once you're ready, click the **Text Generation tab** and enter a prompt to get started!
 
51
 
52
  ## How to use this GPTQ model from Python code
53
 
54
  First make sure you have [AutoGPTQ](https://github.com/PanQiWei/AutoGPTQ) installed:
55
 
56
  `GITHUB_ACTIONS=true pip install auto-gptq`
 
57
 
58
  Then try the following example code:
59
 
@@ -62,7 +64,7 @@ from transformers import AutoTokenizer, pipeline, logging
62
  from auto_gptq import AutoGPTQForCausalLM, BaseQuantizeConfig
63
 
64
  model_name_or_path = "asyafiqe/Merak-7B-v3-Mini-Orca-Indo-GPTQ"
65
- model_basename = "model"
66
 
67
  use_triton = False
68
 
@@ -77,7 +79,7 @@ model = AutoGPTQForCausalLM.from_quantized(model_name_or_path,
77
  quantize_config=None)
78
 
79
 
80
- prompt = "Tell me about AI"
81
  system_message = "Anda adalah asisten AI. Anda akan diberi tugas. Anda harus menghasilkan jawaban yang rinci dan panjang.\n"
82
  prompt_template=f'''SYSTEM: {system_message}
83
  USER: {prompt}
 
44
  4. The model will start downloading. Once it's finished it will say "Done"
45
  5. In the top left, click the refresh icon next to **Model**.
46
  6. In the **Model** dropdown, choose the model you just downloaded: `Merak-7B-v3-Mini-Orca-Indo-GPTQ`
47
+ 7. In the **Model Loader** dropdown, choose ExLlamav2_HF as the model loader.
48
+ 8. Click load.
49
+ 9. Click the **Default** tab
50
+ 10. Copy prompt format mentioned above to the input box.
51
+ 11. Enter a prompt and click generate! Click continue to get longer response.
52
 
53
  ## How to use this GPTQ model from Python code
54
 
55
  First make sure you have [AutoGPTQ](https://github.com/PanQiWei/AutoGPTQ) installed:
56
 
57
  `GITHUB_ACTIONS=true pip install auto-gptq`
58
+ pip install sentencepiece
59
 
60
  Then try the following example code:
61
 
 
64
  from auto_gptq import AutoGPTQForCausalLM, BaseQuantizeConfig
65
 
66
  model_name_or_path = "asyafiqe/Merak-7B-v3-Mini-Orca-Indo-GPTQ"
67
+ model_basename = "Merak-7B-v3-Mini-Orca-Indo-GPTQ"
68
 
69
  use_triton = False
70
 
 
79
  quantize_config=None)
80
 
81
 
82
+ prompt = "Buat rencana untuk menghemat listrik di rumah"
83
  system_message = "Anda adalah asisten AI. Anda akan diberi tugas. Anda harus menghasilkan jawaban yang rinci dan panjang.\n"
84
  prompt_template=f'''SYSTEM: {system_message}
85
  USER: {prompt}