Update README.md
Browse files
README.md
CHANGED
@@ -23,8 +23,13 @@ created by mmnga.
|
|
23 |
You can use gguf model with llama.cpp at cpu only machine.
|
24 |
But maybe gguf model little bit slower then GPTQ especialy long text.
|
25 |
|
|
|
26 |
|
27 |
-
|
|
|
|
|
|
|
|
|
28 |
|
29 |
Currently, models may behave differently on local PC and Colab. On Colab, the model may not respond if you include instructional prompts.
|
30 |
[Colab Sample script](https://github.com/webbigdata-jp/python_sample/blob/main/weblab_10b_instruction_sft_GPTQ_sample.ipynb)
|
@@ -81,6 +86,6 @@ Also, the score may change as a result of more tuning.
|
|
81 |
| *weblab-10b-instruction-sft-GPTQ first tuning* | 69.72 | 74.53 | 41.70 | 89.95 | 72.69 | deleted |
|
82 |
| *weblab-10b-instruction-sft-GPTQ second tuning* | 74.59 | 74.08 | 60.72 | 91.85 | 71.70 | deleted |
|
83 |
| *weblab-10b-instruction-sft-GPTQ third tuning* | 77.62 | 73.19 | 69.26 | 95.91 | 72.10 | current model. replaced on August 26th |
|
84 |
-
| *weblab-10b-instruction-sft-GPTQ 4th tuning* | - |
|
85 |
|
86 |
|
|
|
23 |
You can use gguf model with llama.cpp at cpu only machine.
|
24 |
But maybe gguf model little bit slower then GPTQ especialy long text.
|
25 |
|
26 |
+
### How to run.
|
27 |
|
28 |
+
You can use [text-generation-webui](https://github.com/oobabooga/text-generation-webui) to run this model fast(about 16 tokens/s on my RTX 3060) on your local PC.
|
29 |
+
|
30 |
+
The explanation of [how to install Japanese text-generation-webui is here.](https://webbigdata.jp/post-19926/).
|
31 |
+
|
32 |
+
### simple sample code
|
33 |
|
34 |
Currently, models may behave differently on local PC and Colab. On Colab, the model may not respond if you include instructional prompts.
|
35 |
[Colab Sample script](https://github.com/webbigdata-jp/python_sample/blob/main/weblab_10b_instruction_sft_GPTQ_sample.ipynb)
|
|
|
86 |
| *weblab-10b-instruction-sft-GPTQ first tuning* | 69.72 | 74.53 | 41.70 | 89.95 | 72.69 | deleted |
|
87 |
| *weblab-10b-instruction-sft-GPTQ second tuning* | 74.59 | 74.08 | 60.72 | 91.85 | 71.70 | deleted |
|
88 |
| *weblab-10b-instruction-sft-GPTQ third tuning* | 77.62 | 73.19 | 69.26 | 95.91 | 72.10 | current model. replaced on August 26th |
|
89 |
+
| *weblab-10b-instruction-sft-GPTQ 4th tuning* | - | 14.5 | - | 85.46 | | abandoned |
|
90 |
|
91 |
|