Update README.md
Browse files
README.md
CHANGED
|
@@ -87,7 +87,13 @@ tags:
|
|
| 87 |
line-height: 1.6;">
|
| 88 |
|
| 89 |
See the sample inference code in this repo. You should install exllamv2, this model, and then run the inference script - making sure you have
|
| 90 |
-
pointed to your model's directory
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 91 |
</div>
|
| 92 |
|
| 93 |
# Introduction
|
|
|
|
| 87 |
line-height: 1.6;">
|
| 88 |
|
| 89 |
See the sample inference code in this repo. You should install exllamv2, this model, and then run the inference script - making sure you have
|
| 90 |
+
pointed to your model's directory.
|
| 91 |
+
|
| 92 |
+
MyQuantisation process: clone the latest exllamav2 repository to a new directory, modify the architecture.py (modified version in this repo_) to add the model architecture OpenCUAForConditionalGeneration
|
| 93 |
+
. My quatisation command was
|
| 94 |
+
|
| 95 |
+
python3.9 convert.py -i /home/sujit/OpenCUA-7B -o /home/sujit/OpenCUA-7B-exl2 -c /home/sujit/wikitext_cal_data.parquet -b 4.5 -cf /home/sujit/OpenCUA-7B-exl2-final
|
| 96 |
+
|
| 97 |
</div>
|
| 98 |
|
| 99 |
# Introduction
|