Update README.md

README.md

@@ -9,6 +9,8 @@ First, I trained it on an epoch of https://huggingface.co/datasets/Adapting/empa
 
 Then I trained it on an epoch of https://huggingface.co/datasets/vicgalle/alpaca-gpt4 and landed here, a model that is capable of chatting but very focused on following instructions.
 
+If you would like to run this in 4-bit, you can use the Hugging Face backend in Koboldai (or in a different script, the `load_in_4bit` kwarg when calling `from_pretrained`). GPTQ conversion has so far resulted in broken output for me, YMMV.
+
 **Future Ideas**
 - **This strongly prefers the alpaca prompt format and will try to autocomplete it if you don't provide it.** I'd like to work on removing this fixation and making it more flexible. 
 - Also would like to filter the rows with phrases "AI assistant" and "virtual assistant" from all future runs.