MichaelPape commited on
Commit
ae28e32
1 Parent(s): 661d056

Added VRAM parameters

Browse files
Files changed (1) hide show
  1. README.md +8 -1
README.md CHANGED
@@ -44,7 +44,14 @@ python generate/adapter_v2.py \
44
  --prompt "What temperature should I cook pork at to ensure it is safe?"
45
  ```
46
 
47
- This uses around 14GB of VRAM. If you need to use less you can read [this](https://lightning.ai/pages/blog/falcon-a-guide-to-finetune-and-inference/)
 
 
 
 
 
 
 
48
 
49
  ### Training Data
50
 
 
44
  --prompt "What temperature should I cook pork at to ensure it is safe?"
45
  ```
46
 
47
+ This uses around 14GB of VRAM. If you need to use less VRAM, you can add the parameters
48
+ ```
49
+ --quantize llm.int8
50
+ ```
51
+ or
52
+ ```
53
+ --quantize gptq.int4
54
+ ```
55
 
56
  ### Training Data
57