yujiepan commited on
Commit
f258fb5
1 Parent(s): c9f47b0

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +11 -3
README.md CHANGED
@@ -7,14 +7,22 @@ tags: []
7
 
8
  4-bit weight only quantization by AutoGPTQ on [yujiepan/llama-3-tiny-random](https://huggingface.co/yujiepan/llama-3-tiny-random)
9
 
10
- ```
11
  from transformers import AutoModelForCausalLM, AutoTokenizer, GPTQConfig
12
  import torch
13
 
14
  model_id = "yujiepan/llama-3-tiny-random"
15
  tokenizer = AutoTokenizer.from_pretrained(model_id)
16
 
17
- quantization_config = GPTQConfig(bits=4, group_size=-1, dataset="c4", tokenizer=tokenizer)
18
- model = AutoModelForCausalLM.from_pretrained(model_id, device_map="auto", quantization_config=quantization_config)
 
 
 
 
 
 
 
 
19
  ```
20
 
 
7
 
8
  4-bit weight only quantization by AutoGPTQ on [yujiepan/llama-3-tiny-random](https://huggingface.co/yujiepan/llama-3-tiny-random)
9
 
10
+ ```python
11
  from transformers import AutoModelForCausalLM, AutoTokenizer, GPTQConfig
12
  import torch
13
 
14
  model_id = "yujiepan/llama-3-tiny-random"
15
  tokenizer = AutoTokenizer.from_pretrained(model_id)
16
 
17
+ quantization_config = GPTQConfig(
18
+ bits=4, group_size=-1,
19
+ dataset="c4",
20
+ tokenizer=tokenizer,
21
+ )
22
+ model = AutoModelForCausalLM.from_pretrained(
23
+ model_id,
24
+ device_map="auto",
25
+ quantization_config=quantization_config,
26
+ )
27
  ```
28