helenai commited on
Commit
4373d61
1 Parent(s): b98691f

add compressed weights to README

Browse files
Files changed (1) hide show
  1. README.md +5 -1
README.md CHANGED
@@ -7,7 +7,7 @@ tags:
7
 
8
  # Salesforce/codegen2-1B
9
 
10
- This is the [Salesforce/codegen2-1B](https://huggingface.co/Salesforce/codegen2-1B) model converted to [OpenVINO](https://openvino.ai), for accellerated inference.
11
 
12
  An example of how to do inference on this model:
13
  ```python
@@ -15,8 +15,12 @@ from transformers import AutoTokenizer
15
  from optimum.intel.openvino import OVModelForCausalLM
16
 
17
  tokenizer = AutoTokenizer.from_pretrained("helenai/Salesforce-codegen2-1B-ov")
 
18
  model = OVModelForCausalLM.from_pretrained("helenai/Salesforce-codegen2-1B-ov")
19
 
 
 
 
20
  text = "def hello_world():"
21
  input_ids = tokenizer(text, return_tensors="pt").input_ids
22
  generated_ids = model.generate(input_ids, max_length=128)
 
7
 
8
  # Salesforce/codegen2-1B
9
 
10
+ This is the [Salesforce/codegen2-1B](https://huggingface.co/Salesforce/codegen2-1B) model converted to [OpenVINO](https://openvino.ai), for accelerated inference.
11
 
12
  An example of how to do inference on this model:
13
  ```python
 
15
  from optimum.intel.openvino import OVModelForCausalLM
16
 
17
  tokenizer = AutoTokenizer.from_pretrained("helenai/Salesforce-codegen2-1B-ov")
18
+
19
  model = OVModelForCausalLM.from_pretrained("helenai/Salesforce-codegen2-1B-ov")
20
 
21
+ # Try the version with quantized model weights by changing the line above to:
22
+ # model = OVModelForCausalLM.from_pretrained("helenai/Salesforce-codegen2-1B-ov", revision="compressed_weights")
23
+
24
  text = "def hello_world():"
25
  input_ids = tokenizer(text, return_tensors="pt").input_ids
26
  generated_ids = model.generate(input_ids, max_length=128)