| language: | |
| - en | |
| tags: | |
| - openvino | |
| # Salesforce/codegen2-1B | |
| This is the [Salesforce/codegen2-1B](https://huggingface.co/Salesforce/codegen2-1B) model converted to [OpenVINO](https://openvino.ai), for accelerated inference. | |
| An example of how to do inference on this model: | |
| ```python | |
| from transformers import AutoTokenizer | |
| from optimum.intel.openvino import OVModelForCausalLM | |
| tokenizer = AutoTokenizer.from_pretrained("helenai/Salesforce-codegen2-1B-ov") | |
| model = OVModelForCausalLM.from_pretrained("helenai/Salesforce-codegen2-1B-ov") | |
| # Try the version with quantized model weights by changing the line above to: | |
| # model = OVModelForCausalLM.from_pretrained("helenai/Salesforce-codegen2-1B-ov", revision="compressed_weights") | |
| text = "def hello_world():" | |
| input_ids = tokenizer(text, return_tensors="pt").input_ids | |
| generated_ids = model.generate(input_ids, max_length=128) | |
| print(tokenizer.decode(generated_ids[0], skip_special_tokens=True)) | |
| ``` | |