katuni4ka commited on
Commit
c7f3f7a
1 Parent(s): 0f308d1

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +21 -1
README.md CHANGED
@@ -7,4 +7,24 @@ language:
7
  tags:
8
  - openvino
9
  pipeline_tag: text-generation
10
- ---
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
7
  tags:
8
  - openvino
9
  pipeline_tag: text-generation
10
+ ---
11
+
12
+ # databricks/dolly-v2-3b
13
+
14
+ This is the [databricks/dolly-v2-3b](https://huggingface.co/databricks/dolly-v2-3b) model converted to [OpenVINO](https://openvino.ai), for accellerated inference.
15
+
16
+ An example of how to do inference on this model:
17
+ ```python
18
+ from optimum.intel.openvino import OVModelForCausalLM
19
+ from transformers import AutoTokenizer, pipeline
20
+
21
+ # model_id should be set to either a local directory or a model available on the HuggingFace hub.
22
+ model_id = "katuni4ka/dolly-v2-3b-ov"
23
+ tokenizer = AutoTokenizer.from_pretrained(model_id)
24
+ model = OVModelForCausalLM.from_pretrained(model_id)
25
+ pipe = pipeline("text-generation", model=model, tokenizer=tokenizer)
26
+ result = pipe("hello world")
27
+ print(result)
28
+ ```
29
+
30
+ More detailed example how to use model in instruction following scenario, can be found in this [notebook](https://github.com/openvinotoolkit/openvino_notebooks/tree/main/notebooks/240-dolly-2-instruction-following)