Update README.md
Browse files
README.md
CHANGED
@@ -7,4 +7,24 @@ language:
|
|
7 |
tags:
|
8 |
- openvino
|
9 |
pipeline_tag: text-generation
|
10 |
-
---
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
7 |
tags:
|
8 |
- openvino
|
9 |
pipeline_tag: text-generation
|
10 |
+
---
|
11 |
+
|
12 |
+
# databricks/dolly-v2-3b
|
13 |
+
|
14 |
+
This is the [databricks/dolly-v2-3b](https://huggingface.co/databricks/dolly-v2-3b) model converted to [OpenVINO](https://openvino.ai), for accellerated inference.
|
15 |
+
|
16 |
+
An example of how to do inference on this model:
|
17 |
+
```python
|
18 |
+
from optimum.intel.openvino import OVModelForCausalLM
|
19 |
+
from transformers import AutoTokenizer, pipeline
|
20 |
+
|
21 |
+
# model_id should be set to either a local directory or a model available on the HuggingFace hub.
|
22 |
+
model_id = "katuni4ka/dolly-v2-3b-ov"
|
23 |
+
tokenizer = AutoTokenizer.from_pretrained(model_id)
|
24 |
+
model = OVModelForCausalLM.from_pretrained(model_id)
|
25 |
+
pipe = pipeline("text-generation", model=model, tokenizer=tokenizer)
|
26 |
+
result = pipe("hello world")
|
27 |
+
print(result)
|
28 |
+
```
|
29 |
+
|
30 |
+
More detailed example how to use model in instruction following scenario, can be found in this [notebook](https://github.com/openvinotoolkit/openvino_notebooks/tree/main/notebooks/240-dolly-2-instruction-following)
|