matthayes commited on
Commit
6b63048
1 Parent(s): b7fbc6d

Update README.md

Browse files

Add pipeline usage instructions

Files changed (1) hide show
  1. README.md +26 -0
README.md CHANGED
@@ -24,6 +24,32 @@ high quality instruction following behavior not characteristic of the foundation
24
  [EleutherAI’s](https://www.eleuther.ai/) [Pythia-12b](https://huggingface.co/EleutherAI/pythia-12b) and fine-tuned
25
  on a ~15K record instruction corpus generated by Databricks employees and released under a permissive license (CC-BY-SA)
26
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
27
  ## Known Limitations
28
 
29
  ### Performance Limitations
 
24
  [EleutherAI’s](https://www.eleuther.ai/) [Pythia-12b](https://huggingface.co/EleutherAI/pythia-12b) and fine-tuned
25
  on a ~15K record instruction corpus generated by Databricks employees and released under a permissive license (CC-BY-SA)
26
 
27
+ ## Usage
28
+
29
+ To use the model with the `transformers` library on a machine with GPUs:
30
+
31
+ ```
32
+ from transformers import pipeline
33
+
34
+ instruct_pipeline = pipeline(model="databricks/dolly-v2-12b", trust_remote_code=True, device_map="auto")
35
+ ```
36
+
37
+ You can then use the pipeline to answer instructions:
38
+
39
+ ```
40
+ instruct_pipeline("Explain to me the difference between nuclear fission and fusion.")
41
+ ```
42
+
43
+ To reduce memory usage you can load the model with `bfloat16`:
44
+
45
+ ```
46
+ import torch
47
+ from transformers import pipeline
48
+
49
+ instruct_pipeline = pipeline(model="databricks/dolly-v2-12b", torch_dtype=torch.bfloat16, trust_remote_code=True, device_map="auto")
50
+ ```
51
+
52
+
53
  ## Known Limitations
54
 
55
  ### Performance Limitations