matthayes commited on
Commit
82efc4c
1 Parent(s): 357740e

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +50 -0
README.md CHANGED
@@ -64,6 +64,56 @@ model = AutoModelForCausalLM.from_pretrained("databricks/dolly-v2-3b", device_ma
64
  generate_text = InstructionTextGenerationPipeline(model=model, tokenizer=tokenizer)
65
  ```
66
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
67
 
68
  ## Known Limitations
69
 
 
64
  generate_text = InstructionTextGenerationPipeline(model=model, tokenizer=tokenizer)
65
  ```
66
 
67
+ ### LangChain Usage
68
+
69
+ To use the pipeline with LangChain, you must set `return_full_text=True`, as LangChain expects the full text to be returned
70
+ and the default for the pipeline is to only return the new text.
71
+
72
+ ```
73
+ import torch
74
+ from transformers import pipeline
75
+
76
+ generate_text = pipeline(model="databricks/dolly-v2-3b", torch_dtype=torch.bfloat16,
77
+ trust_remote_code=True, device_map="auto", return_full_text=True)
78
+ ```
79
+
80
+ You can create a prompt that either has only an instruction or has an instruction with context:
81
+
82
+ ```
83
+ from langchain import PromptTemplate, LLMChain
84
+ from langchain.llms import HuggingFacePipeline
85
+
86
+ # template for an instrution with no input
87
+ prompt = PromptTemplate(
88
+ input_variables=["instruction"],
89
+ template="{instruction}")
90
+
91
+ # template for an instruction with input
92
+ prompt_with_context = PromptTemplate(
93
+ input_variables=["instruction", "context"],
94
+ template="{instruction}\n\nInput:\n{context}")
95
+
96
+ hf_pipeline = HuggingFacePipeline(pipeline=generate_text)
97
+
98
+ llm_chain = LLMChain(llm=hf_pipeline, prompt=prompt)
99
+ llm_context_chain = LLMChain(llm=hf_pipeline, prompt=prompt_with_context)
100
+ ```
101
+
102
+ Example predicting using a simple instruction:
103
+
104
+ ```
105
+ print(llm_chain.predict(instruction="Explain to me the difference between nuclear fission and fusion.").lstrip())
106
+ ```
107
+
108
+ Example predicting using an instruction with context:
109
+
110
+ ```
111
+ context = """George Washington (February 22, 1732[b] – December 14, 1799) was an American military officer, statesman,
112
+ and Founding Father who served as the first president of the United States from 1789 to 1797."""
113
+
114
+ print(llm_context_chain.predict(instruction="When was George Washington president?", context=context).lstrip())
115
+ ```
116
+
117
 
118
  ## Known Limitations
119