Update README.md

README.md

@@ -37,7 +37,30 @@ Fine-tuning the base GALACTICA models on the 52k instruction-response pairs in t
 ## How to Use
 
 The GALPACA weights are made available for use with the `transformers` library.
-TODO: add example inference usage.
+
+<details>
+  <summary> Click to expand </summary>
+
+```python
+# pip install accelerate
+from transformers import AutoTokenizer, OPTForCausalLM
+
+tokenizer = AutoTokenizer.from_pretrained("GeorgiaTechResearchInstitute/galpaca-30b")
+model = OPTForCausalLM.from_pretrained("GeorgiaTechResearchInstitute/galpaca-30b", device_map="auto", torch_dtype=torch.float16)
+
+# see the original Alpaca repo for more information about the prompt templates
+no_input_prompt_template = ("Below is an instruction that describes a task. "
+                            "Write a response that appropriately completes the request.\n\n"
+                            "### Instruction:\n{instruction}\n\n### Response:")
+prompt = "Write out Maxwell's equations and explain the meaning of each one."
+formatted_prompt = no_input_prompt_template.format_map({'instruction': prompt})
+
+tokenized_prompt = tokenizer(formatted_prompt, return_tensors="pt").input_ids.to(model.device)
+out_tokens = model.generate(tokenized_prompt)
+
+print(tokenizer.batch_decode(out_tokens, skip_special_tokens=False, clean_up_tokenization_spaces=False))
+```
+</details>
 
 ## Training Resources