Create README.md

README.md

@@ -0,0 +1,37 @@
+# Instruct-GPT-J
+
+Use:
+
+```python
+import torch
+from peft import PeftModel, PeftConfig
+from transformers import AutoModelForCausalLM, AutoTokenizer
+
+peft_model_id = "crumb/gpt-j-6b-lora-alpaca"
+config = PeftConfig.from_pretrained(peft_model_id)
+model = AutoModelForCausalLM.from_pretrained(config.base_model_name_or_path, return_dict=True, load_in_8bit=True, device_map='auto', revision='sharded')
+tokenizer = AutoTokenizer.from_pretrained(config.base_model_name_or_path)
+
+# Load the Lora model
+model = PeftModel.from_pretrained(model, peft_model_id)
+
+# This example is in the alpaca training set
+batch = tokenizer("Below is an instruction that describes a task. Write a response that appropriately completes the request. ### Instruction: How can we reduce air pollution? ### Response:", return_tensors='pt')
+
+with torch.cuda.amp.autocast():
+  output_tokens = model.generate(**batch, max_new_tokens=50)
+
+print(tokenizer.decode(output_tokens[0], skip_special_tokens=True))
+# One way to reduce air pollution is to reduce the amount of emissions from vehicles. This can be done by implementing stricter emission standards and increasing the use of electric vehicles. Another way to reduce air pollution is to reduce the amount of waste produced by industries.
+```
+
+A function to turn an instruction into a prompt for the model could be written as follows
+
+```python
+def prompt(instruction, input=''):
+  if input=='':
+    return f"Below is an instruction that describes a task. Write a response that appropriately completes the request. ### Instruction: {instruction} ### Response: "
+  return f"Below is an instruction that describes a task. Write a response that appropriately completes the request. ### Instruction: {instruction} ### Input: {input} ### Response: "
+```
+
+Where input would be an input for the model to act on based on the instruction.