Create README.md

README.md

@@ -0,0 +1,44 @@
+---
+datasets:
+- Open-Orca/OpenOrca
+---
+
+To use, do:
+
+```
+from peft import PeftModel, PeftConfig
+from transformers import AutoTokenizer
+ref_model = AutoModelForCausalLM.from_pretrained("EleutherAI/pythia-70m-deduped-v0", torch_dtype=torch.bfloat16)
+peft_model_id = "w601sxs/pythia-70m-instruct-orca-chkpt-64000"
+
+config = PeftConfig.from_pretrained(peft_model_id)
+model = PeftModel.from_pretrained(ref_model, peft_model_id)
+tokenizer = AutoTokenizer.from_pretrained(config.base_model_name_or_path)
+
+model = model.to('cuda:0')
+model.eval()
+
+
+inputs = tokenizer(prompt, return_tensors="pt")
+
+with torch.no_grad():
+    outputs = model.generate(input_ids=inputs["input_ids"].to("cuda"), max_new_tokens=10)
+    print(tokenizer.batch_decode(outputs.detach().cpu().numpy(), skip_special_tokens=True)[0]
+
+
+```
+
+
+
+
+### Prompt format
+
+```
+context: < ... >
+question: < ... >
+answer: < ... >
+```
+
+For e.g.
+
+"context: <You are an AI assistant. You will be given a task. You must generate a detailed and long answer.>\n question: <Just when we thought Pharrell Williams couldn’t get any cooler, he becomes the first guy to star in Chanel ’s upcoming handbag campaign. Williams has already appeared in a short video by Karl Lagerfeld and even walked the Chanel runway, and now he’s modelling the Gabrielle bag from the fashion house’s spring collection. Set to join Kristen Stewart, Caroline de Maigret and Cara Delevingne in the campaign - with Lagerfeld taking on the role of photographer - Williams star turn is set to debut on Monday 3 April.\n\nBased on that paragraph can we conclude that this sentence is true?\nLagerfeld will be taking pictures of the event where Pharrell along with three others in the fourth month of the year.\n\nChoose from:\n a). Yes;\n b). It's impossible to say;\n c). No;>\n "