Update README.md

README.md

@@ -29,6 +29,39 @@ Parameter-Efficient Fine-tuning (PEFT) is a technique used to improve the perfor
 ## Training Data
 the model is trained on 'b-mc2/sql-create-context' dataset upto 5000rows
 
+## Usage:
+please install `transformers`, and `peft`:
+```
+!pip install transformers peft
+```
+To use the model, you can run the following:
+
+```py
+import torch
+from peft import PeftModel, PeftConfig
+from transformers import AutoModelForCausalLM,  AutoTokenizer
+
+config = PeftConfig.from_pretrained("Naveengo/gpt2-medium-on-sql-create-context")
+model = AutoModelForCausalLM.from_pretrained(config.base_model_name_or_path, return_dict=True, load_in_8bit=False)
+tokenizer = AutoTokenizer.from_pretrained(config.base_model_name_or_path)
+
+
+# Load the Lora model
+model = PeftModel.from_pretrained(model,"Naveengo/gpt2-medium-on-sql-create-context")
+
+from IPython.display import display, Markdown
+
+def make_inference(question, context):
+  batch = tokenizer(f"Below is an SQL instruction that describes a task, paired with an input that provides further context. Write an SQL query that appropriately completes the request using your expertise in SQL.  ### Instruction: {question}### Input: {context}### Response:", return_tensors='pt')
+  
+  with torch.cuda.amp.autocast():
+    output_tokens = model.generate(**batch, max_new_tokens=100)
+
+  display(Markdown((tokenizer.decode(output_tokens[0], skip_special_tokens=True))))  
+#give question and context to function
+make_inference(your_question_here, your_context_here)
+```
+
 ## Training procedure