SaffalPoosh/system_design_expert

This is llama2 7B finetuned using qlora with bf16 as compute dtype. The dataset has been generated using open-ai api with samples semantics oriented towards abstract explanation of system design.

lora has been merged into the original model, 3 peochs have been trained with batch size of 16.

from google.colab import drive
from transformers import AutoModelForCausalLM, AutoTokenizer
from transformers import pipeline

model_path = "SaffalPoosh/system_design_expert"

model = AutoModelForCausalLM.from_pretrained(model_path)
tokenizer = AutoTokenizer.from_pretrained(model_path)


prompt = "Design an application like Whatsapp with tech stack you will use"
gen = pipeline('text-generation', model=model, tokenizer=tokenizer)
result = gen(prompt)
print(result[0]['generated_text'])