This is llama2 7B finetuned using qlora with bf16 as compute dtype. The dataset has been generated using open-ai api with samples semantics oriented towards abstract explanation of system design.
lora has been merged into the original model, 3 peochs have been trained with batch size of 16.
from google.colab import drive
from transformers import AutoModelForCausalLM, AutoTokenizer
from transformers import pipeline
model_path = "SaffalPoosh/system_design_expert"
model = AutoModelForCausalLM.from_pretrained(model_path)
tokenizer = AutoTokenizer.from_pretrained(model_path)
prompt = "Design an application like Whatsapp with tech stack you will use"
gen = pipeline('text-generation', model=model, tokenizer=tokenizer)
result = gen(prompt)
print(result[0]['generated_text'])
- Downloads last month
- 10
This model does not have enough activity to be deployed to Inference API (serverless) yet.
Increase its social visibility and check back later, or deploy to Inference Endpoints (dedicated)
instead.