DisgustingOzil/phi-2-riddler

Model Card for Model ID

Model Details

This is the model card of a Phi-2 model trained on a synthetic data set to solve step by guid to solve a riddle or answer any kind of question

Requiremnts

!pip install -U transformers bitsandbytes einops accelerate peft datasets wandb

Direct Use

import torch
from transformers import AutoModelForCausalLM, BitsAndBytesConfig, set_seed

# set seed
set_seed(42)

# Load model

modelpath = "DisgustingOzil/phi-2-riddler"
model = AutoModelForCausalLM.from_pretrained(
    modelpath,
    device_map="auto",
    quantization_config=BitsAndBytesConfig(
        load_in_4bit=True,
        bnb_4bit_compute_dtype=torch.float16,
        bnb_4bit_quant_type="nf4",
    ),
    torch_dtype=torch.float16,
)
from transformers import AutoTokenizer

tokenizer = AutoTokenizer.from_pretrained(modelpath, use_fast=False)
question = "Why life is so difficult of life?"
messages = [
    {"role": "user", "content": question},
]
        
input_tokens = tokenizer.apply_chat_template(
    messages, 
    add_generation_prompt=True,
    return_tensors="pt"
).to("cuda")
output_tokens = model.generate(input_tokens, max_new_tokens=200)
output = tokenizer.decode(output_tokens[0])

print(output)