Instructions to use nanditab35/phi-2-jokebot-peft with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use nanditab35/phi-2-jokebot-peft with PEFT:

from peft import PeftModel
from transformers import AutoModelForCausalLM

base_model = AutoModelForCausalLM.from_pretrained("microsoft/phi-2")
model = PeftModel.from_pretrained(base_model, "nanditab35/phi-2-jokebot-peft")

Transformers

How to use nanditab35/phi-2-jokebot-peft with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="nanditab35/phi-2-jokebot-peft")

# pip install -U transformers accelerate
# Load model directly
from transformers import AutoModel
model = AutoModel.from_pretrained("nanditab35/phi-2-jokebot-peft", device_map="auto")

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use nanditab35/phi-2-jokebot-peft with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "nanditab35/phi-2-jokebot-peft"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "nanditab35/phi-2-jokebot-peft",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker

docker model run hf.co/nanditab35/phi-2-jokebot-peft

SGLang

How to use nanditab35/phi-2-jokebot-peft with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "nanditab35/phi-2-jokebot-peft" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "nanditab35/phi-2-jokebot-peft",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "nanditab35/phi-2-jokebot-peft" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "nanditab35/phi-2-jokebot-peft",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Docker Model Runner
How to use nanditab35/phi-2-jokebot-peft with Docker Model Runner:
```
docker model run hf.co/nanditab35/phi-2-jokebot-peft
```

Model Details

Fine-tuned "microsoft/phi-2" model using QLoRA PEFT.

Model Description

Developed by: https://huggingface.co/nanditab35
Language(s) (NLP): English
License: Apache License-2.0
Finetuned from model [optional]: microsoft/phi-2
Repository: Github repo coming soon

Uses

The main purpose behind this fine-tuned model is building a Joke Generation app from some fixed categories.

Direct Use

HF Space coming soon

Bias, Risks, and Limitations

This project is still under development. Efforts are being put to generate safe jokes first. The preference is boring jokes are better than offensive ones. All the jokes generated by this model mayy not be yet funny :). It is still in-progress.

Recommendations

It is recommened to use this Model through the corresponding HF Space (in-progress), which is also coming soon. Because the gradio app in the HF space will allow only the 4 categories of jokes to be generated not any other category, thus adding a layer of safety.

Training Details

The model is fine-tuned using QLoRA (Parameter Efficient Fine-tuning) on the base model ("microsoft/phi-2").

Training Data

This model is trained using joke dataset that is created by using DeepSeek chat with some amount of prompt engineering. Mainly 4 categories of jokes are there in this dataset: ["tech", "coffee", "foodie", "animals"].

Downloads last month: 9

Model tree for nanditab35/phi-2-jokebot-peft

Base model

microsoft/phi-2

Adapter

(994)

this model

nanditab35
/

phi-2-jokebot-peft