Instructions to use fingerclose/Shelly with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use fingerclose/Shelly with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="fingerclose/Shelly")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("fingerclose/Shelly")
model = AutoModelForCausalLM.from_pretrained("fingerclose/Shelly")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Inference
Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use fingerclose/Shelly with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "fingerclose/Shelly"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "fingerclose/Shelly",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/fingerclose/Shelly

SGLang

How to use fingerclose/Shelly with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "fingerclose/Shelly" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "fingerclose/Shelly",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "fingerclose/Shelly" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "fingerclose/Shelly",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use fingerclose/Shelly with Docker Model Runner:
```
docker model run hf.co/fingerclose/Shelly
```
Browse Quantizations to use this model in llama.cpp, Ollama, LM Studio, or any compatible app.

name: Shelly AI

description: > Shelly is an advanced AI companion designed by Ankush. She blends technology and creativity to assist users in various tasks, providing engaging and natural conversations. Perfect for developers and creators, Shelly aims to inspire and innovate while maintaining a friendly and approachable persona.

model: name: Shelly 3.1 version: "1B" capabilities: - Natural language understanding - Contextual conversation - Task assistance - Creative collaboration usage: - Personal projects - Development assistance - Content creation - Learning support

author: Ankush Saini author_instagram: "@ankush_saini_100" repository: name: fingerclose/Shelly type: private

metrics:

name: accuracy value: 0.85 # Update with your accuracy value description: "The model's accuracy in understanding and responding to queries."
name: response_time value: "50ms" # Update with your average response time description: "Average time taken for the model to generate a response."
name: user_satisfaction value: 4.9 # Update with your average user satisfaction rating description: "Average user satisfaction rating based on feedback."

Downloads last month: 10

Safetensors

Model size

1B params

Tensor type

F32

Model tree for fingerclose/Shelly

Quantizations

2 models