Instructions to use Syradevops/Selene-3B-Instruct with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use Syradevops/Selene-3B-Instruct with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="Syradevops/Selene-3B-Instruct")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# pip install -U transformers accelerate
# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("Syradevops/Selene-3B-Instruct")
model = AutoModelForCausalLM.from_pretrained("Syradevops/Selene-3B-Instruct", device_map="auto")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=256)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use Syradevops/Selene-3B-Instruct with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "Syradevops/Selene-3B-Instruct"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Syradevops/Selene-3B-Instruct",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/Syradevops/Selene-3B-Instruct

SGLang

How to use Syradevops/Selene-3B-Instruct with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "Syradevops/Selene-3B-Instruct" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Syradevops/Selene-3B-Instruct",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "Syradevops/Selene-3B-Instruct" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Syradevops/Selene-3B-Instruct",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Unsloth Desktop
Docker Model Runner
How to use Syradevops/Selene-3B-Instruct with Docker Model Runner:
```
docker model run hf.co/Syradevops/Selene-3B-Instruct
```

Uploaded finetuned model

Developed by: Syradevops
License: apache-2.0
Finetuned from model : unsloth/Qwen2.5-3B-Instruct-unsloth-bnb-4bit

About Selene

Selene is a fine-tuned language model based on Qwen2.5-3B-Instruct, developed by Alibaba and optimized for efficient inference. Through fine-tuning, Selene was designed to provide a friendlier, gentler, and more approachable conversational experience while maintaining the capabilities of its base model.

Built with resource efficiency in mind, Selene is intended to be accessible on more modest hardware, making local AI more practical for a wider range of devices and environments.

Selene is the first version of an AI model intended for integration into services, hardware, and software developed by @SyraDevOps. It represents the beginning of an AI ecosystem designed to bring intelligent, accessible, and personalized interactions to Syra's products.

This is an initial release, and future versions may introduce further improvements to conversational quality, efficiency, and integration capabilities.

Sobre a Selene

Selene é um modelo de linguagem ajustado por fine-tuning a partir do Qwen2.5-3B-Instruct, desenvolvido pela Alibaba e voltado à geração eficiente de respostas. O treinamento foi realizado para proporcionar uma experiência de conversação mais amigável, gentil e acessível, preservando as capacidades do modelo-base.

Desenvolvida com foco em eficiência de recursos, a Selene busca oferecer uma alternativa viável para execução local em hardwares mais modestos, tornando a inteligência artificial mais acessível a diferentes dispositivos e ambientes.

A Selene é a primeira versão de um modelo de IA destinado à integração com serviços, hardwares e softwares desenvolvidos pela @SyraDevOps. Ela marca o início de um ecossistema de inteligência artificial voltado a oferecer interações inteligentes, acessíveis e personalizadas nos produtos da Syra.

Esta é uma versão inicial do projeto. Versões futuras poderão trazer melhorias na qualidade das conversas, na eficiência de execução e nas possibilidades de integração.

This qwen2 model was trained 2x faster with Unsloth and Huggingface's TRL library.

Downloads last month: 26

Safetensors

Model size

3B params

Tensor type

BF16

Model tree for Syradevops/Selene-3B-Instruct

Base model

Qwen/Qwen2.5-3B

Finetuned

Qwen/Qwen2.5-3B-Instruct

Quantized

unsloth/Qwen2.5-3B-Instruct-unsloth-bnb-4bit

Finetuned

(46)

this model