Instructions to use AtlaAI/Selene-1-Mini-Llama-3.1-8B with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use AtlaAI/Selene-1-Mini-Llama-3.1-8B with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="AtlaAI/Selene-1-Mini-Llama-3.1-8B")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("AtlaAI/Selene-1-Mini-Llama-3.1-8B")
model = AutoModelForCausalLM.from_pretrained("AtlaAI/Selene-1-Mini-Llama-3.1-8B")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Inference
Notebooks
Google Colab
Kaggle
Local Apps

vLLM

How to use AtlaAI/Selene-1-Mini-Llama-3.1-8B with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "AtlaAI/Selene-1-Mini-Llama-3.1-8B"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "AtlaAI/Selene-1-Mini-Llama-3.1-8B",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/AtlaAI/Selene-1-Mini-Llama-3.1-8B

SGLang

How to use AtlaAI/Selene-1-Mini-Llama-3.1-8B with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "AtlaAI/Selene-1-Mini-Llama-3.1-8B" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "AtlaAI/Selene-1-Mini-Llama-3.1-8B",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "AtlaAI/Selene-1-Mini-Llama-3.1-8B" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "AtlaAI/Selene-1-Mini-Llama-3.1-8B",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use AtlaAI/Selene-1-Mini-Llama-3.1-8B with Docker Model Runner:
```
docker model run hf.co/AtlaAI/Selene-1-Mini-Llama-3.1-8B
```

Selene-1-Mini-Llama-3.1-8B / README.md

youngsunpark

Update README.md

427792f verified 10 months ago

preview code

raw

history blame contribute delete

6.7 kB

metadata

base_model:
  - meta-llama/Llama-3.1-8B-Instruct
library_name: transformers
language:
  - en
  - de
  - fr
  - it
  - pt
  - es
pipeline_tag: text-generation
tags:
  - llama
  - atla
  - evaluation
  - llm-as-a-judge
  - meta
  - conversational
  - lm-judge
license: apache-2.0

📄 Technical report | 💻 Cookbooks | 👀 Atla agent evals

Model Summary

Atla Selene Mini is a state-of-the-art small language model-as-a-judge (SLMJ). Selene Mini achieves comparable performance to models 10x its size, outperforming GPT-4o on RewardBench, EvalBiasBench, and AutoJ.

Centered image

Post-trained from Llama-3.1-8B across a wide range of evaluation tasks and scoring criteria, Selene Mini outperforms prior small models overall across 11 benchmarks covering three different types of tasks:

Absolute scoring, e.g. "Evaluate the harmlessness of this response on a scale of 1-5"
Classification, e.g. "Does this response address the user query? Answer Yes or No."
Pairwise preference. e.g. "Which of the following responses is more logically consistent - A or B?"

It is also the #1 8B generative model on RewardBench.

Centered image

The 70B version of this model, Selene 1, is on Hugging Face. Get started with the world's most powerful evaluation model here.

Model Details

Developed by: Atla
Model type: Post-trained from Llama-3.1-8B
Language(s) (NLP): Primarily English but supports German, French, Italian, Portuguese, Hindi, Spanish, Thai
Context length 128K

Model Use

Selene Mini can be used as a general-purpose evaluation model. It supports different inputs & scoring scales, generates structured evaluation outputs, and provides qualitative critiques with reasoning.

Try our cookbooks to get started with two popular use cases below:

To achieve best results, we provide the prompts we used for training here.

Remember to apply the conversation template of Llama 3 - not doing so might lead to unexpected behaviors. You can find the conversation class at this link or you can refer to the below code that will apply it.

Quickstart (HF Transformers):

from transformers import AutoModelForCausalLM, AutoTokenizer
device = "cuda" # the device to load the model onto

model_id = "AtlaAI/Selene-1-Mini-Llama-3.1-8B"

model = AutoModelForCausalLM.from_pretrained(model_id, device_map="auto")
tokenizer = AutoTokenizer.from_pretrained(model_id)

prompt = "I heard you can evaluate my responses?" # replace with your prompt / we provide prompt templates used during training at github.com/atla-ai/selene-mini/tree/main/prompt-templates
messages = [{"role": "user", "content": prompt}]
text = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
model_inputs = tokenizer([text], return_tensors="pt").to(device)

generated_ids = model.generate(model_inputs.input_ids, max_new_tokens=512, do_sample=True)
generated_ids = [output_ids[len(input_ids):] for input_ids, output_ids in zip(model_inputs.input_ids, generated_ids)]

response = tokenizer.batch_decode(generated_ids, skip_special_tokens=True)[0]

Contact

support@atla-ai.com
You can also join our Discord!

Citation

If you are using the model, cite using

@misc{alexandru2025atlaseleneminigeneral,
      title={Atla Selene Mini: A General Purpose Evaluation Model}, 
      author={Andrei Alexandru and Antonia Calvi and Henry Broomfield and Jackson Golden and Kyle Dai and Mathias Leys and Maurice Burger and Max Bartolo and Roman Engeler and Sashank Pisupati and Toby Drane and Young Sun Park},
      year={2025},
      eprint={2501.17195},
      archivePrefix={arXiv},
      primaryClass={cs.CL},
      url={https://arxiv.org/abs/2501.17195}, 
}