Instructions to use fdtn-ai/antares-1b with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use fdtn-ai/antares-1b with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="fdtn-ai/antares-1b")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("fdtn-ai/antares-1b")
model = AutoModelForCausalLM.from_pretrained("fdtn-ai/antares-1b", device_map="auto")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use fdtn-ai/antares-1b with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "fdtn-ai/antares-1b"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "fdtn-ai/antares-1b",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/fdtn-ai/antares-1b

SGLang

How to use fdtn-ai/antares-1b with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "fdtn-ai/antares-1b" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "fdtn-ai/antares-1b",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "fdtn-ai/antares-1b" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "fdtn-ai/antares-1b",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use fdtn-ai/antares-1b with Docker Model Runner:
```
docker model run hf.co/fdtn-ai/antares-1b
```

MBP M5 24GB Runs

by whiskeysquirrel - opened 19 days ago

Discussion

whiskeysquirrel

19 days ago

I got this working on my MacBook M5 24GB. Originally tried MLX but hit issues with the model ending without an explicit final submission. Converted to GGUF f16 and served with llama.cpp instead, and it seems to work well:

My MBP ran at about 70 tokens/s and did a 20-CWE sweep in 5 minutes with 1 worker.

Definitely leverage the .antares.toml file to ignore anything you want to exclude — packages, build output, and especially legacy code. .gitignore isn't used, so you have to list it yourself. Excluding a dead Express directory cut a big chunk of my findings, since the model only gets a repo snapshot and 15 read-only commands and can't tell what's actually deployed.

Ultimately about half the findings were worth acting on. Pairing it with a harness or something that has repo context is where I think it really starts to shine.

AmanPriyanshu-Cisco

Cisco Foundation AI org 18 days ago

Thanks for writing this up, and glad the CLI's working for you.

The MLX-to-GGUF note is useful for us: that "ends without a final submission" issue on MLX is hard to catch from the inside, so knowing llama.cpp + GGUF f16 runs clean helps. We've been seeing how wide the GGUF use has been, so we'll look into making the models more adoptable in those setups for future releases.

And your last point is the one we'd endorse hardest: pairing it with a harness that carries repo context is the sweet spot at this stage, and it's a direction we'll definitely look into improving in future iterations.

Thank you again!!

AmanPriyanshu-Cisco changed discussion status to closed 18 days ago

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

· Sign up or log in to comment