Instructions to use fdtn-ai/antares-1b with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use fdtn-ai/antares-1b with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="fdtn-ai/antares-1b")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("fdtn-ai/antares-1b")
model = AutoModelForCausalLM.from_pretrained("fdtn-ai/antares-1b", device_map="auto")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use fdtn-ai/antares-1b with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "fdtn-ai/antares-1b"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "fdtn-ai/antares-1b",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/fdtn-ai/antares-1b

SGLang

How to use fdtn-ai/antares-1b with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "fdtn-ai/antares-1b" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "fdtn-ai/antares-1b",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "fdtn-ai/antares-1b" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "fdtn-ai/antares-1b",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use fdtn-ai/antares-1b with Docker Model Runner:
```
docker model run hf.co/fdtn-ai/antares-1b
```

Looking for line level localization

by sardsec - opened 16 days ago

Discussion

sardsec

16 days ago

Hello Found AI team,

First off, congratulations on the release, the coupling - model + cli is quite good.
It produces some findings quite well. Ran it on juice shop and some custom code for testing.

I noticed that there is not line level findings, i.e it just says file x is vulnerable - high/med/low for a CWE number. Any ideas on how to actually get the lines that are vulnerable.
I am looking into the traces folder for now, trying to corelate the final ranked output and the traces , if it exists - pick up the string from there.

Also, the 3B Model might make better regexes, not sure, this one fails quite a bit on regexes (like me!).

Please let me know if you've got ideas.

Thank you!

AmanPriyanshu-Cisco

Cisco Foundation AI org 11 days ago

Hi @sardsec ,

Thanks for the kind words, and for running it against juice shop and your own code. On line-level: you've read it right, the model works at file granularity, that's what it was trained and rewarded against (our labels mark affected files, not spans), so it doesn't have a "line X is the vuln" to hand you yet. Correlating the ranked output with the traces, like you're already doing, is honestly the best handle right now. The commands the agent ran on its way to flagging a file tend to be the closest proxy for where it was looking, so that's the direction I'd go too.

The bigger signal for us is clear though: people want tighter-than-file output, and getting to line or region level is high on the list for the next iteration, so hang tight. On the regexes, yeah, that tracks. One thing that might help you get more out of it in the meantime: we've seen performance swing a fair bit with the inference setup, vLLM version especially (we ran against 0.19.1, and mismatches there have moved numbers noticeably for us and others), so pinning that plus the suggested max output tokens is worth a shot. Appreciate you writing all this up so clearly, this is exactly the kind of feedback that shows us where to push next, so keep it coming.

AmanPriyanshu-Cisco changed discussion status to closed 11 days ago

sardsec

6 days ago

Hi @AmanPriyanshu-Cisco ,

Thank you for taking the time to respond.

If line level localization is indeed the next iteration, I have one research direction that is worth looking into: Graph Transformers.

Being hardware poor I cant explore it on the scale that is needed.
But since you all have access H100s - I assume you can finesse a solution using coding harnesses, and some data engineering.

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

· Sign up or log in to comment