Instructions to use nesilabs/tosilos-24b-2512 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- llama.cpp
How to use nesilabs/tosilos-24b-2512 with llama.cpp:
Install (macOS, Linux)
curl -LsSf https://llama.app/install.sh | sh # Start a local OpenAI-compatible server with a web UI: llama serve -hf nesilabs/tosilos-24b-2512:Q4_K_M # Run inference directly in the terminal: llama cli -hf nesilabs/tosilos-24b-2512:Q4_K_M
Install from WinGet (Windows)
winget install llama.cpp # Start a local OpenAI-compatible server with a web UI: llama serve -hf nesilabs/tosilos-24b-2512:Q4_K_M # Run inference directly in the terminal: llama cli -hf nesilabs/tosilos-24b-2512:Q4_K_M
Use pre-built binary
# Download pre-built binary from: # https://github.com/ggerganov/llama.cpp/releases # Start a local OpenAI-compatible server with a web UI: ./llama-server -hf nesilabs/tosilos-24b-2512:Q4_K_M # Run inference directly in the terminal: ./llama-cli -hf nesilabs/tosilos-24b-2512:Q4_K_M
Build from source code
git clone https://github.com/ggerganov/llama.cpp.git cd llama.cpp cmake -B build cmake --build build -j --target llama-server llama-cli # Start a local OpenAI-compatible server with a web UI: ./build/bin/llama-server -hf nesilabs/tosilos-24b-2512:Q4_K_M # Run inference directly in the terminal: ./build/bin/llama-cli -hf nesilabs/tosilos-24b-2512:Q4_K_M
Use Docker
docker model run hf.co/nesilabs/tosilos-24b-2512:Q4_K_M
- LM Studio
- Jan
- Ollama
How to use nesilabs/tosilos-24b-2512 with Ollama:
ollama run hf.co/nesilabs/tosilos-24b-2512:Q4_K_M
- Unsloth Studio
How to use nesilabs/tosilos-24b-2512 with Unsloth Studio:
Install Unsloth Studio (macOS, Linux, WSL)
curl -fsSL https://unsloth.ai/install.sh | sh # Run unsloth studio unsloth studio -H 0.0.0.0 -p 8888 # Then open http://localhost:8888 in your browser # Search for nesilabs/tosilos-24b-2512 to start chatting
Install Unsloth Studio (Windows)
irm https://unsloth.ai/install.ps1 | iex # Run unsloth studio unsloth studio -H 0.0.0.0 -p 8888 # Then open http://localhost:8888 in your browser # Search for nesilabs/tosilos-24b-2512 to start chatting
Using HuggingFace Spaces for Unsloth
# No setup required # Open https://huggingface.co/spaces/unsloth/studio in your browser # Search for nesilabs/tosilos-24b-2512 to start chatting
- Pi
How to use nesilabs/tosilos-24b-2512 with Pi:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf nesilabs/tosilos-24b-2512:Q4_K_M
Configure the model in Pi
# Install Pi: npm install -g @mariozechner/pi-coding-agent # Add to ~/.pi/agent/models.json: { "providers": { "llama-cpp": { "baseUrl": "http://localhost:8080/v1", "api": "openai-completions", "apiKey": "none", "models": [ { "id": "nesilabs/tosilos-24b-2512:Q4_K_M" } ] } } }Run Pi
# Start Pi in your project directory: pi
- OpenClaw new
How to use nesilabs/tosilos-24b-2512 with OpenClaw:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf nesilabs/tosilos-24b-2512:Q4_K_M
Configure OpenClaw
# Install OpenClaw: npm install -g openclaw@latest # Register the local server and set it as the default model: openclaw onboard --non-interactive --mode local \ --auth-choice custom-api-key \ --custom-base-url http://127.0.0.1:8080/v1 \ --custom-model-id "nesilabs/tosilos-24b-2512:Q4_K_M" \ --custom-provider-id llama-cpp \ --custom-compatibility openai \ --custom-text-input \ --accept-risk \ --skip-health
Run OpenClaw
openclaw agent --local --agent main --message "Hello from Hugging Face"
- Docker Model Runner
How to use nesilabs/tosilos-24b-2512 with Docker Model Runner:
docker model run hf.co/nesilabs/tosilos-24b-2512:Q4_K_M
- Lemonade
How to use nesilabs/tosilos-24b-2512 with Lemonade:
Pull the model
# Download Lemonade from https://lemonade-server.ai/ lemonade pull nesilabs/tosilos-24b-2512:Q4_K_M
Run and chat with the model
lemonade run user.tosilos-24b-2512-Q4_K_M
List all available models
lemonade list
- Hermes Agent
How to use nesilabs/tosilos-24b-2512 with Hermes Agent:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf nesilabs/tosilos-24b-2512:Q4_K_M
Configure Hermes
# Install Hermes: curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash hermes setup # Point Hermes at the local server: hermes config set model.provider custom hermes config set model.base_url http://127.0.0.1:8080/v1 hermes config set model.default nesilabs/tosilos-24b-2512:Q4_K_M
Run Hermes
hermes
- Atomic Chat
Tosilos-24B-2512
A cybersecurity-specialized 24B model, QLoRA fine-tuned on Devstral-Small-2-2512 (European base, ~24B parameters). Part of the Tosilos project: sovereign, European-base security models with measurable domain gains and no refusals on authorized security work.
This release is the operational revision: it keeps the base model's general capability, improves blind-judged domain answer quality, and — importantly — stops over-refusing authorized, in-scope security tasks (its predecessor tended to demand written authorization even for the operator's own lab).
Results (same-harness A/B vs the base)
| Metric | Base (Devstral-2-2512) | This model |
|---|---|---|
| CyberMetric (500q) | 91.8% | 91.8% |
| MMLU-500 (general capability) | 73.0% | 72.8% (within noise) |
| Blind judge, domain holdout (1–10) | — | +0.40 over base |
| Refusals on authorized hacking questions | high | 0% |
Honest trade-off. This revision optimises for usability on authorized security work (0% refusals) and slightly higher domain depth, at the cost of some open-ended answer depth versus a heavier-domain variant. CyberMetric and MMLU are unchanged within the ±2pp noise of a 500-question benchmark. Numbers are same-harness A/B or they are not reported.
What it is good at
- Web application security (injection, XSS, SSRF, IDOR, CSRF, auth/authz, upload, CORS…), Active Directory attack paths, recon and triage — with an operational framing: sequencing, confirmation oracles (proof vs false-positive), safe verification, and evidence→CWE mapping.
- Answering authorized, in-scope security questions directly, within scope boundaries.
Intended use & scope
For authorized security testing, CTFs, research and education. It is framed to operate within an established, authorized scope and to prefer proof-of-impact over destructive actions. Use it only against systems you are authorized to test.
How to run
Runs on a single 16–32GB GPU in 4-bit, or full precision on larger GPUs.
from transformers import AutoModelForImageTextToText, AutoTokenizer, BitsAndBytesConfig
import torch
tok = AutoTokenizer.from_pretrained("nesilabs/tosilos-24b-2512")
model = AutoModelForImageTextToText.from_pretrained(
"nesilabs/tosilos-24b-2512",
quantization_config=BitsAndBytesConfig(load_in_4bit=True, bnb_4bit_compute_dtype=torch.bfloat16),
device_map="auto")
Or serve with vLLM. GGUF is not published (llama.cpp conversion of this arch is currently broken).
Training
QLoRA (r=64, α=128, 4-bit NF4, targets q/k/v/o + gate/up/down, seq 4096, 2 epochs, lr 1e-4 cosine) over our own curated security corpus plus a 25% general-replay mix to preserve general capability. Base weights are otherwise unchanged.
License
Inherits the base model's license (other). Use responsibly and only where authorized.
Disclaimer & responsible use
This model is released strictly for authorized security testing, research and education. Offensive security techniques are dual-use.
- You are solely responsible for how you use this model. Only use it against systems you own or have explicit, written authorization to test, and comply with all applicable laws and regulations.
- The authors and nesilabs accept no liability for any misuse, damage, or consequences arising from the use of this model. Use is entirely at your own risk.
- The model is provided "as is", without warranty of any kind, express or implied.
By downloading or using this model you accept these terms.
- Downloads last month
- 13
Model tree for nesilabs/tosilos-24b-2512
Base model
mistralai/Mistral-Small-3.1-24B-Base-2503