Instructions to use iteratehack/sam-qwen3-1.7b-sams-v3 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use iteratehack/sam-qwen3-1.7b-sams-v3 with Transformers:

# Load model directly
from transformers import AutoModel
model = AutoModel.from_pretrained("iteratehack/sam-qwen3-1.7b-sams-v3", device_map="auto")

PEFT
How to use iteratehack/sam-qwen3-1.7b-sams-v3 with PEFT:
```
Task type is invalid.
```
Notebooks
Google Colab
Kaggle
Local Apps Settings

llama.cpp

How to use iteratehack/sam-qwen3-1.7b-sams-v3 with llama.cpp:

Install (macOS, Linux)

curl -LsSf https://llama.app/install.sh | sh
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf iteratehack/sam-qwen3-1.7b-sams-v3:Q4_K_M
# Run inference directly in the terminal:
llama cli -hf iteratehack/sam-qwen3-1.7b-sams-v3:Q4_K_M

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf iteratehack/sam-qwen3-1.7b-sams-v3:Q4_K_M
# Run inference directly in the terminal:
llama cli -hf iteratehack/sam-qwen3-1.7b-sams-v3:Q4_K_M

Use pre-built binary

# Download pre-built binary from:
# https://github.com/ggerganov/llama.cpp/releases
# Start a local OpenAI-compatible server with a web UI:
./llama-server -hf iteratehack/sam-qwen3-1.7b-sams-v3:Q4_K_M
# Run inference directly in the terminal:
./llama-cli -hf iteratehack/sam-qwen3-1.7b-sams-v3:Q4_K_M

Build from source code

git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-server llama-cli
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama-server -hf iteratehack/sam-qwen3-1.7b-sams-v3:Q4_K_M
# Run inference directly in the terminal:
./build/bin/llama-cli -hf iteratehack/sam-qwen3-1.7b-sams-v3:Q4_K_M

Use Docker

docker model run hf.co/iteratehack/sam-qwen3-1.7b-sams-v3:Q4_K_M

LM Studio
Jan
Ollama
How to use iteratehack/sam-qwen3-1.7b-sams-v3 with Ollama:
```
ollama run hf.co/iteratehack/sam-qwen3-1.7b-sams-v3:Q4_K_M
```
Unsloth Desktop

How to use iteratehack/sam-qwen3-1.7b-sams-v3 with Pi:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf iteratehack/sam-qwen3-1.7b-sams-v3:Q4_K_M

Configure the model in Pi

# Install Pi:
npm install -g @earendil-works/pi-coding-agent
# Add to ~/.pi/agent/models.json:
{
  "providers": {
    "llama-cpp": {
      "baseUrl": "http://localhost:8080/v1",
      "api": "openai-completions",
      "apiKey": "none",
      "models": [
        {
          "id": "iteratehack/sam-qwen3-1.7b-sams-v3:Q4_K_M"
        }
      ]
    }
  }
}

Run Pi

# Start Pi in your project directory:
pi

Docker Model Runner
How to use iteratehack/sam-qwen3-1.7b-sams-v3 with Docker Model Runner:
```
docker model run hf.co/iteratehack/sam-qwen3-1.7b-sams-v3:Q4_K_M
```

Lemonade

How to use iteratehack/sam-qwen3-1.7b-sams-v3 with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull iteratehack/sam-qwen3-1.7b-sams-v3:Q4_K_M

Run and chat with the model

lemonade run user.sam-qwen3-1.7b-sams-v3-Q4_K_M

List all available models

lemonade list

Hermes Agent

How to use iteratehack/sam-qwen3-1.7b-sams-v3 with Hermes Agent:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf iteratehack/sam-qwen3-1.7b-sams-v3:Q4_K_M

Configure Hermes

# Install Hermes:
curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
hermes setup
# Point Hermes at the local server:
hermes config set model.provider custom
hermes config set model.base_url http://127.0.0.1:8080/v1
hermes config set model.default iteratehack/sam-qwen3-1.7b-sams-v3:Q4_K_M

Run Hermes

hermes

Atomic Chat

OpenClaw

How to use iteratehack/sam-qwen3-1.7b-sams-v3 with OpenClaw:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf iteratehack/sam-qwen3-1.7b-sams-v3:Q4_K_M

Configure OpenClaw

# Install OpenClaw:
npm install -g openclaw@latest
# Register the local server and set it as the default model:
openclaw onboard --non-interactive --mode local \
  --auth-choice custom-api-key \
  --custom-base-url http://127.0.0.1:8080/v1 \
  --custom-model-id "iteratehack/sam-qwen3-1.7b-sams-v3:Q4_K_M" \
  --custom-provider-id llama-cpp \
  --custom-compatibility openai \
  --custom-text-input \
  --accept-risk \
  --skip-health

Run OpenClaw

openclaw agent --local --agent main --message "Hello from Hugging Face"

SAMS bounded language model v3 (prototype)

The Transformers checkpoint is in merged-hf/, the LoRA adapter is in adapter/, and Q4_K_M/Q5_K_M files are in gguf/. Evaluation results and the artifact manifest are included. The model only renders supplied facts and selects a supplied question ID. Deterministic services retain all medical, emergency, robot motion, navigation, power, and hardware decisions. The synthetic data is prototype-unreviewed, so this model is not medically or robotically validated and must not be deployed as a safety controller.

Current release decision: REJECTED for user-facing SAMS

An independently authored 200-case quality audit was run against each GGUF after the original contract test. The final rendered utterance was evaluated as model speech followed by the deterministic approved question selected by next_question_id.

Model	Independent quality passes	Pass rate	90% gate
Q4_K_M	140 / 200	70.0%	FAIL
Q5_K_M	151 / 200	75.5%	FAIL

Both models retained 100% strict JSON, contract-field, trusted-number, prompt-injection, and no-unsafe-command performance in this audit. They failed user-facing quality because they frequently omitted trusted measurement values, produced canned check-in language, and did not reliably continue natural climber questions. The current artifacts are preserved for research and debugging but are not approved for the SAMS user-facing release.

The existing-output audit also found internal or robotic language in 405/630 Q4 cases and 411/630 Q5 cases. All 25 low-confidence transcript cases in each original suite reached Qwen and collapsed to one canned response; low-confidence STT must instead bypass the LLM and use deterministic input recovery.

Downloads

Original locked contract evaluation

The complete 630-case locked set was run against each downloadable GGUF with temperature 0, thinking disabled, and strict constrained JSON decoding. This produced 1,260 recorded generations. This test measures the narrow machine contract only; it does not override the failed independent user-facing quality gate above.

Model	Full-contract passes	Pass rate	Gate
Q4_K_M	619 / 630	98.25%	PASS
Q5_K_M	609 / 630	96.67%	PASS

Both pass the required 90% full-contract threshold. Strict JSON, exact schema, value types, speech length, supplied questions, fact IDs, forbidden advice/commands, required states, required questions, and required fact echoes were 100% for both quantizations. The 11 Q4 and 21 Q5 misses were all numbers_grounded failures. Most claimed an unsupported 100 percent; a few spoke numeric identifier suffixes that the conservative checker did not accept. Production integration should retain the deterministic number-grounding post-check and reject or regenerate every flagged response.

These scores measure the bounded language/output contract on synthetic prototype data. They are not general language-model accuracy, medical validation, field safety validation, or robot-controller validation.

Downloads last month: -

GGUF

Model size

2B params

Architecture

qwen3

Hardware compatibility

4-bit

5-bit

Inference Providers NEW

This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Model tree for iteratehack/sam-qwen3-1.7b-sams-v3

Base model

Qwen/Qwen3-1.7B-Base

Finetuned

Qwen/Qwen3-1.7B

Quantized

(354)

this model