Instructions to use barha/granite-switch-4.0-350m-demo with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use barha/granite-switch-4.0-350m-demo with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="barha/granite-switch-4.0-350m-demo")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoModelForCausalLM
model = AutoModelForCausalLM.from_pretrained("barha/granite-switch-4.0-350m-demo", dtype="auto")

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use barha/granite-switch-4.0-350m-demo with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "barha/granite-switch-4.0-350m-demo"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "barha/granite-switch-4.0-350m-demo",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/barha/granite-switch-4.0-350m-demo

SGLang

How to use barha/granite-switch-4.0-350m-demo with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "barha/granite-switch-4.0-350m-demo" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "barha/granite-switch-4.0-350m-demo",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "barha/granite-switch-4.0-350m-demo" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "barha/granite-switch-4.0-350m-demo",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use barha/granite-switch-4.0-350m-demo with Docker Model Runner:
```
docker model run hf.co/barha/granite-switch-4.0-350m-demo
```

Granite Switch 4.0 350M — 3-Adapter Demo

A single Granite Switch checkpoint built on ibm-granite/granite-4.0-350m with three task LoRA adapters embedded in one model. Each adapter is activated by a control token, so one deployed checkpoint serves three different tasks — no adapter swapping, no separate model loads.

This is the multi-adapter companion to the single-adapter barha/granite-switch-4.0-350m-cti.

Embedded adapters

#	Adapter	Control token	Task	Source LoRA
1	`cti-technique-mapping`	`<\|cti-technique-mapping\|>`	Map a CTI description → MITRE ATT&CK technique ID	barha/granite-cti-technique-mapping-350m-lora
2	`text-to-json`	`<\|text-to-json\|>`	Natural language + schema → schema-conforming JSON	barha/granite-text-to-json-350m-lora
3	`genai-attack-vector`	`<\|genai-attack-vector\|>`	Classify a GenAI security incident into 1 of 14 attack-vector classes	barha/granite-genai-attack-vector-350m-lora

All three adapters share an identical LoRA shape (rank 16, alpha 32, on the fused q/k/v/o attention projections and the input_linear / output_linear MLP projections), which is what lets them stack cleanly into one switch checkpoint.

Control token IDs: 100352, 100353, 100354 (3 new tokens; vocab 100355).

Per-adapter evaluation (on granite-4.0-350m)

Adapter	Metric	Score	n
text-to-json	Key-F1 (headline)	98.4	2000
genai-attack-vector	Accuracy	74.8% (166/222)	222
cti-technique-mapping	—	see source repo	—

These are the standalone scores of each source LoRA; embedding them in the switch does not change adapter weights.

How adapter selection works

Granite Switch routes per request via the control token: place the adapter's control token in the prompt (the chat template handles placement) and the switch activates that adapter for the turn. With no control token, the base model runs unmodified.

Usage

Compose / inference follow the standard Granite Switch flow — see the granite-switch repo and its tutorials. The checkpoint loads as a GraniteSwitchForCausalLM; adapter_index.json lists the adapter → control-token mapping and io_configs/<adapter>/io.yaml carries each adapter's I/O contract.

Build

Composed with granite_switch.composer.compose_granite_switch:

python -m granite_switch.composer.compose_granite_switch \
  --base-model ibm-granite/granite-4.0-350m \
  --technology lora \
  --adapters cti-technique-mapping text-to-json genai-attack-vector \
  --output ./granite-switch-4.0-350m-demo

Base params 352M → composed 359M (+2.0%). See BUILD.md and compose_report.json in this repo for the full composition report.

License: Apache-2.0

Downloads last month: 75

Safetensors

Model size

0.4B params

Tensor type

I64

BF16

Model tree for barha/granite-switch-4.0-350m-demo

Base model

ibm-granite/granite-4.0-350m-base

Finetuned

ibm-granite/granite-4.0-350m

Adapter

(8)

this model