Instructions to use Gadsdencode/Granite-4.2-8B-ICDU-DPO with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use Gadsdencode/Granite-4.2-8B-ICDU-DPO with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="Gadsdencode/Granite-4.2-8B-ICDU-DPO")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# pip install -U transformers accelerate
# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("Gadsdencode/Granite-4.2-8B-ICDU-DPO")
model = AutoModelForCausalLM.from_pretrained("Gadsdencode/Granite-4.2-8B-ICDU-DPO", device_map="auto")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=256)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use Gadsdencode/Granite-4.2-8B-ICDU-DPO with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "Gadsdencode/Granite-4.2-8B-ICDU-DPO"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Gadsdencode/Granite-4.2-8B-ICDU-DPO",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/Gadsdencode/Granite-4.2-8B-ICDU-DPO

SGLang

How to use Gadsdencode/Granite-4.2-8B-ICDU-DPO with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "Gadsdencode/Granite-4.2-8B-ICDU-DPO" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Gadsdencode/Granite-4.2-8B-ICDU-DPO",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "Gadsdencode/Granite-4.2-8B-ICDU-DPO" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Gadsdencode/Granite-4.2-8B-ICDU-DPO",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use Gadsdencode/Granite-4.2-8B-ICDU-DPO with Docker Model Runner:
```
docker model run hf.co/Gadsdencode/Granite-4.2-8B-ICDU-DPO
```
Browse Quantizations to use this model in llama.cpp, Ollama, LM Studio, or any compatible app.

Granite-4.2-8B-ICDU-DPO

ICDU supervised fine-tuning followed by tool-preference DPO. Complete standalone text-only BF16 checkpoint (17.59 GB bundle), with both SFT and DPO adapters merged.

Training provenance

Original base: ibm-granite/granite-4.2-8b, revision f8de16cdcdbc6c779ca517604e050d82cc119e44.
SFT: 240 training and 60 validation records, 3 epochs, 90 optimizer steps. The selected best checkpoint was step 60, with validation loss 1.4124904. Learning rate 1e-4, effective batch size 8.
DPO: 900 source records yielded 764 preference pairs; 136 disabled-tool records were skipped. Prepared pairs were not truncated or dropped.
DPO: 100 optimizer steps, effective batch size 4, approximately 0.524 epochs. Learning rate 5e-6, beta 0.1. Final logged window loss: 0.0185257. Logged training preference accuracy reached 100%; this is not held-out task accuracy.
Both stages used 4-bit NF4 QLoRA, BF16 mixed precision, FlashAttention-2, LoRA rank 16, alpha 32, dropout 0.05, seed 42, and maximum sequence length 2,048.
The DPO adapter was merged into the merged SFT checkpoint on CPU in FP32, then exported as BF16 before GGUF conversion.

Usage and context

Use Transformers supporting GraniteForCausalLM / granite. The producing environment used Transformers 5.17.0, PyTorch 2.14.0, and PEFT 0.21.0. No custom model code or remote-code trust is required. Tokenizer, generation configuration and chat template are included. The smaller Q4_K_M GGUF is separate.

Render prompts using the included chat template with add_generation_prompt=True and enable_thinking=False to match training. The same saved template is embedded in the GGUF. Its SHA-256 is 4231f7ddfef1b88340640143c3a7dbbcf9e4880e920bf28f81a0c44731efeda4.

The model configuration retains the base model's 131,072-token context limit. The 2,048-token training length does not reduce that architectural limit, but long-context quality has not been evaluated for this fine-tune. The included Ollama Modelfile defaults to a 2,048-token runtime context; a larger runtime setting requires sufficient memory and separate quality checks.

Validation and limitations

The Q4_K_M file passed a real llama.cpp load and generation smoke test and answered the synthetic arithmetic prompt "What is 2 + 2?" correctly. Six synthetic tokenizer parity probes matched the admitted Hugging Face tokenizer and llama.cpp token IDs, including Unicode, code, tool markup and a rendered chat prompt. These checks are not an ICDU generalization benchmark or a measure of quantization quality loss. No held-out DPO evaluation or sealed test-set evaluation was performed. Tool schemas and execution must be supplied by the hosting application. Quantization and application chat-template handling may change outputs.

Files and license

SHA256SUMS records packaged file hashes; provenance.json records portable lineage. Datasets, training examples, trainer checkpoints, credentials and local filesystem paths are excluded. The base model declares Apache-2.0 in its pinned model card. The standard Apache-2.0 license text is included; the base repository has no separate LICENSE file at the pinned revision.

Downloads last month: 233

Safetensors

Model size

9B params

Tensor type

BF16

Model tree for Gadsdencode/Granite-4.2-8B-ICDU-DPO

Base model

ibm-granite/granite-4.1-8b-base

Finetuned

ibm-granite/granite-4.2-8b

Finetuned

(18)

this model

Quantizations

1 model