Instructions to use jylin001206/SII-Decision-27B-LoRA with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use jylin001206/SII-Decision-27B-LoRA with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="jylin001206/SII-Decision-27B-LoRA")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
pipe(text=messages)

# pip install -U transformers accelerate
# Load model directly
from transformers import AutoProcessor, AutoModelForMultimodalLM

processor = AutoProcessor.from_pretrained("jylin001206/SII-Decision-27B-LoRA")
model = AutoModelForMultimodalLM.from_pretrained("jylin001206/SII-Decision-27B-LoRA", device_map="auto")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
inputs = processor.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=256)
print(processor.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use jylin001206/SII-Decision-27B-LoRA with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "jylin001206/SII-Decision-27B-LoRA"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "jylin001206/SII-Decision-27B-LoRA",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/jylin001206/SII-Decision-27B-LoRA

SGLang

How to use jylin001206/SII-Decision-27B-LoRA with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "jylin001206/SII-Decision-27B-LoRA" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "jylin001206/SII-Decision-27B-LoRA",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "jylin001206/SII-Decision-27B-LoRA" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "jylin001206/SII-Decision-27B-LoRA",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use jylin001206/SII-Decision-27B-LoRA with Docker Model Runner:
```
docker model run hf.co/jylin001206/SII-Decision-27B-LoRA
```

SII-Decision-27B-LoRA

SII-Decision-27B-LoRA belongs to the SII-Decision-27B family of decision models. It returns a probability distribution over user-supplied options with thinking disabled. Choice, binary/probabilistic Noul, and ordered Score tasks share the same answer-slot interface. This training optimizes decision distributions; it does not provide general chat SFT supervision.

Model and training

Item	Setting
Original model	Qwen/Qwen3.8-27B
Adaptation	LoRA, rank 64 / alpha 128 / dropout 0; exported as merged weights
Training questions	39,279
Validation questions	7,506
Teacher	Original same-size model, two reasoning readouts averaged
Student	Thinking disabled; original tokenizer and ordered option labels
Learning rate	5e-05
Global batch	64
Seed	42
Exported step / one-epoch schedule	492 / 614
Updated parameters during training	400,556,032
Weight format	Complete BF16 safetensors; no adapter required for inference

Only the listed language linear layers are adapted. Embeddings, the output head, normalization, and vision parameters remain frozen. Full-rank adaptation and LoRA use the same layer coverage. The architecture retains the original vision components, but SII-Decision decision training and the supplied inference interface use text inputs.

The per-question loss is:

w * CE(q, p) + 0.5 * w * I(Score) * ((E[p]-E[q])/(K-1))² + kappa * KL(p0 || p).

Choice/Noul/Score weights are 1 / 1.1666667 / 0.8. Kappa is 0.2 for hard binary Noul, and 0.05 otherwise. Gold labels are used for validation and identifying hard binary tasks; they do not replace teacher targets. See training_recipe.json for the exact settings.

Evaluation

The associated BF16 checkpoint scores 80.1517 / 100 on the fixed 58-benchmark decision suite, with equal weight per benchmark. This is one checkpoint's score, not a multi-seed mean or a confidence bound. Validation metrics determine the checkpoint within its run. The LoRA score was measured before BF16 weight merging; a full 58-benchmark evaluation of the merged distribution has not been repeated. NF4/INT8 results may differ from the BF16 evaluation and have not been assigned this score.

Load with Transformers

The complete merged weights can be loaded without PEFT or a separate base model. Install torch, accelerate, and transformers==5.14.1.

import torch
from transformers import AutoTokenizer, Qwen3_5ForConditionalGeneration

repo = "jylin001206/SII-Decision-27B-LoRA"
tokenizer = AutoTokenizer.from_pretrained(repo)
model = Qwen3_5ForConditionalGeneration.from_pretrained(
    repo, dtype=torch.bfloat16, device_map="auto"
)

Loading in BF16 requires sufficient RAM or GPU memory. This example loads the original precision; the RTX 4090 launchers below use the stated deployment precision. Training code and training data are publicly available at the repository links below.

Inference

Use the decision scripts from https://github.com/amazingljy1206/SII-Decision; generic free-form generation does not implement the decision scoring interface.

git clone https://github.com/amazingljy1206/SII-Decision.git
cd SII-Decision
pip install -e '.[quantization]'
bash scripts/infer_27b_single_4090.sh \
  jylin001206/SII-Decision-27B-LoRA examples/decisions.jsonl predictions.jsonl

The scripts work with either a Hugging Face repository ID or a local model directory. 4B uses BF16 on one RTX 4090; its two-GPU script runs two independent replicas. 27B uses NF4 on one RTX 4090 and INT8 by default on two RTX 4090 GPUs. Two 24-GB cards cannot hold the complete 27B BF16 model; the two-GPU script uses model sharding with quantized weights. Inputs are rejected rather than truncated when they exceed the configured token limit.

Data and reproduction

Training data: jylin001206/SII-Decision-27B-Training-Data. Training code, exact layer lists, four method configs, and merge code: https://github.com/amazingljy1206/SII-Decision. The tokenizer, chat template, configuration, license, and safetensors shards are included here.

License

The model retains the Apache-2.0 license of the original Qwen weights. See LICENSE and NOTICE. Training-data source materials retain their upstream terms, as described in the dataset card.

Downloads last month: 133

Safetensors

Model size

27B params

Tensor type

BF16

Model tree for jylin001206/SII-Decision-27B-LoRA

Base model

Qwen/Qwen3.8-27B

Finetuned

(797)

this model

jylin001206
/

SII-Decision-27B-LoRA