Instructions to use NikolaKang/AppliancePlan with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use NikolaKang/AppliancePlan with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("image-text-to-text", model="NikolaKang/AppliancePlan")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
pipe(text=messages)

# Load model directly
from transformers import AutoProcessor, AutoModelForMultimodalLM

processor = AutoProcessor.from_pretrained("NikolaKang/AppliancePlan")
model = AutoModelForMultimodalLM.from_pretrained("NikolaKang/AppliancePlan", device_map="auto")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
inputs = processor.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(processor.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use NikolaKang/AppliancePlan with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "NikolaKang/AppliancePlan"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "NikolaKang/AppliancePlan",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker

docker model run hf.co/NikolaKang/AppliancePlan

SGLang

How to use NikolaKang/AppliancePlan with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "NikolaKang/AppliancePlan" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "NikolaKang/AppliancePlan",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "NikolaKang/AppliancePlan" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "NikolaKang/AppliancePlan",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Docker Model Runner
How to use NikolaKang/AppliancePlan with Docker Model Runner:
```
docker model run hf.co/NikolaKang/AppliancePlan
```

AppliancePlan

Fine-tuned VLM for manual-grounded appliance manipulation (ACM MM ’26):

Scaling Manual-Grounded Appliance Manipulation with Data Synthesis and Unified Planning
https://doi.org/10.1145/3767308.3836196

Supports part grounding, open-loop planning, key-page / auxiliaries, and closed-loop adjustment.


Code	https://github.com/Niko-kang/AppliancePlan
Training data (UseAppliance)	https://huggingface.co/datasets/NikolaKang/UseAppliance
Paper (DOI)	https://doi.org/10.1145/3767308.3836196

Paths used by the release scripts

After download, set:

# this model repo
export MODEL_PATH=/path/to/NikolaKang/AppliancePlan   # or: hf download NikolaKang/AppliancePlan --local-dir ./AppliancePlan-Model

# companion dataset
hf download NikolaKang/UseAppliance --repo-type dataset --local-dir ./UseAppliance
cd ./UseAppliance && tar -xf images.tar   # creates ./images/

export IMAGE_PATH=/path/to/UseAppliance
export ROOT_PATH=$IMAGE_PATH
# annotations:  $IMAGE_PATH/Train_data/*.json
# images:       $IMAGE_PATH/images/...   (JSON image fields are relative to IMAGE_PATH)

Then follow train/eval in the GitHub README (scripts/train_flywheel.sh, eval/).

Citation

@inproceedings{long2026applianceplan,
  title     = {Scaling Manual-Grounded Appliance Manipulation with Data Synthesis and Unified Planning},
  author    = {Long, Yuxing and Kang, Lei and Yu, Ziyan and Gao, Yuzheng and Cheng, Bin
               and Zhang, Jiyao and Li, Xiaoqi and Yang, Haolin and Li, Dongjiang
               and Shen, Hui and Dong, Hao},
  booktitle = {Proceedings of the 34th ACM International Conference on Multimedia (MM '26)},
  year      = {2026},
  doi       = {10.1145/3767308.3836196}
}

Downloads last month: 19

Safetensors

Model size

8B params

Tensor type

BF16

NikolaKang
/

AppliancePlan

AppliancePlan

Related links

Paths used by the release scripts

Citation