Renzhong11/mllm-edge-speculative-weights

MLLM Edge Speculative Decoding Weights

This repository provides the trained draft models and companion components used by the MLLM Edge Speculative Decoding project.

The two myVispec-* directories are trained draft models for multimodal speculative decoding:

• myVispec-Qwen2.5-VL-7B-Instruct is the draft model trained for Qwen/Qwen2.5-VL-7B-Instruct.
• myVispec-llava-v1.6-vicuna-7b-hf is the draft model trained for llava-hf/llava-v1.6-vicuna-7b-hf.

The other checkpoints in this repository are the corresponding image-adaptor and language-model-head components required by the ViSpec runtime. These assets are not standalone vision-language models and must be used with the matching base model and project code.

Repository contents

Model family	Trained draft model	Image-adaptor component	LM-head component	Draft precision
Qwen2.5-VL-7B-Instruct	myVispec-Qwen2.5-VL-7B-Instruct/	img_adaptor_vispec_qwen.pth	lm_head_qwen.safetensors	BF16
LLaVA-v1.6-Vicuna-7B	myVispec-llava-v1.6-vicuna-7b-hf/	img_adaptor_vispec.pth	lm_head_only.safetensors	FP16

Each draft-model directory contains:

config.json
model.safetensors
model_1.safetensors

The published draft configurations use one decoder layer. The Qwen draft has a hidden size of 3584 and the LLaVA draft has a hidden size of 4096. The weight shards must remain together with their matching config.json.

Intended use

These weights are intended for research and benchmarking of multimodal speculative decoding with the ViSpec client/server stack. In this setup, the draft model proposes candidate tokens on the client, while the corresponding base vision-language model validates them on the server.

The weights are not intended for direct use through the standard Hugging Face inference widget or as replacements for the full Qwen2.5-VL or LLaVA base models.

Download

Install the Hugging Face CLI if necessary:

python -m pip install -U huggingface_hub

Download all published assets into the project's models/ directory:

hf download Renzhong11/mllm-edge-speculative-weights \
  --local-dir models

Qwen2.5-VL assets only

hf download Renzhong11/mllm-edge-speculative-weights \
  myVispec-Qwen2.5-VL-7B-Instruct/config.json \
  myVispec-Qwen2.5-VL-7B-Instruct/model.safetensors \
  myVispec-Qwen2.5-VL-7B-Instruct/model_1.safetensors \
  img_adaptor_vispec_qwen.pth \
  lm_head_qwen.safetensors \
  --local-dir models

LLaVA assets only

hf download Renzhong11/mllm-edge-speculative-weights \
  myVispec-llava-v1.6-vicuna-7b-hf/config.json \
  myVispec-llava-v1.6-vicuna-7b-hf/model.safetensors \
  myVispec-llava-v1.6-vicuna-7b-hf/model_1.safetensors \
  img_adaptor_vispec.pth \
  lm_head_only.safetensors \
  --local-dir models

After downloading the complete repository, the relevant layout is:

models/
├── img_adaptor_vispec.pth
├── img_adaptor_vispec_qwen.pth
├── lm_head_only.safetensors
├── lm_head_qwen.safetensors
├── myVispec-llava-v1.6-vicuna-7b-hf/
│   ├── config.json
│   ├── model.safetensors
│   └── model_1.safetensors
└── myVispec-Qwen2.5-VL-7B-Instruct/
    ├── config.json
    ├── model.safetensors
    └── model_1.safetensors

Hugging Face may also create models/.cache/huggingface/; this directory only contains download metadata.

Using the weights with ViSpec

Download the required full base model separately, then set the following setting in the YAML configuration files in config folder for the selected family.

Qwen2.5-VL

server:
  base_model_path: models/Qwen2.5-VL-7B-Instruct
  img_adaptor_weights_path: models/img_adaptor_vispec_qwen.pth

client:
  draft_model_path: models/myVispec-Qwen2.5-VL-7B-Instruct
  base_model_lm_head_path: models/lm_head_qwen.safetensors
  tokenizer_path: models/Qwen2.5-VL-7B-Instruct
dataset:
  base_model_path: models/Qwen2.5-VL-7B-Instruct

LLaVA-v1.6-Vicuna-7B

server:
  base_model_path: models/llava-v1.6-vicuna-7b-hf
  img_adaptor_weights_path: models/img_adaptor_vispec.pth

client:
  draft_model_path: models/myVispec-llava-v1.6-vicuna-7b-hf
  base_model_lm_head_path: models/lm_head_only.safetensors
  tokenizer_path: models/llava-v1.6-vicuna-7b-hf
dataset:
  base_model_path: models/llava-v1.6-vicuna-7b-hf

Use the matching draft model, adaptor, LM head, tokenizer, and base model as one set. Mixing Qwen and LLaVA components is unsupported.

Training information

Both draft models were trained specifically for the corresponding base-model family using the training pipeline in the project repository and the Vispec project. They are compact draft networks used to propose speculative token trees; final outputs remain subject to validation by the full base model.

Detailed training-data provenance, hyperparameters, hardware, and final evaluation results are not included in this initial release.

License and upstream terms

The accompanying project source code is licensed under Apache License 2.0. These artifacts are designed to work with third-party base models. Users must also review and comply with the licenses and acceptable-use terms of Qwen2.5-VL, LLaVA, Vicuna, and their underlying model components as applicable.