ParametricSkills

ParametricSkills converts textual skills into parametric adapters for language models. This repository provides the ParametricSkills component checkpoints for Qwen3.5-4B and Qwen3.5-9B.

Checkpoint	Required base model	Directory
ParametricSkills-Qwen3.5-4B	Qwen/Qwen3.5-4B	Qwen3.5-4B
ParametricSkills-Qwen3.5-9B	Qwen/Qwen3.5-9B	Qwen3.5-9B

Files

Each model directory contains:

• metanetwork.pth: hypernetwork state dictionary.
• metalora.pth: shared meta-LoRA parameters.
• mem_tokens.pt: learned memory-token parameters.
• special_token_embeddings.pt: learned special-token embedding rows.
• inference_config.yaml: portable architecture and inference settings.
• checkpoint_config.json: component metadata and SHA256 checksums.

These are skill-conditioned hypernetwork component checkpoints. The Qwen backbone weights and tokenizer are obtained separately from the corresponding base model. The components are used with the matching ParametricSkills Qwen3.5 runtime to generate skill-specific LoRA adapters; they are not themselves a pre-generated PEFT adapter or a complete Transformers language-model checkpoint.

Download

pip install -U huggingface_hub
hf download DiaryOfBacon/ParametricSkills --local-dir ParametricSkills

To download only one variant:

from huggingface_hub import snapshot_download

snapshot_download(
    repo_id="DiaryOfBacon/ParametricSkills",
    allow_patterns=["Qwen3.5-4B/*", "load_checkpoint.py"],
    local_dir="ParametricSkills",
)

Replace Qwen3.5-4B with Qwen3.5-9B for the other variant.

Load the component checkpoint

The supplied helper validates file checksums and loads tensors on CPU using PyTorch's weights_only=True mode. Install PyTorch 2.6 or newer, then run:

python ParametricSkills/load_checkpoint.py --model Qwen3.5-4B

Programmatic loading:

import sys
sys.path.insert(0, "ParametricSkills")
from load_checkpoint import load_components

config, components = load_components("ParametricSkills/Qwen3.5-4B")
# components contains the four checkpoint objects under their original filenames.
# Supply them to the matching ParametricSkills runtime together with the base model.

The helper loads checkpoint components; generating a skill adapter additionally requires the custom ParametricSkills Qwen3.5 model and hypernetwork implementation. The portable inference YAML records the relevant model class names and architecture. Use the backbone revision recorded in checkpoint_config.json for reproducible loading. Retain the checkpoint filenames when passing a model directory to that runtime.

License and acknowledgments

The release is distributed under Apache License 2.0. The corresponding Qwen base models are distributed separately under their own Apache-2.0 licenses. ParametricSkills builds on the SHINE hypernetwork implementation and initialization; upstream attribution and license notices are provided in THIRD_PARTY_NOTICES.md.