README.md

---
license: mit
tags:
  - task-arithmetic
  - task-vectors
  - model-editing
  - model-merging
  - clip
---

# Task Vector Bases — Checkpoints

Pre-built task vector **bases** for
[Task Vector Bases: A Unified and Scalable Framework for Compressed Task Arithmetic](https://openreview.net/forum?id=zkc7u3mIaE).

**Code:** https://github.com/uiuctml/TaskVectorBasis

Each basis compresses `T` task vectors (one per dataset) into `M = T/2` basis vectors,
built from CLIP image encoders fine-tuned on the standard vision benchmark. Loading a
basis lets you do task **addition** and **negation** at a fraction of the storage of
keeping all `T` task vectors.

## Naming

Folders are named `{model}_{method}_M{M}_{T}task`:

- `model` ∈ {`ViT-B-16`, `ViT-B-32`, `ViT-L-14`}
- `method` ∈ {`AE` (Autoencoder / Gram), `PCA`}
- `M` = number of basis vectors (`= T/2`)
- `T` ∈ {8, 14, 20} tasks (seed 0)

e.g. `ViT-B-32_AE_M4_8task`, `ViT-L-14_PCA_M10_20task`.

## Contents of each folder

| file | purpose |
|---|---|
| `basis_vectors.pt` | the `M` basis vectors — used for **addition** (required) |
| `method_info.json` | method, hyperparameters, and the **dataset order** used at build time |
| `AWB.pt` / `pca_components.pt` | method-specific artifact — needed to recover per-task vectors for **negation** |

## Usage

Clone the [code repo](https://github.com/uiuctml/TaskVectorBasis), set up its environment, then:

```python
from huggingface_hub import snapshot_download
from src.basis_vectors import BasisMethod
from src.task_vectors import NonLinearTaskVector
from src.basis_pipeline import load_and_recover_from_saved_basis

name = "ViT-B-32_AE_M4_8task"
local = snapshot_download("cindy2000sh/TaskVectorBasis-checkpoints", allow_patterns=[f"{name}/*"])
basis_dir = f"{local}/{name}"

# Task addition: sum the M basis vectors into one merged task vector.
merged = sum(NonLinearTaskVector(vector=bv) for bv in BasisMethod.load_basis_vectors(basis_dir))
# image_encoder = merged.apply_to("checkpoints/ViT-B-32/zeroshot.pt", scaling_coef=0.4)

# Task negation: recover the per-task vectors from the basis, then negate.
recovered = load_and_recover_from_saved_basis(run_dir=basis_dir)
# neg = -NonLinearTaskVector(vector=recovered[0])
```

Or use the helper script:

```bash
python scripts/load_basis.py --hf-repo cindy2000sh/TaskVectorBasis-checkpoints --hf-subdir ViT-B-32_AE_M4_8task
```

## Citation

```bibtex
@article{zeng2025task,
  title={Task Vector Bases: A Unified and Scalable Framework for Compressed Task Arithmetic},
  author={Zeng, Siqi and He, Yifei and Liu, Meitong and You, Weiqiu and Hao, Yifan and Tsai, Yao-Hung Hubert and Yamada, Makoto and Zhao, Han},
  journal={arXiv preprint arXiv:2502.01015},
  year={2025}
}
```