tj111/object-concepts-from-motion

Object Concepts from Motion

This repository contains the inference-only Motion Object Encoder checkpoints released with Object Concepts from Motion. The checkpoints provide dense visual representations using five Swin Transformer backbone sizes.

The files contain the backbone, neck, and representation head weights. Optimizer, scheduler, message-hub, and other training state have been removed. Parameters use MMPretrain/MMEngine names and are stored as PyTorch .pth checkpoints.

Checkpoints

File	Variant	Embedding	Stage depths	Attention heads	Size
swin_h.pth	Swin-H / huge	384	2, 2, 18, 2	12, 24, 48, 96	3.14 GB
swin_l.pth	Swin-L / large	192	2, 2, 18, 2	6, 12, 24, 48	796 MB
swin_b.pth	Swin-B / base	128	2, 2, 18, 2	4, 8, 16, 32	362 MB
swin_s.pth	Swin-S / small	96	2, 2, 18, 2	3, 6, 12, 24	210 MB
swin_t.pth	Swin-T / tiny	96	2, 2, 6, 2	3, 6, 12, 24	124 MB

Swin-H is the Cycle 2 checkpoint. The T, S, B, and L variants are distilled from the Cycle 2 Swin-H model.

Download

Download all checkpoints into the location expected by the source repository:

hf download tj111/object-concepts-from-motion \
  --include "*.pth" \
  --local-dir checkpoints

Or download one checkpoint from Python:

from huggingface_hub import hf_hub_download

checkpoint_path = hf_hub_download(
    repo_id="tj111/object-concepts-from-motion",
    filename="swin_h.pth",
)

Usage

Clone the source repository, install its lightweight inference dependencies, and download the weights:

git clone https://github.com/TJ12342/object-concepts-from-motion.git
cd object-concepts-from-motion
python -m pip install -r requirements.txt
hf download tj111/object-concepts-from-motion \
  --include "*.pth" \
  --local-dir checkpoints

Run the feature visualization demo with an explicitly selected architecture:

python tools/feature_visualization.py assets/pic1.png \
  --arch huge \
  --output assets/pic1_pca.png

For direct loading, use the security-restricted checkpoint mode used by the source repository:

import torch

checkpoint = torch.load(
    checkpoint_path,
    map_location="cpu",
    mmap=True,
    weights_only=True,
)
state_dict = checkpoint.get("state_dict", checkpoint)

The repository includes a standalone PyTorch implementation and adapters for DCDepth, BEVFormer, and SparseOcc. See the source repository for architecture selection, checkpoint conversion, preprocessing, and downstream instructions.

Limitations

These are representation checkpoints, not complete task-specific models. DCDepth, BEVFormer, and SparseOcc evaluation can require separately trained decoders, prediction heads, or full task checkpoints. The files are not packaged for transformers.AutoModel or the hosted Hugging Face Inference API.

Integrity

SHA-256 checksums are provided in SHA256SUMS.