COMODO-AnyMo-Bench

[COMODO Paper] [Original COMODO Code] [AnyMo Paper] [AnyMo Bench] [AnyMo Project]

This model is an implementation of COMODO trained and evaluated on AnyMo Bench. It transfers knowledge from a frozen video teacher to a wearable IMU encoder and supports motion embedding extraction and closed-set activity classification.

Building on the head-IMU setting used in COMODO's original Ego4D and EgoExo4D experiments, this implementation adds left-wrist and right-wrist IMUs, giving three synchronized body locations and 18 input channels. It evaluates COMODO on AnyMo Bench's 50-class Core50 and 150-class Fine150 taxonomies under unseen-subject and unseen-subject-plus-cross-device protocols.

The added wrist signals provide more motion coverage, while the finer activity taxonomy and explicit subject/device shifts introduce more demanding generalization conditions. These results belong to the AnyMo Bench adaptation; they should be interpreted within its protocol rather than compared directly with accuracies from the original datasets. The original COMODO paper also studies cross-dataset generalization.

Variants

Both variants are available under one model ID, CRUISEResearchGroup/COMODO-AnyMo-Bench.

Variant	Subfolder	Classes	US test configuration	USCD test configuration
Core50	core50	50	AnyMo-Bench-50-US	AnyMo-Bench-50-USCD
Fine150	fine150	150	AnyMo-Bench-150-US	AnyMo-Bench-150-USCD

Select variant="core50" or variant="fine150" when loading (default: core50). Only the selected variant is downloaded; its classifier is loaded when using classify. Both variants use the same trained IMU encoder. Each keeps its taxonomy-specific classifier and label mapping together.

Model Components

The repository groups two complete IMU inference bundles:

CRUISEResearchGroup/COMODO-AnyMo-Bench/
├── README.md
├── requirements.txt
├── config.json                 Variant selection
├── configuration_comodo.py      Shared AutoClass entry points
├── modeling_comodo.py
├── processing_comodo.py
├── pipeline_comodo.py           Shared pipeline entry point
├── variants.json               Available variants and dataset configurations
├── core50/
│   ├── model.safetensors       Shared COMODO student
│   ├── config.json
│   ├── processor_config.json
│   ├── normalization_stats.json
│   ├── labels.json             50-class label mapping
│   ├── configuration_comodo.py
│   ├── modeling_comodo.py
│   ├── processing_comodo.py
│   ├── pipeline_comodo.py
│   ├── classifier/
│   │   ├── svm.joblib
│   │   ├── scaler.joblib
│   │   └── config.json
│   ├── training_config.json
│   ├── evaluation_results.json
│   ├── provenance.json
│   └── README.md
└── fine150/
    └── Same encoder weights, with the Fine150 configuration and 150-class classifier

The repository root supplies the shared model card, AutoClass source files, pipeline entry point, and dependencies. Each variant keeps its model configuration, AutoClass source files, preprocessing assets, and classifier together. The pipeline entry point is also copied into each variant for standalone local-folder loading.

The video teacher is needed during distillation. Both inference modes operate on IMU alone.

Core50 and Fine150 share identical trained encoder tensors and preprocessing statistics. Each variant has its own fitted SVM and label mapping. The same classifier is used for US and USCD within each variant.

Using the Released Model

Install the tested dependencies (Python 3.11), then load the custom Transformers pipeline:

python -m pip install -r https://huggingface.co/CRUISEResearchGroup/COMODO-AnyMo-Bench/resolve/main/requirements.txt

For a local copy, install its local requirements.txt and replace the model ID below with the repository directory.

import numpy as np
from transformers import pipeline

comodo = pipeline(
    "comodo",
    model="CRUISEResearchGroup/COMODO-AnyMo-Bench",
    variant="core50",
    trust_remote_code=True,
    device=0,  # use -1 for CPU
)

# One 60 Hz IMU window, shape [T, 18], in the channel order below.
imu = np.load("imu.npy").astype(np.float32)

# Mode 1: normalized backbone features for representation learning.
embedding = comodo(imu, mode="embed", sampling_rate=60)
print(embedding["embeddings"].shape)  # [1, 4608]

# Mode 2: closed-set activity recognition with the bundled scaler and SVM.
classification = comodo(
    imu,
    mode="classify",
    sampling_rate=60,
    top_k=5,
)
print(classification["predictions"][0])
# Top-k entries contain: label_id, label, score.

In classify mode, top_k=5 returns the five highest-probability classes with their labels and scores. Use top_k=1 to return only the top prediction.

Prediction IDs follow the selected model's labels.json. When evaluating on AnyMo Bench, match reference labels by name.

To use Fine150, set variant="fine150" in the same call. The input format and the two modes remain the same.

The embed mode returns L2-normalized backbone features. The classify mode passes these same features through the saved StandardScaler and SVM. Probability columns must be mapped through svm.classes_ before resolving label names.

Input Format

For one window, input has shape [T, 18]. Batched input uses [B, T, 18]; batch members must have the same length. The channel layout is fixed:

head_acc_x, head_acc_y, head_acc_z,
head_gyro_x, head_gyro_y, head_gyro_z,
left_wrist_acc_x, left_wrist_acc_y, left_wrist_acc_z,
left_wrist_gyro_x, left_wrist_gyro_y, left_wrist_gyro_z,
right_wrist_acc_x, right_wrist_acc_y, right_wrist_acc_z,
right_wrist_gyro_x, right_wrist_gyro_y, right_wrist_gyro_z

• Input sampling rate: 60 Hz.
• The published AnyMo Bench dataset export provides windows of up to five seconds (300 timesteps at 60 Hz).
• Input units, matching AnyMo Bench/Nymeria: accelerometer m/s²; gyroscope rad/s.
• Required locations: head, left wrist, right wrist, in the order above.

Preprocessing follows the AnyMo Bench implementation used to train these checkpoints. The processor automatically normalizes each channel using its saved training-set mean and standard deviation, then resizes each window to 512 time points. Short windows are resized directly without zero padding.

Pass AnyMo Bench's unnormalized IMU windows directly to the processor. There is no need to apply this normalization yourself. This per-channel z-score standardization follows the preprocessing setting used for the AnyMo Bench baselines.

Loading an AnyMo Bench Sample

from datasets import load_dataset

dataset = load_dataset(
    "CRUISEResearchGroup/AnyMo-Bench",
    "AnyMo-Bench-50-US",
    split="test",
    streaming=True,
)
row = next(iter(dataset))

result = comodo(row["imu"], mode="classify", sampling_rate=60, top_k=5)
print("Reference label:", row["label"])
print("Predictions:", result["predictions"][0])

For cross-device evaluation, select AnyMo-Bench-50-USCD and use the same Core50 model, normalization statistics, scaler, and SVM. For Fine150, set variant="fine150" and use AnyMo-Bench-150-US or AnyMo-Bench-150-USCD.

Transformers Interface

The lower-level interface exposes the processor and neural encoder separately:

import torch
from transformers import AutoModel, AutoProcessor

processor = AutoProcessor.from_pretrained(
    "CRUISEResearchGroup/COMODO-AnyMo-Bench", variant="core50", trust_remote_code=True
)
model = AutoModel.from_pretrained(
    "CRUISEResearchGroup/COMODO-AnyMo-Bench", variant="core50", trust_remote_code=True
).eval()

inputs = processor(imu, sampling_rate=60, return_tensors="pt")
with torch.inference_mode():
    features = model(**inputs)["embeddings"]  # [1, 4608]

Use variant="core50" or variant="fine150" to select the matching encoder, processor, and classifier. The earlier subfolder interface remains supported; the pipeline also accepts model_kwargs={"subfolder": "core50"}.

Use pipeline("comodo", ...) for direct classification because it additionally loads the fitted scaler, SVM, and label mapping. The AutoModel interface exposes the neural feature extractor.

Model Architecture

• IMU student: MantisV2.
• Video teacher: TimeSformer-Base fine-tuned on Kinetics-400, kept frozen during distillation.

Results

Results reported in AnyMo Bench. All values are percentages. Bold highlights COMODO.

DeepConvLSTM and MantisV2 use supervised activity recognition training. COMODO learns its IMU representation through self-supervised video-to-IMU distillation; its downstream SVM classifier is fitted using labeled training examples.

Fine150 / Unseen Subject

Model	Acc@1	Acc@5	Macro-F1
DeepConvLSTM	35.3	63.0	17.2
MantisV2	38.5	65.2	22.8
COMODO	37.8	65.2	16.0

Fine150 / Unseen Subject + Cross Device

Model	Acc@1	Acc@5	Macro-F1
DeepConvLSTM	1.9	9.5	0.4
MantisV2	14.4	39.7	8.6
COMODO	24.0	50.6	8.0

Core50 / Unseen Subject

Model	Acc@1	Acc@5	Macro-F1
DeepConvLSTM	43.2	75.4	34.5
MantisV2	45.8	76.8	41.3
COMODO	46.2	78.8	37.3

Core50 / Unseen Subject + Cross Device

Model	Acc@1	Acc@5	Macro-F1
DeepConvLSTM	1.8	12.1	0.6
MantisV2	16.6	48.7	18.4
COMODO	32.6	67.8	23.3

Citation

If you find this model or benchmark useful, please consider citing COMODO and AnyMo.

@article{chen2026comodo,
  author = {Chen, Baiyu and Wongso, Wilson and Li, Zechen and Khaokaew, Yonchanok and Xue, Hao and Salim, Flora D.},
  title = {COMODO: Cross-Modal Video-to-IMU Distillation for Efficient Egocentric Human Activity Recognition},
  year = {2026},
  issue_date = {June 2026},
  publisher = {Association for Computing Machinery},
  address = {New York, NY, USA},
  volume = {10},
  number = {2},
  doi = {10.1145/3810218},
  journal = {Proc. ACM Interact. Mob. Wearable Ubiquitous Technol.},
  month = jun,
  articleno = {33},
  numpages = {29}
}

@article{chen2026anymo,
  title   = {AnyMo: Geometry-Aware Setup-Agnostic Modeling of Human Motion in the Wild},
  author  = {Chen, Baiyu and Li, Zechen and Wongso, Wilson and Li, Lihuan and Lin, Xiachong and Xue, Hao and Tag, Benjamin and Salim, Flora},
  journal = {arXiv preprint arXiv:2605.22715},
  year    = {2026}
}

Please also follow AnyMo Bench's citation guidance for the source Nymeria dataset. The backbone is MantisV2.

License

The released model weights and bundled inference artifacts are distributed under the Apache 2.0 License (LICENSE). The COMODO-AnyMo-Bench Python inference code is distributed under the MIT License (LICENSE-CODE). The upstream MantisV2 weights are Apache 2.0 licensed; original COMODO code is MIT licensed. See NOTICE for attribution and adaptation details.

AnyMo Bench and its Nymeria source data remain subject to their own licenses and access terms, including the benchmark's CC BY-NC 4.0 terms. See dataset terms for source-data licensing and citations.