jbohnslav/deepethogram-tinymotionnet

DeepEthogram TinyMotionNet

TinyMotionNet is an optical-flow generator pretrained on Kinetics700 videos for DeepEthogram. This repository contains this model's checkpoint and historical training hyperparameters. See the DeepEthogram collection for the other models.

Model details

Property	Value
Architecture	TinyMotionNet
Task	optical-flow
Input	11 RGB frames stacked as (N, 33, H, W)
Output	a three-scale optical-flow pyramid; highest-resolution shape (1, 20, 32, 32) for the example
Pretraining data	Kinetics700, as recorded in the historical hyperparameters
Framework	PyTorch; DeepEthogram

The model predicts motion fields from video frames. It does not classify behaviors. Hugging Face has no dedicated optical-flow pipeline category in its current task registry; this repo uses the explicit optical-flow tag and DeepEthogram library metadata.

Download and load

Install DeepEthogram and the Hugging Face CLI. Download only this model into an existing DeepEthogram project's pretrained directory:

hf download jbohnslav/deepethogram-tinymotionnet --revision v1.0 --local-dir /path/to/project/models/pretrained_models/200221_115158_TinyMotionNet

From that run directory, verify the artifacts with shasum -a 256 -c SHA256SUMS. The checkpoint.pt and hyperparameters.yaml at its root let DeepEthogram discover this model using its existing project layout.

To load directly in Python, set run_dir to the directory downloaded above:

from pathlib import Path
import torch
from deepethogram import utils

from deepethogram.flow_generator.models.TinyMotionNet import TinyMotionNet
model = TinyMotionNet()
run_dir = Path('/path/to/project/models/pretrained_models/200221_115158_TinyMotionNet')
state_dict, _, _ = utils.load_state_dict_from_file(run_dir / 'checkpoint.pt')
model.load_state_dict(state_dict, strict=True)
model.eval()
with torch.inference_mode():
    output = model(torch.zeros(1, 33, 64, 64))

The zero-valued example checks shapes and loading. For real videos use DeepEthogram's video preprocessing and normalization. Historical settings are in hyperparameters.yaml; model_config.json summarizes architecture and input/output conventions.

Intended use and limitations

Historical YAML files contain old filesystem paths and legacy serialization tags; they are provenance records, not portable project configs. Checkpoints retain their original PyTorch serialization. Use DeepEthogram, rather than Transformers AutoModel, to load them.

This release contains initialization weights, not a complete trained behavior inference run: newer inference helpers expect config.yaml and associated metrics from your fine-tuned run. The complete historical plots, metrics, and additional stream snapshots remain in the legacy archive.

Validation

SHA256SUMS covers the checkpoint, training hyperparameters, license and model configuration. Validated with PyTorch 2.4.1 and DeepEthogram source revision 1541eec6958f90673b79c402e10b434c8cb7c868: strict parameter loading and finite CPU outputs on the synthetic input above. This is a compatibility check, not an accuracy benchmark.

License and citation

LICENSE reproduces the original DeepEthogram Harvard Academic and Non-Commercial Research Use Software License and Terms of Use unchanged. It restricts commercial use and end-user redistribution. This mirror grants no additional permissions and does not relicense the assets; commercial inquiries should go to Harvard OTD as described in the license.

Bohnslav et al. (2021). DeepEthogram, a machine learning pipeline for supervised behavior classification from raw pixels. eLife 10:e63377. doi:10.7554/eLife.63377.