bghani/perch2-pytorch-weights

Perch v2 — PyTorch backbone weights

PyTorch-converted backbone weights for Google's Perch v2 bioacoustic foundation model, for use with perch2-pytorch — a PyTorch port built for frozen-feature extraction, linear probing, and full deep fine-tuning (unlike the official ONNX/tflite releases, which are inference-only and have no autograd graph).

What this is

• Architecture: stock timm tf_efficientnet_b3, single-channel input (in_chans=1), classification head stripped — returns a pooled 1536-dim embedding.
• Source: extracted from Google's original JAX/Flax SavedModel (infer.graph.variables) and converted into a PyTorch state dict compatible with timm's tf_efficientnet_b3.
• File: perch_v2_backbone_timm.pt — backbone weights only (no classification head).

Known limitation

This is an independent community conversion, not an official release from Google. Per-block validation against the original (stem, expand blocks, residual blocks, head) showed 0.999+ cosine similarity, but whole-network cosine similarity plateaus around ~0.80, likely from small numerical error accumulating across the 26 sequential MBConv blocks. Treat these weights as a strong pretrained initialization for fine-tuning, not a bit-exact reproduction of Google's model.

If you need bit-exact frozen embeddings rather than a trainable backbone, use the official ONNX/tflite build instead.

Usage

1. Frozen features

import torch
from huggingface_hub import hf_hub_download
from perchv2_pytorch import Perch2Embedder

weights_path = hf_hub_download(repo_id="bghani/perch2-pytorch-weights", filename="perch_v2_backbone_timm.pt")

embedder = Perch2Embedder(weights_path=weights_path)
embedder.eval()

waveform = torch.zeros(4, 160_000)  # 5s clips @ 32kHz, batch of 4
with torch.no_grad():
    embeddings = embedder(waveform)  # (4, 1536)

2. Linear probing

from perchv2_pytorch import PerchFrontend, Perch2Classifier

mel = PerchFrontend()
model = Perch2Classifier(
    num_classes=42,
    mel=mel,
    weights_path=weights_path,
    mode="linear_probe",   # backbone frozen, only the new head trains
)

3. Full fine-tuning

model = Perch2Classifier(
    num_classes=42,
    mel=mel,
    weights_path=weights_path,
    mode="finetune",   # backbone unfrozen -- this is the whole point of this repo
)

See the main repo for full usage — frozen embeddings, linear probing, and full fine-tuning, with runnable examples and a walkthrough notebook.

License

Apache 2.0, inherited from the original Perch v2 release. See NOTICE in the main repo for the full derivative-work attribution.

Citation

If you use these weights in published work, please cite the original Perch v2 paper (arXiv:2508.04665) — check the official repo for the current preferred citation — and note that these specific weights are a community PyTorch conversion.