ARIMA: Reconstruction-Grounded Predictive Representation Learning for Symbolic Music

ARIMA (Autoregressive Representation for Symbolic Music Analysis) is a JEPA-inspired, latent-level representation model for symbolic piano performance. It encodes two-second MIDI windows, predicts future latent representations with a causal predictor, and uses onset, sustain, chroma, and velocity reconstruction to preserve musically useful structure.

The released model is the final paper variant without explicit VICReg variance regularization. It has approximately 38.5 million parameters and was trained for 42,000 steps on ATEPP performance MIDI.

Files

• best_val.pt: inference-only PyTorch checkpoint.
• training_config.yaml: final training configuration.

The checkpoint contains:

model
config
step
best_val_loss

It does not contain optimizer state. The release checkpoint was selected at step 42,000 with a validation objective of approximately 0.8881.

Load the model

Clone or install the ARIMA codebase, then:

import torch

from models import SymbolicMusicWorldModel

checkpoint = torch.load("best_val.pt", map_location="cpu", weights_only=True)
model = SymbolicMusicWorldModel(checkpoint["config"])
model.load_state_dict(checkpoint["model"])
model.eval()

Use the repository preprocessing and representation-extraction pipeline to prepare MIDI and obtain window- or piece-level representations.

Paper and code

• Paper: ARIMA: Reconstruction-Grounded Predictive Representation Learning for Symbolic Music
• Code: github.com/AndyWeasley2004/symbolic_music_wm

The model accompanies the paper by Mingyang Yao and Zhaoxiang Feng, available as arXiv:2607.10003.

Citation

@misc{yao2026arima,
  title         = {ARIMA: Reconstruction-Grounded Predictive Representation Learning for Symbolic Music},
  author        = {Mingyang Yao and Zhaoxiang Feng},
  year          = {2026},
  eprint        = {2607.10003},
  archivePrefix = {arXiv},
  primaryClass  = {cs.SD},
  url           = {https://arxiv.org/abs/2607.10003}
}

Scope

This model was developed and evaluated for piano performance MIDI. Its behavior on multi-instrument symbolic music has not been established. No standalone license has yet been specified for this checkpoint; consult the authors before redistribution or commercial use.