Melchior

This model combines the strengths of selective state spaces (Mamba) and transformers for accurate RNA basecalling. It achieves SOTA accuracy while maintaining efficient inference speeds through its hybrid architecture, with further advantages described in the paper.

Model Details

Model Type: Hybrid Mamba-Transformer
Task: RNA Basecalling
Parameters: 134M
Architecture: isotropic 20-layer alternating Mamba and MHA backbone, with 8 attention heads
Framework: PyTorch

Training

Training dataset: Arabidopsis thaliana, Homo sapiens, Caenorhabditis elegans, Escherichia coli, and synthetic constructs from Epinano
Hardware used: One A100
Training time: 9 days
Optimization: Adam, stochastic weight averaging, linear warmup followed by cosine decay

See full code at github.com/elonlit/Melchior.

Citation

@article{melchior2025,
  title={Melchior: A Hybrid Mamba-Transformer RNA Basecaller},
  author={Litman, Elon},
  journal={bioRxiv preprint bioRxiv:2025.01.11.632456},
  year={2025}
}

License

MIT

Contact

For any inquiries, contact elonlit@stanford.edu.