README.md

metadata

license: apache-2.0
tags:
  - audio
  - speech
  - language-model
  - auristream
  - discrete-diffusion
library_name: transformers

AuriStream Parallel - Speech Language Model

AuriStream Parallel is a discrete diffusion speech language model by Greta Tuckute and Klemen Kotar.

This repository contains shared model code for AuriStream Parallel checkpoints.

Overview

AuriStream Parallel uses:

• bidirectional transformer attention
• grouped token projection (group_size=4 by default)
• parallel token heads
• partial-masking diffusion objective

Usage

Load a checkpoint repository that references this base code:

from transformers import AutoModel

model = AutoModel.from_pretrained(
    "TuKoResearch/AuriStreamParallel100M_Group4_BigAudioDataset_180k",
    trust_remote_code=True,
)

Files

• configuration_auristream_parallel.py - Configuration class
• modeling_auristream_parallel.py - Model implementation