amalia-llm/AMALIA-DPO-FALA

Model card

AMALIA-DPO-FALA is a Portuguese ASR checkpoint based on the SLAM-ASR architecture. It connects a pre-trained speech encoder to a large language model through a learned projection module, allowing the LLM to generate text transcriptions from speech representations.

Architecture

The model follows the SLAM-ASR design:

audio → speech encoder → projector → LLM → transcription

The speech encoder extracts acoustic representations from the input audio. These representations are mapped by a trainable projector into the embedding space expected by the language model. The LLM then autoregressively generates the transcription.

Components

• Base architecture: SLAM-ASR
• Speech encoder: AMALIA-speech-encoder, also available on Hugging Face
• Language model: AMALIA-9B-1225-DPO
• Task: Automatic Speech Recognition
• Language: Portuguese, with focus on European Portuguese

Intended Use

This model is intended for research and experimentation on European Portuguese ASR, especially in settings where speech representations are connected to an LLM instead of using a standalone encoder-decoder ASR model.

Example use cases include:

• transcribing Portuguese speech;
• evaluating LLM-augmented ASR systems;
• comparing SLAM-style ASR with conventional ASR models;
• research on European Portuguese speech processing.

AMALIA-FALA Usage

AMALIA-FALA models are split into two parts:

1. Model repository: contains the model weights and example assets.
2. Loader repository: contains the lightweight Python code used to load and run the model.

The model weights are not cloned manually by the user. They are downloaded automatically from Hugging Face the first time the model is loaded.

---

1. Install the loader package

Install the lightweight AMALIA-FALA loader directly from Hugging Face:

pip install git+https://huggingface.co/amalia-llm/amalia-fala-loader

---

2. Run inference on your own audio file

from amalia_fala import pipeline

asr = pipeline(
    "amalia-llm/AMALIA-DPO-FALA",
    device="cuda:0",
)

result = asr("/path/to/audio.wav")
print(result["text"])

The checkpoint is downloaded automatically from:

amalia-llm/AMALIA-DPO-FALA

and cached locally by Hugging Face. Future runs reuse the cached checkpoint.

---

3. Load the model directly

If you need access to the underlying model and tokenizer:

from amalia_fala import load_model

model, tokenizer, model_config, dataset_config = load_model(
    "amalia-llm/AMALIA-DPO-FALA",
    device="cuda:0",
)

---

4. Use the SFT model instead

To use the AMALIA-SFT-FALA model, change the repository name:

from amalia_fala import pipeline

asr = pipeline(
    "amalia-llm/AMALIA-SFT-FALA",
    device="cuda:0",
)

result = asr("/path/to/audio.wav")
print(result["text"])

---

Notes

• The loader package contains only the Python loading code.
• The model repositories contain the actual checkpoints.
• The checkpoint is downloaded automatically on first use.
• The Hugging Face cache is reused after the first download.
• A GPU is recommended for practical inference speed.

Training Data

The model was developed in the context of European Portuguese ASR experiments using FalAR and CAMÕES ASR data. FalAR contains more than 5000 hours of parliamentary speech from the Assambleia da República. CAMÕES consists of a curated collection of up to 14 sub-corpora containing different domains and speech styles, totaling approximately 425 hours of speech. The encoder AMALIA-speech-encoder was trained separately and reused as the speech encoder in this SLAM-ASR setup.

Limitations

This checkpoint is intended as a research artifact. Performance may vary depending on audio quality, speaker domain, recording conditions, and transcription style. The model may be less reliable on noisy audio, long-form speech, code-switching, or domains that differ from the training data.

Citation / Acknowledgements

This model builds on the SLAM-ASR idea of connecting a speech encoder to a large language model for automatic speech recognition.