YirongSun/AuroLA-7B-Dual-LoRA-SPEAR

AuroLA-7B Dual-LoRA for SPEAR retrieval

This private research artifact contains a PEFT LoRA adapter for the official Jazzcharles/AuroLA-7B retrieval checkpoint. It adapts both the AuroLA audio encoder and the language model for paired fine-grained speech-caption retrieval.

This is an audio-text retrieval adapter, not a standalone model and not a dialogue or text-generation checkpoint. Downstream dialogue-data pipelines may use it to retrieve or rank source audio/text pairs, but another model is needed to generate dialogue.

Frozen base identity

• Base repository: Jazzcharles/AuroLA-7B
• Required base revision: 575f9b4a914069392a43c2e7409806b32bc469f4
• AuroLA source revision used by the adapter implementation: 331e8c6768b0f2fcf6b4b3a74ce0ba5753785dda
• Adapter framework: PEFT LoRA
• Adapter tensors: 778 FP32 tensors / 417,955,840 elements
• Adapter SHA-256: fc12294046231b01604dd5220a8663557664943800007ba992a614ec5c1e11ad

The base checkpoint is not included in this repository. Load the exact base revision separately and then attach this adapter with PEFT. Do not merge this adapter into a different AuroLA size, PT checkpoint, reranker, or base revision.

Retrieval input contract

The adapter was trained and evaluated with AuroLA's released retrieval semantics:

• Audio: 16 kHz mono waveform, followed by Summarize above audio in one word:
• Text: the unmodified caption followed by Summarize above sentence in one word:
• Assistant content: <emb>.
• Embedding: the final-layer hidden state immediately before the unique <emb> token, followed by FP32 L2 normalization
• Similarity: text embedding dot audio embedding

Changing the prompts, pooling position, normalization, or base revision changes the model contract and was not evaluated.

Adapter loading

The adapter requires the multimodal AuroLA base architecture, its processor, and the AuroLA retrieval embedding code. A minimal attachment step is:

import torch
from peft import PeftModel
from transformers import Qwen2_5OmniThinkerForConditionalGeneration

base = Qwen2_5OmniThinkerForConditionalGeneration.from_pretrained(
    "Jazzcharles/AuroLA-7B",
    revision="575f9b4a914069392a43c2e7409806b32bc469f4",
    torch_dtype=torch.bfloat16,
    attn_implementation="sdpa",
)
model = PeftModel.from_pretrained(
    base,
    "YirongSun/AuroLA-7B-Dual-LoRA-SPEAR",
    is_trainable=False,
)
model.eval()

Attaching the adapter is only the model-loading step. Use the exact retrieval prompt and <emb>-1 extraction contract above when computing embeddings.

Reference environment: Python 3.10, PyTorch 2.5.1, Transformers 4.57.1, PEFT 0.17.1, and qwen-omni-utils 0.0.8.

Training

• Data: 12,718 one-audio/one-caption SPEAR training pairs
• Objective: symmetric single-positive InfoNCE
• Temperature: fixed at 0.05
• LoRA: rank 128, alpha 256, dropout 0.05, no bias
• Targets: 193 audio-encoder Linear modules and 196 language-model Linear modules; original base parameters and vision modules remained frozen
• True contrastive pool: 64, with a 46-pair final pool per epoch
• Selected learning rate: 5e-5
• Best checkpoint: epoch 3, optimizer step 597

SPEAR evaluation

The full 5,451-row SPEAR test set was used for learning-rate and checkpoint selection. These numbers are therefore test-tuned and are not unbiased holdout estimates.

Direction	R@1	R@5	R@10	MRR	mAP@10	Mean rank	Median rank
Text to audio	74.5368%	89.5982%	93.7076%	81.2912%	80.9979%	4.55	1

An independent fresh-process evaluation reproduced all 5,451 target ranks, top-1 identities, stored scores, and text/audio embedding fingerprints exactly. The paired row is the sole positive for each query. No prompt engineering, reranking, multi-positive masking, or additional captions were used.

Limitations and access

• This adapter was optimized and selected on the reported SPEAR test set.
• Exact overlap between AuroLA's upstream training data and SPEAR has not been established; no claim of contamination-free evaluation is made.
• The upstream AuroLA software repository and checkpoint model card did not state a software or checkpoint license when this artifact was prepared.
• Private repository access does not grant rights to redistribute or deploy the upstream base model or this derived adapter. Users must establish appropriate permissions for their use case.
• This repository is intended only for authorized internal research collaboration and must not be made public or redistributed without a separate license review.

See training_metadata.json, provenance.json, and SHA256SUMS for the frozen artifact and experiment identities.