Configuration Parsing Warning:In adapter_config.json: "peft.task_type" must be a string

homi - an open decision model for audio

A Pareto-frontier open decision model that answers typed questions about a given audio clip (inspired by Jev). Unlike Jev, Clef and pplx-decider, which only take in text, homi takes an audio clip and a question, and returns a probability for every option. It beats Gemini Flash on VocalGrad and is comparable to Gemini Flash on MMSU. The model can be used for monitoring and evaluating large-scale audio at low cost.

Problem

A lot of paralinguistic and phonological questions about audio cannot be answered by looking at a transcript. This model takes in audio natively and answers those questions, e.g. "How does the speaker feel?" with the options angry / happy / sad / fear / disgust.

Just like Jev, there are three question types:

Type	You give	You get
`noul`	a yes/no question	P(yes)
`choice`	a question and 2 to 255 options	a probability per option
`score`	a question and 2 to 10 ordered levels	a probability per level, and the expected level

Model

It is Qwen3-Omni-30B-A3B-Instruct with frozen weights, a decision head in place of the language-model head, 17M parameters of LoRA, and one calibration temperature. The base weights are downloaded from Qwen's repository; this repository holds only the adapter, the head and the code.

Code, evaluation and training data builders: egma-ai/homi.

Quick start

One GPU with about 70 GB free (bf16). Tested with Python 3.12, torch 2.14.1, transformers 5.18.0, peft 0.21.2.

pip install torch==2.14.1 transformers==5.18.0 peft==0.21.2 accelerate safetensors soundfile scipy numpy
hf download egma-ai/homi --local-dir homi
python homi/predict.py clip.wav

import sys; sys.path.insert(0, "homi")
from predict import AudioDecider

decider = AudioDecider("homi")
decider.ask("clip.wav", {
    "angry": {"type": "noul", "instructions": "Does the speaker sound angry?"},
    "emotion": {"type": "choice", "instructions": "Which emotion does the speaker express?",
                "criteria": {"anger": None, "happy": None, "neutral": None, "sad": None}},
    "pace": {"type": "score", "instructions": "How fast is the person speaking?",
             "criteria": ["slow", "normal pace", "fast"]},
})

On an angry clip from RAVDESS:

{"angry": {"type": "noul", "noul": 0.7024},
 "emotion": {"type": "choice", "choice": "anger", "confidence": 0.9609,
             "probabilities": {"anger": 0.9609, "happy": 0.0124, "neutral": 0.0154, "sad": 0.0113}},
 "pace": {"type": "score", "score": 1.0611, "confidence": 0.4426,
          "legend": {"0": "slow", "1": "normal pace", "2": "fast"},
          "probabilities": {"0": 0.2481, "1": 0.4426, "2": 0.3093}}}

Options can carry descriptions: "criteria": {"low": "below 60 dB", "high": "60 dB or more"}.
Text context can go with the audio: decider.ask(audio, questions, text="...").
All questions about a clip run in one batched forward pass: about 75 ms for three questions on a short clip (H200).
Load it with predict.py. The adapter targets the Qwen3-Omni thinker, so AutoPeftModel will not find it.

How it works

The prompt is the audio, then the question with lettered options (A, B, C, ...). The decision head reads the hidden state at the last prompt position and gives one score per answer code; codes beyond the question's options are masked, and a softmax with the fitted temperature gives the probabilities. The head starts as the base model's own output rows for the answer-code tokens.

Results

Same audio, question and options for every model. Unanswered counts as wrong. Our cost is GPU time on one H200 ($4.54/h, batched); API costs are reported tokens at list prices (Gemini 3.8 Flash at its promotional price).

VocalGrad (7,200 questions: does pitch, speed or volume rise or fall within a clip? Chance is 50%.)

Model	VocalGrad	Speed	Pitch	Volume	$ per 10k questions
homi	73.9	74	61	87	0.12
Gemini 3.8 Flash	63.0	78	56	55	12.10
Qwen3-Omni-30B-A3B	50.3	50	50	50	0.11
Gemini 3.5 Flash-Lite	50.9	51	51	50	1.30
OpenAI gpt-audio	33.8	32	35	34	37.03

MMSU (4,981 questions, 47 tasks)

Model	MMSU	Perception	Reasoning	$ per 10k questions
Gemini 3.8 Flash	85.7	82.4	89.2	14.80
homi	83.1	80.4	86.0	0.28
Qwen3-Omni-30B-A3B	72.2	60.5	84.5	0.19
Gemini 3.5 Flash-Lite	67.4	55.1	80.3	2.07
OpenAI gpt-audio	60.3	46.5	74.9	41.69

gpt-audio did not answer 34% of VocalGrad and 13% of MMSU; on the questions it did answer it scores 51.2 and 69.5. Raw answers and per-task tables are in the code repository.

Calibration (does confidence match accuracy?). "Sure" means confidence of at least 0.9.

Model	VocalGrad ECE	VocalGrad Brier	VocalGrad: accuracy when sure (share sure)	MMSU ECE	MMSU Brier	MMSU: accuracy when sure (share sure)
homi	0.157	0.399	82.8% (67%)	0.076	0.259	91.7% (73%)
Gemini 3.8 Flash	0.237	0.570	70.2% (37%)	0.052	0.238	93.7% (63%)
Qwen3-Omni-30B-A3B	0.337	0.740	52.5% (38%)	0.182	0.439	84.4% (73%)
Gemini 3.5 Flash-Lite	0.229	0.622	55.0% (15%)	0.133	0.512	72.1% (14%)
OpenAI gpt-audio	0.269	0.657	50.1% (8%)	0.081	0.459	84.8% (11%)

Gemini and OpenAI probabilities are the ones the models report in their answers. gpt-audio is scored only on the questions it answered. On VocalGrad, homi is the only model with a Brier score better than guessing evenly, but it is overconfident: its most confident answers (98% on average) are right 85% of the time. Full tables: eval/results/CALIBRATION.md in the code repository.

Limitations

Counting distinct speakers is weak: 19% on MMSU's speaker counting. It usually answers one too few.
Weaker than Gemini on long conversations with several people: roles, emotion between speakers, content.
Rhyming couplets (52%) and reasoning from pauses (61%) on MMSU are below the base model.
Real-vs-synthetic speech detection is not reliable. Do not use it to detect deepfakes.
Emotion was learned mostly from acted speech.
Mostly English.
One temperature for every question type. Check calibration on your own data before setting thresholds.

Training

Data: public, permissively licensed audio and text, plus synthetic audio with labels exact by construction. No customer data, and none of the datasets MMSU builds its audio from. Full list with licences: DATA.md.

LoRA rank 16 (alpha 32, dropout 0.05) on q/k/v/o of the language model's 48 layers and q/k/v of the audio encoder's 32 layers. Base weights frozen.
Soft-target cross-entropy at the last prompt position. AdamW, 64 questions per step, cosine schedule, 8 GPUs.
Two passes: 5,654 steps at learning rate 1e-4 on the core data, then 2,136 steps at 5e-5 on newer data plus a 20% replay of the core data. 498,491 training passes in total.
Text questions blend 50/50 with soft labels from perplexity-ai/pplx-decider-v1.1-27b where it is right.
Temperature T = 1.01, fitted on 16,997 calibration questions with sources weighted equally.

Intended use

Answering questions about recorded audio where a probability is more useful than free text: tone and emotion, speaking rate, pauses and turn-taking, background sounds, audio quality, what was said.

Not for: identifying or verifying who is speaking; decisions about people without human review; inferring sensitive traits from a voice; detecting lies or deepfakes; analysing people who have not consented where consent is required.

Licence

Apache-2.0 (LICENSE, NOTICE). The base model is Apache-2.0. CREMA-D: "This model was trained using CREMA-D (Cao et al., 2014), made available under the Open Database License (ODbL v1.0); contents under the Database Contents License (DbCL v1.0)." Per-clip and per-track credits: ATTRIBUTION_FSD50K.txt, ATTRIBUTION_FMA.txt, ATTRIBUTION_MUSAN.txt. One known licence exception is described at the end of DATA.md.

@misc{homi_2026,
  title        = {homi: typed questions about audio, with probabilities},
  author       = {{Outlier Humans Inc.}},
  year         = {2026},
  howpublished = {\url{https://huggingface.co/egma-ai/homi}}
}