BUDDY-0912/streamsonic-v5.4-base

StreamSonic v5.4 — Base (v4loud)

On-device streaming audio perception — silent by default, speak once at a salient event, don't repeat, correctly alert on danger. 端侧流式音频感知 —— 平时沉默、显著事件处开口一次、不重复念、对危险正确告警。

What this repo is / 这个仓库是什么

This is the base model of StreamSonic v5.4: Qwen2.5-Omni-3B, Thinker only, fine-tuned to the v4loud stage (streaming, 1.5s chunk, "always-describe"). Weights are merged (ready to load). 这是 StreamSonic v5.4 的基座模型：Qwen2.5-Omni-3B，仅 Thinker，微调到 v4loud 阶段（流式、1.5s chunk、随时会描述）。权重已 merged，可直接加载。

Full v5.4 system / 完整系统

v5.4 = this base + two small heads (hosted on GitHub): v5.4 = 本基座 + 两个小头（放在 GitHub）：

Part 部件	Where 位置	Size
Base 底座 (v4loud, frozen)	this repo 本仓库	~8.8 GB
Main decision head 主决策头	GitHub: weights/head.pt	4.1 M
Secondary suppressor 二级抑制头	GitHub: weights/sup.pt	0.9 K

Code & slides / 代码与幻灯片: https://github.com/Buddy-svg/streamsonic

Inference / 推理

python src/train/s2_infer.py --audio <clip>.wav \
  --base BUDDY-0912/streamsonic-v5.4-base \
  --head weights/head.pt --sup weights/sup.pt \
  --sup-thr 0.70 --thr 0.75

Runtime knobs / 运行时旋钮: thr (main head 主头), sup-thr=0.70 (suppressor 抑制强度).

Method (short) / 方法简述

• perceive → respond → decide. Self-built tri-state labeling (positive / negative / ignore) grounded on real acoustic change verified by three detectors (AST / CLAP / JSD) + source-clip review.
• Respond: teach the model to always describe each 1.5s chunk (Projector + LLM LoRA + control-token embeddings; audio tower frozen).
• Decide: a frozen-backbone decision head + a 0.9K secondary suppressor that halves repeated narration.
• 自建三态标注（正 / 负 / ignore），建立在被三检测器投票 + 源片段审查验证过的真实声学变化上；先教"随时会描述"，再学"何时开口 / 不重复"。

Author: Wenbo Gao (Intern) · Manager: Yuzhou Liu · Mentor: Gordan Han · Week 1 → Week 12 (v5.4).