Resp-Agent: An Agent-Based System for Multimodal Respiratory Sound Generation and Disease Diagnosis
Authors: Pengfei ZHANG
Tianxin Xie
Minghao Yang
Li Liu
Venue: ICLR 2026 Poster (2026)
Keywords: Respiratory sounds
Multimodal learning
Medical agents
Controllable audio synthesis
Active curriculum learning
Primary area: applications to computer vision, audio, language, and other modalities
TL;DR: We propose Resp-Agent, the first agent-based system for respiratory intelligence, integrating controllable sound synthesis and multimodal diagnosis to achieve state-of-the-art performance on ICBHI and our new large-scale Resp-229k benchmark.

Abstract: Deep learning-based respiratory auscultation is currently hindered by two fundamental challenges: (i) inherent information loss, as converting signals into spectrograms discards transient acoustic events and clinical context; (ii) limited data availability, exacerbated by severe class imbalance. To bridge these gaps, we present **_Resp-Agent_**, an autonomous multimodal system orchestrated by a novel Active Adversarial Curriculum Agent (Thinker-A²CA). Unlike static pipelines, Thinker-A²CA serves as a central controller that actively identifies diagnostic weaknesses and schedules targeted synthesis in a closed loop. To address the representation gap, we introduce a modality-weaving Diagnoser that weaves clinical text with audio tokens via strategic global attention and sparse audio anchors, capturing both long-range clinical context and millisecond-level transients. To address the data gap, we design a flow matching Generator that adapts a text-only Large Language Model (LLM) via modality injection, decoupling pathological content from acoustic style to synthesize hard-to-diagnose samples. As a foundation for this work, we introduce **_Resp-229k_**, a benchmark corpus of 229k recordings paired with LLM-distilled clinical narratives. Extensive experiments demonstrate that Resp-Agent consistently outperforms prior approaches across diverse evaluation settings, improving diagnostic robustness under data scarcity and long-tailed class imbalance. Our code and data are available at [https://github.com/zpforlove/Resp-Agent](https://github.com/zpforlove/Resp-Agent).

Published as a conference paper at ICLR 2026
RESP-AGENT: AN AGENT-BASED SYSTEM FOR MULTI- MODAL RESPIRATORY SOUND GENERATION AND DIS- EASE DIAGNOSIS
Pengfei Zhang, Tianxin Xie, Minghao Yang, Li Liu" The Hong Kong University of Science and Technology (Guangzhou)
ABSTRACT
Deep learning-based respiratory auscultation currently hindered by two funda- mental challenges: (i) inherent information loss, converting signals into spectro- grams discards transient acoustic events and clinical context; (ii) limited data avail- ability, exacerbated by severe class imbalance. To bridge these gaps, we present Resp-Agent, an autonomous multimodal system orchestrated by a novel Active Adversarial Curriculum Agent (Thin Unlike static pipelines. Thinker- AACA serves as a central controller that actively identifies diagnostic weaknesses and schedules targeted synthesis closed loop. To address the representation gap, we introduce modality- -wweaittywwevinng Diagnoser that weaves clinical text with audio tokens via strategic global attention and sparse audio anchors, capturing both long- range clinical context and millisecond-level transients. To address the data gap, we design a flow matching Generator that adapts text-only Large Language Model (LLM) via modality injection, decoupling pathological content from acoustic style to synthesize hard-to to-d--no- samples. As foundation for this work, we introduce Resp-229k, benchmark corpus of 229k recordings paired with L.M-ddstilled clin- ical narratives. Extensive experiments demonstrate that Agent consistently outperforms prior approaches across diverse evaluation settings, improving diag- nostic robustness under data scarcity and long-tailed class imbalance. Our code and data are available https: /ttttp::/githuube wwtp:/ttww...coooooootttoott/oee
com wwwpo///wwoooww.oooooooooooooooe www.rforooooeeeee--eaeaee
INTRODUCTION
Respiratory auscultation is fundamental component of clinical diagnosis, providing critical acoustic evidence for assessing pulmonary health (Heitmann et al., 2023 Bohadana et al. 2014). Accurate and automated analysis of respiratory sounds holds substantial clinical value for the early screening, diagnosis, and monitoring respiratory discases (Rocha et al.) 2019). Although deep learning has driven significant progress in this domain, existing methods remain constrained by fundamental limitations that hinder both performance and practical deployment (Huang et al. 2023
Xia et al.
2022 Coppock et al.) 2024).
The first challenge is unimodal representational bottleneck. Audio models often convert signals into mel-spectrograms for image-style CNNs (Bae et al., 2023 He al. (024), which discards phase and blurs fine temporal structure, obscuring transient events such as crackles (Paliwal et al. 2011). Conversely, text-only models capture electronic health record (EHR) context but lack objective acoustic evidence. limiting discrimination between conditions with similar narratives but distinct auscultatory patterns. Notably, large-scale health acoustic pretraining (Baur al. 2024) remains unimodal; without deep multimodal fusion, performance and reliability saturate.
The second limitation is the lack of large, well- annotated multimodal datasets. Most public respiratory- sound corpora are small, cover only few conditions, and lack systematic curation (Zhang et al. 2024a). Even when auxiliary metadata such as demographics and symptoms is available, existing approaches rely basic fusion techniques and task task-specific designs, limiting the development of generalized multimodal models (Zhang et al. 2024b).
"Corresponding author; avrilliu@ hiilliiuhhkuss-z.ccd..cm

---

Published as a conference paper at ICLR 2026
A third challenge lies in the disconnect between analysis and generation. Current research is heavily skewed towards diagnostic tasks like classification and detection (Huang et al. 2023: Xia et al..
2022), leaving the potential of generative modeling largely unexplored (Kim et al. to synthesize respiratory sounds with specific pathological characteristics would not only support medical education, data augmentation, and interpretability research but also
2023p. The ability
stringent test of a system's 's multimodal understanding. However, no existing framework unifies analysis and synthesis within a single coherent system (Zhang et al. 2024a b).
To systematically address these challenges, we introduce Resp-Agent, a multimodal agent framework inspired by the design philosophy of intelligent agents. Resp-Agent decomposes complex functional- ity into specialized modules coordinated by a central controller that plans and schedules tasks. This design enables unified processing of respiratory sounds for both diagnostic analysis and generative synthesis, advancing the state of multimodal respiratory intelligence.
In summary, we propose Resp-Agent, a closed-loop framework that turns passive analysis into generation < diagnosis co-design. The contributions of our method are:
1) Resp-229k: A large-scale, clinically contextualized benchmark. Resp-229k contains approximately 408 hours and 229k respiratory recordings spanning 16 diagnostic categories. We pair each sample with a clinical narrative synthesized from EHR records using LLMs and refined for accuracy. The benchmark features nource-disjoint splits to rigorously test model generalization, while the correspondence between text and audio supports multimodal modeling and transparent verification.
2) Controllable Synthesis. We design Generator that augments a compact LLM to synthesize high-fidelity respiratory audio. Disease semantics are conditioned on text, while acoustic style is captured by BEATs((hhen et al.) 2023) tokens. A conditional flow-matching decoder reconstructs waveforms with high fidelity. This design is instantiated as RESP-MLLMM, to the best of our knowledge, the first multimodal large language model trained with aligned text-audio supervision for controllable respiratory sound synthesis. Flow matching ensures stable, phase-aware reconstruction of transient events, and the BEATs-derived style tokens, which model device and timbre factors, are essential for clinical realism.
3) Robust Diagnosis. We introduce Diagnoser based on modality weaving, which interleaves audio embeddings with text. A strategically designed global-attention mechanism enables the model jointly condition on fused text while parsing the acoustic stream 800s resolution, capturing fleeting events that characterize respiratory sounds. By leveraging a Longformer(Beltagy et al., 2020) backbone capture long-range dependencies, our approach yields superior performance and improved generalization across varying domains.
2 RELATED WORK
Respiratory Sound Classification (RSC) has largely relied on audio-only pipelines trained on small, single-source datasets, which limits out-of-domain generalization. Standard approaches utilize pre- trained backbones like PANNs (Kong et al. 2020) and AST (Gong et al.) 202I) to mitigate data scarcity. The OPERA benchmark has begun to close the data gap via domain-specific pretrain- ing (Zhang et al. 2024a); however, most systems remain constrained by single-modality supervision and in-distribution evaluation. Our work departs from this paradigm in two key ways. (i) Multimodal fusion. We weave EHR-style textual tokens and acoustic tokens within a long-context Transformer. Unlike RespLLM (Zhang et al. 2024b), which feeds concatenated modality tokens through dense full attention, we introduce Strategic Global Atlention with sparse audio anchors, drawing on ideas from efficient Transformers (Beltagy et al. 2020 Zaheer et al. 2020) to route clinical context to transient acoustic events at sub-quadratic cost. We evaluate on source-disjoint splits to stress-test generalization under realistic distribution shifts (Koh et al., 2021) and label imbalance (Johnson & Khoshgoftaar] 2019). (ii) Targeted augmentation. Recent generative models enable high-fidelity
audio synthesis (Borsos et al., 2023 Lipman et al., 2023 Liu et al. 2023 Peebles & Xie 2023). but existing augmentation strategies are typically untargeted, relying generic perturbations such as SpecAugment (Park et al. 2019) or unconditional generation (Kim et al. 2023). We introduce Resp-Agent, closed-loop system which an LLM- Thinker-A CCA diagnoses model failures and requests condition-controll synthesis from a flo-mmatcher generator, turning augmentation into a precise instrument for adversarial edge-case creation and distribution balancing.
2

---

Published as a conference paper at ICLR 2026
Contrilable Syathras
Clisical
(8)
Audio Patks
EaRR Table
Target Class
[PIAGNOSIS] -Diccse> (Text)
Refervnce Audio (BEATs (colures)
[DESCRIPTION]
BEATS Franse Fentures (30N Audia, Allgaed)
(LIIK Spremary)
Style-radts Trkmm
Midality Weaving
(Mobilly lijatios)
(Early Token Pusioa)
Strectured
Arguments
Rop-MLLM [UAGNOSS) DDseac (I. DAUDAUO 5-1]
Longformncr Eneoder
Thisker-A&CA (Derppheci.V3.2-E..
[DESCRUPTION]
Divrrte Accusiiz Usits
Parsing
Audio Archurs
DIT-ayyle Flan-Marchhng Recoder (M IME-spptiogmmo
Plan-Execute Tool Routing
Spane
(Choue Laup)
Neural Vocoder
Deterministc LD
Schemes
10t
Conciso
(Labeeh
Crafideace
Recycle Rationako, Ernor Profites Calibrated Confidunce
Blas Subangurnt Artions Targsted Syatheda)
Feedbakk Thinker
Figure I: Overview of Resp-Agent. The framework functions as a closed-loop system composed of three interacting modules: (a) Thinker: A compute-awan planner (Thinker- - CA) that parses semantic intents and routes tasks to other agents based on recycled error profiles and calibrated confidence. (b) Generator: A synthesis module utilizing modality injection condition the Resp- MLLM on both textual diagnosis and reference acoustic style, decoding discrete units via conditional flow matching. (c) Diagnoser: A clinical inference module employing modality weaving to fuse EHR summaries with audio features early in the network, leveraging sparse global attention for robust cross-modal reasoning.
RESP-229K: A LARGE-SCALE, MULTI-SOURCE, CROSS-DOMAND BENCHMARK
We introduce Resp-229k to address the scarcity of multimodal supervision and the lack of robust cross-domain evaluation in respiratory sound analysis. Unlike existing datasets, Resp-229k provides paired audio with standardized clinical summaries, converting diverse metadata into a format suitable for multimodal modeling. We also establish a strict out-of-domain evaluation protocol to explicitly test model generalization. We aggregate 238,074 raw clips from five public databases; after discarding corrupted or too-short recordings, 229,10 200 quality-controlled samples remain, spanning 16 classes (15 conditions and I control), and are used in all main main-paper paper experiments.
core contribution is the textual supervision. Instead of full electronic health records, each clip is paired with standardized clinical summary, concise paragraph synthesized from available source fields. Summaries adapt to source coverage: when demographics and symptoms exist, they are included: when only auscultation events and acquisition context are present, the summary focuses on those. Concretely, we retain two typical regimes as a modeling challenge: technical/event--riven summaries (auscultatory events, site, sensor/filter, phases, wheezes/crackles) and clinically enriched summaries (demographics, smoking status, comorbidities, symptoms, past medical history).
We programmatically convert heterogeneous CSV/TXT/JSON fields and filename-derived codes into standardized summaries using DeepSeek-R RI- -Distill-Qwen-7B (Guo et al., (025) as a lightweight data-to-text engine. The model does not interpret audio; instead, it consolidates existing metadata into a schema- schema-grounded paragraph with consistent style across sources, enabling reproducible, low-cost annotation refreshes while preserving diagnostically relevant heterogeneity.
To mitigate hallucination and governance risks, all LLM--generaedd clinical summaries undergo second-stage audit that combines rule-based consistency checks, critique from a stronger reasoning model acting verifier, and sampling-based human review. This process ensures that only summaries that pass the pipeline, or are rewritten and reverified after being flagged, are retained in Resp-229k. A detailed description of the auditing pipeline is provided in Appendix E
To standardize comparisons, we specify two tasks and metrics: (i) multimodal disease classification, reporting accuracy and macro-Fi; and (ii) controllable audio generation conditioned on disease semantics, reporting objective acoustic similarity and clinical-event fidelity. We report both in- domain validation results and strictly out-of-domain test results. For evaluation, Resp-229k enforces
3

---

Published as a conference paper at ICLR 2026
Table Resp-229k overview: split statisties and source datasets. The dataset identifiers correspond
to UK COVID-19 (Coppock et al. 2024, Budd et al., 2024 Pigoli et al. 2022). ICBHI (Rocha et al.
2017). SPRSound (Zhang et al. 2022). COUGHVID (Orlandic et al. 2021 and KAUH (Fraiwan
et al. 2021
(a) Resp-229k split statistics (effective samples)
Split
#Files
Hours
Mean (s)
Max (s)
Train
196,654
341
6.2
86
Valid
16,931
31
6.6
71
Test
15,516
36
8.4
30
Total
229,101
6.4
(b) Source datasets for the curation Resp-229k
Name
Role
Device
Sample Rate (kHz)
Mean Duration (8)
UK COVID-19
Train/Validatinn
Microphone
48
ICBHI
Train/ Validation
Stethoscope
4
44.1
22,2
SPRSound
Train/Vakle Validdttin
Stethoscope
8
11.0
COUGHVID
Test
Microphone
48
6.9
KAUH
Test
Stethoscope
4
15.0
strict cross-domain split: training/validationn on ICBHI, SPRSound, and UK COVID-19, and testing exclusively on KAUH and COUGHVID (unseen during training). This design assesses robustness across institutions, sensors, and collection protocols. Concise split statistics and per-dataset metadata appear in T A complete specification of the 16-class label space (15 disease categories plus healthy control group) is provided Appendix
4 RESP-AGENT: AN LLM--OCHESTRAATED LOOP FOR UNIFIED DIAGNOSIS AND CONTROLLABLE SYNTHESIS
The overall architecture of Resp-Agent depicted in Figure Given the paired text-audio supervision and the cross-domain split established by Resp-229k, Resp-Agent is designed centrally planned, compute-aware multi-agent system that integrates standalone audio and NLP modules into closed loop. A compute-effin efficient planner, Thinker-A*CAA (DeepSeek- V3.2-Exp: (Luu al. 2025)), performs semantic intent parsing and plan-execute tool routing using structured arguments (audio paths, EHR tables, and target classes), enforcing deterministic 1/O schemas and emitting concise, instrumented logs. Beyond dispatch, the controller reuses model rationales, error profiles, and calibrated confidence to bias subsequent actions (e.g., targeted synthesis for failure modes), thereby coupling data generation and diagnosis under tight accelerator budgets without compromising coverage reproducibility. The Thinker coordinates two task-specific agents detailed below: a Generator (Section 4.1) that synthesizes controllable respiratory audio via modality injection and conditional flow matching, and a Diagnoser (Section 4.2) that fuses clinical narratives with audio tokens through modality weaving and strategic global attention.
4.1 GENERATOR: DISCRETE-UNT PLANNING AND CFM RECONSTRUCTION
We target controllable respiratory-soune synthesis that disentangles pathological content (what to generate) from timbral style (how it should sound). The Generator follows a two-stage design. Stage I retools a unimodal LLM into a multimodal unit generator conditioned on diagnosis semantics and a reference style, as illustrated in Figure 2 Stage reconstructs high-fidelit audio from the predicted discrete units via conditional flow matching (CFM) and a neural vocoder.
4.1.1 STYLE-CONDITIONED UNIT MODELING WITH A RETOOLED LLM
We retool light text-only backbone (Qwem3-0.68) Boeen3-o.B--Buse/ Yang al. 2025 into truly multimodal unit generator and denote the trained model Resp-MLLM. The conversion relies on modality injection with trainable style projector while leaving the language backbone architecture intact. Let Z R
4

---

Published as a conference paper at ICLR 2026
DIAGNOSISI
Prompt Construction
Pneumonia
BEATS Encoder (Frozen)
Features
Temporal Pooling (Pool)
Descriptors
Style Projector (Trainable 2-layer MLP)
Reference Audio
(10s,16HHZ)
Style Embeddings
Inputs: Disentangled Content Style
Input Embedding Layer (Modality Injection)
Diagnosis Semantics
[DIAGNOSIS] Disense Semantics (d)
[AUDIO, [J..[AUDIOO 1-1
Orange Trainable Component
Blue Data, Tokens Purple Embeddings
Content
Style
Resp-MLLM Core
Output Head
(Trainable Qwen3-0.6B-Bass Backbone)
(Softmax over BEATs
Codebook size V)
Leak-free Conditioning (Training Mechanism)
Predicted Discrete
[END]
Acoustic Units
(3/1 Mas Mc)
CARRAS MASK
Prevents pecking oracle
Random Mask
1) token when predicting yı
Sampling 10%)
Figure 2: Detailed architecture of Resp-MLLLM (Stage 1 of the Generator). The model functions as style-conditioned multimodal unit generator. Top: modality injection mechanism fuses textual diagnosis semantics with acoustic style embeddings (projected from temporally pooled BEATs features) to prompt the Qwen3-0.6P 1-0--006--Bass backbone. Bottom: leak-free conditioning strategy is employed during training: random mask sampling (M ss 10%) prevents the model from peeking at oracle tokens, ensuring robust autoregressive prediction of discrete acoustic units.
be framewise BEATS features from 10s, 16 kHz reference (T'=966). We compress Z into K style descriptors and map them to the LLM hidden space via two-layer MLP:
P Pool k (2) E KxD
Estyle = StyleProj(P) E K*H
(1)
In the input, we reserve K placeholders [AUDICo], [AUDIOX-1] and replace their embeddings with rows of Estyle The mixed prompt
[IAGNOSIS]
[AUDIO)] ... (.DDIOO)-..AUUIIOKKK.
content
style
binds disease semantics a reference timbre without modifying the language stack. Resp-MLLM
then autoregressively predicts sequence of discrete acoustic units = (V1....,30 U2) from BEATS codebook of size V:
LResp
Σ
log PResp (v: yes. d, Estyle),
(2)
1=1
s.t. yı E (....., V V-13 -1..
5

---

Published as a conference paper at ICLR 2026
To avoid inadvertent teacher forcing while preserving causal decoding, we apply a lightweight masked- input scheme. Let 7 index the target unit segment and M C T be a random subset (e.g., = 10%). For each & M we replace the preceding input embedding by dedicated vector (ReATs Mass)
Ct-1 f- CIBCATs
t M.
so the model cannot peek the oracle (t-1) token when predicting Vt- Sequences terminate with [END], and padding positions excluded from the loss. This keeps the content-style interface clean and stabilizes training of Resp-MLLM.
4.1.2 CONDITIONAL FLOW MATCHING FOR HIGH-FFIDELITY WAVEFORMS
The predicted units serve as content for a CFM decoder parameterized by Diffusion Transformer (DiT), which reconstructs mel-spectrograms: waveforms are then obtained using Vocos (Siuzdak) 2024). Let X2 be the target mel and X6 (O, I) noise prior. Flow matching learns velocity field By along the linear path x, (1-1)xo + Ix1.
dx = vo(x,, c), E [0. dt
The training objective minimizes the mean-squared discrepancy between the predicted and target velocities:
LCFM (x.
[x c) (x1 - -
(5)
The condition с follows dual-path design: (i) a content stream obtained by emhedding unit indices and temporally interpolating them to the mel frame rate; and (ii) global timbre stream formed by time-averaging BEATS features and broadcasting them across time. During training, we additionally expose a short reference prefix of the ground-trut -tuthh mel in the conditioning branch (zero- padded to full length) to encourage accurate continuation while keeping the main path free-form.
(zero-padded
This two-stage design separates what generate (units governed by diagnosis and style prefixes) from how it should sound (timbre and continuation in CFM), enabling precise and editable control while remaining compatible with standard causal L.LM training and vocoder back ends.
4.2
DIAGNOSER: MODALITY WEAVING WITH STRATEGIC GLOBAL ATTENTION
4.2.1
INPUT-LEVEL MODALITY WEAVING
The architecture of Diagnoser is illustrated in Figure 3 Prior work often performs late fusion by concatenating audio and text after separate encoders, which weakens alignment and underutilizes long-context transformers. We instead weave modalities at the input: EHR tokens and fixed audio block form a single sequence so cross-modal dependencies are modeled from the first layer.
Concretely, after tokenizing the EHR, we place contiguous block of 49- audio placeholders (AUDIO_EMBED] ] at a known span. Let x be the waveform kHz, 10s via crop/pad), (BEATT(()) e R the pretrained BEATs features, and Align(-) R Tx + > deterministic crop/pad to T steps. At the Longformer embedding layer we replace the audio placeholders in place by a learned projection:
E < Aligní (BEATS(()) W,
WER E
(6)
Here [A] denotes the audio span; all other tokens (e.g., [CLS], [DESCRIPTION] EHR text, [SEP] )) use standard embeddings. This modality weaving yields single, token-aligned stream in which audio and text interact natively. For robustness we apply light token/frame dropout to text/audio before attention (defaults Piex Paudio
4.2.2 STRATEGIC GLOBAL ATTENTION
Longformer combines efficient sliding-window window attention with a sparse set of global tokens. We allocate global attention three roles: (i) the classifier [CLS]] ; (ii) sentinel for EHR context

---

Published as a conference paper at ICLR 2026
Input-Level Modality Werving (The Foundation)
Stream
BEATS far
Crp (T-406)
Leamed Projection
Text
Tokentzer
[CLS]
[ESCRRPPIIN
Text Tokenss..
[SEP]
(10%, lókilz)
(p-0.2)
Deopout
Text Embedkings
Audio
Finbecdings
Dropout 0p-0.1)
Woven Modality Sequence (Single. Token-Aligned Stream)
DESCRIPTION
Text
Audio Block Emdeeddings
Modality Weaving:
are fused single sequence the input layer
interaction.
Strateglc Global Attention (The Framework)
Global Tokens (G)
Longformer Attention Pattern
Purple Lines: Global Attention
(Mutual entire sequenoe)
Sliding- Window Attention (Local, Ffflicient)
Aadio Anchors (A)
(CLS)
DESC REECRIPPIONI
Text Tokens...
ISEP
Strategic Attention:
sliding-windor window attention spanse of globally attentive tokens ([CLS),
[DECRIPPTIO]] U A) to model long-range dependencies with lincar complexity.
Audio
(Detniled Function)
Audio Block Block (T-196
stops, 10s total)
Frime Gimmlarity 20.16 (Local Attention)
Global Audio Anchors
A a1:2u & 0, 1,...
Anchor Spacing såt 80.6 ms (Global Resolition)
Cros-Modggl Function
Text Token (egg. "wheeze")
Key
Sub-100ms Sensilivity anchor grid)
Lineue-Time Memory (svaids quadratic cost)
Direct Cross- acoustic through
Cross-Modal Modal Query via Anchor: Distant text symptoms can directly query avoiding long-range sliding window
Cross-Modal Routing
Impeuved Observability
limitations.
Figure 3: Diagnoser Architecture: Modality Weaving with Strategic Global Attention. The framework comprises three key mechanisms: (1) Input-Level Modality Weaving: EHR text tokens and projected audio features (extracted via BEATS) are fused into a single token-aligned sequence the input layer, enabling native cross-modal interaction. (2) Strategic Global Attention: A Longformer backbone combines efficient
attention with a sparse set of global tokens, which includes textual sentinels and distributed Audio Anchors to model long-range dependencies with linear complexity. (3) Audio Anchor Mechanism: Anchors act as cross cross-modal hubs spaced at 888ms intervals, allowing distinct text symptoms (e.g., "wheeze") to directly query transient acoustic events, thereby capturing fine- tin-igrinedd temporal structures without quadratic computational costs.
[DESCRIPTION] and (iii) stride-sampled audio "anchors" within the woven block. Let 01, 01,..., index audio positions and s be the stride (default s=4). The global set
A {ai+ks k 0, ... [(T (1/1s//s]
(7)
G = {pos( (pos((CLS)), pos( (DESCRIPTION]] U A.
7

---

Published as a conference paper at ICLR 2026
There is mutual attention between tokens in 9 and the entire sequence; all others remain local. This pattern preserves linear-time memory while creating cross-modal hubs that support long-range evidence flow. Textual symptoms (c.g., "nocturnal dry cough") can directly query transient acoustic events even when far apart.
For 10s segment embedded into T=496 time steps, the per-step hop is A 10,000 ms/ /496 ~ 20. 2031613 ms. With stride s=4, adjacent global anchors are spaced by 8 At = x 20.1613 = 80.6452 ms. Thus, the globally queryable alignment grid affords an 80.6 ms temporal resolution, with worst-case deviation < s21/2 s 40.3 ms when snapping an event to its nearest anchor. Meanwhile, local attention continues to represent features at the frame granularity At = 20.16ms. This strategically sparse global pattern yields sub-100 ms sensitivity to brief, low-energy respiratory phenomena (e.g., wheeze onsets, crackles) while avoiding quadratic costs. contrast to designs that rely solely on [CLS] purely local windows, anchor-based global tokens enable predictable cross-modal routing and improve the observability of rare transients, as shown Table S
5 EXPERIMENTS
We empirically evaluate Resp-Agent on two complementary benchmarks: (i) ICBHI 4-class res- piratory sound classification under the official 60-40% split, and (ii) Resp-229k, our large-scale, cross-domain, 16-class benchmark with a strict held-out test set (Test-CD). Our experiments are organized around three questions: (Q1) Does the proposed multimodal Diagnoser outperform strong unimodal and shallow-fusion basclines? (Q2) Is Thinker-guided controllable synthesis necessary beyond traditional imbalance remedies and simpler planners, and does generalize under severe domain shift? (Q3) Are the Generator and Diagnoser architectures themselves responsible for the observed gains, rather than incidental implementation choices?
Table 2: RSC performance on the ICBHI dataset using the official 60 60-40% train-test split. In the Pretraining Data column, IN, AS, LA, HF, and SPR refer to ImageNet (Deng et al. 2009). AudioSet (Gemmeke et al. 2017), LAION-Audio-633 2021). and SPRSound, respectively. * denotes the previous state-of-the of-ee--f-the-ar the-art ICBHI Score. The SOTA and second-bes best results are highlighted bold and by underlining, respectively.
(Wu et al. 2023),
(Hsu et al.
Method
Backbone
Pretraining Data
Sp (5))
S. (S)
Score (%)
SE+SA Yang al.
ResNet18
81.25
17.84
49.55
LungRN=NI.
et
2020
ResNer-NL
63.20
41.32
$2.26
RespireNer Gairola Garrlla al
2122
ResNer34
72.30
40.10
$6.20
Chang et al. Chang ct al
2022
CNN8-dilated
69.92
35.85
$2.89
Ren et al. Ren
2022
CNN8-Pt
72.96
27.78
50,37
Wang et ol. Wang & Wang
2022
ResNeSi
IN
70.40
40.20
55,30
Late-Fuusion Phum al. 2022
Inc-03 VGG14
IN
85.60
30.00
57.30
Nguyen et al. Nguyen & Pernkopf 2022
ResNet50
IN
79.34
37.24
58.29
Moummad al. Moummad Farrugia
2023
CNN6
AS
75.95
39.15
$7.55
[Bac
2023
AST
IN+AS
81.66
43.07
62.37
Kim Kim 2023
AST
IN+AS
80.72
42.86
61.79
Kim
al.
20244
AST
IN+AS
79.87
43.55
61.71
Kim et al. Kim et.
20246
AST
IN+AS
82.47
40.55
61.51
BTS [Km al 2024c
CLAP
$1.40
45.67
63.54
Wang et
Wang ci al.
2024
HTS-AT
IN+AS
79.61
48.77
64.19
MVST [He ef al. 21124
AST
IN+AS
81.99
$1.10
66.55
Dong et al. Dong ct
2025
AST
IN+AS
85,99
49.11
67.55'
Resp-Agent(Ours)
LLMMooogformer
HF+SPR
79.29
66.10
72.70
Main diagnostic performance. On ICBHI, Resp-Agent attains a score of 72.7 (Sp = 79.3, Se 1). surpassing the best prior audio models by more than 5 absolute points on the official leaderboard while using a distinct LLM+Longformer backbone. This indicates that anchor-aware multimodal reasoning is competitive even against heavily pre-trained audio transformers. On Resp-229k, Table g in Appendix B summarizes performance under the original imbalanced and canerator-balancee regimes. The audio-only Conformer(Gllati et al., 2020) baseline reaches 0.720/0.72 782 Accuracy and 0.1935/0.5360 Macro-F1 (original/balanced)) while the full Resp-Agent Diagnoser achieves 0.8494/0.888 8870 Accuracy and 0.21 18/0.9999 Macro-F1. The balanced regime thus converts generative
8

---

Published as a conference paper at ICLR 2026
augmentation into substantial macro-Fl gains minority conditions without sacrificing overall accuracy.
We next fix the Diagnoser and Generator architectures and vary only the planner policy that allocates a synthetic budget B over label and domain combinations on Resp-229K/TestCDD The consolidated results reported in Table
Table 3: Summary of downstream Diagnoser performance on Test-CD under different planner policies and imbalance remedies. Macro-Fl tail computed over the 8 rarest classes. "_" indicates that LoSO results are not reported for that setting due to the computational cost of retraining across all five folds.
Setting
Method
B (k)
Acc
Macro-F1
Macro-Flum
LoSO Macro-FI / tail
Planner policies under matched budget (Exp.
Test-CD
No-Synth (CE)
0.849
0.212
0.074
0.237 0.086
Test-CD
Random
50
0.869
0.442
0.291
Test-CD
Class-Prior
50
0,876
0.512
0.349
0.473 / 0.334
Test-CD
Uncertainty-Star Static
50
0,881
0.546
0.376
Test-CD
Thinker-A CCA
50
0.887
0.598
0.421
0.532 0,383
Factorized planner variants (Exp.
Test-CD
Rare-only Hard-Casc-only
30
0.873
0.489
0.381
Test-CD
30
0.878
0.512
0.371
Test-CD
Hard-Domaie Domain-only
30
0.876
0.506
0.356
Test-CD
Rare x Hard-Dom.
30
0,881
0.528
0.397
Test-CD
Thinker-A CCA
30
0.883
0.541
0.409
Non-generative vs. generative imbalance remedies (Exp. 4)
Test-CD
CE (Baseline)
0,849
0.212
0,074
Test-CD
Class- Weighte CE
0.842
0.248
0.114
Test-CD
Focal Loss (y=2)
0.839
0.267
0.129
Test-CD
CE + Class-Prior
50
0.876
0.512
0.349
Test-CD CE + Thinker-A
50
0.887
0.598
0.421
Experiment evaluates generative planners under matched budget of B = 50k synthetic clips. All planners consistently improve over the no-synthesis baseline (Macro-F1 0.212). Class-prior rebalancing reaches Macro-F1 of 0.512, and static uncertainty sampling further improves it to 0.546, confirming that both label balancing and error-aware targeting are effective. However, the Active Adversarial Curriculum Agent (Thinker-A' CCA) achieves a Macro-Fl of 0.598 and a Macro- Flail of 0.421, yielding sizable gains of +0.052 Macro-FI and +0.045 Macro-Fl, Flait over the strong static-uncertainty baseline at the same budget.
Experiment decomposes Thinker-A into single- factor planners under a matched budget of B = 30k. Focusing only rare labels (Rare-only) already lifts Macro-F1 from 0.212 to 0.489, while targeting hard cases or hard domains yields Macro-F1 scores of 0.512 and 0. 0.506, respectively. Combining rarity and domain difficulty (Rare x Hard-Domain) is the strongest single-facte heuristic (Macro-Fl 0.528; Macro-F1a, cail 0.397). Yet Thinker-A ACAA still improves further to 0.541/0.409, demonstrating that its iterative, multi-factor planning cannot be reduced any single handcrafted heuristic.
Experiment studies sample efficiency by sweeping B E (0, 10k, 20k, 30k, 50k). All planners exhibit diminishing returns, but Thinker-A is markedly more sample-efii cample-efficient: at B = 10k, it already achieves a Macro-F1 of 0.412 (sz 52% of its total gain over the baseline), compared with 0.378 for class-prior rebalancing and 0.331 for random sampling. This suggests that small number of high-value, Thinker-selected clips more beneficial than a much larger pool drawn by simpler policies.
Experiment compares Thinker-guided synthesis against non non-generattiv imbalance remedies at B 0. Class-weighted cross-entropy and focal loss (Lin et al. 2017) improve Macro-F1 from 0.212 to 0.248 and 0.267, respectively, and nearly double Macro-FI. ai (from 0.074 to 0.1 129). However, the best non-generative method remains 0.331 Macro-F1 below the CE Thinker-A combination at B = 50k (Macro-Fi 0.598), indicating that the dominant source of improvement is the synthetic data itself rather than loss re-geigh

---

Published as a conference paper at ICLR 2026
Table 4: Compact summary of Generator content-style disentanglement
Generator disentanglement (Exp. 6)
Test
Style-Sim
P-Ace t
FAD +
Style-swap (avg over 4 styles)
0.91
97.9%
1.18
Content-swap (avg over 4 labels)
0.93
96.1%
1.19
Diagnoser ablations on Test-CD (Exp. 7)
Config
Acc
Macro-Fi
Late Fusion, Raw Metadata, no anchors Late Fusion, LLM EHR, no anchors
0.780
0.145
0.790
0.160
Modality Weaving, Raw Metadata, anchors
0.640
0.175
Modality Weaving. LLM EHR, anchors Modality Weaving. Metadata, anchors Full Resp-Agent Diagnoser (LLM EHR anchors)
0,650
0.189
0.835
0.195
0.849
0.212
Experiment conducts a rigorous Leave-One-SSurce--e (LoSO) evaluation across the five con- stituent datasets (UK COVID-19, ICBHI, SPRSound, COUGHVID, and KAUH). Averaged over folds. Macro-F1 / MMcro-Flaa tail is 0.237 / 0.086 for the no-synthesis baseline, 0.473 0.334 for class-prior rebalancing, and 0.532 / 0.383 for Thinker-A&CA (Table 3 The consistent ordering Thinker-A CCA > Class-Prior > No-Synth across all held-out sources confirms that the planner's benefits are robust across sources rather than split-specific.
Experiment 6 validates that the Generator can independently control pathological content and acoustic style. the style-swap setting, we fix the pathology to rare label ("Bronchiolitis") and vary the style reference across four cross-domain clips. The Generator achieves high style similarity (Style-Sim 0.89,,.92), while the held-out Diagnoser preserves the target pathology with Pathology-Ace of 97.5-98. 1% and a Fréchet Audio Distance (FAD) of 1.14-1.21. In the complementary content- swap setting, a single "Control Group" style reference held fixed while synthesizing four different pathologies. Style-Sim remains stable at = 0.93 0.94,, and Pathology-Acc remains high 9448-99.22% with FAD in the range of 1.17-1.22 Together, these tests provide quantitative evidence that the two-stage Generator disentangles semantic content from recording style and can reliably instantiate rare pathology-style combinations.
Experiment 7 ablates the Diagnoser architecture Test-CD by varying text quality, fusion strategy, and attention anchors. Table Jsoows that replacing raw metadata with LLM-rendered EHR yields modest but consistent gains (Accuracy modality weaving are present). A simple late-fusion baseline with LLM EHR reaches an Accuracy of 0.790 and Macro-FI of 060 Modality weaving without anchors improves Macro-F1 destabilizes the architecture, reducing Accuracy to 0.650. Reintroducing strategic audio anchors in the full Resp-Agent Diagnoser restores stability and further improves performance to an Accuracy of 0.849 and Macro-F1 I of 0.212. These results confirm that the gains arise from deliberate co-design: high-quality clinical text, tight modality weaving, and anchor- based global attention are all necessary to fully exploit the synthetic data delivered by the Thinker-guice Generator.
and Macro-F1
when anchors and
189 but
CONCLUSION
We present Resp-Agent, centrally orchestrated, closed-loop multi-agent framework that unifies con- trollable, figh-fidelity respiratory sound synthesis with multimodal disease diagnosis. Our framework underpinned by Resp-229k, large-scale cross-domain benchmark with strict evaluation protocol. curriculum-awar Thinker-A planner decomposes diagnostic goals and allocates them between controllable multimodal Generator and a modality-weaving Diagnoser, turning previously isolated modules into closed analyze-synthesize loop. From a methodological standpoint, Resp-Agent instantiates general principle for data-scarce medical domains: tightly coupling where the model fails with what the generator synthesizes transforms augmentation from passive remedy into an active, self-correcting training curriculum. We envision clinician-in-ther -tthe---opp deployments where Resp-Agent synthesizes edge-case exemplars and delivers audio-in andio-iforrmed decision support at the point of care, contributing toward trustworthy and equitable medical-audio AI.
10

---

Published as a conference paper at ICLR 2026
ETHICS STATEMENT
All authors have read and adhere to the ICLR Code of Ethics. This work relies exclusively on publicly available, previously de-identified data; no new human-subject data were collected, and no Personally Identifiable Information (PII) or Protected Health Information (PHI) was accessed or processed at any stage. Below, we detail the provenance, licensing, and intended usage of the data ensure transparency and reproducibility.
Data Provenance and Privacy. The Resp-229k benchmark is curated from multiple public respiratory-sound corpora, including ICBHI, SPRSound, UK COVID-19, COUGHVID, and KAUH. We additionally use HF Lung_V V1 as an external, publicly available pretraining resource for model initialization; is not included in Resp-229k. Each recording retains explicit provenance, including the source dataset identifier, device metadata (e.g., electronic stethoscope vs. microphone), and sampling rate. ensure rigorous evaluation, we enforce a strict cross-institution and cross-device protocol: the training and validation sets are derived exclusively from ICBHI, SPRSound, and UK COVID-19, whereas the test set consists solely of unseen recordings from KAUH and COUGHVID. All source data were de-identified by their original custodians prior to public release.
Licensing and Compliance. We strictly adhere to the original licenses and terms of use for all constituent datasets. Specifically, our usage complies with the Open Government Licence v3.0 (UK COVID-19), Creative Commons Attribution 4.0 International (CC BY 4.0) (COUGHVID, HF Lung_V II, KAUH, SPRSound), and CCO Public Domain Dedication (ICBHI Our derived code, models, and synthetic examples have been released under open-source terms that are compatible with these licenses to facilitate reprod reproducible research while preserving data privacy.
Intended Use and Safety. The Resp-Agent system and the Resp-229k benchmark are developed strictly for research purposes. This system is not certified medical device and has not undergone regulatory approval for clinical use. It must not be deployed to make diagnostic decisions or influence patient care without appropriate clinical validation and regulatory oversight.
REPRODUCIBILITY STATEMENT
We have taken extensive steps facilitate reproducibility. All source code, including training and inference scripts and configuration files with exact commands to reproduce reported results, publicly available hht:ps:/ Resp-229k dataset is released at attttt::/httttiiteeeeo resp-agent-datasett Trained Trained model checkpoints are hosted at co co/AAstinnhaag/ressg agent models) Architectural and algorithmic details are specified the main text, while complete hyperparameters, optimizer settings, and training schedules are consolidated in the appendix.
com/ wttp////fooooeeeeeepeppeep Agent The curated hotaeeeetttttttttttstttttt.omo
REFERENCES
Sangmin Bac, June-Wee Woo Kim, Won- Yang Cho, Hyerim Back, Soyoun Son, Byungjo Lee, Changwan Ha. Kyongpil Tac. Sungnyun Kim, and Seyoung Yun. Patch-Mix contrastive learning with audio spectrogram transformer on respiratory sound classification. In International Speech Communica- fion Association (Interspeech), pp. 1443-5540, 2023.
Sebastien Baur, Zaid Nabulsi, Wei-Hung Weng, Jake Garrison, Louis Blankemeier, Sam Fishman, Christina Chen, Sujay Kakarmath, Minyoi Maimbolwa, Nsala Sanjase, et al. HeAR health acoustic representations. arXiv preprint arX1v:2403.00522. 2024.
Iz Beltagy, Matthew E Peters, and Arman Cohan. Longformer: The long-document transformer. arXiv preprint arX:::2004.0 Xiv:2004.05/500. 2020.
'The ICBHI 2017 Respiratory Sound Database is described as "freely available for research" on its official
wehsite (https: //bbi /hahichhhha 1 med. auth, gr ICBH 2011_217__00 Cha lenge). The CCO designa- tion follows the license annotation adopted by prior benchmarking work (Zhang et al. (024a).
11

---

Published as a conference paper at ICLR 2026
Abraham Bohadana, Gabriel Izbicki, and Steve S Kraman. Fundamentals of lung auscultation. New England Journal of Medicine, 370(8):744-75)) -51, 2014.
Zalán Borsos, Raphaël Marinier, Damien Vincent, Eugene Kharitonov, Olivier Pietquin, Matt Sharifi, Dominik Roblek, Olivier Teboul, David Grangier, Marco Tagliasacchi, et al. AudioLM: language modeling approach to audio generation. JEEE/ACM Transactions Audio, Speech, and Language Processing, 31:2523-20 1122532-253,,.0 2023.
Jobie Budd, Kieran Baker, Emma Karoune, Harry Coppock, Selina Patel, Richard Payne, Ana Tendero Canadas, Alexander Titcomb, David Hurley, Sabrina Egglestone, et al. A large-scale and PCR-referenced vocal audio dataset for COVID-19. Scientific Data, 11(1):700, 2024.
Yi Chang, Zhao Ren, Thanh Tam Nguyen, Wolfgang Nejdl, and Björn W. Schuller. Example-based explanations with adversarial attacks for respiratory sound analysis. In International Speech Communication Association (Interspeech), pp. 4003-4007, 2022.
Sanyuan Chen, Yu Wu, Chengyi Wang, Shujie Liu, Daniel Tompkins, Zhuo Chen, Wanxiang Che, Xiangzhan Yu, and Furu Wei. BEATs: audio pre-training with acoustic tokenizers. Proceedings of the International Conference on Machine Learning (ICML), pp. 5178-5193, 2023.
Harry Coppock, George Nicholson, Ivan Kiskin, Vasiliki Koutra, Kieran Baker, Jobie Budd, Richard Payne, Emma Karoune, David Hurley, Alexander Titcomb, et al. Audio-based Al classifiers show no evidence of improved COVID-19 screening over simple symptoms checkers. Nature Machine Intelligence, 6(2):229-244) 2024.
Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. ImageNet: A large-scale hierarchical image database. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 248-255, 2009.
Jacob Devlin, Ming-Weiee Wei Chang, Kenton Lee, and Kristina Toutanova. BERT: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT)) pp. 4171 4186, 2019.
Gaoyang Dong, Yufei Shen, Jianhong Wang, Mingli Zhang, Ping Sun, and Minghui Zhang, Respira- tory sounds classification by fusing the time-domain and 2d spectral features. Biomedical Signal Processing and Control, 107: 107:90,790, 2025.
Zach Evans, Julian D Parker, CJ Carr, Zack Zukowski, Josiah Taylor, and Jordi Pons. Stable audio open. In Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 1-5, 2025.
Mohammad Fraiwan, Luay Fraiwan, Basheer Khassawnch, and Ali Ibnian. A dataset of lung sounds recorded from the chest wall using electronic stethoscope. Data in Brief, 15:106913, 2021.
Siddhartha Gairola, Francis Tom, Nipun Kwatra, and Mohit Jain. RespireNet: A deep neural network for accurately detecting abnormal lung sounds in limited data setting. In Proceedings of the Anmual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), pp. 527-530, 2021.
Jort F Gemmeke, Daniel PW Ellis, Dylan Freedman, Aren Jansen, Wade Lawrence, R Channing Moore, Manoj Plakal, and Marvin Ritter. Audio Set: An ontology and human-labeled dataset for audio events. In Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 776-780, 2017.
Yuan Gong, Yu-An Chung, and James Glass. AST: Audio spectrogram transformer. In International Speech Communication Association (Interspeech), pp- 571-575, 2021.
Anmol Gulati, James Qin, Shung-Cheng Chiu, Niki Parmar, Yu Zhang, Jiahui Yu, Wei Han. Shibo Wang, Zhengdong Zhang, Yonghui Wu, et al. Conformer: Convolution-augmeenee transformer for speech recognition. In International Speech Communication Association (Interspeech), pp. 5036-5040, 2020.
12

---

Published as a conference paper at ICLR 2026
Daya Guo, Dejian Yang. Haowei Zhang, Junxiao Song, Peiyi Wang, Qihao Zhu, Runxin Xu, Ruoyu Zhang. Shirong Ma, Xiao Bi, et al. DeepSeek-RI: Incentivizing reasoning capability in llms via reinforcement learning. arXiv preprint 2i:1: 2025.
Wentao He, Yuchen Yan, Jianfeng Ren, Ruibin Bai, and Xudong Jiang. Multi-view spectrogram trans- former for respiratory sound classification. In Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 8626 8630, 2024.
Julien Heitmann, Alban Glangetas, Jonathan Doenz, Juliane Dervaux, Deeksha M. Shama, Daniel Hin- jos Garcia, Mohamed Rida Benissa, Aymeric Cantais, Alexandre Perez, Daniel Müller, et al. DeepBreath-auommade detection of respiratory pathology from lung auscultation in 572 pediatric outpatients across 5 countries. NPJ Digital Medicine, 6(1):104, 2023.
Fu-Shun Hsu, Shang- Ran Huang, Wen Huang, Chao-Jung Huang, Yuan- Cheng, Chun- Chich Chen, Jack Hsiao, Chung-Wet Chen, Li-Chin Chen, Yen-Chun Lai, et al. Benchmarking of eight recurrent neural network variants for breath phase and adventitious sound detection on a self-developed open-access lung sound database- HF Lung V1. PLOS ONE, 16(7)xe0254134 2021.
Dong-Min Huang, Jia Huang, Kun Qiao, Nan-Shan Zhong, Hong-Zhou Lu, and Wen-Jin Wang. Deep learning-based -baed lung sound analysis for intelligent stethoscope. Military Medical Research, 10(1): 44, 2023.
Justin M Johnson and Taghi M Khoshgoftaar. Survey on deep learning with class imbalance. Journal of Big Data, 6(1):27, 2019.
June- June-Woo Kim, Chihyeon Yoon, Miika Toikkanen, Sangmin Bae, and Ho- Young Jung. Adver- sarial fine-tuning using generated respiratory sound to address class imbalance. arXiv preprint arXiv:2311.06-48 2023.
June-Woo Kim. Sangmin Bae, Won- Yang Cho. Byungjo Lee, and Ho- Young Jung. Stethoscope- guided supervised contrastive learning for cross-domain adaptation on respiratory sound classi- fication. In Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 1431-1435, 2024a.
June-Wor Kim, Miia Toikkanen, Sangmin Bae. Minscok Kim, and Ho- Young Jung. RepAugment: Input-agnostic representation-leve augmentation for respiratory sound classification. In Proceed- ings of the Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), pp. 1-6, 2024b.
June-Wor Kim, Miia Toikkanen, Yera Choi, Scoung- Eun Moon, and Ho- Young Jung. BTS: Bridging text and sound modalities for metadata-aided respiratory sound classification. In International Speech Communication Association (Interspeech), pp- 1690-1694, 2024c.
Pang Wei Koh, Shiori Sagawa, Henrik Marklund, Sang Michael Xie, Marvin Zhang, Akshay Bal- subramani. Weihua Hu, Michihiro Yasunaga, Richard Lanas Phillips, Irena Gao, et al. WILDS: benchmark of in-the-wild distribution shifts. In Proceedings of the International Conference on Machine Learning (ICML). pp. 5637-5664, 2021.
Qiuqiang Kong, Yin Cao, Turab Iqbal, Yuxuan Wang, Wenwu Wang, and Mark D. Plumbley. PANNs: Large-scale pretrained audio neural networks for audio pattern recognition. /EEE/ACM Transactions on Audio, Speech, and Language Processing. 18:2880-2894 2020.
Chae Young Lee, Anoop Toffy, Gue Jun Jung, and Woo-Jin Han. Conditional WaveGAN. arXiv preprint 18Xi 10636, 2018
Conglong Li, Zhewei Yao. Xiaoxia Wu, Minjia Zhang, Connor Holmes, Cheng Li, and Yuxiong He. DeepSpeed data efficiency: Improving deep learning model quality and training efficiency via efficient data sampling and routing. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, pp. 18490-18498, 2024.
Tsung- Yi Lin, Priya Goyal, Ross Girshick. Kaiming He, and Piotr Dollár. Focal loss for dense object detection. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pp. 2980-2988, 2017.
13

---

Published as a conference paper at ICLR 2026
Yaron Lipman, Ricky T. Q. Chen, Heli Ben-Hamu, Maximilian Nickel, and Matt Le. Flow matching for generative modeling. In Proceedings of the International Conference on Learning Representa- tions (ICLR), 2023.
Aixin Liu, Aoxue Mei, Bangcai Lin, Bing Xue, Bingxuan Wang. Bingzheng Xu, Bochao Wu, Bowei Zhang, Chaofan Lin, Chen Dong, al. -33. Pushing the frontier of open large language models. arXiv preprint 25Xiv:2/2..055:5 2025.
Haohe Liu, Yi Yuan, Xubo Liu, Xinhao Mei, Qiuqiang Kong, Qiao Tian, Yuping Wang, Wenwu Wang, Yuxuan Wang, and Mark D Plumbley. AudioLDM Learning holistic audio generation with self-supervised pretraining. IEEE/ACM Transactions on Audio, Speech, and Language Processing, 32:2871-2883, 2024.
Xingchao Liu, Chengyue Gong, and Qiang Liu. Flow straight and fast: Learning to generate and transfer data with rectified flow. In Proceedings of the International Conference on Learning Representations (ICLR), 2023.
Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. RoBERTa: A robustly optimized BERT pretraining approach, arXiv preprint arXiv: 1692, 2019.
Yi Ma, Xinzi Xu, and Yongfu Li. LungRN+NL: An improved adventitious lung sound classification using non-local block resnet neural network with mixup data augmentation. In International Speech Communication Association (Interspeech), pp. 2902-2906, 2020.
Ilyass Moummad and Nicolas Farrugia. Pretraining respiratory sound representations using meta- data and contrastive learning. In Proceedings of the IEEE Workshop Applications of Signal Processing to Audio and Acoustics (WASPAA), pp. 1-5, 2023.
True Nguyen and Franz Pernkoff. Lung sound classification using co-tuning and stochastic normal- ization. IEEE Transactions on Biomedical Engineering, (9(9)22772-888)) 2022.
Lara Orlandic, Tomas Teijeioo, and David Atienza. The COUGHVID crowdsourcing dataset, corpus for the study of large-scale cough analysis algorithms. Scientific Data, 8(1):166, 2021.
Kuldip Paliwal, Kamil Wojcicki, and Benjamin Shannon. The importance of phase in speech enhancement. Speech Communication, 53(4),465 494, 2011.
Daniel S Park, William Chan, Yu Zhang, Chung-Chert Chiu, Barret Zoph, Ekin D Cubuk, and Quoe Le. Sugment: A simple data augmentation method for automatic speech recognition. In International Speech Communication Association (Interspeech), pp. 2613-2617, 2019.
William Peebles and Saining Xie. Scalable diffusion models with transformers. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV) pp. 4195 4205, 2023.
Lam Pham, Dat Ngo, Khoa Tran, Truong Hoang, Alexander Schindler, and Ian McLoughlin. An ensemble of deep learning frameworks for predicting respiratory anomalies. In Proceedings of the Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), pp. 4595 4598, 2022.
Davide Pigoli, Kieran Baker, Jobie Budd, Lorraine Butler, Harry Coppock, Sabrina Egglestone, Steven G Gilmour, Chris Holmes, David Hurley, Radka Jersakova, et al. Statistical design and anal- ysis for robust machine learning: a case study from COVID-19. arXiv preprint 2x21:2211..08571, 2022.
Alec Radford, Jong Wook Kim, Tao Xu, Greg Brockman, Christine McLeavey, and Ilya Sutskever. Robust speech recognition via large-scale weak supervision. In Proceedings of the International Conference on Machine Learning (ICML), pp. 28492-28518. 2023.
Zhao Ren, Thanh Tam Nguyen, and Wolfgang Nejdl. Prototype learning for interpretable respiratory sound analysis. In Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 9087-9091, 2022.
14

---

Published as a conference paper at ICLR 2026
BM Rocha, Dimitris Filos, Lea Mendes, loannis Vogiatzis, Eleni Perantoni, Evangelos Kaimakamis, P Natsiavas, Ana Oliveira, C Jácome, A Marques, et al. A respiratory sound database for the development of automated classification. In Proceedings of the International Conference Biomedical and Health Informatics, pp. 33-37. Springer, 2017.
Bruno M Rocha, Dimitris Filos, Luís Mendes, Gorkem Serbes, Sezer Ulukaya, Yasemin P Kahya, Nikša Jakovljevie, Tatjana L Turukalo, Ioannis M Vogiatzis, Eleni Perantoni, al. An open access database for the evaluation of respiratory sound classification algorithms. Physiological Measurement, 40(3):33001, 2019.
Hubert Siuzdak. Vocos: Closing the gap between time-domain and fourier-based neural vocoders for high-quality audio synthesis. In Proceedings of the International Conference Learning Representations (ICLR), 2024.
Jianhong Wang, Gaoyang Dong, Yufei Shen, Minghui Zhang, and Ping Sun. Lightweight hierarchical transformer combining patch-random and positional encoding for respiratory sound classification. In Proceedings of the International Conference on Signal and Image Processing (ICSIP), pp.
580-584, 2024.
Zijie Wang and Zhao Wang. A domain transfer based data augmentation method for automated respiratory classification. In Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 9017-9021, 2022.
Yusong Wu, Ke Chen, Tianyu Zhang, Yuchen Hui, Taylor Berg-Kirkpatriiee Kirkpatrick, and Shlomo Dubnov. Large-scale contrastive language-audio pretraining with feature fusion and keyword-to-caption augmentation. In Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 1-5, 2023.
Tong Xia, Jing Han, and Cecilia Mascolo. Exploring machine learning for audio-based respiratory condition screening: A concise review of databases, methods, and open issues. Experimental Biology and Medicine, 247(22):2053-206) 2022.
An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, et al. Qwen3 technical report. arXiv preprint 000392505.09388 2025.
Zijiang Yang, Shuo Liu, Meishu Song, Emilia Parada-Cabaled and Bjöm W. Schuller. Adventi- tious respiratory classification using attentive residual neural networks. In International Speech 2020.
Communication Association (Interspeech), pp.
Manzil Zaheer, Guru Guruganesh, Kumar Avinava Dubey, Joshua Ainslic, Chris Alberti, Santiago Ontanon, Philip Pham, Anirudh Ravula, Qifan Wang. Li Yang. et al. Big Bird: Transformers for longer sequences. Advances in Neural Information Processing Systems, 33:17283- 2020.
Qing Zhang, Jing Zhang, Jiajun Yuan, Huajie Huang, Yuhang Zhang, Baoqin Zhang, Gaomei Lv, Shuzhu Lin, Na Wang, Xin Liu, et al. SPRSound: Open-source SJTU paediatric respiratory sound database. IEEE Transactions on Biomedical Circuits and Systems, 16(5):867-881, 2022.
Yuwei Zhang. Tong Xia, Jing Han, Yu Yvonne Wu, Georgios Rizos, Yang Liu, Mohammed Mosuily, Jagmohan Chauhan, and Cecilia Mascolo. Towards open respiratory acoustic foundation models: Pretraining and benchmarking. Advances Neural Information Processing Systems, 37:27024- 27055, 2024a.
Yuwei Zhang, Tong Xia, Aaqib Saced, and Cecilia Mascolo. RespLLM: Unifying audio and text with multimodal Ims for generalized respiratory health prediction. arXiv preprint 10.0v:2410.05361, 2024b.
15

---

Published as a conference paper at ICLR 2026
Table 5: Unified 16-class taxonomy and sample distribution for Resp-229k (raw N 238,074), The counts highlight the severe long-tail tail imbalance.
Class name (unified)
Total count
Control Group
156,527
COVID-19
77.994
Pneumonia
1,909
COPD
820
Asthma
324
Bronchitis
188
Bronchiectasis
103
Hemoplysis
65
Other respiratory discases
49
URTI
42
Bronchiolitis
18
Pulmonary hemosiderosis
13
Chronic cough
11
Airway foreign body
Kawasaki discase
LRTI
Total
238,074
A LABEL SPACE
Taxonomy Note. The raw Resp-229k corpus is constructed by aggregating heterogeneous public respiratory-sound datasets and inheriting their original diagnostic labels. Across sources, this yields an initial 20-class label space, including several synonymous or overly fine- fine-grained categories (e.g.. severity-specife pneumonia labels). The raw corpus contains 238,074 clips; after discarding corrupted or too-short recordings, 229,101 101 quality-controlled clips remain and are used for all main-paper experiments.
To enhance clinical coherence and facilitate a more robust analysis label imbalance, we program- matically consolidate the original 20-class label space into clinically unified 16-class taxonomy. The unification map is applicd once at preprocessing time before any dataset splitting, ensuring that each recording is assigned unique, consistent diagnosis across training, validation, and test splits. Concretely, the mapping is defined as:
Bronchiectasis: all samples labeled "Bronchiectasia" are relabeled as "Bronchiectasis".
URTI: all samples labeled "Acute upper respiratory infection" are relabeled as "URTI".
Pneumonia: all severity-specifi pneumonia labels (e.g.. "Pneumonia (non-severe)", "Pneu- monia (severe)", "Pneumonia (unspecified)") are relabeled to the parent class "Pneumonia".
After applying this unification, a full recount of the 238,074 raw recordings yields the class distribution summarized in Table The resulting label space comprises 15 disease categories and one healthy control group: Airway foreign body, Asthma, Bronchiectasis, Bronchiolitis, Bronchitis, COPD (Chronic Obstructive Pulmonary Disease), COVID-19, Chronic cough, Hemoptysis, Kawasaki disease, LRTI (Lower Respiratory Truct Infection), Pneumonia, Pulmonary hemosiderosis, URTI (Upper Respiratory Tract Infection), Other respiratory diseases, and Control Group (healthy).
This distribution makes explicit the extreme class imbalance present in Resp-229k, with the majority of diagnostic categories residing in the long tail. This motivates the generative, agent-based rebalancing strategy pursued in the main paper.
B SUPPLEMENTAL RESULTS
The Dual Challenge: Data Scarcity and Transient Event Localization. Under the strict cross- domain protocol (KAUH+COUGHVID held out), the evaluation reveals two fundamental barriers to real-world deployment: (i) the representation gap, where standard encoders fail to localize brief, low-energy events (e.g., crackles) within the 16-class taxonomy (Appendix A). and (ii) the data gap,
16

---

Published as a conference paper at ICLR 2026
causing minority underdiagnosis due to severe label skew. To rigorously validate our architectural and generative solutions, we conducted comprehensive ablation studies.
B.1 VALIDATION OF UNIMODAL BASELINES AND FUSION STRATEGIES
Robustness of Text Baselines (Experiment 1). To preclude the possibility that the multimodal gains of Resp-Agent stem merely from weak textual baseline, we benchmarked strong Transformer-based text encoders against the audio-only Conformer. As detailed Table while modern Transformers (BeRT et al. 2019), RoBERTa(Liu al.. 2019), Longformer- -eetmer---ee))))) significantly outperform the LSTM baseline (0.0401 + 0.0813 Macro-FI), they remain substantially inferior to the audio-only Conformer (0. 1935 Macro-F1). This large modality gap confirms that textual clinical summaries alone are insufficient for accurate diagnosis and that our system's 's performance derives from effective cross-modal synergy rather than "strawman" text baseline.
Table 6: Performance comparison of text-only, audio-only, and multimodal models on the cross- domain test set (original, imbalanced data). Text-only Transformers improve over LSTM but fail to match audio-only baselines, justifying the need for multimodal fusion.
Model
Modality
Accuracy
Macro-F1
LSTM (main paper baseline)
Text
0.0912
0.0401
BERT-base
Text
0.1420
0.0710
RoBERTa-base
Text
0.1513
0.0742
Longformer-base (text-only)
Text
0.1585
0.0813
Conformer (audio-only)
Audio
0.7200
0.1935
Resp-Agent Diagnoser (Ours)
Audio + Text
0.8494
0.2118
Efficacy of Modality Weaving and Audio Backbones (Experiment 2). We further scrutinized the contribution of our fusion architecture versus the choice of audio backbone. Table 7 compares our Modality Weaving against standard late-fusion strategies (concatenation of embeddings and logit voting). Simple fusion yields marginal gains over the audio baseline (Macro-Fl = 0.20). In contrast, our deep weaving mechanism achieves superior integration (Macro-F1 0.2118), confirming that architectural interleaving is crucial for grounding textual symptoms in acoustic features. Furthermore, replacing the Conformer with a improvement in the audio-only setting (0. (0.2010 Macro-FI) and does not close the gap to the full multimodal Diagnoser. This indicates that the system's 's robustness driven primarily by the Modality Weaving and Strategic Global Attention mechanisms rather than the specific acoustic encoder.
et al., 2023) encoder yields only slight
Table 7: Ablation fusion strategies and audio encoder backbones (original, imbalanced data). Deep Modality Weaving outperforms shallow fusion methods, and the choice of fusion architecture outweighs marginal gains from changing the audio backbone.
Model / Fusion Strategy
Modality
Accuracy
Macro-Fl
Conformer (audio-only, main paper)
Audio
0.7200
0.1935
Whisper-Sman Small (audio-only)
Audio
0.7310
0.2010
Conformer + L.TMM (Concat-ML.P
Audio Text (Late)
0.8012
0.2003
Conformer + BERT (Concat-MLP))
Audio + Text (Late)
0.8124
0,2040
Conformer + BERT (Logit- Voting)
Audio + Text (Late)
0.8043
0.1992
Resp-Agent Diagnoser (Ours)
Audio Text (Weaving)
0.8494
0.2118
B.2 ARCHITECTURAL AND DATA-SIDE SOLUTIONS
Architectural Solution: Anchored Attention for Transient Event Localization. Beyond fusion strategy, the mechanism of attention proves critical. Anchor-based global attention the audio stream directly targets transient event localization. Removing anchors degrades performance to 0.6495 Accuracy and 0.1890 Macro-Fl (Table 8). falling below even the audio-only Conformer. With T-496 frames per 10s, one anchor every four frames establishes an 8=80.6 ms grid (guaranteeing alignment within <40.3 ms of any transient). This enables attention heads to precisely lock onto
17

---

Published as a conference paper at ICLR 2026
Table 8: Summary of performance on Resp-229k under the original (imbalanced) and class-balanced regimes. The substantial gains in the balanced regime highlight the efficacy of the Generator, while the multimodal improvements confirm the value of the Diagnoser's architecture.
Accuracy
Macro-F1
vs. Conformer (audio-oaly)
Model
Original
Balanced
Original
Balanced
AAc (Orig.)
AAcc (Bal.)
AFI (Orig.)
AFI (Bal.)
LSTM (text-only)
0,0912
0.3020
0,0401
0.2140
-0.6288
-0.4800
-0.1534
-0.3220
Conformer (audio-only)
0.7200
0.7820
0.1935
0.5360
0.0000
0.0000
0,0000
0.0000
Longformer (no anchors)
0.6495
0.7630
0.1890
0.5200
-0.0705
-0.0190
-0.0455
-0.0160
Ours: Resp-Agent (multimodal)
0.8494
0.8870
0.21188
0.5980
+0.1294
+0.1050
40.0183
+0.0620
Note: A columns are absolute improvements over the Conformer baseline under the regime.
fleeting wheezes crackles and align them with clinical text. The ablation gap indicates that anchors are not merely additive but essential for stable cross-modal reasoning in this domain.
Data-Side Solution: Controllable Synthesis to Overcome Scarcity. While architecture solves representation, it cannot invent missing data distributions. Balancing classes with our diagnosis- conditioned Generator elevates the multimodal Longformer to 0.8870 Accuracy and 0.5980 Macro-FI (Table 8 AFI +0.3862). Notably, this gain is structurally consistent, with the audio-only Conformer also improving significantly (0.1935 - 0.5360) when trained on our synthetic data. Conversely, perturbations (duplication, pitch/time shift, noise) actively harm cross- domain minority sensitivity (Conformer 0.1935 + 0.1688; Appendix [.I). Our Generator, which offers superior fidelity and controllability (FAD 1.13, style cosine 0.92: Appendix (D.2). outperforms e- L-WaveGAN(Lce et al.) 2018) and AudioLDM 2(Liu et al.] 2024) under matched
naive, pathology-agnostee
budgets (Appendix D.3), establishing content-aware balancing the decisive lever for minority-class resilience.
Synthesis: A Data-Model Co-Design for Robust Diagnosis. Taken together, the evidence supports a synergistic co-design:
(1) Anchored global attention essential. It furnishes a precise architectural mechanism for transient localization and stable texte-audio interaction at clinically meaningful timescales. Controllable synthesis is decisive. It supplies diagnostically informative examples for rare classes, converting architectural observability into large Macro-F1 gains.
(3) System-level synergy is paramount. Orchestrated by the Thinker, model-side anchors (where) and data-side generation (what) act jointly to produce models that are accurate on common cases, sensitive to rare events, and robust under domain shift.
C EXPERIMENTAL SETUP
Tasks and Metrics. We evaluate respiratory disease classification on the Resp-229k benchmark under two regimes: (i) the original, class-imbalanced split, and (ii) a label-balanced variant created using our Generator. We report Accuracy for overall performance and Macro-F1 Score to specifically assess performance on minority classes, which is crucial under label skew.
Baselines and Proposed Model. We compare our multimodal approach against two strong single- modality baselines. All models were trained for 10 epochs using the Adam optimizer (3) = 0.9, B2 0.999, weight decay of 0.01) and Cross-Entropy Loss. The BiL.TMM and Conformer baselines used a batch size of 32 and a Cosine Annealing learning rate scheduler with peak LR of x 10-4, Our Longformer model was trained using DeepSpeed (Li et al., 2024), with gradient checkpointing enabled, and OneCycleLR schedule with a maximum learning rate of 1 x 10-5, For reproducibility, all random seeds were fixed. We report the performance of the checkpoint with the best validation loss for all models.
Text-only (BiLSMM). Clinical summaries are tokenized into a vocabulary built with a minimum word frequency of 2. Sequences are padded or truncated to fixed length of 100 tokens. The model consists of a 256-dim embedding layer, followed by an 8-layer bidirectional LSTM with a hidden dimension of 512 and dropout rate of 0.5. The final hidden states passed to linear head for classification.
Audio-only (Conformer). We process 10-second, 16 audio clips to compute 128-bin log-mel mel spectrograms. The STFT uses a 1024-point FFT, window length of 1024 samples, and hop length
18

---

Published as a conference paper at ICLR 2026
of 160 samples, covering frequencies from 50 to 8,000 Hz. The spectrograms then mean-std normalized. Our Conformer architecture comprises an 8-layer encoder with a model dimension of 512 and 8 attention heads, followed by adaptive average pooling for classification.
Multimodal (Ours). Our diagnostic agent based on the longformer-base-UU0
model. As described in Section 4.2 we employ "modality weaving" by replacing 496 placeholder tokens with projected BEATs features. To enhance robustness, we apply token dropout to text (p- = 0.2) and frame dropout audio features (p 0.1) during training. Sparse global attention is strategically assigned to the [CLS] and [DESCRIPTION] tokens, as well as to audio anchors sampled with a stride of 4.
ICBHI Evaluation Protocol. We use the official ICBHI 60-40% train-test split for four-class RSC (Normal, Crackle, e, Wheeze, Both); metrics are Specificity (Sp), Sensitivity (Se), and the ICBHI Score (Sp + Se). For fairness, on ICBHI we use plain cross-entropy with class reweighting or resampling to match prior practice, and reported results follow the official metric definitions. Optimization uses AdamW with OneCycleLR schedule peaking at 1 x 10-5, and gradient checkpointing is enabled for memory efficiency; unless otherwise stated, all remaining hyperparameters (token budgets, feature checkpoints, anchor stride) are specified in our released configs and scripts. We pretrain on HF Lung V1 and SPRSound with Focal Loss to emphasize minority pathologies while using an LLM to render heterogeneous metadata into standardized clinical summaries paired with audio, and during ICBHI fine-tuning we remove focal/balancing heuristics for strict apples-to to-apples comparisons.
D DETAILED EXPERIMENTAL VALIDATION OF THE GENERATIVE AGENT
This appendix provides comprehensive evaluation of the Generator component within the Resp- Agent system. We systematically validate its necessity and efficacy through series of rigorous experiments. Our analysis is structured around two independent "chains of evidence": (i) objective similarity metrics that quantify the fidelity and controllability of the generated audio, and (ii) the downstream clinical value of the synthetic data when used to train diagnostic models. We further include detailed ablation studies to dissect the contributions of key architectural choices. All evalua- tions conducted under the strict cross-domain protocol defined in the main paper to ensure that our findings reflect true generalization capabilities.
D.1 THE INADEQUACY OF NAIVE AUGMENTATION FOR CROSS-DOMAIN GENERALIZATION
Rationale. A foundational premise of our work is that sophisticated generative modeling is not merely an alternative but a necessity for robustly handling the severe class imbalance in respiratory sound data. To establish this, we first conducted counterfactual experiment to determine whether naive, traditional audio augmentation techniques can improve cross-domait generalization. Such methods include over-sampling, pitch/time shifting, and noise injection. Our hypothesis that these transformations distort crucial, low-energy diagnostic cues (e.g., the transient structure of crackles and wheezes) and fail to introduce meaningful new variations, thereby degrading rather than improving performance on unseen data sources.
Experimental Design.
Model: We used a unimodal Conformer (audio-only). identical to the baseline described in the main paper, to isolate the effect of data quality from multimodal interactions.
Training Sets: We compared two training regimes: (A) the original, imbalanced training data, and (B) balanced version created using naive augmentation techniques (simple duplication, pitch shifting within 15%, time-stretching between 0.85x and 1.15x, and injection of moderately high-SNR noise).
Test Set: Evaluation was performed exclusively on the held-out cross-domain test set
(KAUH + COUGHVID) to measure real-world generalization.
Metrics: We report Accuracy and Macro-F1 Score, with the latter being particularly sensitive to performance on rare classes.
Results and Discussion. As shown in Table 9 naive augmentation leads to a clear degradation in performance the cross-domain test set. Both Accuracy and Macro-F1 score decreased, with the
19

---

Published as a conference paper at ICLR 2026
Fl-score dropping by 0.0247. This result provides strong empirical evidence for central claim: simplistic augmentations, while balancing class counts, actually amplify dataset-specific biases and destroy diagnostically salient audio micro-structure They teach the model to overfit to superficial features of the limited minority-class samples, which do not generalize to different devices, patient populations, clinical environments. This negative result firmly establishes the need for more intelligent, content-aware data generation strategy.
Table 9: Degradation of Cross-Domain Performance with naive Augmentation. The model trained on the balanced set created by traditional augmentation techniques performs than the model trained on the original imbalanced set, highlighting the failure of these methods to produce generalizable synthetic data.
test data
Training Data Strategy
Test Acc.
AAce
F1-Macro
AF1
Original Imbalanced naive Augmentation Balanced
0.7200
0.1935
0.6914
-0.0286
0.1688
-0.0247
D.2 EVIDENCE CHAIN OBJECTIVE FIDELITY AND STYLE CONTROLLABILITY
Rationale. The first pillar of validation assesses the Generator's technical capabilities: it produce audio that is not only high-fidelity but also accurately conditioned on a specific pathological class (content) while matching the acoustic characteristics of reference audio (style)? We compare our proposed Generator against strong,
audio generation baselines.
Experimental Design. We compare four generative models under unified individualized- reconstruction protocol:
Generative Models. We benchmark:
c-WaveGAN: a conditional waveform GAN baseline trained on disease labels only. AudioLDM 2 (fine-tuned): a modern text-to-audio diffusion model adapted to our condite-codd prompts.
StableAudio Open (fine-tuned): a strong contemporary text-to-audee model, fine- tuned on Resp-229k disease+style pairs using the same diagnosis prompts and reference- style conditioning protocol as our Generator.
Resp-Agent (Ours): our proposed generator, conditioned on both the disease label (content) and a style embedding extracted from the reference audio.
Metrics.
Cosine Similarity: we measure the cosine similarity between the BEATs embedding of the reference audio and the generated audio. Higher values indicate better style adherence and individualized timbre control.
Fréchet Audio Distance (FAD): distributional perceptual-quality metric between generated and real recordings; lower values are better.
Results and Discussion. Table 10 presents the results. The fine-tuned StableAudio Open(Evans ctt al. 2025) bascline substantially improves over caWaveGAN and AudioLDM 2, reaching a cosine similarity of 0.83 + 0.08 and FAD of 1.54. However, our Resp-Agent Generator still achieves the best overall quality, with a cosine similarity of 0.92 + 0.04 and the lowest (best) FAD of 1.13. This demonstrates that, even against a strong contemporary text-to-audio system, our content/style disentanglement and flow-matching decoder yield more faithful style preservation and higher-fidelity waveforms. These objective results confirm that Resp-Agent superior at producing controllable. high-fidelity and clinically relevant respiratory audio.
D.3 EVIDENCE CHAIN II: DOWNSTREAM CLINICAL VALUE OF GENERATED DATA
Rationale. While objective similarity is important, the ultimate measure of a medical data generator its downstream value-its ability to improve the performance of clinical diagnostic model. This
20

---

Published as a conference paper at ICLR 2026
Table 10: Objective evaluation of individualized audio reconstruction. Our Generator (Resp-Agent) achieves the highest style adherence (Cosine Similarity) and best perceptual fidelity (FAD), outper- forming strong generative baselines including a fine-tuned StableAudio Open model.
Generative Model
Cosine Similarity t
FAD |
<-WaveGAN
0.61 + 0.15
2.85
AudioLDM 2 (fine-tuned)
0.76 + 0.11
1.92
StableAudio Open (fine-tuned)
0.83 + 0.08
1.54
Resp-Agent (Ours)
0.92 = 0.04
1.13
is the gold standard for evaluating its utility. We therefore create class- balanca training sets using each generative method and measure the performance of downstream classifiers trained on these sets.
Experimental Design.
Task. Multimodal and unimodal respiratory disease classification on the cross-domain Test- CD split (KAUH COUGHVID), following the strict source-disooint protocol described in the main paper.
Training Sets. We construct six distinct training sets:
Original Imbalanced data (control).
naive Augmentation Balanced: balanced set obtained via classical audio augmentation (time/pitch perturbation, noise injection) without generative modeling.
3. c- WaveeGAN Balanced: a class-balanced set created by sampling from c-WaveGAN.
4. Audiol.DM 2 Balanced: class-balanced set synthesized by fine-tuned Audiol.DM 2 model.
StableAudio Open Balanced: a class- class-balanced set synthesized by a fine-tuned StableAu- dio Open model using the same diagnosis prompts and style-conditioning protocol as our Generator.
Resp-Agent Balanced: a Blass-balancee set created by our Resp-Agent Generator under the Thinker-A planning policy.
Downstream Models. For each of the six datasets above, we train two diagnostic models: unimodal Conformer (audio-only) and (ii) our multimodal Longformer (audio + text) Diagnoser, both configured exactly as in the main paper. This allows us assess the utility generated audio in both purely acoustic and multimodal settings.
Results and Discussion. Tables 11 and 12 show the results for the Longformer and Conformer models, respectively. The findings are consistent and robust:
Across both diagnostic models, the training set balanced by our Resp-Agent Generator yields the highest performance in terms of both Accuracy and, most critically, Macro-F1. Generative balancing with e- --WveGAN, AudioLDM 2, and StableAudio Open already produces large gains over the imbalanced and naive-augmentation baselines, but Resp-Agent consistently provides the strongest improvements.
For the multimodal Longformer, Resp-Agent's synthetic data boosts the Macro-Fl from 0.2118 (original imbalanced) to 0.5980, a relative increase of +0.3862. The fine-tuned StableAudio Open model achieves a stronger improvement than prior generative baselines (+0.3502 vs. +0.3147 for AudioLDM 2 and +0.2402 for c-WaveGAN), yet still trails Resp-Agent. This indicates that higher-fidelity, style-consistent synthesis directly translates into greater clinical utility.
A similar trend is observed for the audio-only Conformer. Balancing with StabieAudio Open raises the Macro-Fl 0.5050, outperformigg c-WaveGAN and AudioLDM 2, but the Resp-Agent Balanced regime remains best with a Macro-Fl of 0.5360. This shows that our Generator not only improves the multimodal Diagnoser but also strengthens purely acoustic classifier, underscoring the generality of the gains.
These results provide a powerful second chain of evidence. The synthetic audio from Resp-Agen is not only objectively superior in fidelity and controllability, but also contains more diagnostically
21

---

Published as a conference paper at ICLR 2026
salient information than that produced strong baselines, including fine-tuned StableAudio Open model. It successfully teaches downstream models to recogmize rare diseases, dramatically improving their clinical utility and fairness challenging, unseen test set.
Table II: Performance on the cross-domain Test-CD set using different balanced training sets (multimodal Longformer Diagnoser). Balancing with Resp-Agent yields the largest improvement in Macro-F1 over the imbalanced baseline, even compared to strong StableAudio Open baseline.
Training Set Strategy
Accuracy
FI-Macro
Relative AFI (vs. Imbalanced)
Original Imbalanced
0.8494
0.2118
naive Augmentation Balanced
0.7520
0,1720
-0.0398
C- C-WaveGAN Balanced
0.8650
0,4520
+0.2402
AudioLDM Balanced
0.8781
0.5265
+0.3147
StableAudio Open Balanced
0.8830
0.5620
+0.3502
Resp-Agent Balanced
0.8870
0.5980
+003862
Table 12: Performance the cross-domain Test-DD set using different balanced training sets (unimodal Conformer). StableAudio Open improves substantially over older baselines, but the Resp-Agent Balanced regime remains strongest.
Training Set Strategy
Accuracy
F1-Macro
Original Imbalanced
0.7200
0.1935
naive Augmentation Balanced
0.6914
0.1688
C-WaveGAN Balanced
0.7420
0.4010
AudioLDM Balanced
0.7560
0,4760
StableAudio Open Balanced
0.7700
0.5050
Resp-Agent Balanced
0.7820
0.5360
D.4 ABLATION AND ROBUSTNESS ANALYSIS
To ensure our model's design well-justified, we performed targeted ablation studies on its key components.
D.4.1 IMPACT OF STYLE PREFIX LENGTII (K)
Rationale. The number of style tokens, K, is a critical hyperparameter that mediates the trade-off between style representation capacity and model complexity. We investigated its impact both generative quality and downstream task performance.
Experimental Design. We varied K in the 20, 4, 8). where K=0 disables style conditioning entirely. We evaluated its effect on the individualized reconstruction task (Similarity/FAD) and the final downstream F1-Macro score of the Longformer model trained on data generated with the corresponding K value.
Results and Discussion. Table 13 shows clear trend. Increasing from 8 monotonically improves performance across all metrics: style similarity increases, FAD decreases, and the down- stream Macro-Fl score rises. This validates our architectural choice use style prefix tokens for conditioning and confirms that richer style representation (K=8) allows the Generator to produce more effective training data.
Table 13:
Ablation Study the Number of Style Tokens (K). Performance improves consistently with larger style prefix, validating the effectiveness our style conditioning mechanism.
(Style Tokens)
Similarity (Cosine)
FAD
Longformer F1-Macro 1
0 (Style Disabled)
0.80 + 0.09
1.52
0.542
0.85 1 0.08
1.38
0.563
0.87 1 0.06
1.29
0.577
8 (Default)
0.92 1 0.04
1.13
0,591
22

---

Published as a conference paper at ICLR 2026
D.4.2 CHOICE OF DECODER PARADIGM: KLOW-MATCHING VS. DIFFUSION
The second stage of our Generator relies on a decoder to transform discrete tokens into continuous waveform. We compared our choice, Conditional Flow-Matching (CFM), against traditional Denoising Diffusion Probabilistic Model (DDPM) to validate its superiority in both quality and efficiency.
Rationale.
Experimental Design. We trained two decoders with identical conditioning inputs and a fixed inference budget of 32 steps. We compared their FAD, style similarity, and relative inference latency.
Results and Discussion. Table 14 demonstrates the advantages of CFM. At the same step count, CFM achieves better FAD (1.13 1.31) and higher similarity (0.92 0.90), indicating superior sample quality and fidelity. Crucially, it achieves this with approximately 40% less inference time (<s 0.6x latency). This efficiency is vital for practical applications, enabling faster generation of large-scale balanced datasets. This result confirmss that CFM is more effective and efficient choice for the waveform reconstruction stage in our architecture.
Table 14: Comparison of Decoder Paradigms. Conditional Hlo--Matching (CFM) surpasses the traditional DDPM in both audio quality (FAD, Similarity) and inference speed.
Decoder Type
Steps
FAD L
Similarity
Inference Latency
DDPM
1.31
0,90
1.0x
CFM (Ours)
32
1.13
0.92
x0066
E AUDIT AND VALIDATION OF LLM-GENERATED CLINICAL SUMMARIES
To ensure the reliability of the multimodal framework, we conducted a comprehensive audit of the LLM-generated clinical summaries paired with the Resp-229k audio. This experiment quantifies the fidelity of the eext-generation process, verifying that summaries derived from heterogeneous metadata accurate and free from critical hallucinations.
Methodology. The clinical summaries were not synthesized from raw audio (audio-to-text) but were generated from existing structured metadata (data-to-text) using 7B parameter model (DeepSeek-RI 1- Distill-Qwen-B This schema- grouuddd approach ensures that summaries strictly rephrase existing fields (e.g., demographics, symptoms, auscultatory findings) rather than inventing new information. To guarantee high fidelity, we implemented distinct, two-stage quality assurance (QA) pipeline operating on all 238,074 generated summaries:
Stage 1: Heuristic Pre-screening. All descriptions were first passed through a heuris- tic filter to identify suspicious records, specifically flagging empty or truncated text (EMPTY_OR_TRE) TRUNCATEE) overly long text (OVERLONG), or leakage of instructional prompts (P (PROMPT_LEAKS,
LLM-bbseed QA and Audit. All suspicious records identified in Stage 1, plus 1% random sample of heuristically "clean" records, were forwarded to separate validator LLM (DeepSeek- V..k-- V3.2-Exp). This QA model operated under strict prompt forbidding the invention of patient metadata (e.g., age, sex, comorbidities) or the alteration of high-level pathology labels.
Stage 2:
Quantitative Results. The QA pipeline audited the entire set of 238,074 records. The results, detailed in Table [5] confirm the high initial quality of the data-to-text synthesis. The process yielded an effective rewrite rate of only 0.7451 %, indicating that fewer than in 130 descriptions required modification.
Error Typology and Verification. A breakdown of the flagged issues is presented in Table 16 The analysis reveals that the vast majority of interventions (1,747 out of 1,774 rewrites) were triggered by technical artifacts, specifically OVERLONG OR PROMPT LEAK, rather than substantive clinical emror or hallucinations.
23

---

Published as a conference paper at ICLR 2026
Table 15:
Quantitative results of the two-stage summarization QA pipeline (N 238,074). The low rewrite rate confirms the high fidelity of the initial synthesis.
Metric
Value
Total records
238,074
Heuristic suspicious 1EN0T///EEELLLOGOOOOTTTTT... LEEA)
3,356
Randomly audited (heuristic-clean samples)
2,300
Total sent to QA LLM
5,656
LLM kept as OK
3,766
LLM rewrote (suspicious only)
1,774
LLM flagged random audit (no edit applied)
116
API/ parse errors
Effective rewrite rate
0.7451 (1,774 / 238,074)
Table 16: Breakdown of heuristic flags and errors were technical artifacts (c.g., prompt leaks) rather than clinical hallucinations.
error types during the audit. The primary
Heuristic Category (Suspicious)
LLM-Identified Error Type (All Audited)
Type
Count
Error Type
Count
OVERLONG
3,031
OK
3,766
FROMFTLLEAK
300
1,747
OMPTY.OR.TRUNCATE
25
OTHER. TWW.RRRUAALLTTTCCC IIIIUEE
113
EMP TY ,OR, TRUNCATED
30
Key Findings.
High Fidelity: The data-to-text generation process is highly reliable, with a rewrite rate below 0.75%.
Dominant Error Type: The primary issues identified were technical artifacts (e.g., prompt leakage or verbose boilerplate) rather than clinical hallucinations.
Human-in-the-Loop: As final safeguard, all 1,774 L----ooooeed rewrites underwent manual review by the authors to ensure consistency with the original structured metadata and disease labels, preventing the introduction of ungrounded patient information.
F LLM USAGE STATEMENT
We utilized Large Language Models (LLMs) for three distinct purposes this work, spanning core system architecture, data curation, and writing assistance:
As described in Section 4 the central controller of proposed Resp-Agent framework, denoted as Thinker-A"CA. is instantiated using DeepSeek-V3. 2-Exp. This LLM serves as the reasoning core responsible for semantic intent parsing, tool routing, and dynamic planning of the analysis-snntheiis loop.
1. Core System Architecture (The "Thinker" Agent):
2. Data Curation and Validation (Resp-229k Benchmark): As detailed in Section 3 and Ap-
pendix B we employed LLMs to construct and validate the textual component the dataset:
- to synthesize standardized clinical narratives from heterogeneous structured metadata.
Tata-to-Text Generation: We used DeepSeek-Rl-Distte
Quality Assurance: We used DeepSeek-Ve w.w. 2-Exp to audit, flag, and correct potential quality issues in the generated summaries.
3. Writing Assistance:
We utilized general- purpose LLMs to assist with polishing grammar, refin- ing stylistic elements, and formatting BTEX tables. The authors reviewed and retain full responsibility for all text and data presented in this manuscript.
24


Reviews:
[{"review_id": "W07qLzy5tG", "reviewer": "Reviewer_u8tM", "summary": "The paper introduces Resp-Agent, a multimodal agent system tackling the challenges of deep learning-based respiratory assessment by developing both a generator and a classifier. To address the scarcity of high-quality data and the omission of fine acoustic dynamics, the system first transforms a text-only LLM into a multimodal generator via modality injection, producing BEATs tokens conditioned on diagnostic text and style embeddings, which are then reconstructed into waveforms. For diagnosis, it employs a Longformer-based model with strategic global attention to fuse EHR text and acoustic features at the input level, allowing it to capture long-range cross-modal dependencies while accurately detecting brief acoustic events like coughs. This work is supported by the creation of Resp-229k, a massive 408-hour multimodal corpus that pairs audio with expert-level EHR annotations, enabling the system to achieve superior performance in robust respiratory disease prediction.", "questions": "What will happen if you change Conformer to Whisper?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 2, "contribution": 2, "strengths": "The most practical and immediate strength is the size and richness of the new corpus. At 408 hours and 229,000 recordings, this scale is required for a specialized medical domain. Crucially, by linking the acoustic data to expert-level annotations distilled from Electronic Health Records (EHRs), this provides the necessary clinical context, which is often missing in public datasets, thus bridging the gap between raw audio and real-world diagnostic complexity. \n\nThe system is capable of generating high-fidelity audio (using LLM-based modality injection to create BEATs tokens). By generating discrete acoustic units (BEATs tokens) and using a specialized flow-matching decoder, it retains the high-fidelity transient events critical for accurate diagnosis. \n\nThe diagnostic model integrates two disparate data types, which are EHR text and respiratory features, at the input level. The introduction of strategic global attention is a good contribution, as it specifically addresses the difficulty of simultaneously capturing broad clinical context (long-range dependencies) and tiny, clinically vital acoustic cues (brief, low-energy events like wheezes or rales).\n\nThe paper is reasonably well written and focuses on solving an important issue of accurate diagnosis in the respiratory health domain.  \n\nThe datasets and evaluation seem adequate. The performance improvements over compared baselines are good.", "weaknesses": "The papers fall into the category of applying ML for healthcare. I find it interesting as it combines multiple concepts, but incremental. \n\nThe evaluation part is weak. The choice of LSTM for the text baseline is weak, and using an older attention mechanism for fusion is also suboptimal.  I would suggest comparing against a Transformer-based text encoder (e.g., BERT, RoBERTa, or even a small version of the Longformer) for the text-only task. This isolates whether the performance gain is from the fusion or just using better text features. \n\nTo demonstrate the novelty and power of the proposed fusion, the authors should have included multimodal baselines that employ simpler fusion strategies. For example, use the Conformer to extract audio embeddings and the LSTM to extract text embeddings. Concatenate these two vectors and feed the result into a simple Multi-Layer Perceptron (MLP) classifier. The other way can be to do averaging or weighted voting for the final prediction. \n\nThe paper compares audio generative quality with Audio LDM2 and c-WaveGAN. I find StableAudio or StableAudio Open missing from the paper. It will make sense to compare Resp-Agent generative quality against a fine-tuned StableAudio.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "47mo3PSVq4", "reviewer": "Reviewer_WjjZ", "summary": "The paper tackles a clinically meaningful problem with a coherent, end-to-end system and introduces a cross-domain benchmark plus two technically interesting modules (modality-injected unit generator with CFM; Longformer with anchor-based global attention). Futher, the proposed system demonstrates credible ICBHI gains and compelling rebalancing benefits on Resp-229k.", "questions": "The questions for the authors are as follows,\n\n- Class taxonomy: Are Bronchiectasia vs Bronchiectasis and Acute URI vs URTI duplicates? If so, how are they merged across sources and guaranteed disjoint across splits? Please provide the final class map and counts per class per split. \n- Imbalance baselines: Can you compare against strong non-generative remedies under the same cross-domain protocol (class-balanced sampling, effective number reweighting, LDAM-DRW, focal loss in fine-tuning, mixup/Manifold mixup), with CIs?\n- Text summary quality & governance: What safeguards ensure LLM-rendered EHR-style summaries do not hallucinate fields or introduce systematic biases across sources/devices? Provide sampling audits, error types, and licenses/provenance for all sources.\n- Where does the gain come from? Please isolate the modality weaving vs late fusion, anchors vs no anchors, and text quality (raw metadata vs LLM-rendered text) in a single ablation grid on the cross-domain split.\n- Cross-dataset generalization: In addition to source-disjoint evaluation, can you train on subset A and test on B (e.g., train on ICBHI and SPRSound, test on UK-COVID; or leave-one-source-out) to confirm robustness to label/device/site shifts?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "The paper strengths can be summarised as follows,\n\n- Ambitious, unified scope (which includes dataset, generation, diagnosis, and agent loop). The paper introduces a cross-domain multimodal corpus (Resp-229k) with source-disjoint splits. This anchors the contribution in real distribution shift rather than in-domain evaluation.\n- Clear architectural ideas on both sides of the loop. The Generator upgrades a compact LLM via modality injection (BEATs-derived style tokens) to autoregress discrete acoustic units, then reconstructs waveforms with a DiT-style conditional flow-matching decoder and vocoder. This results in a clean disentanglement of content (diagnosis) vs style (timbre/device).\n- Generation for targeted rebalancing appears useful. Using diagnosis-conditioned synthesis to balance classes lifts macro-F1 sharply.\n- The paper reports objective similarity (e.g., FAD, style-cosine) and comparisons to c-WaveGAN / AudioLDM-2 under matched budgets, supporting the claim that content-aware augmentation helps more than generic perturbations.", "weaknesses": "The paper weaknesses can be summarised as follows,\n\n- Low macro-F1 in the natural (imbalanced) setting. Before synthetic balancing, macro-F1 is 0.2118 despite high accuracy... this tell me that there might be substantial minority under-diagnosis. However, the paper relies on its own generator to fix this; stronger baselines (e.g., cost-sensitive losses, reweighting, focal/LDAM, mixup/Manifold mixup, class-balanced sampling) should be compared under the same cross-domain split to show generation is superior beyond conventional imbalance remedies.\n- Potential evaluation confounds on generation. The CFM decoder is conditioned not only on discrete units and global timbre but also \"a short reference prefix of the ground-truth mel during training/validation\" to encourage continuation.\n- Dataset governance and text synthesis risks. Clinical \"summaries\" are LLM-generated from heterogeneous metadata; while the model doesn’t interpret audio, it can still standardize or hallucinate metadata beyond the original fields. The paper should provide audits (error rates, inter-annotator checks) and licenses/provenance per clip, especially with mixed microphone/stethoscope sources used for high-stakes labels.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "Gr3ovNfr00", "reviewer": "Reviewer_MVWu", "summary": "The paper present Resp-agent, an agent-based system for respiratory sound generation and disease diagnosis. The whole system includes a cascaded pipeline leveraging LLMs for content semantics learning and followed by the generator and flow-matching based reconstruction to capture the sound-features. Since there's no existing paired EHR - audio datasets, they create a multimodal benchmark names Resp-229k. Finally, the paper evaluated on disease diagnosis tasks and reports improvements over baselines.", "questions": "see weakness", "limitations": "", "rating": 4, "confidence": 4, "soundness": 3, "presentation": 2, "contribution": 2, "strengths": "1. The paper targets important question that lacking paired multi-modal datasets especially when the text-modality missing.\n2. The system design is comprehensive and clear to follow its flow.", "weaknesses": "1. Despite the comprehensive and close-loop system. The novelty lies in the assembly of recent advances such as longformer, flow-matching models. The core contribution and innovation is hidden and ambiguous. \n2. For the generator part,  the Generator does make the system closed-loop: the “Diagnoser” classifies real respiratory sounds, then the “Generator” synthesizes new examples. It's not clear is there of a real feedback loop: the Diagnoser doesn’t meaningfully inform or retrain the Generator; Besides, there's no evaluation of generation quality. \n3. Lack of discussion of recent work about respiratory foundation models or other llm-based respiratory model. For example, papers such as Resp-LLM or other LLM-prompted multimodal models already explored: Using LLMs to describe audio (e.g., “This sound has wheezes and crackles”), althought not in the EHR format. Why the EHR-simulation text is chosen or better compared with other prompt style?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "0Nes1K6xSA", "reviewer": "Reviewer_CKZt", "summary": "The paper introduces a multimodal method for high-fidelity respiratory sound generation and disease diagnosis, converting a text-only LLM into an audio generator via modality injection and fusing text with acoustic features using a Longformer with strategic global attention.", "questions": "Can you provide a controlled ablation comparing the Thinker-based planner against simple baseline schedulers (class-prior rebalancing, uncertainty sampling, random selection) while holding all other factors constant same Generator, same Diagnoser, same synthesis budget, same training procedure to check and demonstrate whether the intelligent planning policy itself drives improvement?\n\nCould you present performance curves showing macro-F1 as a function of synthetic data budget to reveal whether most benefits come from a small number of targeted clips or require large volumes of generated audio?\n\nCould you demonstrate the Generator's controllability through disentanglement tests specifically style-swap experiments (fixed pathology, varied recording style) and label-swap experiments (fixed style, varied pathology) to confirm that rare pathology-style combinations can be reliably synthesized without the model conflating correlated attributes?\n\nWhy not compare against actual established baselines in the areas of respiratory monitoring?", "limitations": "", "rating": 2, "confidence": 4, "soundness": 1, "presentation": 2, "contribution": 2, "strengths": "The paper’s originality lies in the closed-loop Diagnoser -> Thinker -> Generator setup and anchor-based multimodal Diagnoser, but these build on familiar ideas (active learning, synthetic augmentation, attention).", "weaknesses": "The paper's central claim that the Thinker module intelligently guides data generation lacks evidence, as no controlled comparison against simple baselines like class-prior rebalancing, uncertainty sampling, or random selection is provided under identical experimental settings. Without isolating the planner's contribution from the sheer effect of adding synthetic data, it remains unclear whether the observed gains stem from sophisticated scheduling or merely from data volume. \n\nFurthermore, the work does not decompose the sources of improvement, whether gains arise from balancing underrepresented labels, targeting hard recording styles, combining rare labels with difficult domains, or simply scaling the dataset. It also does not provide ablation curves showing performance versus synthetic budget to identify when returns saturate.\n\nThe evaluation framework also presents gaps that weaken the claims of robustness and controllability. The paper omits comparisons with established imbalance mitigation techniques such as focal loss, which could achieve similar macro-F1 improvements without generative models, rendering the necessity of the proposed pipeline uncertain. Cross-domain evaluation on only two held-out sources is insufficient; a leave-one-source-out protocol would rigorously test whether robustness generalizes across all recording conditions or benefits from a favorable data split. \n\nFinally, the Generator's ability to independently control pathology content and recording style is assumed but not validated without explicit disentanglement tests (e.g., label-swap experiments), the model may conflate correlated attributes from training data, undermining the loop's premise that rare labels and target styles can be independently specified.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)