Aurelius: Relation Aware Text-to-Audio Generation At Scale
Authors: Yuhang He
He Liang
Yash Jain
Andrew Markham
Vibhav Vineet
Venue: ICLR 2026 Poster (2026)
Keywords: Relation Aware Text-to-Audio Generaion
Audio Event Corpus
Relation Corpus
Primary area: datasets and benchmarks
TL;DR: A new audio event corpus and relation corpus supporting relation aware text-to-audio (TTA) generation task and beyond

Abstract: We present  Aurelius, a new framework that enables relation aware text-to-audio (TTA) generation research at scale. Given the lack of essential audio event and relation corpora, Aurelius contributes a large-scale audio event corpus AudioEventSet and another large-scale relation corpus AudioRelSet. Comprising 110 event categories, AudioEventSet maximally covers all commonly heard audio events and each event is unique, realistic and of high-quality. AudioRelSet consists of 100 relations, comprehensively covering the relations that present in the physical world or can be neatly described by text.  As the two corpora provide audio event and relation independently, they can be combined to create massive <text,audio> pairs with our pair generation strategy to support relation aware TTA investigation at scale. We comprehensively benchmark all existing TTA models from both general and relation aware evaluation perspective. We further provide an in-depth investigation into scaling existing TTA models' relation aware generation by either training from scratch or leveraging cross-domain general TTA knowledge. The introduced corpora and the findings from investigation potentially facilitate future research on relation aware TTA generation.

Published as a conference paper at ICLR 2026
AURELIUS: RELATION AWARE TEXT-TO-AUDIO GENERATION AT SCALE
Yuhang He He Liang' Yash Jain! Andrew Markham Vibhav Vineet
Microsoft Research Department of Computer Science, University of Oxford
Corresponding: yuhangheemicrosoffe com
Code:
e
Project:
ABSTRACT
We present Aurelius, new framework that enables relation aware text-to- audio (TTA) generation research at scale. Given the lack of essential audio event and relation corpora, Aurelius contributes a large-scale audio event corpus Audio- EventSet and another large-scale relation corpus AudioRelSet. Comprising 110 event categories, AudioEventSet maximally covers all commonly heard audio events and each event unique, realistic and of high-quality AudioRelSet consists of 100 relations, comprehensively covering the relations that present in the physical world can be neatly described by text. As the two corpora provide audio event and relation independently. they can be combined to create massive <text, , audios o> pairs with our pair generation strategy support relation aware TTA investigation at scale. We comprehensively benchmark all existing TTA models from both general and relation aware evaluation perspective. We further provide an in-depth investigation into scaling existing TTA models' relation aware generation by either training from scratch or leveraging cross-domain general TTA knowledge. The introduced corpora and the findings from investigation potentially facilitate future research on relation aware TTA generation.
INTRODUCTION
Text-to-audio (hereinafter TTA) generation task aims at generating acoustically high-fidelity audio whose content is inferred from the input text. Owing to the success of generative modeling (e.g., diffusion based (Ho et al., 2020; Xue al., 2024), score based (Vahdat et al., 2021) and flow matching based (Lipman et al., 2023; Guan et al., 2024) methods) and the availability of large <text, audio> pair dataset (e.g., AudioCaps (Kim et al., 2019), AudioSet (Gemmeke et al., 2017)), we have witnessed significant advancement in general TTA task in recent years (Ghosal et al., 2023; Hung et al., 2026; Liu et al., 2024). Despite these achievements, the relation aware TTA generation still remains as a challenging task as it jointly requires audio event generation and relation modeling. Audio events and their relation are two fundamental elements humans rely on for holistic acoustic scene understanding or engaging communication (Zacks et al., 2007; Hirsh al., 1967; Lake et al., 2015). We humans can interpret the relation and audio events within the textual description with ease to decide how the target audio looks like. Enabling TTA models with similar relational reasoning and event interpretation capability is therefore essential for bridging the gap between relation aware TTA model quality and human-level crossmodal reasoning.
The recent preliminary investigation by RiTTA (He et al., 2025) already shows the incapability of existing TTA models in relation aware generation, but the investigation runs on top of small relation and audio event corpora. The data corpora small scale issue naturally hinders further investigation. To enable relation aware TTA at scale, we introduce Aurelius, a novel framework that contributes to relation aware TTA from both dataset benchmark and technical methodology aspects. From the dataset benchmark aspect, we meticulously curate two large-scale corpora: AudioEventSer and AudioRelSet. AudioEventSet is audio event corpus that comprises 1 110 across fine-gn grained event classes across main acoustic categories we commonly hear in our daily lives. In contrast to existing audio event datasets (Gemmeke et al., 2017; Kim et al., 2019; Fonseca et al., 2022) that are either

---

Published as a conference paper at ICLR 2026
noisy, polyphonic or label- AudioEventSer provides coarse-to-fine tree structured audio event corpus that is both internally distinctive and externally comprehensive. Each individual audio event in AudioEventSet is high-quality, realistic and intra-class diverse. AudioRelSet is the large-scale relation corpus with up to 100 detailed relations completely covering the potential relations audio events may present in the 3D physical world or text can describe succinetly. AudioRelSet is also tree structured and can be further scaled up to incorporate more relations. Each relation in AudioRelSer has an "arity" property that is further used to combine relation and audio events together create <t <text, audio o> pairs for relation aware TTA task. AudioEventerr and AudioRelSer are orders of magnitude larger than existing relevant dataset, enabling thorough and in in-depth investigation for relation aware TTA task.
Based on the introduced audio event corpus Au- dioEventSer and relation corpus AudioRelSet, we further introduce a <text, audio> pair generation strategy that can generate essential <text, audios o> pairs with both audio event based and textual description diversity. As the audio event corpus is disentangled from rela- tion corpus, our proposed strategy can gener- ate nearly unlimited <text, audi o> pairs tai- lored for various training requirements. In sum- mary, as illustrated in Fig. I, Aurelius advances relation aware TTA research by contributing turge-scale corpora of audio events and relations, together with dedicated framework for relation aware generation. The explicit disentanglement of audio events and relations, the hierarchical tnee-structured design of each corpus, and the systematic <t <text, audio o> creation strategy collectively provide a strong foundation for curating essential datasets in this domain. Building this foundation, our proposed AudioRelGen framework tackles relation aware TTA by decoupling audio event modeling from relation modeling, offering an essential first step toward structured audio generation. We believe this work will not only establish a new benchmark for relation aware TTA but also inspire future research on modeling complex event-relation dynamics in sound.
AedioRelSet
1. 100 relations
2. main cate
3. scalable
AudioEventSee
Test.Audion Audio
1. 110 events
1. unlimited size
2. 7 main cate
2. highly diverse
3. hierarchical
3. customizable
Aurelius
Figure I: Aurelius contributes to relation aware TTA by introducing an audio event corpus Au- dioEventSer, a relation corpus AudioRelSet and <text, audio> pair generation strategy.
2 RELATED WORK
Text-to-Audio Generation aims at generating the audio waveform that semantically aligns well with the input text. The fast development of generative modeling techniques (Ho et al., 2020; Vahdat et al., 2021; Lipman et al., 2023) in recent years has largely advanced the TTA generation in terms of high-fidelity and highiintelliiibiiiiyy (Liu et al., 2024; 2023; Kreuk et al., 2023; Yang et al., 2022; Ghosal et al., 2023; Liao et al., 2024), alongside other crossmodal generation tasks including but not limited mex-to--muiic (TTM, e-g., MusicGen (Copet et al., 2023) and MusicLM (Agostinelli et al., 2023)), image-to-audio (I2A, egg., RegNet (Chen et al., 2020), Img2Wav (Sheffer & Adi, 2023) and Spec (lashin & Rahtu, 2021) and text-to-image (T2I). Although the promising achievement in generating realistic and semantically text-aligned audio, existing TTA methods still perform poorly in relation aware TTA generation. Prior work like RiTTA (He al., 2025) and CompA (Ghosh et al., 2024) have preliminarily explored relation aware TTA and shown the incapability of existing TTA methods through limited audio event and relation corpora, which inevitably hinders future investigation at scale. Moreover, publicly available audio event corpora (AudioSet) are directly collected from either online video data or audio sharing platform without proper quality check, resulting in audio events being label-missing, noisy and ambiguous. Our work circumvents these barriers by introducing a meticulously curated audio event corpus AudioEvenSer that is high-quality, distinctive and realistic, potentially covering all commonly heard audio events.
Relation Modeling has been widely discussed within modalities, including images (Liu et al., 2022; Zerroug et al., 2022), natural language processing (Wadhwa et al., 2023), and acoustics (Xie et al., 2025a; Ghosh et al., 2024; He et al., 2025). In the context of 2D images, the objects of interest can exhibit compositional and spatial relations (Liu al., 2022; Zerruug et al., 2022). In the context of the
2

---

Published as a conference paper at ICLR 2026
3D physical world, an audio event is the most fundamental acoustic signal, and multiple audio events join together to represent the 3D physical world via more sophisticated relations than image- based relations, ranging from basic spatial, temporal, and perceptual relations to their nested combinations. Prior works (Xie et al., 2025a; Ghosh et al., 2024; He et al., 2025; Xie et al., 2025b) have discussed audio-event relations small scale and with minimal complexity, making them hard to scale up to accommodate the potential relation complexity present in either the 3D physical environment or textual descriptions. To fill this gap, we curate AudioRelSet, a large-scale relation corpus that reflects the relations potentially present in the physical world and can be neatly described by text.
Text-to-Audio Generation Techniques. Existing TTA methods can be divided into two main categories: early methods are diffusion based (Liu et al., 2024; 2023; Kreuk et al., 2023; Yang et al., 2022; Ghosal et al., 2023; Liao et al., 2024; Xue et al., 2024), while recent methods are flow-matching based (He et al., 2025; Hung et al., 2026; Guan et al., 2024). Flow-matching methods are usually faster during both training and inference and can yield better performance than diffusion-bai -baed methods. We comprehensively benchmark these methods our introduced corpora and further provide an in-depth investigation to reveal potential ways to scale up existing TTA methods' relation aware capability.
3
AURELIUS BENCHMARK: AUDIOEVENTSET AND AUDIORELSET
3.1 AUDIO EVENT CORPUS: AudiooEventeet
An audio event refers to an auditory signal oc- curring over specific period of time, typically representing independent, human human-recognizzblle sound. To support relation aware TTA research, the desired audio-event corpus should be: 1. di- verse enough to maximally accommodate the wide variety of audio events potentially present in the 3D physical world; 2. clean and gigh-fiielity to enable reliable in-depth technical investigation; 3. distinctive so that events can be easily distin- guished without ambiguity; 4. hierarchically orga- nized with respect to genre to enable investigation at different granularity. After thorough investiga- tion of existing audio-event datasets, however, we find that all existing datasets fall short of these four properties. As shown in Table 1, existing audio- event datasets (e.g., AudioSet (Gemmeke et 2017), AudioCaps (Kim et al., 2019), AudioTime (Xie et al., 2025a), and FSD50K (Fonseca et al., 2022)) are noisy, label-missing, polyphonic (multiple events temporally overlap). or semantically ambiguous (multiple event classes correspond to the same audio). To address this dilemma, we introduce AudioEventSer, a meticulously curated audio-event corpus that is intrinsically clean, diverse, distinctive, and hierarchically organized.
Table 1: Audio Event Dataset Comparison.
Dataset
Characteristic
AudioSet (2017)
polyphonic, ambiguous, noisy, label-missing
FSDSOK (2022)
AudioCaps (2019)
AudioTime (2025a)
distinctive, high-quality, cleon, hierarchical coarse-to-fir to-f--e intra-class diversity inter-class discriminative
AudioEventSer
Audiotventter ontology is tree-structured with depth three. From the root node to the leaf node, each audio event is organized in a tonree-to-fine granularity. As shown in Fig. 2 and Table in the Appendix, we build on RiTTA (He et al., 2025) to categorize AudioEventSet into 7 main categories: five singular-source categories Animal, Human, Machinery, Music, and Nature, and two interaction- based categories, Hfuman-Object and Object-Object interactions. The seven categories maximally cover commonly heard audio events in the 3D physical world. Each main category is associated with multiple subcategories, each of which is further associated with multiple For example, the Human main category contains human voice, human speech, hands action, group action, and locomotion subcategories, comprchensively categorizing human-centered audio events from various aspects.
event classes.
During AudioEventSet ontology construction, we guarantee each curated audio event is distinctive, unique, and human- dimaa-ddstinguushable. An audio event emitting ambiguous or nondistinctive audio is discarded. For example, engine idling in AudioSet (Gemmeke et al., 2017) audio differs significantly across engines, and it is easily confused with other audio events such as working fan and hairdryer. We thus exclude all of them from the corpus. Moreover, we account for audio event source origin, event
3

---

Published as a conference paper at ICLR 2026
spacch
friction
implication succcesiom
Obg-Ob Interaction
voice Westler
Tunan
dropping
hamoonics
famess
Count
instrument
Urory
Birory
Termay
Nature
Music
saccoccion
Spaniality
singing
wattr
wild domestic
Animal
Human Obj
Internction
mechanism alanm
Machinery
Penceptuaiity
bandinn
Compositioaaing
Temporality
succession
Mary
Combination
Relation
tool work
car horn
negition implication
Audio Event
Audio
A. AudioEventSet Corpus Onickogy Vissalization
B. AudioRellSrr Corpus Ontelogy Visualization
C. Relation Arily Illusiraiion
Figure 2: AudioEventSer and AudioRelSer corpora illustration: we visualize the AudioEventSet ontology in sub-figure A. It is tree- seesstuuc with depth and contains 7 main categories and 110 event categories (leaf nodes) in total. We show only part of the leaf nodes (with red dotted arrow) the Appendix. The AudioRelSet ontology
for clarity. The detailed event ontology is given in Table is shown in sub-fig sub-figure B; it is tree-structured with depth 2 and contains 6 main categories and 100 categories in total. The detailed relation ontology is given in Table II in the Appendix. In sub-ligure C. conceptually illustrate the relation "arity", which is used to connect relations and audio events to generate audio.
category, and the physical mechanisms of audio generation in AudioEventSer ontology construction. For example, in the Object-Object main category, we exhaustively consider impact, friction, dropping, and explosion mechanisms. In summary, we have curated 11 audio events, which is four times larger than the audio-event corpus proposed in RiTTA (He et al., 2025); each leaf audio event associated with around 75 realistic audio snippets ranging from s to 5
For each leaf-node audio event, we collect exemplar audio from either the copyright- freesound. org platform or FSD50K (Fonseca et al., 2022). As most audio from freesound, org and FSDSOK are real recordings shared by volunteers across the globe, the collected audio for each event is diverse and realistic enough to reflect audio events in the physical world. Manual verification adopted to ensure content correctness and label consistency. We argue that the curated Audio can be applied to tasks beyond TTA, and we anticipate much wider usage of the dataset.
3.2 RELATION CORPUS: AudioRelSet
AudioRRlSet
Prior works (Xie al., 2025a; He et al., 2025; Ghosh et al., 2024) have explored audio-event relations from various perspectives, but only at small scale. For example. AudioTime (Xie et al., 2025a) and CompA (Ghosh et al., 2024) discuss temporal relations. RiTTA (He al., 2025) additionally introduces spatial, compositional, and count relations, resulting in a total of 11 relations. In this section, we introduce AudioRelSet, meticulously curated large-scale relation corpus with up to 100 distinct relations. To ensure AudioRelSer exhibits real-scenario practicability, text-manageable complexity, and relation scalability, we follow three guidelines to curate AudioRelSer: maximally cover the potential relations audio events can present in the 3D physical world; 2. include sufficient relation complexity while still being efficiently and neatly described by text; 3. allow the relation corpus scale up accommodate more sophisticated relations. To this end, construct six fundamental relation categories, in which four main relations describe relations present in the 3D physical world, one focuses on TTA model's logical reasoning capability, and the last derives from nested combinations of the five main relations.
AudioRelSet ontology is tree-structured with depth 2. The root node connects six main relations, each of which further associates with multiple sub-relations. Let E = [E, Eg,... Em} denote the audio events in AudioEventSet introduced in Sec. 3. and let R (R), Re.... .., R., denote the relations to be constructed. AudioRelSet is represented as follows:
Temporality describes the sequence or overlap of audio events in the time domain; it contains four sub-relations: Precedence: Ej < Eg (event E₁ occurs before E2); Succession: E₁ > Eg (event E, occurs after Ez); Simultaneity: E, Eg (E, and E, occur concurrently); Repetitiveness: E, (event E occurs repetitively in the time domain).
FSDSOK (Fonseca et al., 2022) data is also sourced from freesound.org org
4

---

Published as a conference paper at ICLR 2026
2. Spatiality defines the relative spatial positions motion status between within audio events; it contains five sub-relations: Proximity: d(E E2) & (E₁ and Eg are within distance 7); Close- ness: d(E,) < E2) (Ej is closer than E2); Farness: d(En) dl E2) (E) is farther than E2); Approaching: de dz (t) 0 (E, is moving closer); Departing: di dz (t) 0 (Ei is moving away).
3. Count focuses on the number of audio events that take place within a period of time: Count:
[8] N,Ne (the cardinality of & is the number).
Perceptuality introduces six acoustic effects to an audio event,
Balancing: Roalance 1,, E2,o) (level balance between E1 and Eg by balancing factor or, so that one event dominates and the other serves as the background audio).
Blending: Reiena F,, F2 0) (mix Ej and Eg together by factor so as to be indistinguishable). Reverberation: Reweer E1) applics a reverberation effect to EJ, as is heard in a canyon.
stre-stretching: Ratmen Ei, ot), where the time-stretching factor and E₁ sounds slower. Amplification: Rampl E1, 9), where is the amplification factor and E₁ sounds louder. Attenuation: Rau E1, 7), where is the attenuation factor and E₁ sounds quieter.
5. Compositionality indicates the logical operations within audio events that TTA models need to reason about before deciding what audio events to generate. contains five sub-relations.
Conjunction: E₁ E2 (both events occur).
Disjunction: E1 Eg (at least one event occurs, or both occur).
Negation: - E1 (the absence of the event E₁ in the generated audio).
Exclusive Or: EE E2) ^ -(E A E2) (either E₁ or E2 occur, but not both).
Implication: Ei E2, -E = E3 (if E₁ occurs, then Eg occurs; otherwise, E' occurs).
Nested Combination is hierarchical structuring multiple basic relations (e.g., the aforemen- tioned Temporality, Spatiality), such that the output of one relation serves as the input context for another, forming directed acyclic relation structure. Nested combination allows for capturing complex relation interactions among audio events. For example, by nesting Implication, Approaching and Conjunction, we can generate a more complex text prompt showing below,
Nest Combination Example: Implication, Approaching and Conjunction
If generated both {A} event and [B} event, - Conjunction then continue to generate {C} audio event, else just generate {D) audio event that is gradually approaching close. Approaching
Mathematically, the relation Rnested F)) resulting from nested combination can be represented as,
Raeseal L)
Rr (Rn (E))
where = {E1,, ....... ) represents a finite of audio events. We combine relations aris- ing from the introduced 5 basic relations to con- struct nested combination relations and have cre- ated 79 nested combination relations.
main colation arity coverage
sub-relarion number wirt anty
40
Relation
Pompoala
Compouttonnlity
12
worth noting that nested combinations are scalable, and we can theoretically construct more complex nested relations (even infinite relations) by involving more basic relations in the nesting process. this work, we constrain nested combinations to involve at most five dio events (Quinary). remains a future re- search topic to explore more complex nested
fomoptiallt
Nix Combination
Cian
relations with higher arity by sested combination
Spatialiy
Anty
Unery
Binery
Temary
Quatemory
Quinary
Homury
Stptonary
Figure 3: Arity coverage in Aedio
5

---

Published as a conference paper at ICLR 2026
combinations, and the key challenge is how to construct concise and precise textual descriptions for highly complex nested relations. Moreover, during the nesting process, we explicitly run internal logical-correetnesss and feasibility checks before accepting a nested relation; any nested relation that violates these rules is abandoned. For example, the combination of Count and Conjunction internally equals Count.
Relation Arity. each relation in AudioRelSet is associated with an "arity" property, which indicates the audio event number requires represent the relation. The visual illustration of arity is shown in Fig. 2 C. The arity coverage across AudioRelSet main relation categories is given in Fig, 3, from which we can see that the arity ranges from across multiple arities. Moreover, the construction of more complex relations introduces higher arity. We use "arity" to create <text, audi o> pairs (see Sec. 3.3) and experiment evaluation (see Sec. 4).
(unary to quinary) and most main relation cuts
3.3 TEXT-AUDIO PAIR CREATION: <Txx Audio
With the constructed audio event corpus in Sec. 3.1 and relation corpus in See. 3.2, we can further construct relation aware <text, audio> pairs. Specifically, as is shown in Fig. 4, we first associate each of the 100 relations in the relation corpus with metic- ulously curated 5 text description templates. We either manually write or query GPT-4o to gen- erate 5 text prompt templates precisely describ- ing the relation and accommodating the large language usage variation (see Fig. line 4-8). Each template contains audio events name place- holder, we instantiate the template by replacing the placeholder with real audio event name to obtain the text prompt. To accommodate the synonymy of audio event name, we maintain synonym list for each audio event name, and ran- domly select one each time when instantiating the template. For example, the audio event name "hammer nailing" can be synonymously replaced by one of [hitting, slapping, smacking, punching).
AndioFFverSct
AudioRelSa.t
Machingry
Animal
Compostiinaalt com
Sputility
mechinimm
livestock
implication
elosenoss
paper printig
mooing
rel2xed somplaization
event instantistium
If ganernted [A), than ganraet
[BI, ese generute (C); neen
zudloo
5. should (N) be created, proceed
audio
create (B), olhersise create (C
madio at
paper prinkng if
Indio
Wadio
atherwise, geat bleating
asstastiated
instantizzed
-intt, Audio
Figure 4: <text, audio> pair generation illus- tration, which can generate nearly unlimited pairs with high diversity.
To accurately describe the audio event with text, we adopt the "Head-Modifier Structure with Progressive Verb Form" approach. In this approach, the description begins with the subject or entity producing the audio (e.g., "food") as the head, emphasizing the primary source of the sound. The action is then specified using its present participle form (e.g., "frying") as the modifier to convey sense of immediacy and highlight that the audio event is ongoing. For instance, instead of describing a sound as "frying food" or "fry food" it is labeled "food frying audio," where the subject ("food")) is foregrounded, and the action ("frying") contextualizes the nature of the audio. This approach ensures clarity, aligns with the temporal context of the audio, and effectively captures the dynamic nature of the event. With the same audio events name, we can retrieve its relevant audio waveform data and generate the audio by following the relation (He et al., 2025).
EXPERIMENT
4.1 DATASET CONSTRUCTION
Following the common setup in existing TTA model, the created audio is 10 second long with sampling rate 16 kHz. Based on the data creation method introduced in Sec. 3.3, in the training phase we randomly construct 360 <text, audio> pairs for each relation, resulting in 36,000 pairs in total. In the testing phase, we randomly construct 100 <text, audio> pairs for each relation, ensuring that constructed pairs appear in the training dataset. Following prior TTA model settings, we set each audio sample to 10 seconds with a 16 kHz sampling rate; the training dataset is 100 hours, and the testing dataset is 28 hours. Since decouple relations from audio events during dataset

---

Published as a conference paper at ICLR 2026
construction, and the training texts differ from the testing texts, the constructed training and testing <text, audio> pairs have no overlap and differ significantly.
4.2 EVALUATION METRIC
We use both classic general evaluation metrics and relation aware evaluation metrics, For general evaluation, we follow traditional TTA works (Liu et al., 2024; 2023; Ghosal et al., 2023; Majumder et al., 2024) and adopt three metrics: Fréchet Audio Distance (FAD), Fréchet Distance (FD) (Heusel et al., 2017), and Kullback-Leibler (KL) divergence. These three metrics measure the overall similarity in embedding space between reference audio and generated target audio without explicitly taking relations into account. Specifically, following prior TTA works, extract embeddings from the VGGish (Hershey et al., 2017) model for FAD and KI. and embeddings from the PANNs (Kong al., 2020) model for FD.
For relation aware evaluation, we adopt the multi-stage relation aware (MSR) evaluation protocol introduced in RiTTA (He et al., 2025). In the MSR protocol, we first explicitly extract audio events and relations (E", RR') from generated audio, and then compare them with reference audio events and relations (E, R). To reflect whether the model has generated, and only generated, the designated audio events and re relations, MSR adopts Presence, Relation correctness, and Parsimony scores to gauge the quality of generated audio from different perspectives. Specifically, we report mAPre, mARel, 1, and mAPar scores for either individual relations or all relations. More detailed information about the MSR metric can be found in RiTTA (He et al., 2025). To extract audio events from generated audio, we fine-tune an audio event detection and tagging model on top of the pre-trained PANNs (Kong et al., 2020) model with 1 amillion-sample training dataset. The mAP on a -0000ll- testing dataset reaches 0.91 for audio event detection, ensuring the fine-tuned model can extract all potential audio events with high precision. To classify acoustic effects, we train another seven-class acoustic-effects classifier top of the pre-trained PANNs model with 1-milfion-sample training dataset. The accuracy on 100 k testing dataset reaches 95%.
4.3 BENCHMARKING METHODS
We exhaustively benchmark 9 of the most recent general TTA models: AudioLDM (Liu et al., 2023), AudioLDM 2 (Liu et al., 2024), MakeAAAAdio (Huang et al., 2023), AudioGen (Kreuk et al., 2023), Tango (Ghosal et al., 2023), Tango 2 Majumder et al. (2024), LAFMA (Guan et al., 2024), Auffusion (Xue et al., 2024), and TangoFlux (Hung et al., 2026). These models are pre-trained on general TTA datasets (Gemmeke et al., 2017; Kim et al., 2019). For benchmarking, we use their released checkpoints to generate 10 second audio from text prompts; detailed configurations are provided in Table III in the Appendix.
We further benchmark two agentic workflow based methods, in which we leverage an open-source Owen-famill LLM as an agent to analyze the input text and output the separate audio events a TTA model needs generate. The purpose of experimenting with this agentic workflow is to see whether we can decompose the relation aware generation task into simpler finll--audo-ewwe generation tasks. Detailed implementation of the agentic workflow provided in Appendix .3.
4.4 BENCHMARKING RESULT ON EXISTING TTA MODELS
The benchmarking results are shown in Table 2, from which we observe that all existing TTA models perform poorly on relation aware TTA generation. Similar to RiTTA (He et al., 2025), we also find contradictory evaluation outcome between general evaluation and relation- aware evaluation, which highlights the specificity of the relation-aware TTA task. Among all benchmarking methods, Audio- Gen (Kreuk et al., 2023) and TangoFlux (Hung et al., 2026) perform best. While AudioGen (Kreuk et al., 2023) achieves the best mAPar (relation parsimony) and mAMSR. TangoFlux (Hung et al., 2026) is best in mAPre and mARel, which means it excels at accurately generating the target audio events and corresponding relations. However, almost all benchmarking methods achieve less than a 10% accuracy rate across all relation aware evaluation metrics, which turn verifies the need to introduce new large-scale benchmark tailored for relation aware TTA research.
Furthermore, both agentic- flow baselines perform poorly; they are substantially worse than most existing TTA approaches. This poor performance highlights critical limitation: simply scaling up
7

---

Published as a conference paper at ICLR 2026
Table 2: Quantitative benchmarking result on our introduced benchmark. mAPre, mARel and mAPar are in 10-2 mAPre and mARel can be treated as presence, relation correctness percentage ratio, they lie in range [0, 100]. mAPar score also lies within [0. 100]. mAMSR (%) lies in range [0.
Eval Way
General Evaluation
Relation Aware Evaluation ()(1)
Model
#Param
FAD
KI. |
FD
mAPre
mARel
mAPar
mAMSR
AudioLDM (s-full) 2023
185 M
4.02
21.23
22.36
3.47
0.91
2.95
0.73
AudioLDM (I-full) 2023
739 M
4.13
22.05
23.03
3.10
0.79
2.63
0.63
AudioLDM 2 (1-full) 2024
844 M
4.54
22.90
30.53
0.35
0.04
0.31
0.03
Zero- Shot
MakeAnAudio 2023
452 M
5.10
50.97
30.49
4.75
0.88
4.05
0.73
AudioGen 2023
1.5
7.97
25.19
32.29
11.3
2.84
9.13
2.22
LAFMA 2024
M
25.85
269.54
65.27
0.96
0.15
0.45
0.07
Auffusion 2024
1.1
4.13
42.59
31.17
6.71
1.41
4.07
0.79
Tango 2023
866 M
7.47
64.10
28.28
4.46
0.98
3.67
0.79
Tango 2024
866 M
9.59
65.24
35.50
9.68
2.48
5.49
1.29
TangoFlux 2026
576 M
6.01
26.73
30.00
12.38
3.34
7.28
1.77
Agen
Qwen2 TaTagooFluxx
9.98
142.87
39.20
3.53
0.77
2.25
0.04
tic
Qwen2.5 32B+ TangoFlux
9.70
140.56
38.65
3.79
0.96
2.41
0.60
current TTA methods without fundamentally enhancing their relation aware modeling capability is unlikely to succeed. In this light, the benchmark introduced in this paper is not merely comparison tool but a catalyst, providing the necessary structure, evaluation, and motivation to drive genuine advances in relation aware TTA research.
4.5 TWo INTUITIVE WAYS TO IMPROVE RELATION AWARE MODELING
Table 3:
Quantitative result comparison
testset between finetuning and training from
scratch (scratch) on curated 100 hours dataset.
Train Strategy
General Evaluation
Relation Aware Evaluation %(()
Model
#Param
FAD
KL
FD
mAPre
mARel
mAPar
mAMSR
Tango 2023
866 M
3.88
33.26
21.30
14.58
4.18
10.16
2.73
finetuning
Tango 2 2024
866 M
4.06
22.39
20.32
15.53
4.63
10.21
2.86
TangoFlux 2026
576
1.29
9.68
16.44
28.57
8.02
20.84
5.58
scratch
Tango 2023
866 M
3.63
22.34
20.16
14.89
3.69
10.98
2.64
TangoFlux 2026
576 M
1.64
17.82
11.72
16.68
3.82
12.01
2.58
Two intuitive strategies to enhance relation aware mod- eling in existing TTA methods are (i) fine-tuning on our curated dataset and (ii) training from scratch. This dual perspective not only tests the feasibility of our benchmark but also evaluates the potential to transfer general TTA- domain knowledge into relation-ave aware settings. To this end, apply both training strategies to three representa- tive baselines: Tango (Ghosal et al., 2023), Tango (Ma- jumder et al., 2024), and TangoFlux (Hung et al., 2026). The results in Table reveal a clear trend: both fine-tuning and training from scratch substantially improve relation- aware performance, validating the effectiveness of our TTA.
Text Prompt: At the beginuing, generate applause sound that spatially distont, thos contimee generate applasse sound spatiall)
Maveform
Spectrou
Reference
TangxFllxx
TangoFilu finetuned
benchmark testing ground for relation- Notably, TangoFlux benefits the most from fine-tuning, indicating that cross-domain TTA knowledge can be ef- fectively transferred to relation aware tasks. In contrast, Tango shows little difference between the two strategies, suggesting that model architecture and inductive bias may affect the extent to which general TTA knowledge can be leveraged. These findings highlight our benchmark's s unique role in uncovering such model-specific behaviors and point to open research direction: how to best exploit general TTAA knowledge to scale up iilation-aware TTA, and conversely, how relation aware training reciprocally improve general TTA. We visualize the generated audio
Figure 5: Qualitative comparison be- tween zero-shot and finetune based Tan- goFlux inference
text prompt.
8

---

Published as a conference paper at ICLR 2026
Compostionaaity
Arly
Compontionlly
Peoporalty
Tomporality
Count
Count
Axteys
Anty 2
Aciey 5
Adcy
Noytr Condiittiin
Nestod Conaiinaiinn
Anity
Ansy's
Auby
Percaptnsliy
Perceptuality
AudioGen
Tango 2
Tangol'has lox
Tango
Trigo
TangoFlux
Figure mAMSR regarding main relation cat- egory and 5 relation Arity in Zero-shot setting.
Figure 7: mAMSR regarding main relation cat- egory and relation Arity finetuning setting.
comparison between TangoFlux in zero-shot and fine-tuned inference modes in Fig. 5; this figure clearly shows that fine-tuning on our curated dataset benefits relation aware modeling.
To further investigate the role of training data size, we extend both finetuning and training- toaning-froo---- -rratsc experi- ments to larger datasets of 200 hours and 300 hours. As shown in Fig. 8, the mAMSR trend reveals two distinct behaviors: finetuning yields strong early gains but quickly saturates as the data size approaches 300 hours, whereas training from scratch continues to improve substantially with increasing data. This divergence underscores an im- portant insight: scaling relation aware TTA models ulti- mately requires massive datasets, and reliance on finetun- ing alone may be insufficient for long-term progress. Our benchmark is therefore essential: it not only provides the controlled scaling environment needed expose these trends, but also offers the first practical platform to system- atically study how training strategy and data size interact in advancing relation aware TTA.
Figure 8: mAMSR variation w.r.t train- ing data size (100 h, 200 and 300 h).
We visualize mAMSR across main relation categories and relation arities for three strong methods in the zero-shot setting (Fig. 6) and the finetuning setting (Fig. 7). From these two figures, we observe that finetuning generally improves relation aware modeling capability. In the zero-shot setting, AudioGen (Kreuk et al., 2023) performs better than the other two main categories including Temporality. Count, and Perceptuality In the finetuning setting. TangoFlux (Hung et al., 2026) becomes the best-performing method. However, all methods in both settings perform poorly on Nested Combination or arity greater than Our proposed benchmark enables researchers to tackle these challenges scale.
4.6 MORE INVESTIGATION INTO EXISTING TTA MODELS
Relation aware TTA demands not only the correct presence of target audio events but also the faithful preservation of their underlying relations. However, current TTA methods (Hung al., 2026; Ghosal et al., 2023; Xue et al., 2024) remain narrowly focused on single-event generation, leaving them ill-equipped to handle multi-event, relation velation-aware aware prompts. Ta-
Table 4: Audio event and relation accuracy of TangoFlux generation under different setting.
Description
Accu.
Event (single event, no relation)
75%
Event ( multi-event, relation aware)
12%
ble makes this gap explicit: while TangoFlux (Hung et al., 2026), the state-of-the oftte-----eerttttt general TTA model, achieves 75% accuracy on single-event prompts, its performance collapses to just 12% for multi-event correctness and mere 3% for relation fidelity. This dramatic degradation exposes fundamental blind spot in existing approaches: relation aware modeling is virtually unaddressed. Our benchmark directly targets this deficiency, offering the first systematic platform to quantify and dissect these failures. By doing so, it not only diagnoses the shortcomings of current TTA methods but also establishes the essential foundation for driving genuine advances in relation-aware TTA.
Relation (multi-event, relation aware)
3%

---

Published as a conference paper at ICLR 2026
5 CONCLUSION AND DISCUSSION
This work presents Aurelius as a benchmark-centricc foundation for advancing relation aware text-to- audio (TTA) generation. Across extensive zero-shot and adapted evaluations, we show that existing general-purpose TTA models remain fundamentally weak at relation modeling: while they can often produce plausible single events, performance drops sharply when prompts require multi-event composition, relational consistency, and higher-arity reasoning. This gap confirms that relation fidelity not a minor extension of general TTA quality, but distinct capability that current methods do not reliably learn.
To address this bottleneck, Aurelius contributes two complementary resources: AudioEvenser (110 clean, diverse, hierarchically structured events) and AudioRelSet (100 relations with explicit arity and compositional structure): together with a scalable <text, audi o> pair construction pipeline and unified evaluation protocol. The benchmark enables controlled analysis across relation type, complexity, training strategy, and data scale, and therefore supports both diagnosis and method development.
We expect Aurelius to serve as a common testhed for relation aware generation, structured prompting, agentic decomposition, and new architectures that explicitly model event-relation dependencies. We also anticipate broader reuse in adjacent tasks within and beyond audio based machine learning. For example, AudioEventSer can be potentially used for adjacent acoustic tasks such as acoustic scene understanding, sound event detection and localization (He et al., 2021) and spatial audio modeling (He al., 2024); AudioRelSet can benefit relation modeling in other research comunities including computer vision, natural language processing and multimodality. In short, Aurelius establishes rigorous, extensible starting point for the next stage of structured TTA research, with the potential to motivate and advance other relevant research questions.
ACKNOWLEDGEMENT
We sincerely thank the reviewers for their constructive feedback and thoughtful engagement with this work, particularly for encouraging detailed clarifications during the multi-round discussion. Their input is invaluable for improving both the quality and clarity of this work.
REFERENCES
Andrea Agostinelli, Timo Denk, Zalán Borsos, Jesse Engel, Mauro Verzetti, Antoine Caillon, Qingqing Huang, Aren Jansen, Adam Roberts, Marco Tagliasacchi, et al. MusicLM: Generating Music from Text. arXiv preprint arXiv:2301.11322 2023. 2
Peihao Chen, Yang Zhang, Mingkui Tan, Hongdong Xiao, Deng Huang, and Chuang Gan. Generating Visually Aligned Sound from Videos. IEEE Transactions on Image Processing (TIP), 2020. 2
Jade Copet, Felix Kreuk, Itai Gat, Tal Remez, David Kant, Gabriel Synnaeve, Yossi Adi, and Alexan- dre Défossez. Simple and Controllable Music Generation. Advances Neural Information Processing Systems (NeurIPS), 2023. 2
Eduardo Fonseca, Xavier Favory, Jordi Pons, Frederic Font, and Xavier Serra. FSD50K: an Open Dataset of Human-labeles Sound Events. JEEE/ACM Transactions on Audio, Speech, and Lan- guage Processing (TASLPP), 90:829-8522 2022. 1,
J. F. Gemmeke, D. P. W. Ellis, D. Freedman, A. Jansen, W. Lawrence, R. C. Moore, M. Plakal, and M. Ritter. Audio Set: An Ontology and Human-Labeled Dataset for Audio Events. In IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2017. 1,
Deepanway Ghosal, Navonil Majumder, Ambuj Mehrish, and Soujanya Poria. Text-to-Audio Generation using Instruction Tuned L.M and Latent Diffusion Model. In ACM International Conference on Multimedia (ACMMM), 2023. 2, 8.
Sreyan Ghosh, Ashish Seth, Sonal Kumar, Utkarsh Tyagi, Chandra Kiran Reddy Evuru, Ra- maneswaran S, S Sakshi, Oriol Nieto, Ramani Duraiswami, and Dinesh Manocha. CompA:
10

---

Published as a conference paper at ICLR 2026
Addressing the Gap in Compositional Reasoning in Audio-Langunn Models, In International Conference on Learning Representations (ICLR), 2024. 2, 3, 4
Wenhao Guan, Kaidi Wang, Wangjin Zhou, Yang Wang, Feng Deng, Hui Wang, Lin Li, Qingyang Hong, and Yong Qin. LAFMA: A Latent Flow Matching Model for Text-to-Audin Generation. In Interspeech, 2024. 3, 7, 17
Yuhang He, Niki Trigoni, and Andrew Markham. SoundDet: Polyphonic Moving Sound Event Detection and Localization from Raw Waveform. In International Conference on Machine Learning (ICML), 2021. 10
Yuhang He, Anoop Cherian, Gordon Wichern, and Andrew Markham. Deep Neural Room Acoustics Primitive. In International Conference on Machine Learning (ICML), 2024. 10
Yuhang He, Yash Jain, Xubo Liu, Andrew Markham, and Vibhav Vineet. RiTTA: Modeling Event Relations Text-to-Audio Generation. In Conference on Empirical Methods in Natural Language Processing (EMNLP), 2025. 3, 4, 6, 16
Shawn Hershey, Sourish Chaudhuri, Daniel P. W. Ellis, Jort F. Gemmeke, Aren Jansen, Channing Moore, Manoj Plakal, Devin Platt, Rif A. Saurous, Bryan Seybold, Malcolm Slaney, Ron Weiss, and Kevin Wilson. CNN Architectures for Large-Scale Audio Classification. In International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2017.
Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler, and Sepp Hochreiter. GANs Trained by a Two Time-Scale Update Rule Converge to Local Nash Equilibrium. In Advances in Neural Information Processing Systems (NeurIPS), 2017.
Hirsh, Milliman, and F Darley. Brain Mechanisms Underlying Speech and Language, 1967. 1
Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising Diffusion Probabilistic Models. Advances in Neural Information Processing Systems (NeurIPS), 2020. 1,2
Rongjie Huang, Jiawei Huang, Dongchao Yang, Yi Ren, Luping Liu, Mingze Li, Zhenhui Ye, Jinglin Liu, Xiang Yin, and Zhou Zhao. Make-an-audion Text-to-audio Generation with Prompt-enhanced Diffusion Models. International Conference on Machine Learning (ICML), 2023. 7, 8, 17
Chia- Yu Hung, Navonil Majumder, Zhifeng Kong, Ambuj Mehrish, Rafael Valle, Bryan Catanzaro, and Soujanya Poria. TangoFlux: Super Fast and Faithful Text to Audio Generation with Flow Matching and Clap-Ranke Preference Optimization. In International Conference on Learning Representations (ICLR), 2026. 1, 3, 7, 8, 9, 14, 17
Vladimir lashin and Esa Rahtu. Taming Visually Guided Sound Generation. In British Machine Vision Conference (BMVC), 2021. 2
Chris Dongjoo Kim, Byeongchang Kim, Hyunmin Lee, and Gunhee Kim. Audiocaps: Generating captions for audios in the wild. In Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLLT), 2019. 1,
Qiuqiang Kong, Yin Cao, Turab Iqbal, Yuxuan Wang, Wenwu Wang, and Mark Plumbley. PANNS: Large-Scale Pretrained Audio Neural Networks for Audio Pattern Recognition. In IEEE/ACM Transactions Audio, Speech, and Language Processing (TASLP), 2020. 7
Felix Kreuk, Gabriel Synnaeve, Adam Polyak, Uriel Singer, Alexandre Défossez, Jade Copet, Devi Parikh, Yaniv Taigman, and Yossi Adi. Audiogen: Textually Guided Audio Generation. International Conference on Learning Representations (ICLR), 2023. 2, 3, 7, 8, 9, 17
Brenden M. Lake, Ruslan Salakhutdinov, and Joshua B. Tenenbaum. Human-leve Concept Learning hrough Probabilistic Program Induction. Science, 330(6266):1332-13333 1338, 2015. 1
Huan Liao, Haonan Han, Kai Yang, Tianjiao Du Rui Yang, Zumnan Xu, Qinmei Xu, Jingquan Liu. Jiasheng Lu, and Xiu Li. BATON: Aligning Text-to-Audie Model with Human Preference Feedback. International Joint Conference on Artificial Intelligence (IJCAI), 2024. 2, 3
11

---

Published as a conference paper at ICLR 2026
Yaron Lipman, Ricky T. Q. Chen, Heli Hemuu Maximilian Nickel, and Matthew Le. Flow Match- ing for Generative Modeling. In International Conference on Learning Representations (ICLR), 2023. 2
Haohe Liu, Zehua Chen, Yi Yuan, Xinhao Mei, Xubo Liu, Danilo Mandic, Wenwu Wang, and Mark D Plumbley. AudioLDM: Text-to-Audio Generation with Latent Diffusion Models. International Conference on Machine Learning (ICML), 2023. 2, 3, 7, 8, 17
Haohe Liu, Yi Yuan, Xubo Liu, Xinhao Mei, Qiuqiang Kong, Qiao Tian, Yuping Wang, Wenwu Wang, Yuxuan Wang, and Mark D. Plumbley. AudioLDM 2: Learning Holistic Audio Generation With Self-Supervised Pretraining. IEEE/ACM Trunsactions on Audio, Speech, and Language
Processing (TASLP), 2024. 2,
Nan Liu, Shuang Li, Yilun Du, Antonio Torralba, and Joshua B Tenenbaum. Compositional Vi- sual Generation with Composable Diffusion Models. In European Conference on Computer Vision (ECCV), 2022.
Navonil Majumder, Chia- Yu Hung. Deepanway Ghosal, Wei-Ning Hsu, Rada Mihalcea, and Soujanya Poria. Tango 2: Aligning Diffusion-based Text-to-Audic Generations Through Direct Preference Optimization. In ACM International Conference on Multimedia (ACMMM). 2024. 7, 8, 17
Roy Sheffer and Yossi Adi. Hear Your True Colors: Image Guided Audio Generation. In IEEE International Conference Acoustics, Speech and Signal Processing (ICASSP), 2023. 2
Arash Vahdat, Karsten Kreis, and Jan Kautz. Score-based Generative Modeling in Latent Space. In Neural Information Processing Systems (NeurIPS), 2021. 1.2
Somin Wadhwa, Silvio Amir. and Byron C. Wallace. Revisiting relation extraction in the era of large language models. In Annual Meeting of the Association for Computational Linguistics (ACL.), 2023. 2
Zeyu Xie, Xuenan Xu, Zhizheng Wu, and Mengyu Wu. AudioTime: A Temporally-alin alignndd Audio- Text Benchmark Dataset. IEEE International Conference on Acoustics, Speech and Signal Pro- cessing (ICASSP), 2025a. 2,
Zeyu Xie, Xuenan Xu, Zhizheng Wu, and Mengyue Wu. PicoAudio: Enabling Precise Temporal Controllability in Text-to-Audio Generation. In IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2025b. 3
Jinlong Xue, Yayue Deng, Yingming Gao, and Ya Li. Auffusion: Leveraging the Power of Diffusson and Large Language Models for Text-to-Audin Generation. JEEE/ACM Transactions on Audio, Speech, and Language Processing (TASLP), 2024. 1, 9,
Dongehao Yang, Jianwei Yu, Helin Wang, Wen Wang, Chao Weng, Yuexian Zou, and Dong Yu. Diffsound: Discrete Diffusion Model for Text-to-sound Generation. IEEE Transactions on Audio, Speech and Language Processing, 2022. 2,
Jeffrey M Zacks, Nicole K Speer, Khena M Swallow, Todd S Braver, and Jeremy R Reynolds. Event Perception: Mind-Brain Perspective, 2007. 1
Aimen Zerroug, Mohit Vaishnav, Julien Colin, Sebastian Musslick, and Thomas Serre. A Bench- mark for Compositional Visual Reasoning. In Advances in Neural Information Processing Sys- tems (NeurIPS), 2022.
12

---

Published as a conference paper at ICLR 2026
A
APPENDIX
.1 AUDIOEVENTSET DETAIL.
The detail of SudioEventSet is given in Table I.
.2
AUDIORELSET DETAIL
The detail AudioRelSet given Table
.3 AGENTIC WORKFLOW
The agentic workflow operates in three stages:
Planning Given the input text and the audio length to generated, we query LLM convert the input text into a JSON-formated plan. The prompt is provided below:
Planner Prompt
You audio scene planner for relation aware texi-to-audio (TTA) generation. Perform all reasoning INTERNALLY and output ONLY valid JSON with the final scheduling results. Never include explanations or thain-of-thought Your JSON MUST be parsable and follow the schema exactly.
OBJECTIVE
Given (1) a natural-language text prompt describing an audio scene with multiple events and relations, and (2) a target total duration in seconds, decompose the scene into concise sub-prompts (each corresponds to one independent audio event to be synthesized by a TTA model), and schedule them on the global timeline with start times and durations.
OUTPUT (JSON ONLY)
"total_duration__e <float>, // equals the requested total curation "sub_prompts":
"id": "E1",
// short unique id
"text": "<short English sub-prompt>",
1/ head present participle; may include light modifiers
"start_sec"; <float>, "duration_sec"; <float>
// >->00
11 0, and start_sec duration_sec total_durationssec
11 ... typically <= items total
LANGUAGE FOR SUB-PROMPTS
Use concise English in "head + present participle" style: e.g., "door bell ringing audio", "footsteps running audio".
Add light, meaningful modifiers when clearly implied by text: "as background", "approaching", "departing", "slight reverberation". "time-stretched", "amplified", "attenuated". "balanced against X", "blended with Y".
Avoid redundant words; keep each sub-prompt single-sentence and & 18 words.
RELATIONS AND How TO REFLECT THEM (IMPLICITLY VIA SCHEDULING + WORDING)
Sequencing: "then/after/nexx/fistt. then.. schedule sequentially with a small gap 0.1s.
13

---

Published as a conference paper at ICLR 2026
Simultaneity / Mix / Background allow overlaps: backgrounds can span large portions of the timeline.
Approaching Departing- - keep wording ("aproacding""drpattnnn'", No
mandatory duration change.
OR / XOR - choose the most natural option; do not include the unchosen one. / prohibit - exclude that event entirely.
NOT
IF...HHEN. ELSE - choose the most sensible branch; output only the chosen branch.
Count - if an explicit number of items/events is requested, match it.
Repetition - instantiate repeated events (e.g.. bell ringing three times) multiple sub-segments or one sustained segment implied.
Farmess reflect via wording only ("distant thunder", "near crowd"). No strict timing rules.
Proximity Closeness
DURATION AND SCHEDULING RULES
Respect total duration: sum of all segments should match the requested duration (+0.25s); if off, adjust proportionally.
Each segment duration_sed MUST be integer.
Choose reasonable segment durations:
transient cues (e.g., bell, door knock, gunshot) - 1-2s
medium actions (e.g., footsteps, typing, sawing) - 2-6s
ambient backgrounds (e.g., rain, wind, crowd murmur) - long spans (often entire duration)
start sec 0: end < total duration.
On conflicts, preserve explicit relations first; compress lightly but keep segments 0.5s.
Keep the number of sub- prompts concise (typically < 5).
FORMATTING RULES
Output VALID JSON only. No comments, no trailing commas, no text outside JSON.
Floats may be given with 1-2 decimals.
Ensure "start.sec duration.see & totallduraation.see"" for all segments.
Segment Synthesis. Each sub-prompt is independently synthesized into audio by using Tan- goFlux (Hung et al., 2026).
Stitching. All generated audio segments are linearly composited by respecting their designated started time to generate the final audio.
.4 TTA MODEL BASELINES INFERENCE SETTING
14

---

Published as a conference paper at ICLR 2026
Table I: AudioEventSer corpus detail. We list all 110 event classes, which are deriving from 7 main categories and 23 sub-categories.
Main Category
Sub-Cann CateCaayg
Names
Description
lion roaring, wolf howling. donkey braying, cricket chirping, frog croaking, horse neighing dog barking, cat meowing, dog growling, cat spurring pig oinking, sheep bleating,
wild ground animal
the wild
Animal (22)
domestic animal
domestic setting
livestock
cow mooing, rooster crowing, duck quacking cuckoo calling, birds chorus. seagull cawing, peacock rattling blue jay whistling, nightingale singing, fly buzzing
domesticated livestock
wild animal
animals in the wild
baby crying. laughing, shouting, whistling, coughing, snoring, sneezing, chewing, burping, farting male speech, female speech, child speech, group talk finger snapping, clapping
human voice
human use vocal tract
Human (21)
human speech
speech audio
hands action
audio by action
group clapping. cheering, group talking running, footsteps
group action
audio by a group
locomotion
audio by movement
siren, door bell, car hom, bicyclc bell, telephone ringing, lelephone dialing, boat horn ratchet and pawl clicking, camera shuttering.
Machinery (13)
alarm
machinery alarming
mechanism
mechanism audio
printer printing, engine revving, clock ticking, paper shredding
hammer nailing, wood sawing, pen writing. wood chopping. rasping dish audio, silverware audio, food frying. vegetable chopping toilet flushing, pouring water, keyhoard typing, door slamming, cupboard open or close, drawer open or close, packing tape, dentist drilling, door knocking
Human -bjj Interaction (18)
tools
human use tools
culinary
kitchen setting
work
audio during work
key jingling, ball bouncing, pen clicking, wind chime emergency braking knife sharpening, sandpaper scraping, plastic scratching, string rubbing coin dropping. glass clinking, metal dropping gunshot, firework, artillery fire
impact audio
impact effect
Obj-Obj Interaction (15)
friction audio
friction effect
dropping audio
dropping effect explosion effect
explosion
plucked string, piano keyhoard, bowed string, wind string, brass,
Music (11)
music instrument
musical instruments
harmonica, accordion female singing, male singing, child singing, group singing
singing
singing audio
water bubbling, ocean wave, water dripping, water flowing. water boiling thunder, wind, rain wood cracking. rustling leaves
water
water movement
Nature (10)
weather
nature weather
nature change
natural change
15

---

Published as a conference paper at ICLR 2026
Table II: AudioRelSet corpus detail. We introduce 21 basic relations, and advanced 79 nested combination relations, resulting in a total of 100 relations - -9 times larger than the relation corpus proposed in RiTTA (He et al., 2025). AudioRelSer maximumly covers all potential relations that audio events may exhibit in either the physical world or linguistic description. It is worth noting that AudioRelSet is open-ended. By nesting existing relations, we can potentially construct massive new relations.
Category
Relation Name
Explanation
Event Arity
Sample prompt
precedence
before after
binary
audio (A) followed by (B)
succession
binary
create audio [A] after [B]
Temporality (4)
simultaneity
same time
binary
{a.} and {B} simultancously
periodicity
cyclic
unary
create audio {a} periodically
closeness
spatial close spatial far
binary
[A] is closer than (2)
farness
binary
[ is farther than audio {B}
Spatiality (5)
proximity
equal-dist
binary
[a] and (B) / the same dist
approaching
moving close moving away
unary
a.) is moving closer
departuring
unary
AA) is moving further away.
Count (1)
count
number
n-ary
3 audios: (A). (B) and {c}
balancing
level balance
binary
{A] dominates, {8}} fades
blending
mix audios
binary
[a. and [B} are mixed
Perceptuality (6)
reverberation
reverberant
unary
generate audio (A] in canyon
time-stretching
speed manipulate
unary
stretch audio (A} in time scale
amplification
become louder less loudly
unary
amplify { to be louder
attenuation
unary
attenuate {A} to be quicter
conjunction
logical AND
binary
create both {^} and {B}
Composition- ality
disjunction
logical OR
binary
create {a} or {B), or both
negation
logical NOT
unary
do not generate audio {A.
(5)
exclusive-or
logical XOR
hinary
generate {A} or (R), not both {A}, then (R), else just {c)
implication
if-then-else
temary
Temp + Spat (4)
Temp + Spat
{A] before approaching (B}
Temp + Percep (8)
Temp Percep
reverb. {A), succeeded by {1})
binary
Percep + Comp (12)
Percep + Comp
stretched {A} or {B}, not both
approaching {A} or {B),
Spat + Comp (4)
Spat + Comp
not both
Temp + Comp (6)
Temp + Comp
[A] first, then (B) or {c)
Nested Combination (79)
Percep + Comp (1)
Percep + Comp
mix [A] with (B), or [c}
Comp Comp (1)
Comp + Comp
ternary
{a} and [B}, or {A} and {c}
Spat Comp (5)
Comp Comp
{2} and (B). (A) and {c}
Spat Comp
Comp Comp
[A] and (B), (A) and {c]
+ Percep (2)
audio {A} or {3} first,
Temp + Comp (4)
Temp + Comp
quatemary
followed by [c} or (D)
[A] (B} first.
Comp Comp (7)
Comp Comp
then {c) or (D}
(a) before (B) first, then (c)
Temp + Comp (3)
Temp + Comp
before (D) or (E)
[a] closer than {B), then
Spat Comp (9)
Spat Comp
quinary
{c} closer than {D), else {E}
{A} and {B}, then {c)
Comp Comp (9)
Comp Comp
and (D) else (E}
if (A). (B), (C), then (D).
Count Comp (4)
Count Comp
else {E)
16

---

Published as a conference paper at ICLR 2026
Methods
Setting
AudioLDM (S-Full) (2023)
guidance_scale=33 randon_ 422 n_candidates=3
AudioLDM (L-Full) (2023)
guidance_scale=5,, randdaa random_seed=42, 3
AudioLDM 2 (L-Full) (2023)
guidance miidance candom_seed-44 c.aadidatee=-3
MakeAnAudio (2023)
ddim_steps 100, scale 3.0 model name: audiogen-medium
AudioGen (2023)
Auffusion (2024)
num steps = 100, guidance=7.5,, num samples=1
LAFMA (2024)
num_steps 200, guidance=3, num_samples=1
Tango (2023)
num_steps 200, guidance=3, num_samples=1
Tango 2 (2024)
num num-steps 200, guidance=3, num samples=1
TangoFlux (2026)
num num-steps = 50, guidance=3, num-samples=|
Table III: Detail setting for each TTA method.
17


Reviews:
[{"review_id": "fo0NLPMnz0", "reviewer": "Reviewer_hVjf", "summary": "This paper introduces Aurelius, a large-scale benchmark framework with two corpora, AudioEventSet and AudioRelSet, that enables systematic evaluation and development of relation-aware text-to-audio generation at scale.", "questions": "Please refer to the weaknesses mentioned above.", "limitations": "", "rating": 4, "confidence": 2, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "- The paper tackles the well-established limitation of current TTA models in generating audio with accurate temporal ordering and relational structures, which is an interesting and important research problem.\n- The framework's approach of combining relation templates with audio events to generate numerous <text, audio> pairs provides excellent flexibility and scalability. The adoption of the \"Head-Modifier Structure with Progressive Verb Form\" (e.g., \"door bell ringing audio\" rather than \"ringing door bell\") ensures syntactic consistency across the dataset.", "weaknesses": "- The GPT-generated templates or synonyms are not always accurate, and some generated texts may not properly correspond to the actual audio events, leading to potential noise in the dataset.\n- Since the generated sounds are synthetic, there is no clear way to assess or guarantee their perceptual quality.\n- Compared to the existing datasets shown in Table 1, the improvement in performance is not clearly demonstrated. There is no head-to-head comparison between models trained on existing datasets and those trained on Aurelius, making it difficult to quantitatively verify the superiority of the proposed dataset.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "mJuVLorC3g", "reviewer": "Reviewer_SfjC", "summary": "The paper presents Aurelius, which contains:\n(1) AudioEventSet, which contains clean clips for audio events\n(2) AudioRelSet, which contains the relative relationship of audio events.\n\nThe authors show that, by combining these two resources, we can generate theoretically unlimited relation-aware text-audio pairs. \n\nThe authors also present the evaluation method (benchmarks) for the relation-aware audio.\n\nIt also shows that training existing TTA models on the proposed dataset can benefit the relation-aware TTA performance.\n\nThe relation-awareness is an important property in current TTA, and this paper is a very good resource.", "questions": "In general, whether the resources would be made public is an important metric for the paper evaluation. Would the authors release them?", "limitations": "", "rating": 4, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "The paper comprehensively discusses the relation-aware TTA generation. It contains good resources, benchmarks and sufficient discussion.\n\nThe relation-awareness is of wide interest in the TTA community. The paper also reveals that the current TTA is not good enough in this direction.", "weaknesses": "(1) The paper is mostly about the resources (data, benchmark) of building relation-ware TTA. For this kind of resource paper, I wonder if the author would make it public.\n(2) In section 3, although the author provides a comprehensive design in the data content, they don't mention (1) why the designs are reasonable and (2) how they ensure the intended design philosophy is well implemented in practice. e.g., how they ensure the audio clips in AudioEventSet are precise and clean enough; why such designs in AudioRelSet are reasonable. Such missing information would compromise the contribution of the work.\n(3) I'm a bit confused about the mAMSR metric: in the Table 2 caption, you mention its range is [0, 1], but numbers in Table 3 are beyond this range.\n(4) Even with this carefully designed data pipeline, the overall mAPre, mARel, etc, are still absolutely low. (e.g., <30%). This very tailor-made dataset seems not to solve the issue very well.\n(5) The authors claim that unlimited data simulation is feasible, but scaling up the simulated data is not very effective, as shown in Figure 6.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "JGoz10oUW6", "reviewer": "Reviewer_2P2L", "summary": "This paper introduces Aurelius, a new framework designed to enable relation-aware text-to-audio (TTA) generation research at scale. The authors also propose two accompanying datasets, AudioRealSet and AudioEventSet, which demonstrate effectiveness in audio generation tasks. Overall, the proposed datasets are valuable contributions to the audio research community. However, the paper lacks novel methodological and technical innovations beyond dataset construction. (But the dataset is a great one!)", "questions": "N.A.", "limitations": "", "rating": 4, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- The release of a large-scale, well-curated dataset is always beneficial to the audio research community.\n\n- The authors conduct comprehensive benchmarking of existing TTA models to evaluate the proposed datasets.\n\n- The distinction between AudioEventSet and AudioRealSet is clearly defined. In particular, AudioRealSet provides rich attribute-level annotations for sound events, which is a valuable addition.", "weaknesses": "- My main concern is that, despite the usefulness of the dataset, there are no novel methodological or technical contributions. The paper reads primarily as a dataset paper rather than a technical paper.\n\n- In the Introduction, the term “relation modeling” is not clearly defined.\n\n- In Section 3.1, the phrase “audio events potentially present in the 3D physical world is unclear” is ambiguous — clarification is needed.\n\n- The authors claim that AudioEventSet is more distinctive than AudioSet, but no supporting evidence or analysis is provided. Since AudioEventSet is also manually designed, how do the authors ensure that the 110 sound classes are well-separated and not confusing?\n\n- The size of AudioEventSet should be reported more clearly. As it is built from Freesound and FSD50K (which contains only ~50k clips), it is unclear how large the final dataset actually is or how scalability is achieved.\n\n- In Section 3.2, is AudioRealSet a subset of AudioEventSet? How are the relations labeled — automatically or manually? The writing in this section could be improved for clarity.\n\n- The experimental section lacks comprehensive evaluation of how the proposed datasets improve performance. For instance, how much improvement is observed when models are trained with these datasets compared to without them?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "W0nkXJcjDE", "reviewer": "Reviewer_P1t4", "summary": "This paper proposes a relation-aware text-to-audio generation framework and introduces two newly developed datasets, AudioEventSet and AudioRelSet, to improve the modeling of relational structures between audio events. The datasets are organized in a tree-structured hierarchy and are designed to enhance compositionality and relation understanding in audio generation. The authors argue that these datasets can serve as new benchmarks for relation-aware audio generation.", "questions": "Previous works such as RiTTA and CompA also address relation-aware text-to-audio generation. Beyond the newly developed datasets, what is the core conceptual or technical difference between this work and prior ones?\n\nHow exactly were the datasets collected — are all audio-text pairs manually constructed or verified? If so, how was data distinctiveness and quality control ensured?\n\nDid the authors listen to and validate all audio compositions to ensure naturalness and correct temporal ordering of events?\n\nThe proposed tree structure seems to play no role in evaluation — could the authors propose or experiment with metrics that explicitly utilize this structure?\n\nWhy was the relation-aware improvement only tested on the Tango model family? Would results generalize to other text-to-audio architectures (e.g., AudioLDM, Make-An-Audio)?\n\nIs there any other way to improve the performance on relation aware?  Not just training on the \"new-proposed dataset\".\n\nAny demo or public data for the dataset (waveform)?", "limitations": "", "rating": 4, "confidence": 4, "soundness": 3, "presentation": 2, "contribution": 2, "strengths": "The paper focuses on an important and relatively underexplored topic — relation-aware text-to-audio generation, which aims to capture richer dependencies between sound events beyond conventional text-to-audio mapping.\n\nThe proposed datasets include explicit relation-level annotations (such as “arity,” count, and compositional structure), which provide more fine-grained relational information than most existing datasets.\n\nThe authors attempt to model hierarchical relations between sound events, potentially offering insights into how complex auditory scenes could be represented in structured data formats.", "weaknesses": "The main contribution lies almost entirely in dataset creation, with no substantial methodological innovation or new generation framework beyond existing relation-aware approaches (e.g., RiTTA or CompA).\n\nThe datasets are not released, and only high-level descriptions are given. Without access to examples or samples, it is impossible to verify the data quality or reproducibility.\n\nThe tree structure design (depth 3 for AudioEventSet and depth 2 for AudioRelSet) lacks theoretical or empirical justification. It is unclear why this specific hierarchy benefits the relation-aware task or how the root/leaf nodes are defined.\n\nThe data generation process appears mostly manual or GPT-assisted, rather than automatic. This makes it resource-intensive, potentially inconsistent, and difficult to scale. The resulting compositions may sound unnatural or ambiguous, especially since all clips are fixed to 10 seconds regardless of content or event duration.\n\nThe evaluation methodology is largely inherited from general text-to-audio works and RiTTA, without introducing new metrics to evaluate the proposed hierarchical or relation-specific structures.\n\nOverall, the paper reads more like an engineering dataset report than a research paper introducing conceptual or algorithmic innovation.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)