OWL : Geometry-Aware Spatial Reasoning for Audio Large Language Models
Authors: Subrata Biswas
Mohammad Nur Hossain Khan
Bashima Islam
Venue: ICLR 2026 Poster (2026)
Keywords: Chain of Thoughts
Spatial Reasoning
Primary area: foundation or frontier models, including LLMs

Abstract: Spatial reasoning is fundamental to auditory perception, yet current audio large
language models (ALLMs) largely rely on unstructured binaural cues and single-
step inference. This limits both perceptual accuracy in direction and distance
estimation and the capacity for interpretable reasoning. Recent work such as BAT
demonstrates spatial QA with binaural audio, but its reliance on coarse categorical
labels (left, right, up, down) and the absence of explicit geometric supervision
constrain resolution and robustness. We introduce the $\textbf{Spatial-Acoustic Geometry
Encoder (SAGE}$), a geometry-aware audio encoder that aligns binaural acoustic
features with 3D spatial structure using panoramic depth images and room-impulse
responses at training time, while requiring only audio at inference. Building on this
representation, we present $\textbf{OWL}$, an ALLM that integrates $\textbf{SAGE}$ with a spatially
grounded chain-of-thought to rationalize over direction-of-arrivals (DoA) and
distance estimates. Through curriculum learning from perceptual QA to multi-step
reasoning, $\textbf{OWL}$ supports o’clock-level azimuth and DoA
estimation. To enable large-scale training and evaluation, we construct and release $\textbf{BiDepth}$,
a dataset of over one million QA pairs combining binaural audio with panoramic
depth images and room impulse responses across both in-room and out-of-room scenarios. Across two benchmark datasets, our new $\textbf{BiDepth}$ and the public SpatialSoundQA, $\textbf{OWL}$ reduces mean DoA error by $\textbf{11$^{\circ}$}$ through $\textbf{SAGE}$
and improves spatial reasoning QA accuracy by up to $\textbf{25}$% over BAT. Our dataset and code are available at: https://github.com/BASHLab/OWL

Published as a conference paper at ICLR 2026
WOWL: GEOMETRY-AWARD SPATIAL REASONING FOR 88
AUDIO LARGE LANGUAGE MODELS
Subrata Biswas", Mohammad Nur Hosssain Khan & Bashima Islam
Department of Electrical & Computer Engineering Worcester Polytechnic Institute Worcester, MA 01609, USA {sbiswas, mkhan, (ialam)ewpie edu
ABSTRACT
Spatial reasoning is fundamental to auditory perception, yet current audio large language models (ALLMs) largely rely on unstructured binaural cues and single- step inference. This limits both perceptual accuracy in direction and distance estimation and the capacity for interpretable reasoning. Recent work such as BAT demonstrates spatial QA with binaural audio, but its reliance coarse categorical labels (left, right, up, down) and the absence of explicit geometric supervision constrain resolution and robustness. We introduce the Spatial-Acoustic Geometry Encoder (SAGE). geometry-ave aware audio encoder that aligns binaural acoustic features with 3D spatial structure using panoramic depth images and simulated room-impulse responses at training time, while requiring only audio at inference. Building on this representation, we present OWL, an ALLM that integrates SAGE with spatially grounded chain-of-thought rationalize over direction-of-ariin (DoA) and distance estimates. Through curriculum learning from perceptual QA multi-step reasoning, OWL supports o' d'llokklevel azimuth and DoA estimation. To enable large-scale training and evaluation, we construct and release BiDepth, dataset of over million QA pairs combining binaural audio with panoramic depth images and room impulse responses across both in-room and out-of-room scenarios. Across two benchmark datasets, our new BiDepth and the public SpatialSoundQA, OWL reduces mean DoA егтоr by 11 through SAGE and improves spatial reasoning QA accuracy by up to over BAT. Our dataset and code are available at: https: /tte://gittti com BASHLab/OHL...
INTRODUCTION
Large language models (LLMs) Achiam et al. (2023); Team et al. (2023): Touvron et al. (0023
have catalyzed rapid progress beyond pure text, inspiring multimodal systems Biswas et al. 2025
that pair an LLM backbone with modality encoders for vision Liu et al. (2023): Li et al. (002)),
audio Gong al. (2023); Ghosh al. (2025); Goel et al. (2025). and other sensors Imran al.
(2024): Leng et al. 2024): ; Ouyang & Srivastava (2024). Through projection layers for cross-modal
over joint representations, and follow instructions in context by training paired (modality, text) data. Within audio, such models learn to align acoustic features with language to parse complex sound scenes, recognize events and speaker attributes, and carry out dialogue conditioned on what they "hear" Early results show strong zero-shot generalization and retrieval capabilities when trained scale, and instruction tuning further enables conversational audio-conditioned queries.
alignment, these systems can process heterogeneous inputs,
Despite this momentum, audio-augmented multimodal LLMS lag behind their vision-language counterparts Liu et al. (2024); Bai et al.) (2023); Li et al. (2024) due to the unique challenges of sound: long-range temporal dependencies, nonstationary noise, and the need to capture geometric cues of the acoustic environment that shape auditory perception. Even advanced models such as Gemini-22 5- flash Comanici et al. (2025) struggle with composite acoustic tasks that require fine-grained spatial reasoning, highlighting persistent gaps in audio-language understanding, Recent
Equal Contribution.

---

Published as a conference paper at ICLR 2026
Cat meow
Music
Gemini-2.5-Flash: From my listening position direction the sound of the cat's meow is very much centered. It doesn't pan significantly left or right.
8 o'clock; down, 2.0
o'clock; down, 1.5
8 o'clock is lelt center and 1'0 clock is not.
OWL: Cat mcow originates from 8 o'clock while music is at 1 o'clock. Therefore, Cat mcow is on the left side of the listener.
Q: Which sound source is left to the listener?
Cat meow
Multi-step Reasoning Path
Figure I: SAGE E encodes binaural audio into spatially grounded representations. OWL detects events, localizes by direction and distance, and applies multi-step reasoning, yielding interpretable rationales for queries such as "Which sound source is left of the listener?"
advancements, such as BAT Zheng al. (2024b), demonstrates spatial reasoning capabilities, but its localization remains coarse, subdividing the scene into only four broad regions (e.g., front-left, left-behind). However, many downstream tasks, such as fine-grained source tracking, relative distance estimation, and multi-source disambiguation, require more precise understanding of spatial cues.
These gaps stem from two fundamental limitations in current audio large language models (ALLMs). (i) Lack of geometric grounding. Existing encoders capture spectral and temporal patterns but overlook crucial spatial cues such as direet-to-reverberatiion ratios, reverberation time (RT60), and room layout that determine how sound propagates. As result, models can recognize sound events but fail at spatial reasoning tasks such as deciding which source is closer or determining whether a sound originates from the left right. (ii) Single-pass reasoning. Current ALLMs map questions directly to answers without intermediate inference steps. This prevents them from decomposing complex acoustic queries into smaller, interpretable subproblems. Consequently, they falter in multi-source scenes and queries that require step-by-step spatial reasoning.
We address these limitations by proposing OWL, a framework powered by the Spatial-Acouste Geometry Encoder (SAGE). Unlike BAT, OWL combines geometry-cov geometry.conditioned conditioned training with spatially grounded chain-of-thhuuht (CoT), enabling finer localization and structured inference for complex queries. SAGE jointly models acoustic and geometric properties of environments, incorporating cues such as directionality, distance-dependent reverberation, and room structure. During training, it leverages binaural room impulse responses (RIRs) and paired panoramic depth images to learn how geometry shapes sound, but at inference it requires only binaural audio, making broadly applicable. While SAGE provides geometry- this with a spatially grounded CoT mechanism that anchors intermediate reasoning steps to source locations. Rather than answering in single pass, OWL localizes and interprets spatial configurations, then performs structured inference. This decomposition from perception (localization and detection) to reasoning leads to more accurate and interpretable responses, for example, generating rationales such as 'sound A at 8 o' clock is left of sound B at 1 o' clock,' as illustrated in Figure []]
acoustic representations, OWL extends
To support this pipeline, we construct BiDepth, a large-scale public dataset that couples binaural audio, binaural RIRs, panoramic depth images, and question-answer annotations. Unlike prior datasets such SoundScape Pano-IR and SpatialSoundQA, BiDepth integrates all three modalities to provide explicit geometric supervision for training SAGE and OWL. It contains over 1.1M questions spanning perceptual QA, spatial reasoning, and CoT-augmented multi-step QA. To reduce template bias and leakage, BiDepth includes linguistic variants and a split design that prevents overlap in rooms and sources between train and test.
We validate our approach on both standard SELD tasks and new spatial reasoning benchmarks. Results show that SAGE consistently outperforms state-of-the-art (SOTA) methods in sound event localization and detection (SELD) Adavanne et al. (2018). achieving 1.71% gain in mean average precision, an 11 reduction in mean angular error, and 33.5% decrease in distance error rate. OWL surpasses BAT by 46.4% on perceptual QA and 24.9% spatial reasoning benchmarks, using sparse variant aligned with BAT's coarse categories; our full model further supports fine-grained 1 12-sector DoA estimation and spatial reasoning. Our contributions can be summarized as follows:
ii

---

Published as a conference paper at ICLR 2026
BiDepth, the first large-scale dataset (=1..IM QA pairs) of (binaural audio, binaural RIR, depth image, QA} 4-tuples with geometric grounding for perception and multi- spatial reasoning.
requirii SAGE, A novel spatially grounded acoustic encoder trained with multimodal supervision, requiring only audio at inference for efficient, geometry-aware deployment.
OWL, spatial ALLM integrating SAGE with spatially grounded CoT reasoning, unifying event detection, localization, and structured inference to achieve SOTA SELD and spatial QA performance.
2 RELATED WORKS
Audio Large Language Models. Contrastive aadio-language pretraining (c.g., CLAP Wu et al. (2023) laid the foundation for retrieval and zero-shot transfer. Subsequent audio LLMs Deshmukh et al. 2023); Gong et al. (2023); Huang et al. (2024); [Tang et al. (2023); Chu et al. (2023 Biswas al. 2025 scaled datasets and training strategies remained perception-oriented. focusing on
recognition QA. The Audio Flamingo family Kong al. (2024); Ghosh et al. (2025) Goel et al. (2025) further broadened task coverage, enhancing performance across diverse applications. Yet, most models still emphasize perception-oriente oriinted classification and QA while overlooking spatial reasoning, which is essential for progress toward audio general intelligence Morris et al. (2023). BhT et al.) (2024b) is notable exception, introducing spatial QA from binaural audio, but reduces scenes coarse bins (front, back, left, right), uses single-step inference, and lacks geometric grounding since its encoder is trained with audio alone. In contrast, approach leverages ssometry-conditione training to align audio with spatial structure for better localization and multi-step reasoning.
CoT Reasoning in Multimodal LLMs. Chain-of-thoughh (CoT) reasoning has been widely adopted to improve stepwise inference in multimodal LLMs. Prior work has exploited graphical cues from images Deng et al. (2024); He al. (2024): Thawakar et al. (2025), logical structures Dong al., (2025) Xiao et al. (2024): Zheng et al. (2024a). and textual prompts Bi et al. (2024); Xu et al. (2024); Chen al. (2024b), achieving stronger interpretability and accuracy in vision-language tasks. In audio, however, CoT is largely absent: our knowledge, Audio Flamingo 3 GoGl et al. (2025) is the only prior attempt and is limited to simple perceptual queries without spatial grounding. Our work fills this gap by introducing geometry-awan CoT for audio, anchoring intermediate reasoning steps to source locations and enabling fine-grained 3D acoustic reasoning.
Sound Event Detection and Localization. Sound event detection and localization (SELD) jointly addresses sound event recognition and direction-of-anne arrvvvl (DoA) estimation. The DCASE challenge standardized this task, with baselines such SELDnet Adavanne et al. (2018) and the ACCDOA formulation Shimada et al. (202T) widely adopted. Later architectures (CRNNs Biswas et al. 2023 Conformers Gulati et al. (2020), GRUs Cho et al. (2014) further improved accuracy. However, SELD methods remain task-specific, rely solely on audio features, and lack explicit geometric grounding. In contrast, our approach leverages an LLM framework that aligns audio with environmental geometry and extends beyond detection and localization to support interpretable, multi-step spatial reasoning.
0"
330
30
300
90*
60"
120
(con)) 2
240
120
AC daak
(3at. F haa)
Di
210
150
0"
180
180"
50 Sac
Figure 2: Example of paired modalities in BiDepth. Left: panoramic depth image D₁ capturing geometric context from the listener's perspective. Right: binaural acoustic simula- tion, we place sound sources at positions and 82 and place the receiver at location r. For illus- tration, cat sound can be positioned at 81.
60 dack
(a) Azimuth distribution
(b) Elevation distribution
Figure 3: Azimuth and elevation angle distributions in BiDepth,, showing source directions relative the listener. Azimuths are nearly uniform, while clevations cluster near the horizontal plane.
iii

---

Published as a conference paper at ICLR 2026
3 BIDEPTH DATASET
Audio QA datasets (e.g., SpatialSoundQA ZZeng et al. (2024b), SpatialVLM Chen et al. (2024a) in- clude binaural RIRs but lack geometric cues, unlike vision-language QA datasets (e.g.. CLEVR John- son et al. (0117), GQA Hudson & Manning (2019) that use depth for relational reasoning. We introduce BiDepth, a synthetic dataset that couples binaural RIRs with panoramic depth and CoT QA corpus with stepwise rationales, comprising 28,000 RIR RIR-depth pairs and 1.1 million QA 4-tuples
3.1 ACOUSTIC-GEOME GEOMETRIC SIMULATION
We adopt a simulation- approach to generate paired acoustic and geometric data, enabling systematic variation across environments with scalability and reproducibility. Unlike real-world measurements that require specialized hardware and cover limited spaces, simulation offers controlled access to diverse layouts, surface materials, and source-receiver configurations. Our dataset built with SoundSpaces v2 Chen et al. (2020 2022) and Matt exportt 3D Chang et al. (2017) (90 buildings, 24 rooms per building, 30 scene types). For each RIR, binaural receiver is placed at
(at, y. 2, f)) with random location (z, 2) and orientation (0). A sound source is uniformly sampled within 10 of the receiver r to remain inside the building. The RIR encodes the acoustic transfer function from r, capturing spatial and reverberant properties of the environment. Figure shows the simulation setup, including the sound source, binaural receiver, and the panoramic depth image Dj capturing room geometry. Let (t) denote monaural input signal emitted from the source at s. The corresponding binaural signal received at r is then given by
[B] ( = & @ (t):); E [1, T] and @ denotes convolution (1) (RIR(((,,,,77))
(t) =
where (t) and (t) are the left and right binaural channels, RIR, s, 7) denotes the room impulse response between the source and the receiver at for channel E {L, RJ, and ? represents the environmental configuration, including geometry, construction materials, and furniture layout. To complement acoustic signals, we render panoramic depth maps from the receiver. Rotating the receiver in 20 increments yields depth images D; with limited fields of view; concatenation produces a panoramic map encoding walls, obstacles, and room structure. This pairing provides explicit alignment between acoustic propagation and 3D geometry.
With the simulated setup illustrated in Figure 2 we generate 28K unique RIR, depth pairs spanning diverse azimuths, elevations, and distances shown in Figure 3 Unlike prior datasets (e.g.. Spatial- SoundQA, SpatialVLM) that provide RIRs without geometry, our dataset explicitly couples acoustics with depth, offering the first large-scale resource for geometry-aware audio reasoning. Representative examples, full rendering details, sampling distributions, and statistics are in Appendix
3.2 DUESTION-ANSWER GENERATION
We construct QA pairs that integrate auditory cues and geometry for spatial understanding. Fol- lowing GAMA Ghosh et al. (2024) and LTU Gong et al. (2023). each entry is a quadruplet binaural audio, binaural RIR, depth image, QA, where audio. RIR, and depth provide multimodal supervision while QA drives perception and reasoning. At inference, models receive only audio, with extra signals used for geometry-ave four categories described below (examples of templates and stepwise rationales are in Appendix AA.4).
training. BiDepth contains over 1.1M QA pairs balanced across
Type I: Event Detection. This task identifies the sound sources present in the scene. We include both single-source cases and dual concurrent-source cases, using monaural clips (M*(I) from AudioSet Gemmeke et al. (0117) spatialized with simulated RIRs.
Type II: Direction Estimation. This task estimates the azimuth, elevation, and distance of sources, requiring models to predict direction and distance either from source to receiver or between sources. The horizontal plane is divided into 12 clock-based sectors, elevation is labeled up down, and distance is expressed in conversational form (e.g.. 3 o' clock; up; 2.5 m'), quantized in 0.5 m steps up to 10 m to reflect human approximation.
Type III: Spatial Reasoning. This task targets spatial reasoning through relational queries involving two sources and the receiver, such as 'Iy source left of source 2?' or 'I source I closer to the receiver
iv

---

Published as a conference paper at ICLR 2026
than source 2?' Formulated as binary Yes/No tasks, these queries move beyond absolute localization to assess whether models can reason over relative spatial relationships in complex acoustic scenes.
Type IV: CoT for Spatial Reasoning. This task extends spatial reasoning by providing CoT rationales. Instead of binary Yes/No labels, answers include concise reasoning steps. For example, for the query "Is <source 1> closer to the receiver than <source 2>?", the rationale may state: source / is 5.0 m away; source 2 is 3.5 m away, therefore source 2 is closer, yielding the answer No. These rationales make the reasoning process explicit and encourage models to ground predictions in structured spatial comparisons rather than direct classification.
4 SAGE: SPATIAL- GEOMETRY ENCODER
SAGE: Spatial - Accusiic Goometry Encoddr
CLS
DoA
DIS
IR Prediction Medule
cat meowing o'clock, about 3.5 meters awny.
Tramnin Inference-
Transposed
Add & Norm
LLM (IT)
CWL. Spatial Audio LLM
Training Ouly
Convolation
Feed Forward
ResNet-18
Add & Norm
Dimaural Audio Emendeer
Decoder
Projector v-())
Multi-Head Attention
Can you locate the sound of
cat meow from the audio?
SAGE (Binaural Audio
ResNet-18
Patch Embedding
Encoder) o.(.)
Encoder
Encoder
Binauarl Audio, (*)
Figure 4: Architecture of OWL and SAGE. The left panel shows SAGE, trained with geometry- aware supervision using RIRs and depth cues. The right panel illustrates the OWL pipeline, where the Binaural Audio Encoder oz() is combined with the LLM II through a projector v-(-) to generate spatially grounded answers. Here, and * represent trainable and frozen components, respectively.
We introduce SAGE, the first geometry- aware encoder that jointly models acoustic signals and scene structure. The key idea is to inject geometric cues through an auxiliary binaural RIR prediction task, providing privileged supervision that regularizes the audio encoder. SAGE comprises two jointly optimized modules: (i) binaural audio encoder for perceptual tasks, and (ii) RIR prediction module that fuses depth features with audio representations. At inference time, only the audio encoder used, ensuring broad applicability without requiring geometric inputs.
Binaural Audio Encoder. The binaural audio encoder (a(-) takes a binaural waveform B E 2 as input, where each channel represents a left or right ear signal of length L. It outputs an embedding h. = 0. (B) E CCT that captures spatial and semantic cues across C feature channels and T temporal frames. This representation supports three prediction tasks: sound event classification, direction-of-arce arrivvl (DoA) estimation, and distance prediction. For DoA, azimuth is discretized into 360 bins (00", 3599)) at resolution and elevation into 180 bins ([0s, 179%)). while for distance the range [0, 10] is uniformly quantized into 21 bins with 0.5 m spacing. The overall training objective a weighted sum of cross-entropy losses for event classification (Lus), distance prediction (Ldis), and DoA estimation (Looa), where the learnable weight coefficients (a,) balance each task's 's contribution.
Leinaual 01 Cos + Co + ag
(2)
RIR prediction module. A ResNet- 18 encoder <a(-)) processes the panoramic depth image D, R W and produces a latent representation h1 = (D,) E RCXT, This depth-derived embedding is fused with the audio features h, and decoded by ResNet- transposed convolutional head to reconstruct the binaural RIR, R = (ha, ha).
Given the truuhd-truth RIR R. reconstruction is supervised by geometric loss that combines E term with an Energy Decay Curve (EDC) loss,
Lgoo
R R| + À LED.CERRR........ (R. R)
where LEDE measures the mismatch between predicted and reference decay curves computed via Schroeder's backward integration algorithm Schroeder (1965). Unlike scalar descriptors such as

---

Published as a conference paper at ICLR 2026
RT60, the EDC loss is differentiable and captures richer reverberant structure, including direct-to- reverberant ratio (DRR) and early decay time (EDT). This auxiliary supervision encourages the encoder to internalize geometry
acoustic features that complement the perceptual tasks.
Overall Training Objective. Following prior work Zheng et al. (2024b), we use Mcl-Spectrograms
and Interaural Phase Difference (IPD) inputs to Pa(+) (see Appendix B.1. The overall training
objective combines perceptual and geometric terms, L 1i1 Ehinaural + 1/2 Loo- Scalar weights (?), al. (2017) events spatialized with
vi2) balance the contributions. Training uses AudioSet Gemmeke AudioSe simulated RIRs to enable joint optimization of acoustic and geometric objectives. At inference, depth images are unavailable and only the audio encoder (a(-) used for downstream tasks, e.g., event classification, DoA estimation, and distance prediction. See Appendix B.2 for further details.
OWL: SPATIAL AUDIO-LLM WITH CHAIN-O--THOUGH REASONING
OWL integrates spatial perception and reasoning over binaural audio by coupling geometry- aware encoder with a large language model (LLM). The overall architecture. illustrated in Figure comprises three components: (i) the binaural audio encoder (a from SAGE, which extracts spatially grounded acoustic features from raw waveforms; (ii) projection module (h(.) based on Q-Former et al. (2023). where Q learnable query tokens perform cross-attentive pooling to align audio features with the LLM embedding space while reducing sequence length; and (iii) decoder I()), instantiated LLaMA-2-7B Touvron et al. (2023), which conditions both projected tokens / and textual prompts to generate task-specific outputs. Formally, oa(t (-) maps a binaural input B' (t) to embedding h, E RCTT, (-(-) projects h, into 2, E R and II(.) decodes 2, together with a text prompt x, to produce the output sequence = [l(y,. x.). This design enables OWL to compress high-dimensional acoustic features into semantically aligned tokens and generate outputs that combine spatial perception with interpretable reasoning.
We adopt Q-Former for its selective cross-attentive pooling, which preserves spatial cues more effectively than lightweight linear or MLP adapters. Following BAT, we use LLaMA-2-7B as the language backbone ensure fair comparison, while our novelty lies in augmenting it with geometry- aware encoder and a curriculum for explicit spatial reasoning and Chain-of- Thought supervision. LoRA Hu et al. (2022) enables parameter-efficien adaptation, and (a (.) is kept frozen to retain geometry-ave aware features learned in SAGE. To our knowledge, this is the first curriculum-traind spatial audio LLM with explicit gcometry-aware CoT supervision.
5.1 TRAINING OF OWL WITH COT SUPERVISION
We train OWL using three-stage curriculum that progresses from perceptual grounding to relational reasoning and finally explicit CoT supervision. This staged approach ensures that the model first acquires low-level perception skills before being challenged with higher-level geometric reasoning.
Table Multi-stage training: single-source warmup, then dual-source inputs for reasoning.
Training Stage
Question
No. Audio
No. of Train
Type
Source
Samples
Single (warmup)
270K
Stage
I,
Dual
270K
Stage 1: Perceptual Pre-Traine Pre-Training The model first trained on Type 1-II QA pairs for event detec- tion and direction estimation. Training begins with songle-source recordings to stabilize event recognition and DoA estimation before introducing dual- source cases, which require disentangling overlapping cues. This stage grounds OWL in basic spatial perception and prevents overfitting to relational shortcuts before learning low-level geometry.
Stage 2
Dual
300K
Stuge
Dual
250K
Stage Relative Geometry Pre-Training. The model is next exposed to Type III QA pairs, which emphasize relational geometry (c.g.. left/right closer/farther) rather than absolute positions. This stage bridges perceptual grounding and CoT supervision by encouraging OWL to internalize structured spatial relations between sources and the receiver. Without this intermediate step, the model struggles to generalize from low-level perception to multi-step reasoning.
Stage 3: CoT Instruction Tuning. Finally, the model is trained on Type IV QA pairs that provide full Chain-of- Thought (CoT) explanations alongside the final decision. Supervising both the intermediate reasoning steps and the final prediction aligns the model's outputs with interpretable spatial logic.
vi

---

Published as a conference paper at ICLR 2026
Table 2: Comparison of SAGE, SELDNet, and Spatial-AST on SpatialSound and BiDepth. Best and second-best results are in bold and underline. Models are trained on their evaluation dataset unless noted otherwise (see footnotes for setups).
Modality
SpatialSound-4 -QA (SSQA)
BiDepth
Method
Audio
Depth
mAP 1
2R 4
MAE
DER
mAP1
ER
MAE.
DER
SELDNet
42.66
25.19
19.21
38.46
39.46
53.21
38.71
53.38
Spatial-AST
50.03
23.89
17.94
32.54
48.97
45.29
32.99
47.82
Spatial-ASTe
49.17
41.94
27.24
39.21
SAGE
x
49.71
26.59
23.19
33.03
49.75
36.89
26.32
17.11
SAGE*
x
49.94
23.67
18.26
32.61
SAGE"
49.93
24.71
18.47
17.84
49.81
28.13
21.67
14.32
Trained SpatialSoundQA. TTrined SpatialSounQAA Pre-trained on BiDepth audio only. then fine-tuned on SpatialSoundQA.
tuned on BiDepth. Trained BiDepth audio only. TTained of BiDepth audio and depth.
Unlike prior work, which supplies only categorical labels, this stage enforces explicit step-by-step reasoning, yielding more accurate responses accompanied by human-readable justifications.
Training Loss. At each stage we minimize the standard auto-regressive cross- cross-entropy loss over the target token sequence. Given a binaural input B' (t))), a question Xt, and a target output y (v1...., )r), the audio encoder (-)-) produces features that are projected by <b(-) into query tokens 2, = (B"((B (t))). The language decoder (I(-) then conditions on both x; and 2, to predict each token. Across the three stage-specific datasets Di D3. and D4, Equation is the unified training objective. During training, (-(-) is trained from scratch while II(-) is fine-tuned with LoRA, and (. remains frozen. Table summarizes the datasets used across all stages, with ablations in Section 6.3 and hyperparameter details in Appendix C
I Σ
C(oa: v. II)
Σ
((B:))))))
logn (v. (bo((B(t)))
(4)
sE(1,2,3,1)
EXPERIMENTS
6.1 IMPLEMENTATION DETAILS
Front-end Audio Processing. Following BAT, we normalize loudness so each 10-s binaural clip has consistent energy, as AudioMAF (window size = 1024, hop size 320), then obtain two-channel mel-spectrograas (128 mel bins). We also extract sine and cosine encodings of the Interaural phase difference (IPD). The mel-spectrogram provides spectral energy cues essential for event detection, while IPD encodes inter-channel phase differences critical for localization. See Appendix B.I for details.
al. (2022). We compute Short-Time Fourier Transforms
Evaluation Metrics. For SAGE, evaluate event detection using mean average precision (mAP), DoA estimation using mean angular error (MAE) and error rate where angular error (azimuth and elevation) exceeds 20 (ER2o=), and distance estimation using the Distance Error Rate (DER), which measures predictions deviating from the ground truth by more than 0.5 m. These metrics jointly assess semantic correctness (mAP) and geometric accuracy (DoA, distance), with results reported on both SpatialSound(AA Zheng et all (2024b) and BiDepth. For OWL, we use the same measures for event detection, DoA, and distance, and additionally report binary accuracy (BA) for spatial reasoning (Type III) and combined detection, direction, and BA for CoT reasoning (Type IV), under both single- and dual-souree conditions (Types I-II). We refer to for details of evaluation metrics.
Baselines. We compare SAGE against SELDNet Adavanne et al. (2018) and Spatial- AST Zheng al.) (2024b), two established baselines for sound event localization and detection. For OWL. we evaluate against both open- and closed-source multimodal LLMss. The open-source baselines include BAT Zheng et al. (2024b), designed specifically for spatial audio reasoning, and general multimodal models such as VideoL Cheng et al.) (2024), RAVEN Biswas al. (2025), and AudioFlamingo2 Ghosh et al. (025)). For closed- source systems, we run Gemini- 1.5-Pro. Gemini-2.5-Prrs and Gemini-2.5-P F.5-aahh under our evaluation setup to benchmark against the latest proprictary models. Together, these baselines cover both task-specific spatial audio systems and broader audio-augmented L.Mss, ensuring fair and comprehensive comparison.
vii

---

Published as a conference paper at ICLR 2026
Table Comparison of OWL with closed- and open-source baselines on BiDepth across four task types: Type I (event detection), Type II (direction estimation), Type III (spatial reasoning), and Type IV (CoT reasoning). OWL consistently surpasses prior open-source models, with further gains from CoT supervision. Best results are in bold.
Typel
Typell
Typelll
TypelV
Detection (mAP)?
DoA (Ace)
Distance (DER)
Method
Single Source
Double source
Single Source
Double
Single Source
Double source
Detection
Direction
BA
BA
source
Closed source Models'
Geminil.5PP
Gemini22.5PM
32.47
12.17
12.01
Gemini2.555 FFaash
Open-source Models
Vida
17.11
5.21
12.23
11.76
68.12
83.78
8.29
5.19
RAVEN
16.29
5.43
13.79
9.39
71.46
82:37
9.76
5.97
Audiollamingu2
27.59
17.74
BAT
24.97
8.73
-171.59'
[355.99'
28.61
45.79
69.46
71.62
78.27
61.29
OWL w/o Col
33.31
17.24
46.15|77.21
34.24|51.67
24.67
31.29
74.29
65.27
OWL CoT
33.37
17.26
46.17
34.31
23.29
29.91
77.89
79.04
86.76
76.53
Gemini models evaluated via API binaural inputs: results are reported only for event detection. Appendix D As BAT uses a 4-bin protocol, we also report 4-bin results for OWL alongside its native 12-bin evaluation (12-bin 4-bmm.
Table 4: Zero-shot Performance of OWL on the SpatialSoundQA across perception and reasoning tasks. OWL consistently outperforms the baselines, with larger gains in spatial reasoning tasks, demonstrating the benefit of the SAGE and CoT instruction tuning. Best results are denoted in bold.
Perception (Type ABCD)
Reasoning (Type E)
Detection (mAP)
DoA (Acc) f
DP (DER)
Model
Direction
Distances
Avg
Type
Type C
Type B
Type
Type
Type D)
Random
0.61
0.59
12.57
12.41
67.33
67.46
50.00
50.00
50.00
Mono BAT
24.15
6.42
14.31
11.93
34.17
56.26
57.69
51.36
54.33
BAT
26.34
9.89
75.54
37.65
29.16
47.90
69.77
84.04
76.89
OWL
26.76
12.73
78.31
43.15
26.14
43.21
71.21
86.91
79.06
6.2
MAIN RESULTS
SAGE performance on SELD. Table 2 compares SAGE with SELDNet and Spatial-AST on SpatialSoundQA and BiDepth. At test time, evaluation uses only binaural audio: depth is incorporated during training auxiliary supervision. We evaluate three SAGE setups: (i) trained on BiDepth audio, zero-shot on SpatialSoundQA: (ii) pre-trained on BiDepth then fine-tuned on SpatialSoundQA: (iii) trained BiDepth with audio & depth, evaluated in-domain and zero-shot. For Spatial-AST. we report models trained on SpatialSoundQA with zero-shot or fine-tuned evaluation on BiDepth. SELDNet is trained independently each dataset and excluded from transfer, as it cannot depth.
Across both datasets, SAGE achieves best or second- best results in nearly every metric. Relative to Spatial-AST, it improves event detection modestly (s= 1.6 1.7%) but yields much larger gains in localization: 2R decreases by 23.61% %. MAE by 25.52%, and DER by 31.34%. In cross-dataset evaluation, SAGE reduces DER by 82% relative SpSt while depth supervision mitigates the smaller drops observed in detection and angular metrics. Two main trends emerge: depth supervision consistently strengthens localization, showing that geometry primarily aids spatial reasoning rather than event detection; and S transfers far more robustly than baselines, whose performance degrades sharply. Even where mAP gains are modest, SAGE remains among the top two models across all metrics, underscoring consistent rather than isolated improvements. Overall, SAGE demonstrates clear advantages localization accuracy and robustness under cross-dataset transfer.
OWL Performance on QA. We evaluate OWL on BiDepth and SpatialSoundQA to assess system- level performance beyond the encoder. On BiDepth (Table OWL outperforms both closed- and open-source baselines across all task types. Gemini models included as high-capacity LLM references but only support event detection. Among open-source baselines, BAT is the closest competitor, while RAVEN, Videol LaMA2, and AudioFlamingo2 provide broader audio-language
viii

---

Published as a conference paper at ICLR 2026
comparisons, Leveraging the geometry SAGE, OWL achieves 46.15% accuracy under fine- grained 12-bin DoA and 77.21% % under coarse 4-bin quadrants. compared to BAT's 71.59% (4-bin). Reporting both protocols ensures fairness while highlighting robustness under stricter evaluation. Distance error rates are also lower (24.67% and 31.29%) than the baselines. Reasoning-hean show the sharpest gains: OWL reaches 65.37-74.29% accuracy single-step reasoning and 76.53- 77.89% in multi-step CoT reasoning, far surpassing all baselines. Adding CoT supervision further improves reasoning accuracy by 11.26% and yields consistent gains in detection and DoA.
tasks
On SpatialSoundQA (Table 4). OWL again surpasses BAT, improving DoA accuracy (from 75.54% 78.31%) 1%) and reducing DER (from 29.16 26.14). CoT tuning provides the largest boost in reasoning, raising direction and distance accuracy to 71.21% and 86.91% (79.06% overall) compared to BAT's 76.89% OWI is stronger in perception and the first to demonstrate geometry-aware multi-step reasoning. Its encoder provides robust spatial features, while CoT supervision yields explicit, interpretable rationales and consistent benchmark gains. This robust zero-shot result on the unseen dataset ensures no data leakage in OWL training. Appendix E presents qualitative examples illustrating CoT efficacy beyond numerical results.
6.3 ABLATION STUDY
Loss Component Ablation for SAGE. T the ef- fect of different loss terms. Train- ing with only the binaural loss Donnaral (72 = 0) yields mAP 49.75 but high localization errors (ER 36.89, MAE 26.32, DER 17.11). Re- moving the CEDC term (A 0) has little effect on detection but higher impact localization, in- dicating that binaural supervision dominates baseline performance.
Table 5: Ablation study of loss components in SAGE. Adding the geometric loss Leen yields substantial gains spatial localization (ER209, MAE, DER) while preserving high event detection mAP.
Loss
mAP f
ER
MAE
DER
Lbinaural (72
49.75
36.89
26.32
17.11
A=
49.73
36.79
26.12
16.71
-e-
1/2 1e-4
49.28
36.13
25.91
21.72
À le
le
49.39
33.64
23.31
17.47
1e"
49.81
28.13
21.67
14.32
Small weights on Lee marginally reduce ER and MAE but destabilize DER. Larger weights (m) 0.01, 1/2 0.001) improve MAE (23.31) without harming detection. The largest gains occur when 1/2 = 0.01 is applied directly, lowering all errors (ER20- = 28.13, MAE 21.67, DER 14.32) while preserving mAP (49.81). Thus, geometry alignment via 7/2 is the main driver of improved localization, while imbalanced or overly small weightings dilute its effect.
Effect of Training Stage of OWL. Table o evaluates the curriculum stages. Without Stagel warmup, detection collapses (mAP 32.92 32.92/8.97) and DoA/distance estimation degrades. Adding Stage recovers detection (33. 1.3.27/17.19) and improves Type tasks (DoA: 45. 45.91/34.21; Distance: 24.39/31.17), showing that single-source pretraining stabilizes learning. Stage 2 adds relative reasoning. boosting Type III BA to 74.29 and Type IV BA to 65.27, confirming the benefit of explicit geometric supervision. The full threc-stage curriculum yiclds the strongest results: Type III BA 77.89 and Type IV (Detection 79.04, Direction 66.76, BA 76.53), demonstrating that gradual progression from perception to reasoning essential for building robust spatial reasoning.
Table 6: Training Stage Ablation of OWL. While warmup stage stabilizes training, progressively adding training stages leads to consistent performance improvements across all task types. denotes without warmup. Best values are denoted in bold.
Training Stages
Typel
Typell
Typel11
TypeIV
Detection (mAP)t
DoA (Ace)t
Distance (DER),
Stage
Stage 2
Stage 3
Single Source
Double source
Single Source
Double
Single Source
Double snurce
Detection
Direction
BA
BA1
source
Vt
x
x
32.92
8.97
41.28
13.71
22.77
61.24
17.19
34.21
/
x
33.31
17.24
46.15
34.24
24.67
31.29
74.29
65.27
33.37
17.26
46.17
34.31
23.29
29.91
77.89
86.76
76.53
ix

---

Published as a conference paper at ICLR 2026
Table 7: OWL on real-world binaural acoustic scene classification.
Street pedestrian
Street traffic
Metro station
Puhlic square
Shopping
Class
Metro
Park
Airport
mall
Bus
Tram
Acuracy
0.66
0.79
0.76
0.68
0.72
0.77
0.84
0.71
0.83
0.77
6.4
REAL-WORLD GENERALIZABILITY
To evaluate whether OWL relies synthetic data and to examine its behavior in realistic acoustic conditions, we conducted additional experiments using the real-world DCASE Binaural Audio Scene Classification Mataset[Mars et al. (2019) and the DCASE SELD Challenge 2021 dataset et al. (2021). These datasets were selected because both are recorded real environments. The first supports general audio perception task, and the second supports spatial localization task. Neither dataset is used during training, and both contain diverse environments that differ from our synthetic domains in microphone characteristics, background noise patterns, and reverberant structure.
Audio Scene Classification. For zero-shot evaluation on the audio scene classification dataset, used the Instruction-based prompt without any adaptation or fine-tuning. The model achieves stable accuracy across all acoustic scenes (Table 7. including street pedestrian 0.66, metro 0.79, park 0.76, street traffic 0.68, metro station 0.72, public square 0.77, airport 0.84, shopping mall 0.71, bus 0.83, and tram 0.77. These results show that OWL generalizes well to real-world binaural recordings, despite being trained primarily on synthetic spatial audio. This behavior indi- cates that the geometry-awee audio encoder and the downstream reasoning components capture structure that transfers across simulation-to-ree domain gaps.
Table 8: OWL on real-world acoustic source lo- calization task.
Type I
Method
Detection (mAP) (1)
DoA (Ace)
Single source
Double
Single souree
Double
source
OWL OWL CoT
57.21 57.19
51.26 $1.09
42.78
31.46
42.63
31.29
Sound Source Localization. To further demonstrate the compatibility of OWL with spatial reasoning tasks real-world audio, we evaluate it on the DCASE SELD Challenge 2021 dataset. Because this dataset is recorded in first-order ambisonic format, we convert the FOA signals to binaural audio using an HRTF transformation Noisternig et al. (0012). This step is required because the SAGE encoder operates on binaural input and no real-world binaural spatial dataset is publicly available. Table reports the performance OWL on this dataset. OWL achieves consistent performance across both single-source and double-source conditions, which shows OWL's effectiveness in these real-world settings.
7 CONCLUSION & FUTURE WORKS
We introduced BiDepth, a large-scale dataset, and OWL, the first spatial audio LLM with geometry- aware Chain-of- Thought reasoning. By coupling binaural audio with panoramic depth supervision, OWL achieves strong gains in event detection, DoA, distance estimation, and higher-order spatial reasoning. Ablations show that geometry chiefly strengthens localization, while CoT alignment is essential for multi-step reasoning, together yielding interpretable and transferable representations.
A key limitation is that BiDepth is insillation-baaede leaving open the question of robustness in real-world acoustic conditions. Moreover, our reasoning tasks are restricted to single-turn QA, whereas human spatial communication is interactive and dialog-based. Future work will extend BiDepth to real recordings, expand reasoning to multi-turn dialogues, and explore richer grounding with vision inertial sensing. Preference- based alignment could also improve the coherence of generated rationales. Taken together, these directions position OWL as step toward embodied agents capable of human-like spatial reasoning.

---

Published as a conference paper at ICLR 2026
ACKNOWLEDGMENT
We acknowledge support from the National Institute on Drug Abuse (IR01DA059422) and the National Institute of Diabetes and Digestive Kidney Diseases (SRO1DK Results in this paper were obtained in part using high-performance computing system acquired through NSF MRI grant DMS-1337943 to WPI. We gratefully acknowledge their support in enabling this work.
REFERENCES
Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, 1lge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. Gpt-4 technical report. arXiv preprini wrXiv22303.08774 2023.
Sharath Adavanne, Archontis Politis, Joonas Nikunen, and Tuomas Virtanen. Sound event localization and detection of overlapping sources using convolutional recurrent neural networks. IEEE Journal of Selected Topics in Signal Processing, 13(1):34-48, 2018.
Jinze Bai, Shuai Bai, Yunfei Chu, Zeyu Cui, Kai Dang, Xiaodong Deng, Yang Fan, Wenbin Ge, Yu Han, Fei Huang, et al. Qwen technical report. arXiv preprint arXiv:2309. 16609, 2023.
Zhenni Bi, Kai Han, Chuanjian Liu, Yehui Tang, and Yunhe Wang. Forest-of-thought Scaling test-time compute for enhancing IIm reasoning. arXiv preprint arXiv:2412.00077. 2024.
Subrata Biswas, Mohammad Nur Hossain Khan, Alex Colwell, Jack Adiletta, and Bashima Islam. Lo- cus: Localization with channel uncertainty and sporadic energy. arXiv preprint .9440999, 2023.
Subrata Biswas, Mohammad Nur Hossain Khan, and Bashima Islam. Raven: Query-guided rep- resentation alignment for question answering over audio, video, embedded sensors, and natural language. arXiv preprint arXiv:2505.17114 2025.
Angel Chang, Angela Dai, Thomas Funkhouser, Maciej Halber, Matthias Niessner, Manolis Savva. Shuran Song, Andy Zeng, and Yinda Zhang. Matterport3d:: Learning from rgb-d data in indoor environments. International Conference on 3D Vision (3DV), 2017.
Boyuan Chen, Zhuo Xu, Sean Kirmani, Brain Ichter, Dorsa Sadigh, Leonidas Guibas, and Fei Xia. Spatialvim: Endowing vision-language models with spatial reasoning capabilities. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 14455-14465, 2024a.
Changan Chen, Unnat Jain, Carl Schissler, Sebastia Vicenc Amengual Gari, Ziad Al-Halah, Vamsi Kr- ishna Ithapu, Philip Robinson, and Kristen Grauman. Soundspaces: Audio-visual navigaton in 3d environments. In ECCV, 2020.
Changan Chen, Carl Schissler, Sanchit Garg, Philip Kobernik, Alexander Clegg, Paul Calamia, Dhruv Batra, Philip W Robinson, and Kristen Grauman. Soundspaces 2.0: simulation platform for visual-acoustic learning. In NeurIPS 2022 Datasets and Benchmarks Track, 2022.
Yongchao Chen, Harsh Jhamtani, Srinagesh Sharma, Chuchu Fan, and Chi Wang. Steering large language models between code execution and textual reasoning. arXiv preprint arXiv:2410.03524, 2024b.
Zesen Cheng, Sicong Leng, Hang Zhang, Yifei Xin, Xin Li, Guanzheng Chen, Yongxin Zhu, Wenqi Zhang, Ziyang Luo, Deli Zhao, and Lidong Bing. Videollama 2: Advancing spatial-tempora, modeling and audio understanding in video-Ilms. arXiv preprint 4rXiv:2406.07476, 2024. URL
wwwp/vivorrgaaooo hwtp://b///aaa wttp:/wwwww.///////....//0 07476
Kyunghyun Cho, Bart Van Merriënboer, Dzmitry Bahdanau, and Yoshua Bengio. On the properties of neural machine translation: Encoder-decoder approaches. arXiv preprint / /259, 2014.
Yunfei Chu, Jin Xu, Xiaohuan Zhou, Qian Yang, Shiliang Zhang. Zhijie Yan, Chang Zhou, and Jingren Zhou. Qwen-audio: Advancing universal audio understanding via unified large-scale audio-language models. arXiv preprint axXiv:2311.07919 2023.
xi

---

Published as a conference paper at ICLR 2026
Gheorghe Comanici, Eric Bieber, Mike Schaekermann, Ice Pasupat, Noveen Sachdeva, Inderjit Dhillon, Marcel Blistein, Ori Ram, Dan Zhang, Evan Rosen, et al. Gemini 2.5: Pushing the frontier with advanced reasoning, multimodality, long context, and next generation agentic capabilities. arXiv preprint arXiv:2507.0626/ 2025.
Naihao Deng, Zhenjie Sun, Ruiqi He, Aman Sikka, Yulong Chen, Lin Ma, Yue Zhang, and Rada Mihalcea. Tables as texts or images: Evaluating the table reasoning ability of Ilms and milms. arXiv preprint arXiv:2402.12242 2024.
Soham Deshmukh, Benjamin Elizalde, Rita Singh, and Huaming Wang, Pengi: An audio language model for audio tasks. Advances in Neural Information Processing Systems. 16:18090-18108 2023.
Yuhao Dong, Zuyan Liu, Hai-Long Sun, Jingkang Yang, Winston Hu, Yongming Rao, and Ziwei Liu. Insight-v: Exploring long-chain visual reasoning with multimodal large language models. In Proceedings of the Computer Vision and Pattern Recognition Conference, pp. 9062-90000 2072, 2025.
Jort F. Gemmeke, Daniel P. W. Ellis, Dylan Freedman, Aren Jansen, Wade Lawrence, Channing Moore, Manoj Plakal, and Marvin Ritter. Audio set: An ontology and human-labeled dataset for audio events. In 2017 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 776-780, 2017. doi: 10.1 1A.1109CCASSP221777752222
Sreyan Ghosh, Sonal Kumar, Ashish Seth, Chandra Kiran Reddy Evuru, Utkarsh Tyagi, Sakshi, Oriol Nieto, Ramani Duraiswami, and Dinesh Manocha. Gama: A large audio-language model with advanced audio understanding and complex reasoning abilities. arXiv preprint arXiv:2406.1170 11668, 2024.
Sreyan Ghosh, Zhifeng Kong. Sonal Kumar, S Sakshi, Jaehyeon Kim, Wei Ping, Rafael Valle, Dinesh Manocha, and Bryan Catanzaro. Audio flamingo 2: An audio-language model with long-audio understanding and expert reasoning abilities. arXiv preprint arXiv:2503.03983 2025.
Arushi Goel, Sreyan Ghosh, Jachyeon Kim, Sonal Kumar, Zhifeng Kong, Sang-gil Lee, Chao- Han Huck Yang, Ramani Duraiswami, Dinesh Manocha, Rafael Valle, et al. Audio flamingo 3: Advancing audio intelligence with fully open large audio language models. arXiv preprint arXiv:2507.08128, 2025.
Yuan Gong, Hongyin Luo, Alexander H Liu, Leonid Karlinsky, and James Glass. Listen, think, and understand. arXiv preprint arXiv:2305. 10790, 2023.
Anmol Gulati, James Qin, Chung- Cheng Chiu, Niki Parmar, Yu Zhang, Jiahui Yu, Wei Han, Shibo Wang, Zhengdong Zhang, Yonghui Wu, et al. Conformer: Convolution-augmie transformer for speech recognition. arXiv preprint arXiv:2005.08/00 2020.
Cejm Constant Hak, Rhe Remy Wenmaekers, and van Lej Renz Luxemburg. Measuring room impulse responses: impact of the decay range on derived room acoustic parameters. Acta Acustica United With Acustica, 98:907-915, 2012. URL lttpps::/ hppp//ppp.///// semantccccoolar.......oocicccaie
Corpus ID: 53339067
Wei He, Zhiheng Xi, Wanxu Zhao Xiaoran Fan, Yiwen Ding, Zifei Shan, Tao Gui, Qi Zhang, and Xuanjing Huang. Distill visual chart reasoning ability from Ilms to milms. arXiv preprint MxXiv:2410118798 2024.
Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang. Lu Wang, Weizhu Chen, et al. Lora: Low-rank adaptation of large language models. ICIR, 1(2):3, 2022.
Yo-Yao Huang, Hu Xu, Juncheng Li, Alexei Baevski, Michael Auli, Wojeiech Galuba, Florian Metze, and Christoph Feichtenhofer. Masked autoencoders that listen. Advances in Neural Information 2022.
Processing Systems, 35:28708-28724
Rongjie Huang, Mingze Li, Dongchao Yang. Jiatong Shi, Xuankai Chang, Zhenhui Ye, Yuning Wu, Zhiqing Hong, Jiawei Huang, Jinglin Liu, et al. Audiogpt: Understanding and generating speech, music, sound, and talking head. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, pp. 23802-2388 23804, 2024.
xii

---

Published as a conference paper at ICLR 2026
Drew A Hudson and Christopher D Manning. Gqa: A new dataset for real-world visual reasoning and compositional question answering. In Proceedings of the IEEE/CVE conference on computer viston and pattern recognition, pp. 6700-6709, 2019.
Sheikh Asif Imran, Mohammad Nur Hossain Khan, Subrata Biswas, and Bashima Islam. Llasa: A multimodal IIm for human activity analysis through wearable and smartphone sensors. arXiv preprint arXiv:2406. 14498, 2024.
Justin Johnson, Bharath Hariharan, Laurens Van Der Maaten, Li Fei-Fei, C Lawrence Zitnick, and Ross Girshick. Clevr: A diagnostic dataset for compositional language and elementary visual reasoning. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 2901-2910, 2017.
Zhifeng Kong, Arushi Goel, Rohan Badlani, Wei Ping, Rafael Valle, and Bryan Catanzaro. Audio flamingo: A novel audio language model with few-shot learning and dialogue abilities. arXiv preprint arXiv:2402.0/88/ 2024.
Zikang Leng, Amitrajit Bhattacharjee, Hrudhai Rajasekhar, Lizhe Zhang, Elizabeth Bruda, Hyeokhyen Kwon, and Thomas Plötz. Imugpt 2.0: Language-based cross modality transfer for sensor-based human activity recognition. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies, 8(3):1-32, 2024.
Junnan Li, Dongxu Li, Silvio Savarese, and Steven Hoi. Blip-2: Bootstrapping language-image pre-training with frozen image encoders and large language models. In International conference on machine learming, pp. 19730-19742. PMLR, 2023.
Zejun Li, Ruipu Luo, Jiwen Zhang, Minghui Qiu, Xuanjing Huang, and Zhongyu Wei. Vocot: Unleashing visually grounded multi-step reasoning in large multi-modal models. arXiv preprint arXiv:2405. 1691::990..6619 2024.
Haotian Liu, Chunyuan Li. Qingyang Wu, and Yong Jae Lee. Visual instruction tuning. Advances in neural information processing systems, 96:34892-34916, 2023.
Haotian Liu, Chunyuan Li, Yuheng Li, and Yong Jae Lee. Improved baselines with visual instruction tuning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 26296-26306, 2024.
Ilya Loshchilov and Frank Hutter. Sgdr: Stochastic gradient descent with warm restarts. arXiv preprint arXiv:1608.0.99 2016.
Ilya Loshchilov and Frank Hutter. Decoupled weight decay regularization. arXiv preprint arXiv:1711.05101, 2017.
Rohith Mars, Pranay Pratik, Srikanth Nagisetty, and Chongsoon Lim. Acoustic scene classification from binaural signals using convolutional neural networks. 2019.
Meredith Ringel Morris, Jascha Sohl-Dickstein, Noah Fiedel, Tris Warkentin, Allan Dafoe, Aleksan- dra Faust, Clement Farabet, and Shane Legg. Levels of agi for operationalizing progress on the path to agi. arXiv preprint arXiv:231 (tt::23/1.02440 1.0262,, 2023.
Markus Noisternig, Alois Sontacchi, Thomas Musil, and Robert Hidrich. A 3d ambisonic based binaural sound reproduction system. Advances in Engineering Software AES, 01 2012.
Xiaomin Ouyang and Mani Srivastava. Limsense:
Harnessing llms for high-level reasoning over spatiotemporal sensor traces. 2024 IEEE 3rd Workshop on Machine Learning on Edge in Sensor Systems (SenSys-ML), pp. 9-14. IEEE, 2024.
Archontis Politis, Sharath Adavanne, Daniel Krause, Antoine Deleforge, Prerak Srivastava, and Tuomas Virtanen. A dataset of dynamic reverberant sound scenes with directional interferers for sound event localization and detection. arXiv preprint vrXiv:2166.06999 2021.
Manfred Schroeder. New method of measuring reverberation time. The Journal of the Acoustical Society of America, 37(3):409-41 1965.
Xiii

---

Published as a conference paper at ICLR 2026
Kazuki Shimada, Yuichiro Koyama, Naoya Takahashi, Shusuke Takahashi, and Yuki Mitsufuji. Acedoa: Activity-coupled cartesian direction of arrival representation for sound event localization and detection. In ICASSP 2-211 IEEE international conference on acoustics, speech and signal processing (ICASSP), pp- 915-919. IEEE, 2021.
Changli Tang, Wenyi Yu, Guangzhi Sun, Xianzhao Chen, Tian Tan, Wei Li, Lu Lu, Zejun Ma, and Chao Zhang. Salmonn: Towards generic hearing abilities for large language models, arXiv preprint
arXiv:2310.13289, 2023.
Gemini Team, Rohan Anil, Sebastian Borgeaud, Jean-Baptiste Alayrac, Jiahui Yu, Radu Soricut, Johan Schalkwyk, Andrew Dai, Anja Hauth, Katie Millican, et al. Gemini: a family of highly capable multimodal models. arXiv preprint arXiv:2312.11805, 2023.
Omkar Thawakar, Dinura Dissanayake, Ketan More, Ritesh Thawkar, Ahmed Heakl, Noor Ahsan, Yuhao Li, Mohammed Zumri, Jean Lahoud, Rao Muhammad Anwer, et al. Llamav-ol: Rethinking step-by-step visual reasoning in Ilms, arXiv preprint arXiv:2501.06 2025.
Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, et al. Llama 2: Open foundation and fine-tuned chat models. arXiv preprint arXiv:2307.09256 2023.
Yusong Wu, Ke Chen, Tianyu Zhang, Yuchen Hui, Taylor Berg- Borg-Kirkpatrick, and Shlomo Dubnov. Large-scale contrastive language-audia pretraining with feature fusion and keyword-to-caation augmentation. In ICASSP 2023-2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 1-5. IEEE, 2023.
Yijia Xiao, Edward Sun, Tianyu Liu, and Wei Wang. Logicvista: Multimodal 11m logical reasoning benchmark in visual contexts. arXiv preprint arXiv:2407.0499 2024.
Guowei Xu, Peng Jin, Ziang Wu, Hao Li, Yibing Song, Lichao Sun, and Li Yuan. Llava-cot: Let vision language models reason step-by-step. arXiv preprint arXiv:2411.1044/ 2024.
Haojie Zheng, Tianyang Xu, Hanchi Sun, Shu Pu, Ruoxi Chen, and Lichao Sun. Thinking before looking: Improving multimodal Ilm reasoning via mitigating visual hallucination. arXiv preprint wrXiv:2411.1259) 2024a.
Zhisheng Zheng, Puyuan Peng, Ziyang Ma, Xie Chen, Eunsol Choi, and David Harwath. Bat Learn- ing to reason about spatial sounds with large language models. arXiv preprint arXiv:2402.0159) 2024b.
xiv

---

Published as a conference paper at ICLR 2026
APPENDIX
A BIDEPTH CURATION DETAILS
A.I DATASET STATISTICS
400
300
Count
200
100
0
50
100
150
200
250
300
350
Degree
(a) Azimuth distribution
600
Count
200
0
0
25
50
75
100
125
150
175
Degree
(b) Elevation distribution
600
Count
400
200
0
2
4
8
10
Meter
(c) Distance
Figure 5: Distributions of azimuth, elevation, and source-receiver distance in BiDepth. Azimuth angles are nearly uniform, elevation skewed toward the horizontal plane (reflecting typical indoor acoustics), and distances peak around 1.8 m within 10 m range. The dataset, comprising 28K binaural RIRs and 1.1M QA pairs, will be made publicly available to ensure reproducibility and facilitate further research.
Statistical analysis spatial distributions validates that BiDepth provides balanced and diverse coverage for training and evaluation of geometry models. Figure the distributions of azimuth, elevation, and distance for sound sources relative to the receiver. Azimuth angles are nearly uniform across (0", 3606),) confirming that the dataset covers the full horizontal plane without bias toward particular directions. Elevation angles follow a unimodal distribution centered near 92". corresponding to sources located close to the horizontal plane of the receiver. While this distribution skewed toward horizontal orientations, it mirrors realistic indoor acoustics where most sound sources (e.g., speakers, televisions, human speech) occur around ear level. Source-receiver distances
XV

---

Published as a conference paper at ICLR 2026
bounded within 10 meters, with the highest density approximately 1.8 meters. This ensures the majority of pairs are in-room, while still including out-of-room placements that require models to reason about occlusion and indirect propagation.
Overall, BiDepth consists of 28K unique binaural RIRs and panoramic depth maps, paired with over 1.1M QA exam- ples. The distribution of QA types (de- tection, direction estimation, spatial rea- soning, and CoT reasoning) is reported in Figure 6 confirming that each reason- ing level well represented. Although synthetic, the scale of BiDepth exceeds prior spatial audio corpora by an order of magnitude, providing the coverage needed for geometry-awan training and evaluation.
CoT reasoning: 23%
Detection (single- source); 12%
Detection (dual- source); 12%
Spatial reusoning; 15%
DoA (single- source); 12%
Distance prediction (dual-source); 7%
DoA (dual- souree); 12%
Distance prediction (single-source):
Elevation distribution is biased toward horizontal sources, potentially underrep- resenting extreme overhead or below- floor conditions. Nonetheless, this bias reflects realistic indoor settings and sup- ports stable training. Extending BiDepth to environments with more diverse verti- cal layouts (e.g., multi-floor outdoor spaces) is a promising direction for fu- ture work. The diversity captured here, including in-room and out-of-room scenes, underpins the generalization results reported in Section O
7%
Figure 6: Distribution of question types in BiDepth, in- cluding detection (single/dual-souree) direction-of-arriye (DoA), distance prediction, spatial reasoning, and chain- of thought (CoT) reasoning, shows balanced representa- tion of each category.
0.3
RT60
0.2
0.1
0.0
bathroom
office
familyroom
porch
entryway-
gym/
stairs
kitchen
garage
laundryroom
hallway
bedroom
coridoor
closet
other room
woilet-
living room-
utilityroom
baleony
lounge
dining room
junk
spa-
theater-styy styae seatiig
outdoor areas
meetingroom
classroom
dining booth
library
bar
rec/ ggame
Room Type
Figure 7: RT60 distribution across different room types, showing the variation in reverberation times with emror bars indicating standard deviation. Higher RT60 values correspond to more reverber- ant environments, while lower values indicate faster sound decay.
Xvi

---

Published as a conference paper at ICLR 2026
0.1
0.25
0.0
0.00
-0.25
0.1
0.06
0.12
0.18
0.24
0.3
0.36
0.42
0.48
0
0.06
0.12
0.18
0.24
0.3
0.36
0.42
0.48
Time (seconds)
Time (scconds)
(a) RIR measured right and right receiver.
Left Channel
Righr Channel
0.0
0.1
-0.7
Tiizu 69)
Time
(b) Binaural Audio right and right receiver.
(c) Panoramic depth image of environments 20" FOV, captured from receiver position
Figure 8: Example binaural RIR-depth image from BiDepth. (a) Binaural room impulse responses (RIRs) at the left and right receivers. (b) Binaural Audio left and right receiver (c) Panoramic depth image constructed from 20 field-of-view sweep at the receiver position.
A.2 RT60 STATISTICS OF DIFFERENT ROOMS
Statistical analysis of reverberation confirms that BiDepth captures realistic acoustic diversity across room types. Figure 7 RT60 values across different Matterport3D Chang et (2017) environments used for simulation.
RT60 is standard acoustic measure defined the time required for the sound pressure level decay by 60 dB Hak et al. (2012). Values in BiDepth range from approximately 0.1-0.4s, consistent with typical furnished indoor spaces such bedrooms, offices, and kitchens, while remaining shorter than highly reverberant environments like concert halls or auditoriums. Variation arises from differences room size, object arrangement, and construction materials. Importantly, the dataset includes both in-room and out-of-room configurations, where indirect propagation paths yield longer decay times, further enriching reverberation diversity.
A limitation is that RT60 values are derived from simulated Matterport3D settings, which may underrepresent extreme real-world cases with very high reverberation. Extending the dataset to cover more reverberant environments, particularly measured in real-world settings, is an important direction for future work.
A.3 EXAMPLE OF BINAURAL RIR AND DEPTH IMAGE
Figure 8 prreennts representative example the paired acoustic and geometric signals generated in our simulation pipeline. Binaural RIRs capture spatialized cues the receiver's left and right ears, while the panoramic depth image encodes the surrounding geometry. These paired signals central to SAGE, where depth- goghh-guiedd supervision regularizes the audio representation and enables geometry-aware learning. While this example is simulated, it reflects realistic variations in propagation and geometry diversity observed in indoor environments.
xvii

---

Published as a conference paper at ICLR 2026
A.4 QUESTION-ANSWER DATASET CURATION DETAILS
We generate four types of question-answer pairs that consist of event detection, direction estimation, spatial reasoning, and reasoning with COT.
Type I. We curate 135K QA pairs for the single-source setting, where each question requires the model to identify the event category. A few representative question formats used in this setting are illustrated below.
Type Questions (Single Source)
Can you tell me what kind of sounds are in this recording?
What categories sounds are present here?
Could you list the types of sounds captured in this audio?
Which sound events audible in this clip?
Please identify all the distinct sounds this recording.
Can you break down the audio into individual sound events?
then curate another 135K QA pairs with two sound sources. this setting, given the location of one source, OWL is required to predict its event category. Representative question formats are shown below, where each query follows the {o' (o'clock). (up/down), {distance m convention.
Type I Questions (Dual Source)
Can you categorize the sounds in the audio that are located to the (). () at an estimated distance of {} meters?
Determine the types sounds present in the audio clip from directions the (). (). approximately { distant.
Enumcrate the sound occurrences in the audio clip that are sourced from the (O. O. around O meters away.
Point the sound sources heard O) O at an approximate distance of { meters,
Can you pick out the sound events originating from O. O approximately O meters away?.
What sound sources be identified from (). { roughly () meters distant?
Type II. We create 135K single-source QA pairs for estimating the direction of arrival given sound source class. The answers follow the format of clock; up/ down; distance m. Representa- tive question types are illustrated below.
Type Questions (Single Source)
Could you identify the likely setting of this sound clip's source?
What is the most likely place or seene producing this sound clip?
What's 's the spatial origin of this sound clip?
Which sound events are audible in clip?
Where do you think this audio clip originates?
Could you pinpoint the approximate region or context of sound clip's source?
We further curate 135K two-source QA pairs for localization. Similar to Type I, the class one sound event is provided, and OWL is required to estimate its location. Representative examples of this question type are shown below, where <CLS> is replaced with the corresponding sound event class.
xviii

---

Published as a conference paper at ICLR 2026
Type Questions (Dual Source)
At what distance and in which direction, is the <CLS> sound originating?
what spot the sound <CLS> audible?
Where, in terms of direction and distance. can the sound of the <CLS> be located"?
Can you estimate bearing and distance of the <CLS>'s sound source?
How would you locate the <CLS>'s 's sound in terms of both distance and direction from you?
What is the direction and range the <CT.s>'s sound origin?
Type III. We curate 300K QA pairs targeting relational reasoning, including leftright, frontback, abovebelow, and distance comparisons between two sources. These questions emphasize richer geometric understanding beyond basic perception. All corresponding binaural audio samples contain two sound sources. The answers to these questions are binaray (Yes/No). Representative question formats are illustrated below. We replace the { with sound event class names.
Type III Questions
Are the sounds { coming from the left of you?
Is the sound of {} coming from overhead?
Are the sounds () coming from your back?
Are the sounds of { coming from the left of the sound of {
Is the sound located in front compared to
Are () and (} both originating from one side of you?
Type IV. We curate 250K Chain-of- Thoough (CoT) QA pairs, where each answer includes stepwise reasoning in addition to the final prediction. These questions extend beyond perception and relational queries by requiring explicit multi-step inference grounded in both acoustic and geometric cues. As with Type III, binaural audio samples contain two sound sources. Representative CoT question- answer pairs format are provided below.
Type IV QA Pairs (left-right)
Question:
Which sound can heard to the left of the receiver?
Answer format (One left): <el> originates from <ulp> while <=2> is <a2p>, Therefore, <s1> on the left side the receiver.
format (One right):
receiver,
comes pp
<s2p>.
the right.
Answer format (Both left): Both <s1> and <>2>> positioned <alp> >> and <a2p>, which lie the left receiver.
Answer format (Both right): Since <sl> is at <alpp and <s2> at <a2p>, and both positions are the right, the sounds lie on the right side of the receiver.
Type IV QA Pairs (front-back
Question: From the receiver's perspective, which sound originates ahead?
Answer format (One front): <sl>> originates from pa while <s2> is at <s2p>, Therefore, <s is on the left side recciver.
Answer format (One back); <s1> located <elp>, while <a2> positioned <a2p>. Therefore, <>2> behind receiver.
from <s2p>, both sounds front the Answer format (Both back): Because <sl>> originates from <slp> and <62> from <p2p> both sources located the back side.
Answer format (Both front): Since
comes from <slp>
receiver.
xix

---

Published as a conference paper at ICLR 2026
Type IV QA Pairs (up-down)
Question: Identify the sound that is located the upper side of the receiver.
Answer format (One up): <a1> <elp>, while <a2> <a2p>. Therefore, <a1> is coming from above the receiver.
Answer format (One down): With respect to the receiver, <a at <slp> and <&2> at <s2p> indicate that <>2>is on the lower side..
Answer format (Both up): Because <&1. originates from <s1p> and <s2> from <s2p>, hoth are situated on the upper side.
Answer format (Both down): Since <s1> is <&1 and <s?> <s2p>, both sounds are located heneath.
We replace <s1>, <s2>, <slp>, and <s2p> with source label, source position, source 2 label, and source 2 position.
A.5
EXAMPLE OF QUESTION-ANSWER PATR
Table 9: Examples of QA pairs from BiDepth across the four task types (I-IV). The samples span event detection, localization, relative spatial reasoning, and Chain-of- ofoo-hhh-thttt (CoT) supervision.
No. of Sound Source
Type
Objective
Question-Answer
Event Detection
Q: Could you describe the various sounds detected here? Male singing; Child singing
Given Direction.
Q: Identify the sound events in the audio clip coming from the one clook, down, upproximately 3.0, meters away. A: Bicycle; Bicycle hell
detect the
From which direction and what distance the sound of the Speech be detected?
Direction Estimation
A: seven o' clock; down; 0.5
Given a sound source, estimate the location
Which way and
off Bird flight, flapping wings sound's origin?
A: six o' o'clock; down; 1.5
Left-right composition
Q: Are the sounds of Speech coming from the left of you?
Yes
Front-buck
Speech sound originating from the rear?
III
2
composition
A: No
Upper-below
Q: the sound of Speech originating from below in relation to Speech?
composition
A: Yes
Q: Does the Music sound arrive from a smaller direct distance than
Comparing distance between two sound
the Rodents, rats, mice sound?
A: Yes
Q: Music coming from nearer point than Music when considering direct path?
A: No
Q: the origin of Boat, Water vehicle sound located closer than
origin Speech sound?
A: No
Q: What sound originates from the receiver's 's left side?
Left-rihtt composition
A: Relative to the receiver, Wheeze and Sawing are detected at eight o' clock and left.
o' clock. Thus, they
Identify sound that is located
lower side receiver.
Front-buck composition
A: Because Pulleys comes from nine clock and Animal
from twn o" clock, the source below is Animal.
Upper-below composition
Q: Which sound can be heard in front of the receiver?
Relative receiver, Speech Speech nine clock clock do not correspond front directions.
Table shows representative QA examples from BiDepth across the four task types. These span event detection (Typell), localization with direction and distance (Typell), relational reasoning (TypellI), and explicit step-by-step rationales (TypeIV, Chain-of offThought), Together, they illustrate the progression from lew-level perception to geometry- aware multi reasoning. The taxonomy aligns with the staged training curriculum of OWL, ensuring that each reasoning level is explicitly supervised. As
XX

---

Published as a conference paper at ICLR 2026
detailed in the main paper, the four types represented in roughly equal scale, allowing controlled evaluation across perception and reasoning.
B MODEL DETAILS
B.1 FEATURE EXTRACTION FROM BINAURAL AUDIO
We follow prior work Zheng et al. 2024b) and extract complementary representations from the binaural waveform (t))). A binaural recording is represented as
B₁ (t) (²(() = (t) =
(t, [RRR(t.s,n.))
@ M*()),
where (() and (t) are the left and right ear signals, (* the monaural input event, RIR, (1, 8, 7) is the room impulse response from source to receiver under configuration 7, and C denotes convolution.
We first compute the short-time Fourier transform (STFT) for each channel R):
N-1
Xe(m, k)
Σ
[n] w/n mf] -j2nkn/N
where wi{ is window of length 1024, 320 is the hop size, m indexes time frames, and indexes frequency bins.
From the STFT magnitudes, we derive the Mel-spectrogram. With filterbank M E R FxX (with F = 128 Mel bands and K = 512 frequency bins), the Mel energy frame and band f
S.
Σ log log((M ((f.k) (m)mm,)))))
To encode spatial cues, we compute the Interaural phase difference (IPD):
IPD(m, k) = (m, k)
(R(m,k) k)
To avoid phase wraparound instabilities, IPD is represented using sine and cosine transforms:
cPD (m, k) cos (IsD) k)),
IPD (m,,)) sin(IPD(m,)))
These are filtered with the same Mel filterbank:
iPD (m, f) Σ I(F.k) (m, k).
iPD (m, f) = Σ M(f. IPD (m, k))
Finally, concatenate the four feature maps into the input tensor:
[SL SR. IPD IPD
RXXMXF
where M is the number of time frames, J is the number of Mel bands, and the leading dimension corresponds to the left Mel, right Mel, and IPD sine/cosine channels. This representation preserves
both semantic information (via Mel energy) and geometric cues (via phase differences), providing strong input basis for SAGE.
B.2 SAGE ARCHITECTURE DETAILS
SAGE consists of two modules: a binaural audio encoder and an RIR prediction module, trained jointly with geometric supervision.
The binaural audio encoder processes through an initial x 2D convolution, batch normalization, and GELU. Features then patch-embedded using 16 16 CNN in both time and frequency,
xxi

---

Published as a conference paper at ICLR 2026
producing non-overlapping tokens. These passed through 12-layer Transformer encoder (hidden size 768, heads). Three separate linear heads, each attached to [CLS] tokens, predict event class, direction of arrival, and distance.
The RIR prediction module processes panoramic depth images using a ResNet-18 encoder. The resulting depth features are fused with the audio encoder's output and passed to ResNet-18 18 decoder with transposed convolution to reconstruct the binaural RIR. This auxiliary task supplies privileged geometric supervision, improving spatial reasoning of the binaural audio encoder of SAGE without requiring depth at inference.
B.3 OWL ARCHITECTRRE DETAILS
We reuse the SAGE's binaural audio encoder as our binaural front end. The network a 12-layer Transformer encoder with 85.52M parameters. Given binaural input, the encoder produces sequence frame-level embeddings that capture spectral, interaural, and temporal cues needed for downstream spatial inference. To interface the acoustic features with the language model, adopt a Q-Former as a learned projector. The module has Q-Former layers and a bank of 64 learnable queries. It is trained from scratch to attend over the encoder outputs and emit a compact set of query-aligned tokens. we use LLaMA-2-77 as the backbone. We fine-tune with LoRA on the Query and Value projection matrices inside the self-attention blocks, using rank r=8 and a=32. This adaptation introduces 4.1M trainable parameters, which is roughly 0.062% % of the total model size, while the remaining weights stay frozen. The resulting tuning strategy preserves the linguistic competence of LLaMA-2-7B and focuses capacity on aligning the model to OWL's 's audio- geometric tokens.
C TRAINING DETAILS
C.I TRAINING PROCEDURE OF SAGE
Table 10: Hyperparameters used to train SAGE, including dataset source, optimizer settings, learning rate schedule, and training hardware configuration.
Description
Value
Sound Source
AudioSet-2M Loudness Yes
Audio Normalization
Augmentation
Weighted Sampling
Yes
Optimizer
Adamm Loshchilov & Hutter 2016
Optimizer Momentum
U.U. 1 0.0001
Weight Decay
Base learning rate
0.001
Learning rate schedule
Half-eycle Cosine Loshchilov & Hutter 2017
GPU
x A100
We train SAGE in two sequential stages to progressively incorporate both classification and geometric supervision.
Stage 1 (Audio Pretraining). This stage focuses solely the binaural audio encoder to ensures stable audio representations before introducing additional supervision. The binaural encoder is initialized from AudioMAE BondioMe/HHHungg classification loss Les- This loss encourages the encoder to learn discriminative features for sound event recognition, serving as stable initialization point for subsequent joint optimization.
al. (022) and fine-tuned for 40 epochs using only the event
Stage 2 (Joint Training). We attach the RIR prediction module to binaural audio encoder and optimize both components together for 60 epochs, enabling the network to strike balance between perceptual classification and geometry-aware learning. The training objective is
L 1 Lxinmural 112 Cgen
where Lpinaural = a Cat + oz2 Cais + Lo We set 7h = 1, 172 = 0.01, and a = for audio pretraining stage and on 1250 in joint training, 02 1, and 03 2 unless otherwise noted in
xxii

---

Published as a conference paper at ICLR 2026
Table 11: Hyperparameters used for training OWL, including optimization settings, LoRA configu- ration, and stage-wise epoch schedule.
Description
Value
Sound Source
AudioSet-2OK Loudness
Audio Normalization
Augmentation
No
Weighted Sampling
No
LLM backbone
LLaMA-2-15 22-B Touvron et al.
(2023
Optimizer
Adam Loshchlov & Hutter (0166)
Stage-1:
Epochs
Stage-2:
Stage-3:
Learning Rate
0.0001
LoRA Rank
8
LoRA alpha
32
Global batch size GPU
8
4 x A100 (80 GB)
ablations. The hyperparameters used for both stages of training are summarized in Table 10 To understand the effect of different hyperparameter configurations, are reported in section 6.3 This design ensures that the encoder not only captures task-relevant event information but also aligns its latent space with geometric cues reflected in impulse responses.
perform ablation studies, which
Optimization. We use A with an initial learning rate of 1 x 10-A, cosine decay, weight decay of 0.01, and gradient clipping 1.0. A linear warm-up is applied for the first 5k steps of Stage 1. Batch size is 64. Training is performed on 4 AX100 GPUs with mixed precision (fp16).
C.2 TRAINING PROCEDURE OF OWL
We adopt the three-stage curriculum described in Section 5.1 progressively transitioning from per- ception to relative reasoning to Chain-of-Thouue supervision. The goals are to stabilize optimization early, introduce progressively harder supervision, and restrict learning to adapters while preserving the pretrained encoders.
Stage 1.
The model is trained for two epochs on Type I-1l QA pairs, with 5000 steps of half-cycle cosine LR warm-up.
Stage 2. The model continues to train for epochs on Type reasoning pairs.
Stage 3. The model is trained for three epochs Type IV QA pairs with CoT rationales.
Frozen vs trainable modules. The binaural encoder (. (.) is frozen throughout the stages. The projection module vi(-) is trained from scratch. The LLaMA-2-7B decoder is adapted with LoRA Hu et al. (2022) applied to query, key, and value projections in all attention layers. LoRA rank is 8, scaling factor 32, and dropout 0.05, yielding ~0.8% trainable parameters. This ensures efficient adaptation with limited parameters. This strategy gradually stabilizes, refines, and consolidates the reasoning ability of OWL Table 11 summarizes the hyperparameters used in training.
Optimization. Training uses AdamW W with initial learning rate 1 x 10-4 cosine decay, weight decay 0.01, and gradient clipping 1.0. Batch size is 32 due LLM memory limits. Training runs on 4x A100 GPUs with mixed precision.
xxiii

---

Published as a conference paper at ICLR 2026
D GEMINI API RESPONSE EXAMPLE
import os
from google import genai
class Gemini:
def _ (self, model):
self.api_ken os. .nvenvi self.client genai.Client(
enviroo ron ["GRMINI IIIMINIIA_PAAA
api key key
self audio_dir
self.model model
def get ett_response binaural_avdiofflle.eee question); :
file = self..lliet.filees.pplooo #ile-binnura___udo__fflleee
response self. www.eensttmddls....mmmmmmm ger.raseeeneaatteooteetttt
model=self.model, contents= question, filel
print (response.text)
if name = main ":
parser argparse ArgumentParser
parser add argument ("--api key", type=str, required-True)
parser.add_ane add_agumnd_ ("-moocl", type-str, required-True) args =
enviren ""MMINI_AAI__AAA [PP__ args.api_ken key
cls Gemini (model-args.model)
file file_path ppath/to/pinnural..o o tile.mp3"
questicn = "Where does the Car's sound originate in relation to your position?"
answer cls.get_rreponne
question-question
For an audio sample where the ground-truth car sound location clock; 5.0 m, obtain the following model response:
The car's s sound originates from a source that is approaching you rapidly passing by and then moving away into the distance.
The strong **Doppler effect** is very clear: the pitch and volume rise sharply it approaches, reach peak it passes your position, and then quickly decrease as it moves away.
xxiv

---

Published as a conference paper at ICLR 2026
E
QUALITATIVE RESULTS
Right
20
-2
10
Tate
Tixe
Scene Deseription: 2 sounds (Music, Flectric Pianc) placed in small room.
Question:
From the receiver's perspective, what sound coming from the left?
GT Answer: Since Music the receiver.
clock and Electric piano
left side of clock, both
Relative the receiver, Keybourd (musiccul) and Music detected of clock and eight of clock. Thus, both lie on the left.
Figure 9: Example left-right spatial reasoning question. Two concurrent sounds (Music and Electric Piano) are placed in small room, and the system identifies both originating from the left side of the receiver.
Left Chamaa
Right Cherndl
0.2
Scene Description: 2 sounds (Crackle, Speech) placed in a medium sized room.
Question: Identify sound located side the
GT Answer: Neither Crackle twelve clock Speech of clock positioned below the receiver.
From their given positions twelve clock and one o' clock. evident that sound source is beneath.
Figure 10: Example of up-down spatial reasoning question. Two sounds (Crackle and Speech) are positioned at twelve o' clock and one o' cloock leading to the conclusion that no source is located beneath the receiver.
Left Channel
Right Channel
Scene Deseription: 2 sounds (Waterfall, Music) placed highly reverberant environmens.
Question:
Which sound heard behind the receiver?
GT Answer: : Becouse Waterfall originates from six o' clock and Music from eight o' clock, both sources are located the side.
Relative to receiver waterfall originates seven o'clock and. Music at eight o d'clock, clear that both sounds originate
receiver..
Figure 11: Example of back-front spatial reasoning question. Two sounds (Waterfall and Music) placed in highly reverberant environment. While reverberation causes slight error in localizing the Waterfall (seven instead of six), the final reasoning still correctly infers that both sources are behind the receiver.
XXV

---

Published as a conference paper at ICLR 2026
F EVALUATION METRIC DETAILS
Mean Average Precision (mAP). Mean Average Precision evaluates ranking quality by averaging precision across all relevant retrievals, For each query, the average precision (AP) computed, and the overall mAP is given by
N
mAP Σ AP..
(5)
N
where is the number of queries. Higher values indicate better consistency in retrieval or classifica- tion.
Direction Arrival (DoA) Estimation. We report two metrics for spatial localization:
Mean Angular Error (MAE) This measures the average angular distance between predicted (f, and ground-truth (0, の) directions:
1 MAE
Σ Dai.
(6)
N
where Aa; computed using the spherical law of cosines.
Error Rate at 20 (ER2u=). ). This is the fraction of samples whose angular error exceeds 20%;
ER ER 20
1 1 > 20%
N
Distance Estimation. We evaluate distance prediction using the Distance Error Rate (DER), defined as the proportion of samples where the predicted distance deviates from ground truth by more than 0.5 m:
DER
Σ1
1 [ de 0.5
(8)
N
where de and de denote the predicted and ground-truth distances, respectively.
Binary Accuracy (BA). Binary accuracy measures the proportion of samples where the predicted binary label matches the ground truth. Formally, given predictions it E [0, 1 and ground truth labels yı {0, 1J. defined as
N Σ
BA
1 [
= v1)
(9)
N
1=1
where N is the total number of samples, and [-] denotes the indicator function.
G USE OF LLM
We used the Google Nono-Banana model to generate the logo and Figure | and the ChatGPT 5 model to generate Figure [2] All other LLM usage is explicitly documented in the paper where relevant.
(acoustic environment),
xxvi


Reviews:
[{"review_id": "KxOoB1JTW4", "reviewer": "Reviewer_2wCv", "summary": "- This paper introduces OWL, an ALLM for geometry-aware spatial reasoning. It integrates an audio encoder with a spatially grounded CoT, enabling it to perform multi-step reasoning over direction and distance estimates. A large-scale dataset, coined BiDepth, is constructed to power this framework.\n- The Spatial-Acoustic Geometry Encoder is trained to align binaural acoustic features with 3D spatial structure using panoramic depth images and simulated RIRs.\n- Evaluated on BiDepth and the public SpatialSoundQA dataset, OWL demonstrates state-of-the-art SELD performance and spatial reasoning QA accuracy compared to major baselines.", "questions": "Please refer to the weaknesses for detail. Almost all citations in the main draft are ill-formatted, making it quite uncomfortable to read.", "limitations": "", "rating": 4, "confidence": 4, "soundness": 2, "presentation": 2, "contribution": 3, "strengths": "- This work provides a comprehensive end-to-end framework for geometry-aware spatial reasoning, addressing all key components from the SAGE encoder and the OWL LLM to the specialized BiDepth training data.\n- The paper introduces BiDepth, the first large-scale dataset of its kind, and its construction is sufficiently detailed in the appendix.\n- The framework demonstrates significantly superior performance against both open-source and closed-source baselines, supported by extensive experimental validation and ablation studies.", "weaknesses": "- My main concern lies in the complexity and potential over-fitting of the proposed training pipeline. The framework requires a two-stage SAGE encoder pre-training, followed by a three-stage curriculum for the OWL model. While the ablation studies confirm this complex, multi-step process is necessary for the reported performance, it gives a strong impression of being tightly tailored to the custom-built BiDepth dataset. This raises significant questions about whether the approach is generally applicable or a highly specific solution.\n- The contribution of using panoramic depth maps feels overstated, as this is a standard modality provided by the underlying SoundSpaces and Matterport simulation platforms. This is further confused by diagrams (Fig. 2) that imply the presence of specific sounding objects in depth maps (e.g., a cat), but the methodology does not discuss augmenting the 3D scenes with such new visual assets, only placing sound sources at coordinates.\n- The design choice of dividing azimuth into 12 \"o'clock\" sectors feels ad-hoc and is not justified against other discretization schemes or as an established practice.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "BXVmb0fh0c", "reviewer": "Reviewer_tQDg", "summary": "The paper introduces SAGE (Spatial-Acoustic Geometry Encoder), a geometry-aware audio encoder, and OWL, an audio large language model (ALLM) that integrates SAGE with a chain-of-thought (CoT) spatial reasoning mechanism for auditory tasks. To support this approach, the authors curate BiDepth, a large-scale simulated dataset of over 1.1 million question-answer (QA) pairs, combining binaural audio, room impulse responses (RIR), panoramic depth images, and spatially grounded QA annotations. SAGE is supervised with paired geometric and acoustic data during training (but requires only audio at inference), and OWL builds on SAGE to enable structured spatial reasoning with interpretable rationales. Experiments across both new (BiDepth) and existing (SpatialSoundQA) datasets show consistent improvements in direction-of-arrival (DoA) estimation, spatial QA, and multi-step reasoning compared to prior work.", "questions": "1. Given the elevation bias displayed in Figure 5, how severe is OWL/SAGE’s performance degradation for sources at highly non-horizontal elevations?", "limitations": "", "rating": 6, "confidence": 5, "soundness": 4, "presentation": 4, "contribution": 4, "strengths": "1. The introduction of SAGE, trained with auxiliary geometric supervision (via depth/RIR) but requiring only audio at inference, effectively bridges a key gap in current multimodal audio-language models (ALLMs), which often lack geometric grounding.\n\n2. BiDepth represents a substantial, balanced, and diverse resource for training and evaluating geometry-aware audio-language systems.\n\n3. Quantitative results demonstrate that SAGE and OWL outperform strong baselines (such as BAT and Spatial-AST) on both event detection and fine-grained DoA and spatial reasoning accuracy.", "weaknesses": "1. Reliance on synthetic data limits real-world generalizability.\n2. The BiDepth dataset’s elevation coverage is heavily skewed toward the horizontal plane (Figure 5); this is realistic for indoor sources but may result in models struggling with uncommon vertical positions.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "y4Tf0j2BjL", "reviewer": "Reviewer_V2cc", "summary": "This paper introduces a novel framework for spatial audio understanding, marked by three core contributions. First, it presents BiDepth, a new large-scale synthetic dataset for spatial audio tasks. Second, it proposes SAGE, a novel spatial audio encoder designed to align binaural acoustic features with 3D spatial structure. Finally, it introduces OWL, an audio LLM that utilizes the SAGE encoder to extract and process acoustic and spatial features. The integrated OWL model is shown to achieve superior performance on both the proposed BiDepth dataset and the existing SpatialSound-QA benchmark.", "questions": "- What is the model's performance on real-world data? For example, on the DCASE Challenge?\n- In the results presented in Table 2, the model using Chain-of-Thought (CoT) reasoning shows improved performance on Type I and Type II tasks. This is counterintuitive, as these tasks do not seem to require complex reasoning. Why CoT is beneficial for these tasks?\n- What is the performance of the SAGE encoder for general audio perception tasks? Besides, what are the advantages of a unified encoder like SAGE, which models audio and spatial information jointly, compared to a multi-encoder architecture that models them separately? For instance, [1] demonstrates success with separate modeling. Have you experimented with combining a state-of-the-art audio encoder with the SAGE encoder?\n\n[1] Tang, C.,et al. Can Large Language Models Understand Spatial Audio? Proc. Interspeech 2024", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- The development of the BiDepth dataset is a significant contribution to the field. A large-scale, well-structured dataset for spatial audio understanding can facilitate further research.\n- The SAGE encoder is also a good design.", "weaknesses": "- Lack of Evaluation on Real-World Data. The model's performance is exclusively validated on simulated data. While the results are encouraging, models trained solely on synthetic data often fail to generalize to real-world scenarios due to the domain gap. The paper would be significantly strengthened by testing the model's robustness and transferability on real-world datasets, such as those from the DCASE Challenge on spatial audio tasks.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)