LLM2Fx-Tools: Tool Calling for Music Post-Production
Authors: SeungHeon Doh
Junghyun Koo
Marco A. Martínez-Ramírez
Woosung Choi
Wei-Hsiang Liao
Qiyu Wu
Juhan Nam
Yuki Mitsufuji
Venue: ICLR 2026 Poster (2026)
Keywords: Music Post Production
Fx Chain Generation
Tool Calling
Primary area: applications to computer vision, audio, language, and other modalities
TL;DR: LLM2Fx-Tools is a framework that uses a multimodal LLM to automatically generate executable audio effect chains (as tools), chain-of-thought reasoning, and natural language responses.

Abstract: This paper introduces LLM2Fx-Tools, a multimodal tool-calling framework that generates executable sequences of audio effects (Fx-chain) for music post-production. LLM2Fx-Tools uses a large language model (LLM) to understand audio inputs, select audio effects types, determine their order, and estimate parameters, guided by chain-of-thought (CoT) planning. We also present LP-Fx, a new instruction-following dataset with structured CoT annotations and tool calls for audio effects modules. Experiments show that LLM2Fx-Tools can infer an Fx-chain and its parameters from pairs of unprocessed and processed audio, enabled by autoregressive sequence modeling, tool calling, and CoT reasoning. We further validate the system in a style transfer setting, where audio effects information is transferred from a reference source and applied to new content. Finally, LLM-as-a-judge evaluation demonstrates that our approach generates appropriate CoT reasoning and responses for music production queries. To our knowledge, this is the first work to apply LLM-based tool calling to audio effects modules, enabling interpretable and controllable music production.

Published as a conference paper at ICLR 2026
LLM2FX-TOOLS: TOOL CALLING FOR MUSIC POST-PRODUCTION
Seungheon Doh
122. Doh
Junghyun Koo?**
Marco A. Martinez-Ramírez?
Woosung Choi?
Wei-Hsiang Liao²
Qiyu Wu
Juhan Nam
Yuki Mitsufuji
KAISS Sony AI
Sony Group Corporation
seungheondoh@kaaist kr.
junghyun. kool
ABSTRACT
This paper introduces LLM2Fx-Tools, a multimodal tool-calling framework that generates executable sequences of audio effects (Fx-chain) for music post- production. LLM2Fx-Tooe uses a large language model (LLM) to understand audio inputs, select audio effects types, determine their order, and estimate pa- rameters, guided by chain-of-thought (Co'T) planning. We also present I.-Fx, a new cntrruction-followiin dataset with structured CoT annotations and tool calls for audio effects modules, Experiments show that LLM2Fx- Tools can infer an Fx- chain and its parameters from pairs of unprocessed and processed audio, enabled by autoregressive sequence modeling, tool calling, and CoT reasoning. We further validate the system in a style transfer setting, where audio effects information transferred from a reference source and applied to new content. Finally, L.M- as-a-judge evaluation demonstrates that our approach generates appropriate CoT reasoning and responses for music production queries. To our knowledge, this is the first work to apply LLM-based tool calling to audio effects modules, enabling interpretable and controllable music production.
INTRODUCTION
The use of Audio effects (Fx) processing constitutes a fundamental component of modern music post-production, where producers systematically apply sequences of effects (Fx-chain) to transform audio signals and achieve desired sound textures (Zölzer et al., 2002; De Man & Reiss, 2013). In most post-production workflows, the application of audio effects is guided by both technical and creative criteria, requiring a high level of expertise from audio engineers. Estimating the appropriate Fx-chain from unprocessed input audio (dry), processed output audio (wet), or by reverse engineer- ing from both requires extensive domain expertise and often involves iterative manual adjustment.
To address this challenge, automatic Fx-chain estimation has emerged as promising approach to reduce barrier and labor-intensive aspects of music production. Previous works focus on signal processing- baased optimization (Barchiesi & Reiss, 2010; Giannoulis al., 2013; Ma et al., 2015), bradient-based optimization (Colonel & Reiss, 2021; Lee et al., 2024; Steinmetz et al., 2024; Koo et al., 2025; Lee et al., 2025; Yu et al., 2025), regression methods (Rämö Välimäki, 2019; Sheng & Fazekas, 2019; Mimilakis et al., 2020; Martinez-Ramíres Hayes et al., 2025), and multitask methods (Mitcheltree & Koike, 2020; Lee et al., 2023; Take et al., 2024). While these methods demonstrate promising performance, they face several fundamental limitations. First, gradient-based methods require differentiable audio effects modules, limiting their applicability specific effects. Second, regression and signal processing-basee methods operate on fixed, predefined configurations and lack the ability to dynamically select effects and determine their ordering. Furthermore, these approaches lack user-level interpretability, as they provide only Fx-chain without numan-readable descriptions or reasoning explain why such decisions are made.
al., 2021; Steinmetz et al., 2022;
Meanwhile, recent advances in large language models (Achiam et al., 2023; Grattafiori et al., 2024; Comanici et al., 2025) (LLMs) have introduced powerful capabilities including instruction follow-
'Equal contribution
I Work done while an intern at Sony AI
Demo available https: http:///uunnnhh.ggiiiiiteeeeeeee io/1 1m/11m222xxx -ooool demo/

---

Published as a conference paper at ICLR 2026
LLM2FX-Tooo Generate FX Chain with CoT & Tool Calling
Instruction
Chain-of-
Please provide audio effect
<thinks For generating an audio effect chain simier the reference audio, Step Identily the category and order of chain... Step 2: Create prediction plan</thinke
similar to the reforence audio, which features plano sounds.
Available Tools
EQ
Distortion
Gain
Comp.
Tool Calling (Tool, Order, Parameters)
Reverb
Delay
Limiter
0..
LLM
<toob>'name': 'pan', 'parame": (paan: 0.4))=/toob <toob>/'name. 'delay, 'params"; ('dalay seconds': 0.1. "eedback: mix. matioo (nx,_aatio:.22))/toooe t/)o)//
Reference Audio
Response
Fx Removal & Fx-Norm
masking
<answer>i lore an effect chain for piano sound: Apply deaa with a delay of 70ms, food- back 17%, and a mix of </anwer//
(Pseudo) Dry Audio
Music Post- Produuctodtti::: Apply Generated Fx Chain for (New) Inout Audio
Input Audio
Preprocessing
AFx Tool Chain (Tool, Order, Params)
Output Audio
Fx- Removvl
Pan
Delay
EQ
Limiter
Fx-Norm
Figure I: An illustration of the L.Mo2Fx framework. The input to LLM2Fx- Tools consists of instruction, available tools, reference audio, and (pscudo) dry audio that is preprocessed with audio effects removal and noramlization (Fx- Rex-oel and Fx-Norm). The framework outputs chain of thought, tool calling procedure, and response. The generated tool calling outputs (Fx-chain) are then combined with tool environments (audio effects modules) to enable the transformation of new audio in the style of the reference audio.
ing (Achiam et al., 2023; Wei et al., 2022a), tool calling (Schick et al., 2023). Chain-of-thought (CoT) enables models to decompose a com- plex task into series of reasoning sub-tasks, providing an interpretable view of its reasoning pro- cess (Wei et al., 2022b). Tool calling enables LLMS to flexibly connect with external modules (e.g., non-differentiable audio effect modules) and knowledge bases to accomplish domain specific tasks (Schick & Schütze, 2020; Gao et al., 2023; Doh et al., 2025b). These capabilities present new opportunities to address the flexibility and interpretability issues in Fx-chain prediction. In the context of music post-production specifically, LLM2Fx (Doh et al., 2025c) predicts audio effects parameters from natural language prompts, but it does not employ explicit tool calling or chain-of- thought reasoning and is limited to single effects (Equalization and Reverb).
reasoning (Wei et al., 2022b), and
In this work, introduce L.LM2Fx- L.LM-based Fx-chain estimation with Tool-calling, multimodal framework that addresses these limitations by enabling flexible Fx-chain prediction through tool calling and enhancing interpretability with cfain-of-thought reasoning. L generates I) thain-of-thought, 2) executable Fx-chain and 3) natural language response. Our key contributions are:
Tool- Cool-Calling for Music Production: We develop the first structured tool-calling approach for Fx- chain generation that enables multimodal LLMs to understand audio conditioning and generate ex- ecutable tool calls for non-differentiabbe audio effects modules.
We utilize a signed for Fx-chain generation that decomposes the complex task into interpretable sub-tasks: effect selection, order determination, and parameter estimation. This intermediate reasoning bridges the gap between user inputs and target Fx-chain, improving both performance and interpretability.
Chain-of-Though for Fx-chain Planning:
(CoT) mechanism de-
Multimodal Eestruction-Following We extend the Fx-chain estimation task from unimodal audio- to-effects mapping to multimodal framework incorporating natural language instructions. Users can specify preferred effect types, musical genres, or instrument characteristics, enabling customized Fx-chains that align with specific user requirements.
Conversational Music Production Dataset: We introduce LP-Fx, LLM-based music production dataset for audio effects tools, containing 101K conversational examples with structured Tool Call- ing, Ch Thoouhh, and Response. Each example comprises 1) user instructions, 2) audio effects tool calls, 3) thain-of-thought, and 4) responses.
2

---

Published as a conference paper at ICLR 2026
2 LLM2FX-TOOLS: FX-CHAIN GENERATION VIA TOOL CALLING
2.1 TASK DEFINITION
Our main task is to estimate the Fx-chain (C) that transforms a dry unprocessed audio (Zary) into reference audio signal (Izet). Formally, given Fary and Trets our goal is to estimate C such that Jet = E(C, Fary); where& & denotes the tool environment that applies C to Fary. For additional controllability, we incorporate natural language instructions (Linstmction) to guide the generation process. Our goal learn the inverse mapping
C = fo (iinttructio Eary, Jyeti T)
(I)
where fo represents an LLM that predicts the Fx-chain C = [ttol,, params, ) which consits of sequence of N tools and their corresponding parameters, from the reference audio zief given the available tool set T. We treat each audio effect module tool E T as an external executable tool.
As implied in Eq. (1), we consider Fx-chain estimation task where both Jarry and Tie available, which is commonly referred to as the reverse engineering (Lee et al., 2025) task. However, Zdry is not always accessible in practical scenarios, corresponding to the blind estimation (Lee et al., 2023) task. While our primary target is the reverse engineering, we propose robust training method to simultaneously handle both reverse engineering and blind estimation within single model, as detailed in Section 2.3.
In addition to Fx-chain estimation, our model also aims to generate chain-of-thought (Iem) and nat- ural language responses (Fresponse). The chain-of-thought (CoTT) reasoning serves an intermediate planning stage that decomposes the complex Fx-chain generation into four sequential components: 1) user imput analysis, 2) audio effects module selection, 3) processing order determination, and 4) parameter planning. In our auto-regressive generation framework, the CoT functions as an context condition (Wei et al., 2022b) for subsequent tool calling, bridging user queries and action plans to support more accurate and interpretable tool execution. Following the tool calling generation, the model produces natural language responses that provide users with conversational interface for music production tasks. The overall framework is depicted in Figure 1.
2.2 ARCHITECTURE
Finetune weights
Text Outputs
To enable LLMs to comprehend audio inputs for tool calling, we adopt multimodal autoregressive generation framework (Liu et al., 2023; Gardner et al., 2023). As illus- trated in Figure 2, since LLMs inherently lack audio pro- cessing capabilities, we bridge this modality gap through a pretrained audio encoder coupled with learnable audio- language adapter. This adapter projects audio represen- tations into the language model's embedding space, for- mally defined as: Caudio = Fudapter (f encodeer (Faudio)), )), where fencoder Re + I xde processes input audio signals Faudio (c channels, & samples) to extract audio represen- tations of dimension denc. and falupter xd - x maps these representations to the language model's embed- ding space of dimension dLIM. The resulting audio embed- dings Eauiio are concatenated with text token embeddings form unified multimodal input sequence. Specifically. the input to the LLM is a combined to- ken sequence consisting of the instruction tokens Ennstruction separation tokens e (implemented as natural-language tokens such as "dry audio" and "reference audio" to create token overlap between Cinstruction and emdiio, and the audio embeddings eary and ere from the input audio signals Tary and Fref- This unified multimodal sequence enables the LLM to generate structured outputs including chain-of-thought reasoning Teat, tool calling sequences C, and natural language responses Tresponse-
Freeze weights
Text LM Head
Multimodal Large Language Model
Cinstraciion
SEEP
Gdzy
PESP
Adapter
Text Embedding
Audio Encoder
Text Tokenzier
Audio Inputs
Text Inputs
Figure 2: Model Architecture
We employ Fx-Encoder++ (Yeh et al., 2025), specialized audio encoder trained with contrastive learning to obtain representations for audio effects processing. This encoder con- sists of ConvNet-based feature extractor, pooling layer, and MLP projection layer. We remove the pooling and MLP projection layers and use the patch embeddings handio E R Xixde as audio representations.
Audio Encoder:
3

---

Published as a conference paper at ICLR 2026
Adapter: Unlike previous work that uses simple linear projection layer for cross-modal align- ment (Liu et al., 2023; Gardner et al., 2023), we employ a transformer- based audio-language adapter (Li et al., 2023) with linear projection layer W E Rd and that utilizes 32 learn- able query embeddings Equery E R32xduM This design uses cross-attention to aggregate audio information into learnable query tokens.
Large Language Model:
We employ Qwen3-4BB (Yang et al., 2025) as our foundation LLM back- bone, which provides inherent capabilities for structured tool calling and chain-of-thought reasoning. We fine-tune the model using Low-Rank Adaptation (LoRA) (Hu et al., 2022) with rank 128 and alpha 256 to efficiently adapt the model our Fx-chain estimation task.
2.3 TRAINING
We employ unified autoregressive next-token prediction objective to train our multimodal LLM.
Cross-Entropy for Next-token Prediction: Given training sample with user instruction Finstruction reference audio Zeef, dry audio Edry, chain-of-though Test, tool calling sequence C, and assistant response Tresponse- we construct the input sequence as a concatenation of conditioning prefix and target sequence to be generated by the model as follows:
Finput
Pinstruction dry Trefs Teots C, Presponse
(2)
Conditioning Prefix
Target Sequence
We train the model with the cross-entropy loss LCF, computed only over the target sequence, while leaving prefix conditioning context follows:
LCF = Σ
LETorgs
where Tunget represents the set of token indices of the target sequence.
Number Token Loss: Standard cross entropy loss treats all incorrect predictions equally, even when some numerical values are closer to the correct answer than others. As result, directly applying it not ideal for parameter estimation.
To address this limitation, we adopt regression- like Number Token Loss (NTL), which uses the Wasserstein-1 distance between the predicted and one- hot number distributions (Zausinger et al., 2025):
ENTL-WSS
Σ
Σ
P.(e))a - val(u)]
(4)
Inum TE V
where Inum is the set of sequence positions where number-token applies, and Vmmm is the subset of tokens corresponding to numeric tokens. For a given position i, P((v) the predicted probability of token v. y: is the ground truth numerical value, and val(u) maps token (strings) to its numerical values (floats). This loss function penalizes predictions based on how far they are from the true numerical value, rather than treating all incorrect tokens equally. Our final loss function combines both objectives: Lioral = LCR + XCTT. where À is a hyperparameter for balancing cross-entropy and number token losses.
Multi-Stage Training: To effectively train our multimodal LLM, we adopt a multi-stage training strategy (Liu al., 2023) that systematically builds capabilities from basic audio-language align- ment to complex reasoning tasks. Our training protocol comprises two distinct phases: 1) modality alignment pre-training and 2) LLM fine-tuning while progressively incorporating task complexity.
We first pre-train the adapter module to bridge the audio modality and text modality. We use only audio inputs and tool calling outputs (Fx-chain) as training data, focusing solely on learning the rela- tionship between dry and reference audio differences and their corresponding Fx-chains. We employ random Fx sampling to maximize the diversity of parameter-audio mappings, ensuring comprehen- sive coverage of the parameter space and understanding of audio representations. In this stage, freeze the LLM parameters and only update the audio-language adapter parameters.
In the fine-tuning stage, we initialize the adapter with the pre-trained weights from the previous stage and update both the adapter and LLM through LoRA adaptation (Hu et al., 2022). This stage incorporates the full complexity of our task by training the complete conversational data, in- cluding user instructions, chain-of-the -ffthhugtth reasoning, natural language responses, and tool calling sequences.
4

---

Published as a conference paper at ICLR 2026
Robust Training Techniques for Distribution Shift: As mentioned in Section 2.1, our goal is to estimate C such that zret = {CC, Tay), assuming we have access to both Tay and Teet- However, training model only dataset of paired (Zany Feet) audio samples for the reverse engineering setup pose distribution shift challenges in real-world scenarios, where &dyy is typically unavailable during inference. Even when it is available, its acoustic environment may differ from the training distribution since recording studios vary significantly in their equipment and environments.
To address this challenge, we preprocess the input audio by employing Fx-Removal (Rice et al., 2023) and Fx-Normalization (Martínez-Ramérez et al., 2022) techniques at both the training and inference stages, in order align environmental distributions and obtain pseudo-dry audio &ay-- Furthermore, we apply dry audio masking during training, randomly omitting dry audio inputs with probability Pmasking to force the model to rely solely on reference audio for the blind estimation setup.
3 DATASET: LP-FX
Following previous works (Doh et al., 2025a; Choi et al., 2025), we adapt the LLM based data synthesis pipeline and further improve it by incorporating an LLM-as-a-judge (Zheng et al., 2023), enabling the systematic generation of high-quality conversational data for Fx-chain generation tasks.
3.1 BASE DATASET AND TOOL ENVIRONMENT
The audio source of LP-Fx is MedleyDB (Bittner et al., 2014; 2016), which provides royalty-frre 196 multitrack recordings. Each recording includes three different levels of audio: (i) unprocessed raw tracks, (ii) stems, which are submixes of raw tracks with audio effects applied, and (iii) a full mix, created by combining the processed stems into a complete mixture. We use unprocessed raw audio as the dry audio zary- We filter out multitracks with bleed using the metadata provided by MedleyDB", resulting in curated set of 2,1 19 raw audio files from 116 multitracks, spanning genres and 80 unique instruments. We use the Peda 1 boardd audio effects library and our custom audio effects modules as our tool environment T. We select modules (compressor, distortion, reverb, delay, limiter, and gain) from the Pedalboard library and 3 modules (three-band equalizer, stereo widener, and panner) from our custom modules, totaling 9 modules and 26 parameters.
3.2 DATA GENERATION PROCESS
As illustrated in Figure 3, our data genera- tion process consists of three sequential stages. In the first stage, we sample Fx-chains within musically plausible ranges to create dry/pro- cessed audio pairs. The second stage generates conversations grounded in these Fx-chains to ensure factual accuracy. The third stage produces chain-of-thought rea- soning that explicitly connects user instruc- tions to the underlying audio effects trans- formations. Finally, we employ LLM-as-a- judge (Chen et al., 2024; Zheng et al., 2023) evaluation to filter the dataset for high-quality samples. The basic framework for utilizing LLMs in our data generation process follows
Multitrack Dataset
Tools (Audio Effects Modules)
EQ
Distortion
Gain
Comp.
RAW Recording
& Metadata
Reverb
Delay
Limiter
iestructioo-followiinn
Edry
Fx-Removal & Fx-Norm
Sampled Fx Chain
Gemini 2.5 Flash-Lite
Instruction
Gemini 2.5
Chain-of- Thought
>
Following
Flash
Instruction Following
Sampled Chain
Gemini 2.5
Chain-ot Thought
Score
Data generation process for LP-Fx the formulation Idama L.MM(Fgrrnu)) P), where Tyrund is the grounded information (e.g., Fx-chain C) and P is the carefully crafted prompts. We further provide details of each stage in Appendix C.
Figure 3:
3.3 STATISTICS
Table presents the statistics of our LP-Fx dataset. Based on the observation that task complexity increases with the number of effects in the chain, we create stratified dataset with 11, 100 training
hnttps:/ ://: thub. www.c/mm//amo....mommmmmm med1 htwe///dddddlb/beeeee/eee wtb.mastee//RRaa md 'https://g thub. com http://ooif/////ee board
5

---

Published as a conference paper at ICLR 2026
samples and 100 test samples for each Fx-chain length from 1 to 9, resulting in total of 100,800 conversations. We ensure no overlap between audio tracks used in training and test sets to prevent data leakage.
Table 1: Statistics of LP-Fx Dataset.
Each example comprises 1) user instructions, 2) umprocessed/processee audio pairs, 3) ex- ecutable audio effects tool calls, 4) chain-of- thought reasoning, and 5) assistant responses. The dataset's rich diversity in musical content makes it particularly effective for LLM fine- tuning. With 2,019 tracks spanning 80 unique instruments across 9 genres.
Statistics
Trainset
Testset
# of Dialogue
99,900
900
# of Tracks
2019
100
# of Instruments
80
33
Avg. Instruction length
28.8
28.2
Avg. Response length
178.6
178,6
Avg. CoT length
251.9
252.7
Min/Max Tool Number
1-9
1-9
EXPERIMENTS
4.1 REVERSE ENGINEERING
Task Definition:
We evaluate our approach on reverse engineering. The task involves predicting Fx- chains C' from reference audio žee given access to the corresponding dry audio Zary- We evaluate this task using the LP-Fx test split, which provides ground truth triplets of (Iáry, Fref, C) for evaluation.
Metries: Our evaluation framework assesses model performance through four complementary per- spectives: 1) Fx-chain Planning. 2) Perceptual Distance, 3) DSP Feature Distance, and 4) Deep Embedding Similarity. For Fx-chain Planning, we use classification accuracy to measure whether the model correctly predicts the presence of target audio effects modules in the ground truth Fx- chain. We then apply Spearman rank correlation to assess how closely the predicted ordering of modules matches the ground truth. Finally, parameter mean absolute error (MAE) is used to quan- tify the precision of parameter prediction. For Perceptual Distance, employ Multi- Musti-Resolution STFT (MRS) distance (Yamamoto et al., 2020) on both left-right (L/R) and mid-side (M/S) chan- nels for stereo-aware processing evaluation. For DSP Feature Distance, utilize Audio Features (AF) (Man et al., 2014; Vanka et al., 2024), including root mean square, crest factor, stereo width, stereo imbalance, and bark spectrum. For Deep Embedding Similarity, we employ audio effects- specific pretrained representations, including classification-basee AFx-Rep (Steinmetz et al., 2024) and contrastive learning-based Fx-Encoder (Koo et al., 2023).
Baselines: We evaluate our approach against several baselines to assess the effectiveness of our instriction-following Fx-chain generation framework.
1) No Fx: A naive baseline that applies no audio effects to the input audio, representing the lower bound performance where the predicted reference audio श.e equals the dry audio Tayy-
2) Random Fx: The random number of effects with randomized ordering and parameters.
3) Regression:
regression-basee approach that directly predicts audio effects parameters from audio features without explicit tool selection or ordering capabilities. Specifically, we first extract embeddings using the Fx-Encoder++ (Ych al., 2025), followed by a 2-layer MLP with ReLU ac- tivations. The regression head outputs a vector of logits corresponding to the number of parameters in the full Fx-chain.
4) Multitask: An enhanced regression model incorporating additional classification heads to address limitations of the pure regression approach. The base architecture identical to the regression model, but additional logits are predicted to classify which audio effects modules are applied.
5) DeepAFx-ST (Steinmetz et al., 2022): This approach employs differentiable audio effects mod- ules that enable backpropagation through signal-level objectives, specifically trained with Multi- Resolution STFT loss (Yamamoto et al., 2020). We include this method as a baseline to assess how close its outputs can be in terms of perceptual distance. In our experiments, extend the original implementation by adapting the differentiable audio effects modules to match the audio effects types used in our system, leveraging the lasp-pytorce repository*.
6) Gemini 2.5 Flash (Comanici et al., 2025): A closed-source multimodal LLM with audio under-
standing, reasoning, and tool calling capabilities.
thttps:///o wtt.:/ccccccceeeeeo http://steeimmmttlaaaaapppppeeee /wtapp/paappppappeeee

---

Published as a conference paper at ICLR 2026
4.2 AUDIO EFFECTS STYLE TRANSFER WITH BLIND ESTIMATION
We evaluate our approach on the audio effects style transfer task, which simulates real-world scenarios where users only have access reference audio with different content. This task comprises two sequential stages: 1) blind estimation inferring the underlying Fx-chain C from processed reference audio 2ref- Unlike traditional blind cstimation approaches, our method pro- vides additional context through Fx Removal and Fx Normalization applied to Zirfs enabling reverse engineering-sty inference. 2) style transfer applying the estimated Fx-chain to another source audio Tiput- This evaluation assesses the model's ability to generalize to unseen audio content.
Task Definition:
Evaluation Protocol: To evaluate generalization across different musical content, we employ Moi- sesDB (Pereira al., 2023) the source of processed reference stems and MedleyDB (Bittner et al., 2014; 2016) as the source of clean input audio. This cross-dataset evaluation protocol ensures that models encounter entirely unseen audio content, providing a test of generalization capabilities. We construct evaluation pairs by matching instrument categories between the two datasets, resulting in an evaluation set of 100 test samples. Given that the reference and input audio contain distinct musical content, we focus our evaluation on feature-based metrics, including DSP feature distance and embedding similarity. We employ the same baseline methods of the reverse engineering task as described in 4.1.
4.3 NATURAL LANGUAGE GENERATION
Task Definition:
Beyond the Fx-chain estimation capabilitiis, LLM2Fx-Tool generates chain-of- thought reasoning and natural language responses, providing interpretability and transparency to users through comprehensive explanations of the audio processing decisions.
Evaluation Protocol:
We evaluate the natural language generation quality of our LLM2Fx-Tools
framework. Following previous works (Gardner et al., 2023; Clemens & Marasović, 2025), we assess the natural language generation capabilities through an LLM-as-a-judge framework (Zheng et al., 2023). We use GPT-5 (OpenAl, 2025) as LLM Specifically, we evaluate three key dimensions: 1) tool calling success, whether the model correctly executes the required Fx-chain, 2) instruction following quality. whether the generated response adequately addresses the user instruction, and 3) chain-of-thought quality, whether the reasoning effectively connects user in- structions to responses through coherent intermediate steps. This process can be formulated as (STP, SCoT) LLMMjdge (Finstmctinn Fresponse: Foot: P.ddge). Pjudge details provided in Appendix F.
Baselines: We compare our approach with LLMs for natural language generation: 1) Qwen2.5- Omni 7B: (Chu et al., 2024) An open-source multimodal LLM without reasoning capabilities, 2) Qwen 2.5 4B: (Yang et al., 2025) A compact open-source LLM without audio understanding, and 3) Gemini 2.5 Flash: (Comanici et al., 2025) A closed-source multimodal LLM with advanced reasoning capabilities.
4.4 TRAINING EVALUATION DETAILS
We utilize Qwen3-4B (Yang al., 2025) our pretrained LLM foundation, which provides instruc- tion following, reasoning and tool calling ability. Training is performed across multi-stage training (MST) with different learning rates and batch sizes. For Stage (modality alignment pretraining), we use a learning rate of le-4 with batch size of 32 and train for 100K steps. Stage 2 (LLM finetun- ing) employs a learning rate of 5e-5 with batch size of 16 and iterated for 400K steps.
5 RESULTS/ / ANALYSIS
5.1 REVERSE ENGINEERING
Comparison on Fx-chain Planning. Table demonstrates that LLM2Fx-Tools achieves superior performance across multiple evaluation dimensions. In Fx-chain Planning, our approach signifi- cantly outperforms all baselines, achieving 80% accuracy in audio effects module classification and 0.56 Spearman correlation for ordering. compared to the MultiTask baseline with 61% accuracy and near-zero correlation. While the regression baseline achieves slightly better parameter MAE (0.20

---

Published as a conference paper at ICLR 2026
Table 2: Fx-chain Estimation Results. We compare with multiple baselines and analyze the con- tribution of key components in LLM2Fx-Tove framework: Chain-of-Thouge (CoT), Number Token Loss (NTL)), and Multi-Stage Training (MST). "DeepAFx-ST was trained with Perceptual Dist. as its training objective.
Fx-chain Planning
Perceptual Dist.
DSP
Embedding Sim.(t)
Acc.(t)
Com,(")
MAE())
L/R(L)
M/S(L)
AF())
Rex
FxEne
No Fx Random Fx
13.11
13.49
14.82
0.50
0.30
52%
-0.01
0.39
8,07
8.90
13.70
0.41
0.34
Regression
55%
-0.03
0.20
3.81
4.12
9.20
0.62
0.64
MultiTask
61%
0.00
0.23
3.17
3.39
8.39
0.63
0.66
DeepAFx-ST
1.75
2.06
3.95
0.62
0.66
Gemini2.5raut
78%
0.54
0.32
3,42
4.24
14.97
0,56
0,50
LLM2Fx-Towe
80 %
0.56
0.23
3.13
3.27
8.29
0.68
0.67
w/o CoT
67%
0.49
0.24
3.34
3.38
8.39
0.64
0.66
w/o NTL
73%
0.51
0.32
3.69
3.52
8.47
0.61
0.63
w/o MST
76%
0.55
0.25
3.21
3.32
8.30
0.67
0.64
vs 0.23), this comes at the cost of substantially worse audio effects module selection and ordering capabilities. As DeepAFx-ST is trained directly with an audio domain objective, exhibits strong performance audio distance metrics; yet. it lacks the capability to utilize non-differeneitable modules. Interestingly, Gemini 2.5 Flash demonstrates strong Fx-chain Planning capabilities with 78% effect classification accuracy and reasonable ordering correlation (0.54). However, it exhibits limitations in parameter estimation, achieving the highest parameter MAE (0.32).
The performance improvements of L.M2FFxTTool stem from two key aspects: 1) instruction- following capabilities leverage natural language understanding to provide additional conditioning audio processing
beyond pure audio comprehension, enabling more precise and semantically decisions; and 2) the autoregressive sequence modeling inherent in LLMs provides fundamental advantage for handling Fx-chain ordering compared models that rely solely on audio features.
Does Fx-chain Planning Lead to Better Acoustic Similarity? For Perceptual Distance, LLM2Fx-Tool achieves the competitive performance on both MRS distances. Our analysis dicates that effective Fx-chain Planning is essential not only for accurate parameter prediction but also for achieving strong perceptual performance. Notably, while the regression baseline achieves the lowest parameter MAE (0.20), this advantage in parameter space does not translate into improved perceptual distance. Because the regression model lacks the ability to selectively apply audio effects modules, it must predict parameters for all predefined modules, even when they are absent in the ref- erence audio. This limitation leads to suboptimal perceptual distance, underscoring the importance of Fx-chain planning for bridging parameter accuracy and perceptual quality.
In contrast, both the MultiTask baseline and our LLM2Fx-Tool. framework, which incorporate au- dio effects module selection capabilities, demonstrate superior performance in both perceptual and DSP distance compared to the base regression approach. Comparing MultiTask vs LLM2Fx-Tood further demonstrates the critical importance of Fx-chain ordering: despite achieving similar DSP distances (8.39 vs 8.29), LLM2Fx-Tools substantial improvement in ordering correlation (0.56 vs 0.00) leads to better perceptual reconstruction (3.13 vs 3.17 L/R MRS). This indicates that correct effect sequencing significantly contributes to audio processing quality. as the order of effects can dramatically alter the final audio output. For Deep Embedding Similarity, LLM2Fx Tools achieves the highest similarity scores (AFx-Rep: 0.68, Fx-Encoder: 0.67), demonstrating that effective Fx- chain outputs more semantically similar to reference audio.
-Lools
The lower portion of Table 2 demonstrates that our core design choices contribute meaningfully to model performance. Chain-of-Thougle (CoT) reasoning significantly aids Fx-chain Planning capabilities, improving effect classification accuracy from 67% to 80% and enhancing ordering correlation from 0.49 to 0.56. Number Token Loss (NTL) notably impacts parameter esti- mation, reducing MAE from 0.32 to 0.23, while also improving overall perceptual and feature-level metrics. MST provides improvements across all metrics. bridging the representations between the pretrained audio encoder and L.LM while leveraging the pretrained capabilities of Qwen3.
Ablation Studies.
8

---

Published as a conference paper at ICLR 2026
100
MUSHRA Score
60
40
20
0
No Fx
Regression
MultiTask
DeepAFX-ST Gemini2.5-Flash LLM2Fx-Tools
Reference
Figure Subjective evaluation on reverse engineering.
Human Listening Test. To evaluate the perceptual quality of reverse engineering results, we con- duct a MUSHRA (MUltiple Stimuli with Hidden Reference and Anchor) listening test (Series, 2014). Participants were asked to rate different mixes based on their similarity to the reference mix in terms of audio effects characteristics. Each test question consists seven different stimuli tracks. We designed the test with a total of questions and 16 participants.
reference track and
Figure presents the MUSHRA test results. The hidden reference achieves a mean score of 81.5, confirming the validity of our test design. Our proposed LLM2Fx-Tove outperforms other meth- ods. Pairwise t-tests reveal that LLM2Fx-Toou (62.8), Gemini 2.5 Flash (56.5), and DeepAFX-ST
(54.8) all significantly outperform the No Fx baseline (39.1) with p 0.05. While Gemini 2.5 Flash and DEePAFX-ST show no statistically significant difference between each other (p 0.747), LLM2FF-Tools significantly outperforms both Gemini 2.5 Flash (p 0.020) and DeepAFX-ST (p 0.006). Interestingly, both MultiTask (34.9) and Regression (16.2) baselines score lower than the No Fx baseline, indicating that incorrect effect application can degrade perceptual quality more than applying no effects at all. This finding reinforces our earlier observation that effective Fx-chain planning, including proper effect selection and ordering, is crucial for achieving perceptually similar audio processing results.
5.2 AUDIO EFFECTS STYLE TRANSFER WITH BLIND ESTIMATION
Table 3 presents the experiment results of audio effects style transfer, designed to evaluate the cross- domain generalization capabilities of each method. We observe similar trends those seen in reverse engineering experiments. The regression baseline, which applies all predefined audio ef- fects regardless their relevance, achieves suboptimal performance with higher AF distance (7.83) and lower embedding similarity scores. The MulúiTask approach shows modest improvements with better AF distance (7.62) and enhanced embedding similarities, highlighting the importance se- lective effect application. However, its lack of ordering capabilities limits further performance gains. DeepAFx-S exhibits worse AF distance in the style transfer experiments, indicating limited gener- alization in this setting.
Among LLM-based approaches, Gemini 2.5 Flash performs poorly, yielding DSP dis- tance of 9.00 and embedding similarities barely above the No Fx baseline, despite its large pa- rameter count. Our reverse engineering ex- periments reveal that its parameter predictions are nearly random, which explains why it fails generalize effectively the style transfer task. In contrast, LLM2Fx-Tooll consistently achieves the best results, with the lowest DSP distance (7.41) and the highest embedding sim- ilarity scores ReFx-Rep: 0.35, Fx-Encoder: 0.49) across all evaluated methods.
Table 3: Audio Effects Style Transfer Results.
DSP
Embedding Sim.
AF(J)
AFx-Rep(")
FxHne(()
No Fx Random Fx
8.69
0.24
0.43
15.22
0.14
0.19
Regression
7.83
0.24
0.31
MultiTask FxcPAFx-ST
7.62
0.29
0.46
10.50
0.26
0.49
Gemini2.5puu.
9.00
0.24
0.27
LLM2Fx- -oolss
7.41
0.35
0.49
5.3 NATURAL LANGUAGE GENERATION
Table 4 evaluates natural language generation capabilities through comprehensive LLM-as-a-Judee assessment. Qwen 2.5 demonstrates limited zero-shot tool calling capabilities, failing to gener-

---

Published as a conference paper at ICLR 2026
Table Natural Language Generation Results. We compare with multiple (multimodal) large lan- guage models on tool calling (TC), instruction following (IF), and chain of thought (CoT).
Params
Multimodal
Reasoning
TC Success
IF Quality
CoT Quality
Qwen 2.50mmi
7B
X 0.2%%
1.46
N/A
Qwen 3
4B
V
73.7%
2.89
2.30
Gemini Flash
N/A
/ 100% %
3.39
3.03
LLM2Fx-
4B
/ 99.8%
3.50
3.05
ate correctly formatted JSON structures. In contrast, Qwen 3-4B achieves substantial tool calling success (73.7%) despite lacking multimodal capabilities, indicating the effectiveness of text-based reasoning for this task. LLM2Fx-Tools, built upon Qwen 3-4B with specialized multimodal training, achieves near-perfect tool calling performance (99.8%) matching the capabilities of state-of-the-art closed-source models such as Gemini 2.5 Flash (100.0%)
The instruction following evaluation reveals substantial quality improvements from Nomainnspeciinn training. LLM2Fx-Tooe -ools outperforms zero-shot baselines, with quality scores increasing from 3.63 to 3.86 compared to the base Qwen 3 model. This improvement demonstrates the critical im- portance of specialized training for music production tasks, where general-purpose models lack domain-specific knowledge about audio effects and their applications.
Additionally, CoT quality evaluation shows that LLM2Fx-Took outperforms other LLM baselines. This can be attributed to LLM2Fx Tools's training on high-quality CoT dataset specifically focused on music production tasks. The training dataset LP-Fx leverages knowledge distillation from Gem- ini2.5 Flash, with quality assurance provided through filtering by Gemini2.5 Pro. This multi-stage data synthesis approach is expected enhance the CoT capabilities of the fine-tuned model, en- abling more coherent and domain-relevant reasoning for audio effects manipulation.
6 LIMITATION
While our framework advances interpretable and controllable Fx-chain estimation, several chal- lenges remain. First, the predicted Fx-chain is interpretable only relative to pseudo-dry audio ob- tained through Fx-removal and Fx-normalization preprocessing. Full interpretability would require integrating these preprocessing steps directly into the Fx-chain representation and reasoning pro- Second, the inherent one-to-mas many mapping in audio effects estimation (Hayes et al., 2025) creates ambiguity where multiple distinct Fx-chains can produce perceptually similar results, par- ticularly in symmetric parameter spaces. Our current evaluation framework does fully account for this fundamental ambiguity, and the limited scale of LP-Fx's evaluation set further constrains experimental validation focuses
the ability comprehensively assess these challenges. Third, exclusively on single-instrument sources, which may limit direct applicability to complex multitrack music production scenarios. Furthermore, we have not evaluated generalization to unseen audio ef- fects modules beyond our training distribution. While our tool-calling framework is designed to be extensible to new VST plugins and audio effects modules, empirical validation of this capability remains future work.
7 CONCLUSION
We present 11M2FFx-Tooss multimodal tool-calling framework for generating executable audio effect chains with interpretable chain-of-thought planning. Our experimental results demonstrate that LLMoFx-Tools outperforms regression and multitask baselines on both reverse engineering and audio effects style transfer tasks. Additional evaluations through LLM-as-a-judde confirm the nat- ural language generation capabilities of our approach, demonstrating strong instruction following and saai--of-thoughe reasoning quality. By emitting structured tool calls over audio effect modules, approach enables controllable and explainable music post-proe prottprrouction applications. This work opens several promising directions for future research, including scaling to richer VST plugin tool- boxes, developing reinforcement learning frameworks, and conducting comprehensive perceptual evaluations with expert music producers to validate real-world applicability.
10

---

Published as a conference paper at ICLR 2026
REFERENCES
Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Iige Akkaya, Florencia Leoni Ale- man, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et. GPT-4 technical report. arXiv preprint (axXxx:3333.8777 2023.
Daniele Barchiesi and Joshua Reiss. Reverse engineering of a mix. Journal the Audio Engineering Society, 58(7/8):563-576, 2010.
Rachel M Bittner, Justin Salamon, Mike Tiemney Matthias Mauch, Chris Cannam, and Juan Pablo Bello. MB: A multitrack dataset for annotation-intensivy mir research. In The 15th Inter- national Society for Music Information Retrieval Conference (ISMIR), volume 14, pp. 155-160, 2014.
Rachel M Bittner, Julia Wilkins, Hanna Yip. and Juan P Bello. MedleyDB 2.0: New data and system for sustainable data collection. ISMIR Late Breaking and Demo Papers, 36, 2016.
Dongping Chen, Ruoxi Chen, Shilin Zhang, Yaochen Wang, Yinuo Liu, Huichi Zhou, Qihui Zhang, Yao Wan, Pan Zhou, and Lichao Sun. audde Assessing multimodal llm-as-a-judee with vision-language benchmark. In Forty-first International Conference on Machine Learning, 2024.
Keunwoo Choi, Seungheon Doh, and Juhan Nam. Talkplaydata 2: An agentic synthetic data pipeline for multimodal conversational music recommendation, arXiv preprint arXiv:2509.09683 2025.
Yunfei Chu, Jin Xu, Qian Yang, Haojie Wei, Xipin Wei, Zhifang Guo, Yichong Leng, Yuanjun Lv,
Jinzheng He, Junyang Lin, al. Qwen2-audio technical report. arXiv preprint arX1v:240..10759, 2024.
Michael Clemens and Ana Marasović. MixAssist: An audio-language dataset for co-creative ai assistance in music mixing. In Conference on Language Modeling (COLM), 2025.
Joseph T Colonel and Joshua Reiss. Reverse engineering of recording mix with differentiable digital signal processing. The Journal of the Acoustical Society of America, 150(1))608-619, 2021.
Gheorghe Comanici, Eric Bieber, Mike Schaekermann, Ice Pasupat, Noveen Sachdeva, Inderjit Dhillon, Marcel Blistein, Ori Ram, Dan Zhang, Evan Rosen, et al. Gemini 2.5: Pushing the frontier with advanced reasoning, multimodality, long context, and next generation agentic capa- bilities. arXiv preprint arXiv:2507.06261 2025.
Brecht De Man. Towards a better understanding of mix engineering. PhD thesis, Queen Mary University of London, 2017.
Brecht De Man and Joshua D Reiss. A knowledge-engineered autonomous mixing system. In Audio Engineering Society Convention 135. Audio Engineering Society, 2013.
Soham Deshmukh, Shuo Han, Rita Singh, and Bhiksha Raj. Adiff: Explaining audio difference using natural language. arXiv preprint Stt:22502.04477 2025.
Seungheon Doh, Minz Won, Keunwoo Choi, and Juhan Nam. Toward universal text-to-music re- trieval. In ICASSP 2023-2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 1-5. IEEE, 2023.
Seungheon Doh, Minhee Lee, Dasaem Jeong, and Juhan Nam. Enriching music descriptions with finetuned-llm and metadata for text-to-music retrieval. In ICASSP 2024-2024 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEBE, 2024.
Seungheon Doh, Keunwoo Choi, and Juhan Nam. Talkplay: Multimodal music recommendation with large language models. arXiv preprint arXiv:2502.13713, 2025a.
Seungheon Doh, Keunwoo Choi, and Juhan Nam. Talkplay-tools: Conversational music recommen- dation with Ul tool calling, arXiv preprint arXiv:2510.0/693 2025b.

---

Published as a conference paper at ICLR 2026
Seungheon Doh, Junghyun Koo, Marco A Berrinez-Rameez, Wei-Hsiang Liao, Juhan Nam, and Yuki Mitsufuji. Can large language models predict audio effects parameters from natural lan- guage? In 2025 IEEE Workshop on Applications of Signal Processing to Audio and Acoustics (WASPAA), 2025c.
Luyu Gao, Aman Madaan, Shuyan Zhou, Uri Alon, Pengfei Liu, Yiming Yang, Jamie Callan, and Graham Neubig. PAL: Program-aided language models. In International Conference on Machine Learning, pp. 10764-10799. PMLR, 2023.
Josh Gardner, Simon Durand, Daniel Stoller, and Rachel M Bittner. LLark: A multimodal instruction-following language model for music. arXiv preprint arXiv:2310.07160, 2023.
Dimitrios Giannoulis, Michael Massherg, and Joshua 1) Reiss. Parameter automation in dynamic range compressor. Journal of the Audio Engineering Sociery, 61(10):716-766, 2013.
Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Alex Vaughan, et al. The Llama 3 herd of models. arXiv preprint arXiv:2407.21783 2024.
Ben Hayes, Charalampos Saitis, and György Fazekas. Audio synthesizer inversion in symmetric parameter spaces with approximately equivariant flow matching. In The 26th International Society for Music Information Retrieval Conference (ISMIR), 2025.
Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, Weizhu Chen, et al. LoRA: Low-rank adaptation of large language models. International Con- ference on Learning Representations, 2022.
Tatsuya Komatsu, Yusuke Fujita, Kazuya Takeda, and Tomoki Toda. Audio difference learning for audio captioning. In ICASSP 2024-2024 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 1456-1460, IEEE, 2024.
Junghyun Koo, Marco A Martinez-Ramírer Wei-Hsiang Liao, Stefan Uhlich, Kyogu Lee, and Yuki Mitsufuji. Music mixing style transfer: contrastive learning approach to disentangle audio effects. In ICASSP 2023-2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 2023.
Junghyun Koo, Marco A Martinez- Martinez-Ramirez, Wei-Hsiang Liao, Giorgio Fabbro, Michele Mancusi, and Yuki Mitsufuji. ITOO-Mastr: Inference-time optimization for audio effects modeling of music mastering processors. In The 26th International Society for Music Information Retrieval Confer- ence (ISMIR), 2025.
Sungho Lee, Jachyun Park, Seungryeol Paik, and Kyogu Lee. Blind estimation of audio processing graph. In ICASSP 2023-2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). pp. 1-5. IEEE, 2023.
Sungho Lee, Marco A Mariin, Wei-Hsiang Liao, Stefan Uhlich, Giorgio Fabbro, Kyogu Lee, and Yuki Mitsufuji. Searching for music mixing graphs: A pruning approach. Proceedings of the International Conference Digital Audio Effects (DAFx), 2024.
Sungho Lee, Marco A Martinez-Ramírez Wei-Hsiang Liao, Stefan Uhlich, Giorgio Fabbro, Kyogu Lee, and Yuki Mitsufuji. Reverse engineering of music mixing graphs with differentiable pro- cessors and iterative pruning. AES: Journal of the Audio Engineering Society, 73(6))344-365, 2025.
Junnan Li. Dongxu Li, Silvio Savarese, and Steven Hoi. BLIP-2: Bootstrapping language-image pre-training with frozen image encoders and large language models. In International conference on machine learning, pp. 19730-19742. PMLR, 2023.
Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. Visual instruction tuning. Advances in neural information processing systems, 26:34892-34911 2023.
Zheng Ma. Brecht De Man, Pedro DL Pestana, Dawn AA Black, and Joshua D Reiss. Intelligent multitrack dynamic range compression. Journal of the Audio Engineering Society, 63(6):412 426, 2015.
12

---

Published as a conference paper at ICLR 2026
BD Man, Brett Leonard, Richard King, Joshua D Reiss, et al. An analysis and evaluation of audio features for multitrack music mixtures. In The 15th International Society for Music Information Retrieval Conference (ISMIR), 2014.
Ilaria Manco, Emmanouil Benetos, Elio Quinton, and György Fazekas. Contrastive audio-language learning for music. arXiv preprint urXiv:22088 /2208, 2022.
Marco A Martinez-Ramirez, Oliver Wang, Paris Smaragdis, and Nicholas Bryan. Differentiable signal processing with blxk-box audio effects. In ICASSP 2021-2021 IEEE International Con- ference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 2021.
Marco A Martínez- Wei-Hsiang Liao, Giorgio Fabbro, Stefan Uhlich, Chihiro Nagashima, and Yuki Mitsufuji. Automatic music mixing with deep learning and out-of-domain data. In The 23rd International Society for Music Information Retrieval Conference (ISMIR), 2022.
Stylianos I Mimilakis, Nicholas Bryan, and Paris Smaragdis. One-shot parametric audio pro- duction style transfer with application to frequency equalization. ICASSP 2020-2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 256-260. IEEE, 2020.
Christopher Mitcheltree and Hideki Koike. White-box audio vst effect programming. NeurIPS 2020 Workshop on Machine Learning for Creativity and Design, 2020.
OpenAl. GPT-5 system card. https://cdnoopenaii.c http://.ooo.ssppsysstmaaarr/r//o 2025.
Igor Pereira, Felipe Araújo, Filip Korzeniowski, and Richard Vogl. Moisesdb: A dataset for source separation beyond 4-stems. arviv preprint arXiv:2307.15913, 2023.
Pedro Duarte Leal Gomes Pestana. Automatic mixing systems using adaptive digital audio effects. Phd thesis, Universidade Catolica Portuguesa, 2013.
Jussi Rämö and Vesa Välimäki. Neural third-octave graphic equalizer. In Proceedings of the Inter- national Conference on Digital Audio Effects (DAFx). University of Birmingham, 2019.
Matthew Rice, Christian Steinmetz, George Fazekas, and Joshua D Reiss. General purpose audio effect removal. In 2023 IEEE Workshop Applications of Signal Processing to Audio and Acoustics (WASPAA), pp. 1-5. IEEE, 2023.
Timo Schick and Hinrich Schütze. Exploiting cloze questions for few shot text classification and natural language inference. arXiv preprint arXiv:2001 /000::200.007666.6 2020.
Timo Schick, Jane Yu, Roberto Dessi, Roberta Raileanu, Maria Lomeli, Eric Hambro, Luke Zettlemoyer, Nicola Cancedda, and Thomas Scialom. Toolformer: Language models can teach themselves to use tools. Advances in Neural Information Processing Systems, 36:68539- 68551, 2023.
Series. Method for the subjective assessment of intermediate quality level of audio systems. International Telecommunication Union Radiocommunication Assembly, 2014.
Di Sheng and György Fazekas. A feature learning siamese model for intelligent control of dynamic range compressor. In 2019 International Joint Conference on Neural Networks (IJCNN), pp. 1-8. IEEE, 2019.
Joint
Janne Spijkervet and John Ashley Burgoyne. Contrastive learning of musical representations. arXiv preprint arXiv:2100 103. 103.021003. 2021.
Christian J Steinmetz, Nicholas J Bryan, and Joshua D Reiss. Style transfer of audio effects with differentiable signal processing. Journal of the Audio Engineering Society, 70(9):708-721, 2022.
Christian Steinmetz, Shubhr Singh, Marco Comunità, Ilias Ibnyahya, Shanxin Yuan, Emmanouil Benetos, and Joshua D Reiss. ST-ITO: Controlling audio effects for style transfer with inference- time optimization. In The 25th International Society for Music Information Retrieval Conference (ISMIR), 2024.
13

---

Published as a conference paper at ICLR 2026
Osamu Take, Kento Watanabe, Takayuki Nakatsuka, Tian Cheng, Tomoyasu Nakano, Masataka Goto, Shinnosuke Takamichi, and Hiroshi Saruwatari. Audio effect chain estimation and dry signal recovery from multi-efffct--rocessse musical signals. In Proc. Int. Conf. Digital Audio Effects (DAFx), pp. 1-8, 2024.
Soumya Sai Vanka, Christian Steinmetz, lean-Baptiste Rolland, Joshua Reiss, and George Fazekas. Diff-MST: Differentiable mixing style transfer. In The 25th International Society for Music In- formation Retrieval Conference (ISMIR), 2024.
Jason Wei, Yi Tay, Rishi Bommasani, Colin Raffel, Barret Zoph, Sebastian Borgeaud, Dani Yo- gatama, Maarten Bosma, Denny Zhou, Donald Metzler, al. Emergent abilities of large language models. arXiv preprint arXiv:2206.000 07682, 2022a.
Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoe V Le, Denny Zhou, et al. Chain-of-thought prompting elicits reasoning in large language models. Advances in neural information processing systems, 05:24824-24831 2022b.
Shangda Wu, Zhancheng Guo, Ruibin Yuan, Junyan Jiang, Seungheon Doh, Gus Xia, Juhan Nam, Xiaobing Li, Feng Yu, and Maosong Sun. Clamp 3: Universal music information retrieval across unaligned modalities and unseen languages. arXiv preprint 250. 2025.
Yusong Wu*, Ke Chen*, Tianyu Zhang* Yuchen Hui Taylor Korg-Kirkpatrick, and Shlomo Dubnov. Large-scale contrastive language-audio pretraining with feature fusion and keyword- to-caption augmentation. In IEEE International Conference on Acoustics, Speech and Signal Processing, ICASSP, 2023.
Ryuichi Yamamoto, Eunwoo Song, and Jae-Min Kim. Parallel WaveGAN: A fast waveform gen- eration model based on generative adversarial networks with multi-resolution spectrogram. In ICASSP 2020-2020 IEEE International Conference on Acousties, Speech and Signal Processing (ICASSP), pp. 6199-6203. IEEE, 2020.
An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, et al. Qwen3 technical report. arXiv preprint N0w:25555009888, 2025.
Yen-Tung Yeh, Junghyun Koo, Marco Martínez-Ramires Wei-Hsiang Liao, Yi-Hsuan Yang, and Yuki Mitsufuji. Fx-Encoder++: Extracting instrument-wise audio effects representations from mixtures. In The 26th International Society for Music Information Retrieval Conference (ISMIR), 2025.
Chin-Yun Yu, Marco A Martínez-Ramíree Junghyun Koo, Wei-Hsiang Liao, Yuki Mitsufuji. and György Fazekas. Improving inference-time optimisation for vocal effects style transfer with a gaussian prior. 2025 IEEE Workshop on Applications of Signal Processing to Audio and Acoustics (WASPAA), 2025.
Jonas Zausinger, Lars Pennig, Anamarija Kozina, Sean Sdahl, Julian Sikora, Adrian Dendorfer, Timofey Kuznetsov, Mohamad Hagog, Nina Wiedemann, Kacper Chlodny, et al. Regress, don't guess-a regression-like loss on number tokens for language models. In International Conference on Machine Learning. 2025.
Lianmin Zheng, Wei-Lin Chiang, Ying Sheng, Siyuan Zhuang, Zhanghao Wu, Yonghao Zhuang, Zi Lin, Zhuohan Li, Dacheng Li, Eric Xing, et al. Judging LLM-as-a-judge with MT-bench and chatbot arena. Advances in neural information processing systems, 36:46595-46623, 2023.
Udo 7ölzer, Xavier Amatriain, Daniel Arfib, Jordi Bonada, Giovanni De Poli, Pierre Dutilleux, Gianpaolo Evangelista, Florian Keiler, Alex Loscos, Davide Rocchesso, et al. DAFX-Digita audio effects. John Wiley & Sons, 2002.
14

---

Published as a conference paper at ICLR 2026
A QUALITATIVE EXAMPLES
In this section, we present a qualitative example below illustrating how LLM22x-Tooke processes user query and generates an appropriate Fx-chain. The example highlights each stage of the system's 's reasoning and output: the user's ss request, the model's 's thain-of-thouglh for Fx-chain modeling and parameter estimation, the corresponding tool calls, and the final response presented the user.
User Query
I'm working on a track with Electronic/Fusion synthesizer sounds, and I'd like emulate the audio effects style of reference audio. Could you suggest a plugin chain with a stereo widener, distortion, and a compressor?
Chain-hf-Thought
For emulating the audio effects style of reference audio for Electronic/Fusion synthesizer sounds using a stereo widener, distortion, and compressor:
Step 1. The identified audio effects in the chain are stdreo_widener, distortion, and compressor, in that specific order.
Step 2. For stereo _widener the width parameter will be adjusted to create expansive and immersive stereo image, enhancing the synth's 's presence.
For distortion, the drive db will be increased to introduce harmonic richness and saturation, adding grit and character suitable for electronic fusion.
For compressor, the threshold_dt will be set to effectively manage the dynamic range of the synth, the ratio will be chosen to provide controlled dynamic reduction, the attack ms will be adjusted to either preserve or shape the initial transients, and the release ms will be set to allow the compressor to disengage smoothly with the synth's 's decay, ensuring a natural sound.
Tool Calling
"<tool_call>> |n namee: 'stereo_ wwdenerr,, 'arguments': ('width': [3]//oo<cooo..cll>>>>
<tool call>> ( nname"; 'distortion', 'arguments': {'drive ddrivve_bb'::/ (nn)/oooooooooo call>
toool call>> (n 'name": 'compressor', 'arguments': { ('threshold -19.0, 'ratio': 4.0,
attack_ms': 250.0, 'release_ms': 250.0)) |n</tool_call>>>
Response
Here is a plugin chain designed to emulate the audio effects style of your reference audio, featuring stereo widener, distortion, and compressor. The stereo widener is set a width of 1.3 to expand the stereo image. Following this, a distortion effect with a drive of 5.0 dB is applied to add harmonic content.
Finally, compressor with a threshold of -9.00 dB, ratio of 4.0, attack of 250.0 ms, and a release of 250.0 ms is used for dynamic control.",
B DETAILED TASK DEFINITION
Borrowing notation from (Rice et al., 2023), we define T as the set of audio effects functions sup- ported by the environment (E). Given dry audio (dary)) a typical audio processing can be formulated as composition of functions that yields the processed audio (x.ee) as follows:
Twet gN (9N-1 (-. ... g2 (gi (Jary (itar:i))::0) @2): ( :ox-)):xx).
(5)
where 9n E T is the n-th effect function and on denotes the parameters of gn. In this paper, assume 95 * 9j for any pair of i j for simplicity. We denote the Fx-chain by C [(),, On ) Equation can be reformulated in terms of & and C as Tweet = {CC, &drry)
= [[(tool,; params,
Our main task is to reverse-engineer the Fx-chain (C) applied to reference audio signal (Frer). Specifically, given a processed reference audio signal Tref we aim to predict the sequence of audio effects and their parameters that were used to create the processed version from an original dry signal (I Formally, we can express this relationship as Feet = E(C, Zay), where the environment (E) applies the Fx-chain (C) to the dry audio (Zary) to produce the processed reference audio (I₁at) For additional controllability, we incorporate natural language instructions (Iinsmetton) to guide the generation process. Our goal is to learn the inverse mapping
15

---

Published as a conference paper at ICLR 2026
fo Tinstruction dry Frefi 7)
(6)
where fo represents an LLM that aims to estimate the original Fx-chain C = [tromm,, I from the reference audio Tret and Tary with an additional input Finstruction the provided environment T. Unlike conventional music captioning tasks (?Gardner et al., 2023) that describe the content or characteristics of a single audio signal, our task focuses on identifying the differences between two audio signals (Komatsu et al., 2024; Deshmukh et al., 2025), the transformations applied to convert dry audio into processed reference
Our secondary task involves generating intermediate chain-of of-thougtt (Zeor) and natural language responses (Irespense). The bhain-of-though reasoning serves as an intermediate planning stage that decomposes the complex Fx-chain generation into four sequential components: 1) user input anal- ysis, 2) effect selection, 3) processing order determination, and 4) parameter planning. In our au- toregressive generation framework, the chain-of-thouge functions subsequent tool calling, bridging user queries and action plans to support more accurate and inter- pretable tool execution. Following the tool calling generation, the model produces natural language responses that provide users with a conversational interface for music production tasks.
in-context condition for
C DETAILED DATASET GENERATION
We detail out each stage of the data generation pipeline for creating LP-Fx below as mentioned in Section 3.2.
Stage 1: Dry/processed audio pairs synthesis. For synthesizing processed reference audio Tief we apply the sampled Fx-chain C to the dry audio Tary from MedleyDB. First, we respectively apply Fx-Removal (Rice al., 2023) and Fx-Normalization (Martinez-Ramírez al., 2022) (in the order of EQ, stereo imager, and loudness) to the dry audio samples and create a pseudo-dry audio day.. We randomly sample parameters within predefined min-max ranges and quantize them to discrete steps that mirror practical knob granularity (Pestana, 2013). We employ two sampling regimes: coarse regime to broadly cover the operating space and a fine regime, which reflect real world production practices (De Man, 2017) (sampling ranges are detailed in Table 6). Consequently, we obtain (airy: Fiet, C) triplets where each triplet contains the original dry audio, the processed reference audio, and the corresponding Fx-chain sequence.
Stage 2: Instruction-followin synthesis. We synthesize natural single-turn conversations between users and assistants for music production scenarios using the Fx-chains generated in Stage 1. For efficient large-scale generation, we employ a distillation LLM, Gemini-2.55Flaan --55--aaeee (Comanici et al., 2025). In this stage, the Fx-chain sequence C [(tool,, params,,) from Stage is paired with task prompts Pehat that describe realistic music production scenarios. The LLM then generates natural language instructions Tinstrucionn and assistant responses Tresponses that preserve the underlying Fx-chain structure while providing contextually appropriate explanations, formally expressed as Finstruction Tresponse LLM( Pehat).
Stage 3: Chain-of-thought generation. To bridge the gap between the Fx-chains C generated in Stage and the instruction-responsee pairs Finstruction Fresponse from Stage we decompose the music production task into a step-by-step manner. We construct chain thain-of-thought reasoming by dividing the tool calling process into four sequential steps: 1) user input analysis, tool se- lection, 3) ordering, and 4) parameter planning. We utilize Gemini-2.5-Flaas (Comanici et al., 2025) with enhanced reasoning capabilities for this stage. This process can be formulated as Jeot LLMM(Iinstructt Fresponse C, Pcot) where Peot represents the task prompts that guide the de- composition of complex audio processing into interpretable reasoning steps.
Quality filtering. To ensure data quality and minimize hallucinations in our syn- thetic dataset, we employ an LLM-as-a-judge evaluation framework using Gemini 2.5 Pro (Co- manici et al., 2025). We implement two dimensional quality assessment evaluating: 1) tool alignment, whether generated conversations accurately align with the grounded tool informa- tion, Fx-chain C, and 2) CoT quality, whether chain of thought reasoning effectively guides
Stage 4:
16

---

Published as a conference paper at ICLR 2026
from queries tool calling sequences. This process can be formulated (Stool ScoT) LLdg (Finstruction Fresponse: Zcot, C, (uddge) where Sool and ScoT are quality scores of tool align- ment and CoT, respectively, and juudge is the evaluation prompts. Both 8oal and SCuT evaluated using a point Likert scale (from 1=poor to 4=excellent), where samples scoring I 2 are flagged for regeneration maintain dataset integrity. Judde details provided in Appendix F.
D ABLATION STUDY ON AUDIO ENCODERS
Table 5 shows an ablation study on the reverse engineering task following the regression approach, as described in Section. 4.1, using different audio encoders. We compare contrastive learning- -based music representations (Spijkervet & Burgoyne, 2021; Manco et al., 2022; Doh et al., 2023; 2024; Wu et al., 2025), including audio effects-specific representations (Koo et al., 2023; Steinmetz al., 2024; Yeh et al., 2025) and audio-text representations (Wu* et al., 2023). We observe that dio epfects--pecific encoders outperform audio-text representation. With prior work identifying Fx- Encoder++ as the strongest audio-effects representation (Yeh et al., 2025), and based on its average performance across our objective metrics, we adopt Fx-Encoder++ as the front-end audio encoder for L.M22X-TTool..
Table 5: Ablation study the reverse engineering task using different audio encoders. "Indicates that the metric was computed using the same audio encoder as the input encoder.
Fx-chain MAE(()
Perceptual Dist.
DSP
Embedding Sim.(1)
Audio Encoder
L/R(J)
M/S(.)
AF(U)
AFx-Rep
FxEnc
FxEnc++
CLAP (Wu* al., 2023)
0.21
3.84
4.21
11.3
0.52
Fx- Ex-Encoder (Koo al,, 2023)
0.20
3.66
4.06
9.68
0.56
0.68
0.56
AFx-Rep (Steinmetz al., 2024)
0.21
3.84
3.90
10.4
0.64
0.58
0.52
ix-Encoder++ (Yeh al., 2025)
0.20
3.81
4.12
9.20
0.62
0.64
0.65*
E EVALUATION METRIC
Audio Effects Module Classification Accuracy: We evaluate the model's ability to correctly iden- tify which audio effects module should be applied using standard classification accuracy:
Acc correct predictions
total predictions
Fx-Chain Order Correlation: We assess Fx-chain ordering (order of audio effects module) capa- bility using Spearman rank correlation between predicted and ground truth orders. Missing values are set to ffx pool + 1 for consistent ranking evaluation.
We calculate Mean Absolute Error for parameter prediction: MAE > pss Pal. where parameters are normalized to [o, 1] before computation. This metric only considers correctly classified effects.
Audio Effects Parameter MAE:
Left/Right MRS:
STFT distance computed separately for stereo channels: ((k)
MRS = "xho (Bret, Kret) + C (fref, Trer)) where h indexes different time-frequency reso- lutions, and
STFT |STT
((k (I, .))
(8)
(TTFT( (x))
[(k) mag
(x. .)
log (x))(((())) - log (S) (
N
Mid/Side MRS: We convert stereo audio to Mid/Side representation and compute MRS distance. Mid-channel captures mono content (addition left and right channels) while Side-channel captures stereo width and spatial characteristics (subtraction of left and right channels).
17

---

Published as a conference paper at ICLR 2026
DSP Feature Distance: We extract digital signal processing (DSP) based low-level deseriptors, including the root mean square and crest factor, stereo width and stereo imbalance and bark spectrum corresponding to the dynamics, spatialization and spectral attributes respectively.
Embedding Similarity: We use pretrained audio effect encoders to extract semantic representations and compute cosine similarity for different types of audio encoders, including AFX-Reps (Steinmetz et al., 2024), and Fx-Encoder (Koo et al., 2023).
F PROMPT DETAILS
We present the detailed prompts used for our dataset generation and LLM-as-aajudge evaluation.
Instruction Instruction-ollooving Generation Prompts: We use two main prompts for generating our dataset. The first prompt guides the model to generate realistic user-assistant conversations with appropriate tool calls.
You are post-production assistane (mixing and mastering) speci al ized in audio processing and vs plugin chains. Complete the following conversation.
Output:
11
"role": "user",
"content": onserinstrructio
1h,
(
"role": "assistant",
"content": [assistant_reeponeed
11
Tools: Ifx _x__chain]
Requirements:
User requests audio effect parameters of the reference audio. cts_uusr_rrupss__pccllll.
jts_user_isstructionn
The reference audio contains (genre) (instrument) sounds. In the assistant message, please keep tool number (tool_numer) and the tool order (tool_order!
In the assistant message, briefly explain the audio effect type, order and paraneters with natural Language description. Please provide objective information, don't use overly subjective words. Please answer with short and concise description.
Chain-of-Thought Generation Prompts: The second prompt specifically focuses on generating chain-of-thought reasoning that bridges multimodal understanding with parameter prediction.
You post-production assistant (mixing and mastering) specialized in audio processing and VST plugin chains.
Given a Audio Effects Chain and a previous tool-based chat conversation, generate the next chain-of-thought plan.
Return ONLY single valid JSON object. Do not include any text before after the JSON. Do not use markdown fences.
Outputs:
{1
"chain "chain_of_ thouughht" "<think>For [task description), Stepl,.. Step2, </think>"
1)
where:
task description: The task description the user request.
18

---

Published as a conference paper at ICLR 2026
chain of thought: A step-by-step explanation that covers:
Step 1. From the reference audic, identify the category and order audio effects in the chain. Do not specify exact values.
Step 2. Create an FX parameter prediction plan that describes the general direction and approach for each effect's parameters without specifying exact values.
Constraints:
Use the provided Audio Eftects Chain for eftect and paraneter names; maten exactly.
Chain of thought reflects the assistant's thinking process for ysal and parameter prediction.
Audio Effects Chain: (vst_info)
conversations:
(conversation)
L.-M-as-a-Judge Promptsl: For evaluate dataset generation, we evaluate for tool alignment and thought quality.
You expert evaluator for audio produproduction conversations involving VST plugin chains. ivaluate the assistant's response the gi ven conversation based on the following criteria.
Use scores show the quality of the response. Here is the detailed scoring rubric for evaluating the quality responses from AI assistants:
Tool Alignment (Order, Directicn, Parameter Accuracy):
Poor (1): significant misalignment with tool chain order, incorrect parameter directions, and highly inaccurate parameter values that would produce undesirable audio results.
Fair : Partial alignment with tcel order but contains noticeable errors in parameter direction or accuracy.
Good (3): Strong alignment with toel order, correct paraneter directions, and accurate parameter values with only minor room for improvement. Excellest lent (4): Perfect alignment with tool chain order, correct parameter directions, and highly accurate parameter values demonstrating expert-level understanding.
Thought Quality:
Poor (1): Illogical chain of thought lacking coherent reasoning about audio processing decisions.
Fair (2): Basic rcasoning but contains gaps in logic limited
understanding of audio processing principles. Good (3) : Strong reasoning with clear understanding effect interactions and good audio processing knowledge. Excellent (4): Expert-level reascning with sophist st cated understanding ng of complex effect interactions.
"tool alignment": 11 "score": [1, 2, 3, 41,
1b, "thought_qualltty" {1 "score": [1, 2, 3, 41,
1h
})
Tool calling ground truth: (fx_chain)
Conversation to evaluate: (conversation)
19

---

Published as a conference paper at ICLR 2026
L.-M-as-a-Judge Prompts2: For natual langauge generation, we evaluate for instruction following and chain of thought quality.
You are an expert evaluator for audio post- production conversations involving VSI plugin chains. Evaluate the assistant's response in the given conversation based the following criteria.
Use scores to show the quality of the response. Here is the detailed scoring rubric for evaluating the quality of responses from assistants:
Instructien Following Quality:
Poor (1): The response does not follow the user's instructions, ignores
key requirements, provides irrelevant information. The answer is not in natural language does not address the task described in the instruction.
Fair (2) The response partially follows the instructions, but misses important details or only addresses some aspects of the user's request. The natural Language answer may be incomplete or only loosely related the instruction.
Good (3) The response follows the instructions well, addresses most requirements, and provides a mostly complete and relevant answer in natural language that matches the task in the instruction, but may lack some detail or
Excellent (4): The response fully fellows the user's instructions, addresses all requi rements detail, and provides clear, relevant, and comprehensive answer in natural language that is directly aligned with the task described in the instruction.
Chain of Thought Quality:
Poor (1): The chain of thought dees not logically connect the user's
query to the assistant's response, lacking coherent reasoning about audio processing decisions. The reasoning fails to demonstrate proper task decomposition, analysis of chain implementation. or the chain thought Fair (2): The reasoning attempts to bridge the user's s query and the
input, and planning for effect empty.
assistant's response but contains gaps in logic or shows limited understanding of audio processing principles. Some evidence of task decomposition and planning handle user input may be present but incomplete or flawed.
Good (3): The chain of thought clearly links the user's query to the assistant's response, demonstrating effective task decomposition and planning. The reasoning provides clear evidence of analysis and systematic planning to handle requirements with mostly sound logic.
input
Excellent (4); The reasoning expertly bridges the user's query and the assistant's response through comprehensive task decomposition and strategic planning. The analysis damonstrates thorough task decomposition, comprehensive planning to handle user input, and expert-level reasoning throughout the process.
I
iintrructionffoooownnng following -ualiitygaaly "score": 11, 2, 3, 41,
11, "chain_of_thoue ttt_thhuuh__uuaattyl. (( "score"; [1, 2, 3, 4],
1h,
1) Conversation to evaluate: (conversation) Chain of thought: (cot)
20

---

Published as a conference paper at ICLR 2026
G PARAMETER RANGE FOR DATASET SAMPLING
Table 6: Parameter space the audio effects used in this study. For each parameter, we define the range and discretized step size for both coarse and fine-grained search spaces.
Coarse
Fine
Parameter
Range
Step
Range
Step
Three-band Equalizer
low gain db low cutoff freq
[-20.0, 20.0]
2
[-6.0, 6.0]
[0.0, 400.0]
20
[60.0, 120.0]
10
low Q factor
[0.0, 6.0]
0.5
(0.5,
0.25
mid gain db
[-20.0, 20.0]
2
[-6.0, 6.0]
mid cutoff freq
[250.0, 6000.0]
250
[250.0,
1000.0]
100
mid Q factor
[0.1, 6.0]
0.5
(0.5,
0.25
high gain db
[-20.0, 20.0]
2
[-6.0, 6.0]
high cutoff freq
[4000.0, 20000.0]
1000
[4000.0,
8000.0]
500
high Q factor
[0.0, 6.0]
0.5
[0.5, 3.0]
0.5
Compressor
threshold db ratio attack mss release ms
[-40.0, -5.0]
5
[-20.0,
1
[0.0, 20.0]
[2.0, 8.0]
0.5
[0.0, 500.0]
5
[1.0, 30.0]
1
[0.0, 1000.0]
[0.0, 500.0]
25
Stereo Widener width
[0.0, 1.5]
0.1
[1.1, 1.5]
0.1
Gain
gain db
[-20.0, 20.0]
[-6.0, 6.0]
Panner
pan
[-1.0, 1.0]
0.1
[-0.6, 0.6]
0.1
Distortion
drive db
[0.0, 20.0]
2
[1.0, 5.0]
0.5
Reverb
room size
[0.0, 0.9]
0.1
[0.3, 0.6]
0.05
damping
[0.0, 0.9]
0.1
[0.3,
0.05
width
[0.0, 0.9]
0.1
[0.3, 0.6]
0.05
mix ratio
[0.0, 1.0]
0.1
[0.1, 1.0]
0.1
Delay
delay seconds
[0.0, 0.7]
0.05
[0.01, 0.2]
0.02
feedback
[0.0, 0.6]
0.05
[0.01, 0.2]
0.02
mix ratio
[0.0, 1.0]
0.1
[0.1, 1.0]
0.1
Limiter
threshold db
[-20.0, -1.00
[-5.0,
0.1
release ms
[0.0, 1000.0]
[0.0, 300.0]
25
21


Reviews:
[{"review_id": "B7fQjc5nan", "reviewer": "Reviewer_Rbd9", "summary": "This paper applies existing LLM tool calling techniques to audio effects chain generation. The system uses chain-of-thought to predict effect sequences from audio. The authors create a 101K synthetic dataset LP-Fx generated by Gemini 2.5. In my opinion, the work is mostly an application of existing techniques to a new domain without significant technical innovation.", "questions": "Recommand using the term \"audio effects (parameter) transfer\" instead of \"audio style transfer\"\n\nGemini 2.5 Flash gets MAE 0.32 vs your 0.23, but it achieves 78% accuracy vs your 80%. Why does such a large model fail so badly on parameters? Does it indicate setup issue?", "limitations": "", "rating": 4, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "1. First work applying structured tool calling to audio effects chains\n2. Comprehensive evaluation across multiple metrics", "weaknesses": "1. The paper misuses terminology. \"Audio style transfer\" has established meaning in audio processing literature (timbre/texture transformation). This work only does audio effects parameter transfer, which is much narrower. This creates confusion with existing work and is misleading.\n2. Limited technical novelty. The method is standard multimodal LLM fine-tuning: audio encoder -> adapter -> LLM with LoRA. This is direct application of existing techniques without methodological contribution.\n3. No human evaluation despite claims about \"interpretability\" and \"controllable music production\". All evaluation is automatic metrics or LLM-as-a-judge, which has known reliability issues.\n4. Missing details: How does the model handle effects outside the 9 trained types? The paper claims \"users can incorporate their own audio plugins\" but provides no evidence.\n5. The work is mostly experimental validation that LLM tool calling works for this task. The technical contribution is limited.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "DNjRsVuCU3", "reviewer": "Reviewer_CSNR", "summary": "The paper presents LLM2Fx-Tools, a novel tool-calling framework which for a given set of audio inputs, provides executable audio effects sequences (Fx-chain), with appropriate CoT reasoning and responses . The paper also introduces LP-FX, a new instruction following dataset with CoT annotations and tools calls for audio effects. The authors provide experimental validation for their approach along with a demo page for subjective verification.", "questions": "For equation 4, $N$ corresponds to the sequence length or training examples? Previous equation 3 has $t$, which is over the sequence, while for $t$ in equation 4 represents the upper range of the number token. It will be helpful if we can improve the notation a little here.\n\nPlease consider having a subjective evaluation and a stronger baseline like (https://arxiv.org/pdf/2207.08759) for the style transfer task. \n\nGemini 2.5 Pro is used both for dataset generation judge in 3.2, and natural language evaluation judge for table 4. Could we use a different judge to remove the bias for this case?", "limitations": "", "rating": 8, "confidence": 3, "soundness": 2, "presentation": 4, "contribution": 3, "strengths": "Originality: The paper's key novelty lies in formulating Fx-chain estimation as a LLM-based tool call problem. The autoregressive modeling for LLMs is able to learn the sequential order of audio effect calls as opposed to systems only based on audio features. \n\nQuality: The paper has detailed experiments around the three evaluation tasks, reverse engineering to show the model can predict tool-chain for paired audios, blind style transfer to show the generalization capability to unseen audios, and natural language language generation to showcase interpretability. Across all the tasks,  LLM2FX-Tools results are strong as compared to the baselines. The authors conduct ablations to show the importance of optimization decisions (CoT, NTL, MST). \n\nClarity: The paper is well written with clear notation and figures, with appendix covering all necessary details for dataset generation, evaluation and LLM prompting.\n\nSignificance: LLM2FX-Tools framework treats the audio effect modules as external non-differentiable tools, which makes the framework flexible to diverse real world scenarios.The authors also present a LP-FX dataset with CoT annotations and tool calls, which is beneficial for future research", "weaknesses": "For the reverse engineering task, the strongest baseline is Multi-task regression, which comes close even without relying on the ordering of Fx-chain, while the LLM is learning that information. The authors can consider adding a pairwise-ordering loss for the 9 audio effects for the multi-task baselines.\n\nFor the style transfer task, the style of the output appears to be mixed between the input and reference audio while listening subjectively to the demo examples. A comparison with differential audio effects style transfer baseline would be quite important to see (https://arxiv.org/pdf/2207.08759). The objective evaluation can benefit on a larger set than 100 test samples\n\nAs covered in the limitation section, the paper relies on Fx-normalization and Fx-removal preprocessing, while ideally, they should be modeling as part of the tool-calling framework. Experimental validation is limited to single instruments, datasets are relatively smaller in size with ~2k tracks.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "nsPS8cJzfi", "reviewer": "Reviewer_A8mk", "summary": "This paper presents LLM2Fx-Tools, a tool-calling framework that generates sequences of audio effects (Fx-chains) for music post-production. The authors also introduce a new dataset, LP-Fx, to support this task. The topic is novel and of clear interest to the audio and music research community. The proposed system builds upon Fx-Encoder++ and fine-tunes Qwen-4B to achieve the goal of automatic Fx-chain generation.", "questions": "- Will the training dataset and training code be released for reproducibility in future work?\n\n- Will the evaluation dataset and evaluation code also be made publicly available?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- The proposed approach to Fx-chain estimation is novel. The integration of Chain-of-Thought (CoT) reasoning into the training framework is also interesting.\n\n- The problem is clearly defined and well motivated.\n\n- The methodology for dataset creation is clearly described and systematically organized.", "weaknesses": "- In Figure 1, the meaning of FxNorm is unclear.\n\n- In Figure 2, why does e_{SEP} consist of two tokens?\n\n- Below Equation (1), what is N? What is param_n?\n\n- In Section 2.1, second paragraph, the authors mention “handle both tasks.” What exactly are the two tasks?\n\n- In Section 2.1, the term “secondary task” is introduced but not clearly defined.\n\n- In Section 2.2 (Audio Encoder), why was Fx-Encoder++ chosen over other possible encoders? How might different audio encoders influence system performance?\n\n- The writing in Section 2.3 (Number Token Loss) and Equation (4) needs improvement for clarity. The statement “a key problem with Cross Entropy is that it treats all incorrect predictions equally” is vague—please elaborate on how this issue is addressed in your proposed loss. Overall, Subsection 2.3 and Equation (4) are difficult to follow.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "AhiFImc0KX", "reviewer": "Reviewer_Cr7x", "summary": "This paper proposes a new framework (including a model, dataset, and overall methodology) for music post-production based on a multimodal LLM. This is evaluated on inferring an Fx-chain, but also on style transfer and using the LLM as a judge.", "questions": "As stated above, I fully agree with the design choices made by the authors in terms of methodology, problem setting, evaluation, and the new dataset based on MedleyDB. The paper is also well structured and well written, and I also appreciate the inclusion of a section on Limitations, which is not something always present in ICLR submissions. \n\nMy only comment is the one stated above, on whether is ICLR the most suitable venue for a work which does not offer any innovation in AI, but rather uses established AI methods with some minor modifications as to support a research question and problem directly situated in the field of audio engineering. As such I might recommend this paper on being marginally out of scope of ICLR.", "limitations": "", "rating": 4, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "* This is a relatively unexplored application domain in terms of using multimodal LLMs for music post-production tasks.\n* The overall choice of models, the task definition, training process, dataset creation, and evaluation methodology are all appropriate and technically sound.", "weaknesses": "* My main source of criticism for this paper is that this work overall uses established AI methods for a new application. There is little to no AI innovation taking place, and I feel that this work would be more suitable for a venue specializing in audio production or audio engineering (e.g. AES conferences or conventions, ICASSP, or DAFx). I do not see any compelling evidence for inclusion in ICLR.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)