MixAssist: An Audio-Language Dataset for Co-Creative AI Assistance in Music Mixing
Authors: Michael Paul Clemens
Ana Marasovic
Venue: COLM 2025 (2025)
Keywords: audio-language dataset
dataset creation
co-creativity
TL;DR: CoMix introduces a novel audio-language dataset of 640 collaborative mixing conversations, designed to train AI assistants for co-creative music production by capturing real-world expert-amateur interactions.

Abstract: While AI presents significant potential for enhancing music mixing and mastering workflows, current research predominantly emphasizes end-to-end automation or generation, often overlooking the collaborative and instructional dimensions vital for co-creative processes. This gap leaves artists, particularly amateurs seeking to develop expertise, underserved. To bridge this, we introduce MixAssist, a novel audio-language dataset capturing the situated, multi-turn dialogue between expert and amateur music producers during collaborative mixing sessions. Comprising 431 audio-grounded conversational turns derived from 7 in-depth sessions involving 12 producers, MixAssist provides a unique resource for training and evaluating audio-language models that can comprehend and respond to the complexities of real-world music production dialogues. Our evaluations, including automated LLM-as-a-judge assessments and human expert comparisons, demonstrate that fine-tuning models such as Qwen-Audio on MixAssist can yield promising results, with Qwen significantly outperforming other tested models in generating helpful, contextually relevant mixing advice. By focusing on co-creative instruction grounded in audio context, MixAssist enables the development of intelligent AI assistants designed to support and augment the creative process in music mixing.

Published as a conference paper at COLM 2025
MixAssist: An Audio-Language Dataset for Co-Creative AI Assistance in Music Mixing
Michael Clemens" & Ana Marasović University of Utah
michael..lemens, ana. wawasovic Seutiiic@@ethhhhcc edu
Abstract
While Al presents significant potential for enhancing music mixing and mastering workflows, current research predominantly emphasizes end- to-end automation or generation, often overlooking the collaborative and instructional dimensions vital for co-creative processes. This gap leaves artists, particularly amateurs seeking to develop expertise, underserved. To bridge this, we introduce MIXASSIST, novel audio-language dataset capturing the situated, multi-turn dialogue between expert and amateur music producers during collaborative mixing sessions. Comprising 431 audio-grounded conversational turns derived from in-depth sessions involving 12 producers, MIX ASSIST provides unique resource for training and evaluating audio-language models that can comprehend and respond the complexities of real-world music production dialogues. Our evalua- tions, including automated LLM-as-a-judge assessments and human expert comparisons, demonstrate that fine-tuning models such as Qwen-Audio on MIXAsSIST can yield promising results, with Qwen significantly outper- forming other tested models generating helpful, contextually relevant mixing advice. By focusing on co-creative instruction grounded in audio context, MIXASSIST enables the development of intelligent Al assistants designed to support and augment the creative process in music mixing.
Introduction
The integration of Artificial Intelligence (Al) into music production offers compelling op- portunities, from automating technical tasks to potentially augmenting creative work- flows (Deruty et al., 2022; Nicholls et al., 2018). Within this domain, music mixing- the intricate process of blending and processing individual tracks into a cohesive whole- remains critical stage heavily reliant both technical skill and artistic judgment (Moffat, 2021). While commercial AT tools provide valuable end-to-end automation for mixing and mastering, simplifying complex processes and reducing costs (Moffat, 2021; Vanka et al., 2023), they often function as "black boxes", abstracting the underlying techniques and potentially offering limited pedagogical value for users seeking improve their own mixing proficiency (Vanka et al., 2023; Li et al., 2024b).
However, significant and growing segment of users amateurs and pro-ams (Leadbeater & Miller, 2004)- desire tools that facilitate learning through doing, providing contextual guidance and co-creative support within their actual mixing environment (Vanka et al., 2023). Moreover, studies involving novice creators in music production highlight how AT, while accelerating ideation, can compress crucial preparation stages and introduce new challenges in selecting, validating, and integrating Al outputs (Fu et al., 2025). Research in Comann-Computer Interaction (HCI) increasingly emphasizes understanding situated practices and user needs in designing human-Al AI collaborative tools, particularly in creative domains like music (McCormack et al., 2020; Eric et al., 2024).
Portions of this work were conducted while Michael Clemens was at New Jersey Institute of Technology:
1

---

Published as a conference paper at COLM 2025
Audio segment being discussed
Listens alongside amateur
Plays audio relevant question
trying blend these drums with this ittle
think torn could around bring It down maybe live
ON feet like that's sounding better already. Do you like crastic? you like bigger?
For this style music, would typically probably
mixing, but with style get really exper
This funtl feel like I'm loaminn much?'
Expert
Amateur
Figure 1: An example of the audio-grounder multi-turn instructional dialogue captured in the MIXAsSIST dataset. An amateur producer plays an audio segment (top) and asks clarifying question about stylistic choices ("Do you like it drastic?") after receiving specific technical advice (equalization suggestion: "low shelf around 150Hz") from the expert. The expert's response further provides justification based on genre conventions ("For this style of music...") "), highlighting the co-creative and pedagogical interaction patterns modeled in the MIXASSIST dataset.
To foster the development of co-creative mixing AI assistants, we introduce MIXASSIST, an audio-language, conversational dataset specifically designed to model the instructional dialogue between expert and amateur music producers. Moving beyond datasets focused on static parameters (De Man & Reiss, 2017), single-turn annotations (captions /tags) (Kim et al., 2019; Drossos et al., 2020; Agostinelli et al., 2023; Law et al., 2009), or general/ mattra///sscc-ppcfffeee question-answering (Gong et al., 2023; Liu et al., 2024; Gardner et al., 2023), MIXASSIST captures the dynamic, turn-y--urn exchange of knowledge grounded in specific audio contexts during live mixing sessions. Figure 1 presents snippet from one of the sessions, showcasing the interplay between audio context, amateur inquiry, and expert guidance. MIXAsSIST contains 431 conversation turns derived from live collaborative mixing sessions involving 12 music producers, featuring temporal alignment between dialogue and the corresponding audio segments being discussed.
Effective music mixing involves both conceptual understanding (the "why", often captured in dialogue) and practical implementation (the "how", reflected in specific settings like equal- izations (FQ) levels or compressor thresholds, known as parameters in music production Our primary focus is on capturing the conversational "wohy" through the audio-grounded dialogues in MIXASSIST. Direct logging of parameters during the data collection sessions was intentionally avoided to preserve the natural workflow and avoid interfering with participants' creative process, as well as to allow participants to use their preferred Digital Audio Workstation (DAW). The nuances related to specific DAW choices and parameter adjustments discussed during these sessions are captured primarily within the dialogue itself (e.g.." "I think that guitar should go up about 3dBs").
Separately, introduce MIXPARAMS, a detailed dataset containing specific parameter settings derived from DAW sessions corresponding to The Mix Evaluation Dataset (De Man & Reiss, 2017), which utilizes the same source songs featured in our MIXAsSIST sessions (See Appendix H for more details). While this parallel use of source material suggests potential avenues for future research exploring links between conversational guidance and technical settings, MIXPARAMS serves primarily as a complementary resource. Our primary focus in this work remains the analysis and modeling of the unique conversational dynamics within MIXASSIST. It is this rich, audio-grounded conversational data that provides the foundation for training AI assistants capable of co-creative dialogue.
Leveraging MIXASSIST, we investigate the potential of state-of-the-art audio language models (ALMs) to function as co-creative mixing assistants. We fine-tune three promi- nent prodell-Qweerr
(Chu et al., 2023), LTU (Gong et al., 2023), and MU-LLaMA (Liu et al., 2024) - chosen for their diverse strengths in general audio un- derstanding, audio reasoning, and music-specific processing, respectively. Using large language model (LLM) ranking generations of these three models, we observe that the fine-tuned Qwen model significantly outperformed the others, achieving the top rank in
2

---

Published as a conference paper at COLM 2025
50.4% of the evaluations. This finding was validated through human expert assessments, leading to the selection of Qwen for ecologically valid in situ user studies where participants interacted with the model during mixing task. These subsequent real-time interactions highlighted Qwen's conversational strengths but also revealed significant limitations in its audio analysis capabilities particular for music mixing, demonstrating the necessity of evaluating such assistive creative agents within realistic user workflows (further details in Section 4 and Appendix F).
Our primary contributions through this work include:
The public release of MIXASSIST, the first audio-grounded, multi-turn conversational dataset specifically capturing expert-amatert instruction in music mixing.
Empirical validation of fine-tuned ALMs for conversational mixing assistance, assessed through automated, expert, and in situ user evaluations.
This research aims to foster the development Al systems that empower artists by enhanc- ing their mixing capabilities with accessible, context-aware guidance, striking balance between automated support and human creative agency, aligning with calls for more human-centric and collaborative AI tools in creative practices (Tsiros & Palladini, 2020).
2 Related Works
Audio-Language Models (ALMs) The field of ALMs has evolved rapidly from models learning joint audio-text representations via contrastive learning (e.g., CLAP; Elizalde et al., 2023), which excel at zero-shot classification and retrieval but lack generative capa- bilities, towards integrating audio perception with the reasoning and generation power of LLMs (Deshmukh et al., 2023). ALMs released in the past few years- including LTU (Gong et al., 2023), Qwen-Audio (Chu et al., 2023), SALMONN (Tang et al., 2023), and Audio Flamingo 2 (Ghosh et al., 2025) employ stronger LLMs (e.g., LLaMA-7B; Touvron et al., 2023), stronger audio encoders (e.g., AST; Gong et al., 2021), and cnerruccionfinnttunii datasets (e.g. OPAMAQA-5MI; Gong et al., 2023). Specialized models like MU-LLaMA (Liu et al., 2024) and LLARK (Gardner et al., 2023) target music-specific understanding and instruction-followeer
-
We select Qwen-Audio andio-nsttruc-711 LTU, and MU-LLaMA as base- lines within our study given their diverse strengths in general audio understanding, audio reasoning, and specialized music processing relevant to the nuances of mixing dialogue.
AI Music Mixing AI approaches to music mixing have spanned knowledge-based sys- tems attempting to codify heuristic mixing rules based on genre conventions or perceptual targets (e.g., EQ balancing or stereo placement) (Schaffer & McGee, 1997), data-driven methods learning parameters for effects like EQ or compression (Moffat & Sandler, 2019b; Martinez Ramirez et al., 2021), and newer techniques using differentiable digital signal pro- cessing (DDSP) to directly control effects or generate parameters from text (Liu et al., 2023; Chu et al., 2025). While these systems advance automation (Moffat, 2021), they predomi- nantly focus on predicting or applying technical parameters, often lacking conversational features. This contrasts with user studies highlighting a need for assistive, co-creative tools that offer explanation and control, facilitating learning and exploration rather than only end-to-end automation (Vanka et al., 2023). Our work directly addresses this gap by focusing on modeling the explanatory, pedagogical dialogue within the mixing process.
Text-and- Audio Datasets Progress in ALMs relies on suitable datasets. Existing resources offer general audio classification/ tags (e-g., AudioSet; Gemmeke et al., 2017), captions (Au- dioCaps; Kim et al., 2019), or single-turn QA such ClothoAQA (Lipping et al., 2022) and OpenAQA-5M (Gong et al., 2023). Music-specific data includes captions/ tags (Music- Caps; Agostinelli et al., 2023), QA pairs (MusicQA; Liu et al., 2024), or instruction--uuing data (LLARK; Gardner et al., 2023)). focuseter-focuse datasets like the Mix Evaluation Dataset (De Man & Reiss, 2017) provide technical settings. While these valuable resources capture descriptive labels, static parameters, or single-turn interactions, they generally lack the dynamic, multi-turn, audio- indio-grounded conversational flow and pedagogical interaction
3

---

Published as a conference paper at COLM 2025
inherent in co-creative, instructional setting such music mixing. MIXASSIST is the first dataset specifically designed to capture this nuanced dialogue between experts and amateurs during active mixing sessions. This provides a unique resource for training ALMs to understand and participate in the co-creative and instructional aspects of music mixing.
For detailed discussion of related literature, please Appendix A.
3 MIXASSIST Data Collection
This section details the creation and analysis of the MIXASSIST dataset. We first outline the motivation and design principles guiding the dataset's development ($3.1), then describe the dataset construction process including participant recruitment, session procedure, and data processing steps ($3.2), and finally analyze the resulting data's characteristics.
3.1 Motivation and Design Considerations
To ground the development of MIXASSIST, we first surveyed music producers (N=5) with diverse expertise levels about their mixing practices and learning needs (see Appendix B for survey details). The survey confirmed prevalent challenges in music mixing and dissatisfac- tion with current learning resources among the respondents. There was strong interest in co-creative AI assistant capable of providing context-aware explanations and guidance, with producers envisioning it as potentially "Very" or "Extremely" useful. Open-ended responses highlighted desires for collaborative development, such wanting to "learn from famous mixes" " (Survey Respondent 4 (S4)) or finding "someone to bounce ideas off of with the goal of fostering development (S1). While producers like S4 voiced concerns about adaptability and ethical sourcing ("scraping of techniques"), the clear interest in AI for guided learning motivated our work. Respondent preferences centered on interactive help inside the DAW, utilizing personalized suggestions, examples, and visuals.
These findings motivated us to propose the task of music-mixing response generation: Given conversation history between an amateur and a human/A AT expert producer collab- oratively mixing music, the amateur producer's latest response, and audio from the most recently played track, the Al assistant must generate contextually relevant, technically correct, and pedagogically helpful response. To support developing and evaluating ALMs for this task, we identify two key desiderata for suitable dataset:
Instructional Dialog: Captures the dynamic, multi-turn pedagogical dialogue between experts and amateurs during the mixing process, a dimension missing from prior datasets focused on tags, captions, or single-turn QA.
Audio Grounded: Each conversational turn is associated with relevant music-only audio segment being discussed.
3.2 Dataset Construction & Analysis
We illustrate the process of constructing the MIXASSIST dataset in Figure 2. We first con- ducted seven hour-long co-creative mixing sessions. While these sessions involved expert- amateur pairings, the total number of unique producers was 12 instead of 14, as two individuals participated in multiple sessions swapping roles, allowing us to capture di- verse learning dynamics. During the sessions, amateurs mixed multitrack audio recordings sourced from The Mix Evaluation Dataset (De Man & Reiss, 2017), chosen for its permis- sively licensed material across various genres (see Table 15 for song details). To structure these sessions and elicit realistic pedagogical interactions, we defined distinct participant roles and tasks. Amateurs used their preferred DAW and employed a think-aloud pro- tocol (Van Someren et al., 1994), verbalizing their mixing process and reasoning, Experts acted as mentors, providing guidance primarily upon the amateur's request. This expert- novice pairing and structured interaction protocol aimed to simulate a natural learning environment while ensuring the collected dialogue focused on instructional exchanges relevant to the mixing task. The specific guidelines provided to each participant type are
4

---

Published as a conference paper at COLM 2025
Raw Data REC
Conversation Transcripts
Speaker-Split & Aligned
Focused Sub- Conversation
Dataset Instances
Hadel Zeput -aput-
v qu+ q +
(Phor Conresation Summang.
Madel-Irqur--
90 mrit
oed codl...
AAl right. thank. TThe lick sounds. pap mork"
Utternnel.
Usseano))
#ccel-Oxtpac:
Utternnce)
Figure 2: Overview of the MIXAsSIST cataset construction pipeline. Raw data from co- creative mixing sessions between expert (E) and amateur (A) producers is first transcribed. These transcripts are then processed to split utterances by speaker and align them with the corresponding audio segments discussed during the session. Aligned turns are grouped into focused sub-conversations based on the mixing topic (e.g., keys, drums, etc). Finally, da dataset instances are created for training audio-language models, pairing conversational context (a generated summary and recent dialogue history ending with an amateur utterance) and the relevant audio segment as input, with the subsequent expert utterance as the target output.
detailed in Appendix G.2. The task was for the amateur to complete mix of the provided song with the expert's s guidance. Direct parameter logging was intentionally avoided to maintain ecological validity and allow participants their preferred DAW; nuances related to parameters were captured within the dialogue itself. Full details on participant recruitment, compensation, and other factors are provided in Appendix G. This study was IRB approved.
The next step was to derive dataset instances from the collected ~7 hours of joint music mixing. Dialog was transcribed using Whisper (Radford et al., 2023) and then manually cleaned by one author, removing filler words while retaining conversational backchannels (e.g., 'uh-huh', 'okay') and using ellipses for natural pauses or fragments. This process included splitting utterances by speaker (amateur/ /aruur/exx expert) and augmenting dialogue with standardized placeholders (e.g., "Please analyze this audio segment" or "I need more infor- mation before I can respond. Please elaborate") where necessary for conversational structure or to account for interaction deviations. Postprocessing also involved removing Personally Identifiable Information (PII) and segmenting relevant music-only audio segments, aligning them with corresponding amateur utterances.
A critical step involved filtering for pedagogical value: Expert responses were manually annotated with a binary 'has content' tag indicating substantive, actionable guidance. While inherently subjective, this labeling employed reflexive practices (Palaganas & Estacio, 2021; Finlay, 2002) to maintain consistency. This tag was crucial not only for selecting target expert outputs but also for curating the expert utterances within the conversational history provided input to the models. Specifically, expert turns within the input dialogue history were also filtered to include only those marked Thas content=True'. This consistent approach was adopted to steer the model's learning towards generating similarly actionable feedback. The final MIXAsSIST instances exclusively use expert responses marked hhas_content=Trus as the target output, ensuring the dataset focuses on meaningful instructional content.
To structure the data, consecutive utterances were grouped into sub-conversations by mixing focus (e.g., drums, vocals), identified through instrument mentions and dialogue context. These sub-conversations represent coherent instructional episodes. For creating turn-level MIXAsSIST instances, summaries providing relevant context from earlier in the session were generated using gpt-4o-ming -o--miii-2224 1eei--2224---111 instead of using the entire preceding dialogue. This approach provides focused historical context across topic shifts while maintaining manageable inputs for efficient processing (see Appendix G.3 for prompt details).
Finally, each MIXASSIST instance was constructed. The input consists of the generated prior session summary, the dialogue within the current corr-ccionrs up to the immediately preceding amateur utterance, and the associated music-only audio segment. The target output is the subsequent expert utterance. This process resulted in the final dataset of 431 expert-authores audio-grod audio-grounded instructional responses. Illustrative examples of these final dataset instances can be found in Appendix G.5.
These 431 instances were partitioned into training (241 examples), development (34 exam- ples), and test (156 examples) sets. To rigorously test generalization, the split was performed
5

---

Published as a conference paper at COLM 2025
using sub-conversations as units, holding out conversations from two complete sessions (the shortest, ID 5 Hard Rock; and second longest, ID 2 Pop Rock) exclusively for the test set. Sub-conversations from the remaining five sessions were distributed across all three splits. This strategy ensures evaluation against unseen producer pairs and promotes testing diverse genres (see Table 14 and Appendix G.1 for further details).
Detailed statistics on the final dataset splits, linguistic properties (e.g,, tech- nical term usage), topic distribu- tion (e.g,, drums, overall mix), and temporal dynamics (e.g,, evidence of amateur vocabulary acquisition) are presented in Appendix G. Ad- ditionally, we also release the com- plete, unprocessed audio recordings from the sessions, containing par- ticipant dialogue and simultaneous DAW playback to support research in areas such as end-to-end conver- sational speech recognition (Prab- havalkar et al., 2023) or the analy- sis of fine-grained interaction dynam- ics. The dataset, including both pro- cessed data and raw recordings, will be publicly available.
MIXASSIST General Data Statistics
Number mixing sessions
Average # of turns per session
61.57
Average # of topic switches per conversation
10.00
Average expert utterance length (# tokens)
36.57
Average amateur utterance length (# tokens)
25.39
Average audio segment duration (# seconds)
19.44
MIxAsSIST Topic Distribution
Drums
40.4%
Overall mix
25.3%
Guitars
15.1%
Vocals
8.6%
Bass
6.5%
Keys
4.2%
Train
Dev
Test
Expert Responses
241
34
156
# Subb-Convessatinns Avg. # Turns / Sub-Convers.
28
11
38
8.61
3.09
4.11
4 Experiments
Table 1: Dataset statistics of MIXASSIST.
This section details our experimental methodology for evaluating ALMs as co-creative mixing assistants using the MIXASSIST dataset, along with results and discussion. We first introduce the baseline ALMs we fine-tuned (64.1). Next, we describe our evaluation approach, which includes LLM-as-a-judge, human preference judgments, and qualitative user studies involving real-time interaction ($4.2). We then present the quantitative and qualitative results (54.3), followed by a discussion interpreting these findings and outlining implications for the future design of Al-assisted music mixing tools (64.4).
4.1 Baselines
Given the multimodal and instructional nature of the task, appropriate baselines require models capable of processing both audio context and dialogue history. Our primary bench- marks are derived from SOTA ALMs specifically fine-tuned on MIXASSIST. We selected three distinct ALMs based their diverse pre-training focus and capabilities relevant to our task:
(Chu et al., 2023) for general audio understand- ing, LTU (Gong et al., 2023) for audio reasoning, and MU-LLaMA (Liu et al., 2024) for music-specific processing via its MERT encoder (Li et al., 2023). To confirm the necessity of fine-tuning, we also evaluated a zero-shot prompting baseline against our fine-tuned model, the results of which are presented in 54.3.
While prompting offers a potential approach, fine-tuning was chosen to adapt these models more deeply to the specific nuances, terminology, and conversational patterns of pedagog- ical music mixing dialogue present in the MIXAssISI dataset, which are less likely be fully captured by zero-shot or few-shot methods alone. For efficient adaptation, we used LoRA (Hu al., 2022) to avoid the computational cost associated with updating all model parameters. Models were trained for single epoch; preliminary experiments with more epochs (e.g., 4) showed signs of overfitting on conversational artifacts (e.g., generating repet- itive acknowledgments like 'Mm-hmm') aligning with recommendations in the original Qwen and LTU fine-tuning documentation. Other hyperparameters followed the default settings provided in the respective model implementations.

---

Published as a conference paper at COLM 2025
Metric
Qwen
LTU
MU-LLaMA
Overall Performance
Avg. Rank (Lower is better)
1.59
1.70
2.71
Avg. Score (out of 10)
7.06
6.48
1.46
Times Ranked #1
126 (50.4%)
111 (44.4%)
13 (5.2%)
Rank Distribution
% Rank 1
50.4%
44.4%
5.2%
% Rank 2
40.4%
40.8%
18.8%
% Rank 3
9.2%
14.8%
76.0%
Table 2: LLM-as-judge (o3-mini) ranking results (250 samples)
4.2 Evaluation
Through our experiments, we sought to have both qualitative and quantitative exploration of the evaluation of these models. Full details of the methodology, survey questions, and results for each stage are available in the appendices referenced below.
N-Gram Overlap Measurements and LLM-as-a-Judde Evaluating generative models for complex, subjective tasks like conversational music mixing advice poses significant challenges regarding scalability and consistency (Li et al., 2024a). Consequently, we em- ployed the LLM-as-a-judge framework (Zheng et al., 2023; Gu et al., 2024) using mul- tiple contemporary models as judges, including gemma3: 4b, qwen3: : 8b, 1lama3. 8b, and o3- 03-mini-2825-011
We used listwise ranking approach (N=3) for each input prompt, asking the judge to rank the outputs based on technical accuracy, helpfulness, and fluency, forcing a clear preference ordering. This methodology, including prompt design and valida- tion against human raters, represents, to our knowledge, the first use of LLLM-as-a-judge in the music mixing context. Detailed methodology and results are presented in Appendix D. Standard automated text generation metrics (e.g., BLEU, ROUGE, BERTScore) were also computed. However, given the known limitations of these metrics for evaluating conver- sational quality (Sai et al., 2022), we prioritize the LLM-aa-a-judge and human evaluation findings.
Human Evaluation: Generated vs. Expert Responses Based its performance in the multi-model LLM-as-a-judge evaluation (see $4.3), Qwen was selected for the subsequent resource-intensive human preference and real-time studies. Recognizing that automated evaluations like LLM-as-a-judge may not fully capture nuanced human preferences (Zheng et al., 2023), we conducted a direct human preference study to provide a gold-standard comparison between the best fine-tuned model (Qwen) and the original human expert responses from the MIXASSIST dataset. This study involved 10 music producers with varying expertise. Participants compared pairs of responses (ground-trutte human expert vs. fine-tuned Qwen output) for 10 prompts randomly selected from the test set, evaluating which response was better given the prompt and audio context, or if both were good/ bad. Further details on the methodology and participant instructions are provided in Appendix
To assess usability and interaction dynamics in a more ecologically valid setting (Kumar et al., 2024; Subramanian et al., 2023), we conducted a real-time interaction study. 10 music producers interacted with the fine-tuned Qwen agent via a chat interface, using it to get guidance on mixing a track their choice for approximately 5 minutes. Participants then completed post-interaction survey assessing conversational naturalness, perceived creative contribution, and novelty of ideas, along with open-ended feedback. Survey questions, results, and analysis are detailed in Appendix F.
Human Evaluation: Real-Time Interaction.
7

---

Published as a conference paper at COLM 2025
4.3 Results
Our multi-stage evaluation yielded insights into the capabilities and limitations of fine-tuned ALMs as co-creative mixing assistants.
Automated Evaluation (LLM-as-a-Judge) Our automated evaluation leverages robust LLM- as-a-judge framework to compare our fine- tuned baselines. To ensure the reliability of our findings, we used multiple contemporary mod- els judges (gemma3; 4b, qwen3: 1lama3. 1: 8b, and 1--mini-2225-0--330
Preference
Count (N=100)
%
Generated Better
40
40.0
Human Better
33
33.0
Both Good Both Bad
12
12.0
The results consis- tently showed that the fine-tuned Qwen model outperformed LTU and MU-LLaMA, achieving the best average rank across all judges. This out- come validated its selection for our human evaluations. We also conducted an experiment comparing our fine-tuned Qwen model against a zero-shot prompted baseline, which demonstrated that line-tuning on MIXASSIST is necessary for achieving high-quality re- sponses. The detailed methodology and full results tables for these evaluations are presented in Appendix D.
15
15.0
Table 3: Human Preference: Gen. vs. Exp.
Human Evaluation (Generated vs. Expert) We present these results in Table 3. When compared directly against the original human expert responses from the MIX dataset, human evaluators (N=10) surprisingly showed a slight preference for the responses gener- ated by the fine-tuned Qwen model (preferred 40% of the time) over the original human expert responses (preferred 33% of the time). Qualitative analysis suggests generated out- puts may have been favored when providing more detailed explanations (e.g,, explaining compression concepts when changed topic) or more structured, direct answers (e-g., concisely suggesting a separate drum reverb vs. lengthy human anecdote about reverb workflows). Conversely, hu- man responses often excelled at leveraging implicit context (e.g., correctly interpreting ambiguous user input such as "0.5") or providing quick, natural conversational feedback. Given the limited sample size (N=100 comparisons), these observations are preliminary and warrant further investigation. See Appendix for detailed breakdown and further discussion including specific examples.
expressed confusion, unlike human response that
Human Evaluation (Real-Time Interaction) Table 13 (Appendix) shows the results of the real-time interaction study (N=10). Participants generally found the Qwen-based agent conversational, with 70% rating the interaction as 'Somewhat natural' or 'Very natural'. 70% also reported the agent suggested novel ideas ('Somewhat' or 'A little') they hadn't 't consid- ered. However, users perceived their own creative contribution as higher than the agent's (60% rated agent contribution 'Slightly less' or 'Significantly less'). Qualitative feedback highlighted good conversational fluency and adaptability but pointed to limitations in the depth and accuracy of its audio analysis and the creativity of its suggestions. Full survey results and thematic analysis are in Appendix F.
4.4 Discussion: Future Design of AI-Assisted Music Mixing
Following the co-creative mixing sessions detailed in $3.2, we conducted semi-structured interviews to further inform the design of co-creative agents. Synthesizing insights from these interviews (detailed in Appendix C.2) and the real-time interaction study (Appendix F), outline key implications for designing effective AI co-creative partners for music mixing.
Explainable Teacher and Assistant: A strong theme emerged favoring AI as an educational tool and technical assistant. Producers expressed desire for agents that "explain industry standards" and demystify processes, like compression (Amateur 3 (A3)). The ideal role was often framed as a guide "'Hey, this needs a compressor, but this is how you achieve that." (Expert 7 (E7)). This educational potential was echoed in the
Desired Role:
8

---

Published as a conference paper at COLM 2025
real-time study, where participants valued the agent's conversational ability (70% found it natural) and its capacity to adapt, with one noting, "I especially liked how when I used non- specific, non-technical terms... the agent matched me" (P1). Beyond explanation, automating tedious tasks like track grouping and initial setup was seen as highly valuable for workflow efficiency (E1, A6, E4). Additionally, users strongly preferred feedback integrated visually within their DAW, wanting the Al to, for example, "show like your computer mouse like dragging, moving it... Like this is what would sound good" (A7), rather than relying solely on text.
Challenge 1: Audio Understanding: While conversational interaction was generally positive, our user studies revealed significant limitations in the current model's audio analysis capabilities, critical factor for mixing assistant. Several participants noted the agent failing to provide meaningful insights based on the audio provided, with one stating, "It would have been more helpful had it been able to analyze the audio" (P5). This work does not claim to have solved this core research challenge; rather, a key contribution of MIXASSIST is providing benchmark to diagnose and address this very limitation. To quantify the impact of our dataset, we conducted a manual analysis which confirmed that fine-tuning on MIXAsSIST significantly enhances the model's ability to provide substantive, task-relevant, and correct guidance compared to a non-fine-tuned baseline. The detailed quantitative results of this analysis are presented in Appendix F
Challenge 2: Balancing Guidance with Creativity: Both studies revealed tension between wanting guidance and preserving creative control. While 70% of real-time study participants felt the agent suggested novel ideas, feedback also indicated desire for "more precise or little more tut-of-thhebox- suggestions (P5), suggesting the current guidance might be perceived as too generic or conservative. This resonates with interview concerns about Al leading to homogenized sounds if strictly enforcing "industry standard[s]" (E3) or stifling creativity by adapting too closely to a user's style without flexibility (E4). Designing agents that can offer technically sound advice while also prompting creative exploration and Seepyaaccctttttt
allowing for "happy accidents" (E2) remains core challenge.
Challenge Ethical Positioning: Ethical concerns, particularly around data provenance and attribution, were consistently raised in interviews. There was strong consensus on the need for transparency regarding training data and fair compensation or attribution for creators whose work informs Al recommendations, with suggestions ranging from royalty systems (A2) to source citations (E5) and explicit consent (E1, A7). Developing ethical frameworks is crucial to build trust and ensure responsible Al deployment in creative fields.
Future Directions: Based on our findings, future work should prioritize enhancing ALMs' audio grounding to improve their ability to understand and reason about complex audio mixtures beyond basic feature detection when applied to music mixing. Concurrently, developing multimodal interaction paradigms that integrate visual feedback within DAWs, as strongly desired by users, is another potential direction. Future research can leverage the parallel structure between MIXASSIST and the MIXPARAMS dataset (aaeeeetApppae (Appendix H) to explore the explicit link between instructional dialogue and concrete mixing parameters, potentially enabling models to translate conversational advice into explicit technical ac- tions. This dialogue-centric approach complements other methods that aim generate parameters directly from textual descriptions in zero-shot manner, such as LLM2Fx (Doh et al., 2025). Further exploration is needed in adaptive creativity support for music mixing that balances supportive scaffolding with suggestions that encourage user agency, perhaps via configurable Al intervention. Finally, implementing transparent Al systems, alongside ethical guidelines for data usage, attribution, and compensation, re- mains critical. We acknowledge our findings' generalizability is constrained by the challenge of accessing diverse, high-quality open-source multitracks. The MIXASSIST methodology provides a blueprint for expanding data collection to build more robust assistants capable of learning the nuances of pedagogical and co-creative interaction grounded in audio.
community-acceptee

---

Published as a conference paper at COLM 2025
5 Conclusion
Music mixing remains a complex craft requiring both technical skill and artistic judgment. While Al offers potential to assist, gap exists in tools that support co-creative learning and collaboration, particularly for amateur music producers. This work introduced MIXASSIST, novel audio-language dataset capturing the multi-tur,, instructional dialogue between ex- pert and amateur producers during live mixing sessions. Derived from 7 sessions involving 12 producers, MIXASSIST provides 431 audio-grounded conversational turns focused on pedagogical interaction.
Our experiments showed that current ALMs, when fine-tuned on MIXAsSIST (with Qwen showing particular promise), can generate contextually relevant conversational mixing advice, sometimes even preferred over original human expert responses in ranked compar- isons. However, user studies also highlighted limitations, notably in audio understanding and creative suggestion capabilities, along with a strong user desire for explainable, control- lable, visually integrated tools that function as teaching assistants instead of black boxes.
By providing the first dataset focused on situated, multi-turn instructional dialogue in music mixing, MIX ASSIST enables future research to address these limitations. It serves resource for developing and evaluating the next generation of AT mixing assistants- sysstes designed not to automate, but to collaboratively empower human creativity and skill development in the art of music production.
Availability
To support further research, the MIXASSIST dataset, including the processed conversational data, associated audio segments, and the raw session recordings, will be made publicly available upon publication. The MIXASSIST dataset accessible on Hugging Face, with the audio sessions and transcripts available from Zenodo, and the evaluation code on GitHub.
Ethics Statement
The development and deployment of MIXASSIST were guided by ethical considerations central to co-creative AI. All 12 participants provided informed consent for the recording and use of their dialogue for research purposes, and all Personallyy Identifiable Information (PII) was redacted during processing (SG). The dataset is built upon publicly available multitrack recordings from "The Mix Evaluation Dataset," " ensuring no copyright infringement ($3.2).
However, we lcdn that the development of advanced Al assistants carries broader societal risks. While our pedagogical approach aims to augment human skill, such tools could be perceived as threat to producers who earn income by coaching others. Fur- thermore, we must be cognizant of risks to creative originality. Research has shown that using LLMs can lead to semantic homogenization at a group level (Anderson al., 2024). This highlights the challenge that Al assistants must enhance human creativity rather than diminish linguistic diversity or personal style (Chakrabarty et al., 2025).
Despite these valid concerns, we believe the benefits of well-designed, human-centric Al assistants are significant. The goal of MIXASSIST is not to prescribe a single correct path, but to aid the creation of agents that act collaborative partners to scaffold learning, demystify complex concepts, and empower artists (Gabriel et al., 2024). By focusing on collaborative, human-in -aaan---tthe-log thop--oe-opp models, our aim is to develop systems that empower users to develop their unique artistic voice with greater confidence and skill, fostering a co-creative process that values both human ingenuity and Al support.
Acknowledgements
We wish to express our sincere gratitude to the 12 music producers who generously con- tributed their time and expertise. Their participation for the creation of the MIXASSIST dataset. We are also grateful for the foundational work of Brecht De Man and his collaborators on The Mix Evaluation Dataset. Their efforts in making this resource publicly available were essential for research (De Man & Reiss, 2017).
our co-mixing sessions was essential
10

---

Published as a conference paper at COLM 2025
References
Andrea Agostinelli, Timo I Denk, Zalán Borsos, Jesse Engel, Mauro Verzetti, Antoine Caillon, Qingqing Huang, Aren Jansen, Adam Roberts, Marco Tagliasacchi, al. Musiclm: Generating music from text. Xiv preprint ar XfX::23001.11325, 2023.
Barrett R Anderson, Jash Hemant Shah, and Max Kreminski. Homogenization effects of large language models on human creative ideation. In Proceedings of the 16th conference on creativity & cognition, pp. 413-425, 2024.
Satanjeev Banerjee and Alon Lavie. METEOR: An automatic metric for MT evaluation with improved correlation with human judgments.. In Jade Goldstein, Alon Lavie, Chin-Yew Lin, and Clare Voss (eds.), Proceedings of the ACL Workshop on Intrinsic and Extrinsic Evaluation Measures for Machine Translation and/or Summarization, pp. 65-72, Ann Arbor, Michigan, June 2005. Association for Computational Linguistics. URL https://aclantte wwr.aaatnhoooooooo////e--00
Virginia Braun and Victoria Clarke. Toward good practice in thematic analysis: Avoid- ing common problems and be (com) ing a knowing researcher. International journal of fransgender health, 24(1):1-6, 2023.
Nick Bryan-Kinns, Berker Banar, Corey Ford, Courtney N Reed, Yixiao Zhang, and Jack Ar- mitage. Explainable ai and music. In Artificial Intelligence for Art Creation and Understanding, pp. 1-29. CRC Press, 2024.
Tuhin Chakrabarty, Philippe Laban, and Chien-Sheng Wu. Can ai writing be salvaged? mitigating idiosyncrasies and improving human-ai alignment the writing process through edits. In Proceedings of the 2025 CHI Conference on Human Factors in Computing Systems, pp. 1-33, 2025.
Annie Chu, Patrick O'Reilly, Julia Barnett, and Bryan Pardo. Text2fx: Harnessing clap em- beddings for text-guided audio effects. In ICASSP 2025-2025 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 1-5. IEEE, 2025.
Yunfei Chu, Jin Xu, Xiaohuan Zhou, Qian Yang, Shiliang Zhang, Zhijie Yan, Chang Zhou, and Jingren Zhou. Qwen-audio: Advancing universal audio understanding via unified large-scale audio-language models. arXiv preprint ar arXi:2231..07919 2023.
Jacopo de Berardinis, Lorenzo Porcaro, Albert Meroño-Peñueke Angelo Cangelosi, and Tess Buckley. Towards responsible ai music: an investigation of trustworthy features for creative systems. Xiv preprint ar XiX::2500311881 2025.
Brecht De Man and Joshua D Reiss. The mix evaluation dataset. In 20th Int. Conf. on Digital Audio Effects (DAFx-17), 2017.
Emmanuel Deruty, Maarten Grachten, Stefan Lattner, Javier Nistal, and Cyran Aouameur. On the development and practice of ai technology for contemporary popular music production. Transactions of the International Sociefy for Music Information Retrieval, 5(1), 2022.
Soham Deshmukh, Benjamin Elizalde, Rita Singh, and Huaming Wang. Pengi: An audio language model for audio tasks. Advances in Neural Information Processing Systems, 36: 18090018108, 2023.
Soham Deshmukh, Shuo Han, Rita Singh, and Bhiksha Raj. Adiff: Explaining audio difference using natural language. arXit preprint ar arXiv:2502.04476 2025.
Prafulla Dhariwal, Heewoo Jun, Christine Payne, Jong Wook Kim, Alec Radford, and Ilya Sutskever. Jukebox: A generative model for music. Xiv preprint ar arXiv:2005.00341, 2020.
Seungheon Doh, Junghyun Koo, Marco A Martinez-Ramiree Wei-Hsiang Liao, Juhan Nam, and Yuki Mitsufuji. Can large language models predict audio effects parameters from natural language? ar preprint ar arXitv2505.20770, 2025.
11

---

Published as a conference paper at COLM 2025
Konstantinos Drossos, Samuel Lipping, and Tuomas Virtanen. Clotho: An audio captioning dataset. In ICASSP 2020-2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 736-740. IEEE, 2020.
Benjamin Elizalde, Soham Deshmukh, Mahmoud Al Ismail, and Huaming Wang. Clap learning audio concepts from natural language supervision. In ICASSP 2023-2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 1-5. IEEE, 2023.
Nordström Emil. The collaborative music producer: a comparative study of integrated and separated mixing methods. 2024.
Tron Gianet Eric, Luigi Di Caro, Amon Rapp, et al. Redefining the user human-generative ai collaboration: Insights from music composition. In CEUR Workshop Proceedings, volume 3685, pp. 1-6. CEUR-WS, 2024.
Linda Finlay: Negotiating the swamp: the opportunity and challenge of reflexivity in research practice. Qualitative research, 2(2):209-230, 2002.
Eduardo Fonseca, Xavier Favory, Jordi Pons, Frederic Font, and Xavier Serra. Fsd50k: an open dataset haman-labeled sound events. JEEE/ACM Transactions on Audio, Speech, and Language Processing, 30:829-852 852, 2021.
Yue Fu, Michele Newman, Lewis Going, Qiuzi Feng, and Jin Ha Lee. Exploring the collaborative co-creation process with ai: A case study in novice music production. Xiii preprint wwXt::2501.1526, 2025.
lason Gabriel, Arianna Manzini, Geoff Keeling, Lisa Anne Hendricks, Verena Rieser, Hasan Iqbal, Nenad Tomašev, Ira Ktena, Zachary Kenton, Mikel Rodriguez, et al. The ethics of advanced assistants. arviv preprint #xiv:2404.16244 2024.
Josh Gardner, Simon Durand, Daniel Stoller, and Rachel M Bittner. Llark: A multimodal instruction-folloiinng language model for music. Xiv preprint ar XxX:::2310.0716, 2023.
Jort F Gemmeke, Daniel PW Ellis, Dylan Freedman, Aren Jansen, Wade Lawrence, R Chan- ning Moore, Manoj Plakal, and Marvin Ritter. Audio set: An ontology and human-labeled dataset for audio events. In 2017 IEEE international conference acoustics, speech and signal processing (ICASSP), pp. 776-780, IEEE, 2017.
Sreyan Ghosh, Zhifeng Kong, Sonal Kumar, Sakshi, Jaehyeon Kim, Wei Ping, Rafael Valle, Dinesh Manocha, and Bryan Catanzaro. Audio flamingo 2: An audio-language model with long-audio understanding and expert reasoning abilities. arXiv preprint arXi::2503.03983, 2025.
Yuan Gong, Yu-An Chung, and James Glass. Ast: Audio spectrogram transformer. aiXiv preprint ar Xiv::1004.0000 04.01778, 2021.
Yuan Gong, Hongyin Luo, Alexander H Liu, Leonid Karlinsky, and James Glass. Listen, think, and understand. arXiv preprint arXiv:2305.1077 2023.
Jiawei Gu, Xuhui Jiang, Zhichao Shi, Hexiang Tan, Xuehao Zhai, Chengjin Xu, Wei Li, Yinghan Shen, Shengjie Ma, Honghao Liu, et al. A survey on llm-as-a-judge. arXiv preprint arXio:2411.15553 2024.
Gunnar Harboe, Jonas Minke, Ioana Tlea, and Elaine M Huang. Computer support supportt for collaborative data analysis: augmenting paper affinity diagrams. In Proceedings of the ACM 2012 Conference on Computer Supported Cooperative Work, pp. 1179-1182, 2012.
Edward Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, Weizhu Chen, et al. Lora: Low-rank adaptation of large language models. ICLR, 1(2)3, 2022.
Pegah Karimi, Kazjon Grace, Mary Lou Maher, and Nicholas Davis. Evaluating creativity in computational co-creative systems. Xiv preprint ar Xiv: 1807. (0l.01000.9,,6, 2018.
12

---

Published as a conference paper at COLM 2025
Chris Dongjoo Kim, Byeongchang Kim, Hyunmin Lee, and Gunhee Kim. Audiocaps: Generating captions for audios in the wild. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pp. 119-132, 2019.
Junghyun Koo, Seungryeol Paik, and Kyogu Lee. End-to-end music remastering system using self-supervised and adversarial training. In ICASSP 2022-2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp- 4608 4618. IEEE, 2022.
Junghyun Koo, Marco A Martínez- Martinez-Ramfrez, Wei-Hsiang Liao, Stefan Uhlich, Kyogu Lee, and Yuki Mitsufuji. Music mixing style transfer: A contrastive learning approach to disentangle audio effects. In ICASSP 2023-2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 1-5. IEEE, 2023.
Sushant Kumnr, Sumit Datta, Vishakha Singh, Deepanwita Datta, Sanjay Kumar Singh, and Ritesh Sharma. Applications, challenges, and future directions of human-in-the-loop learning. JEEE Access, 2024.
Edith Law, Kris West, Michael Mandel, Mert Bay, and J Stephen Downie. Evaluation of algorithms using games: The case of music tagging. In ISMIR, pp. 387-392. Citeseer, 2009.
Charles Leadbeater and Paul Miller. The pro-am revolution: How enthusiasts are changing our society and economy, 2004.
Sungho Lee, Marco Partinez-Ramirez Wei-Hsiang Liao, Stefan Uhlich, Giorgio Fabbro, Kyogu Lee, and Yuki Mitsufuji. Grafx: an open-source library for audio processing graphs in pytorcc.. arXiv preprint ar arXiv:2408.03224 2024.
Haitao Li, Qian Dong, Junjie Chen, Huixue Su, Yujia Zhou, Qingyao Ai, Ziyi Ye, and Yiqun Liu. Lims-as-judges: a comprehensive survey on llm-based evaluation methods, arv preprint arXiv:2412.05579 2024a.
Yizhi Li, Ruibin Yuan, Ge Zhang, Yinghao Ma, Xingran Chen, Hanzhi Yin, Chenghao Xiao, Chenghua Lin, Anton Ragni, Emmanouil Benetos, et al. Mert: Acoustic music under- standing model with large-scale self-supervised training. ar preprint arXiv:2306.00107, 2023.
Yupei Li, Manuel Milling, Lucia Specia, and Björn W Schuller. From audio deepfake detection to ai-generated music detection-a pathway and overview. arXiv preprint arXiv:2412.00571, 2024b.
Chin-Yew Lin. ROUGE: A package for automatic evaluation of summaries. In Text Summa- rization Branches Out, pp. 74-81, Barcelona, Spain, July 2004. Association for Computa- tional Linguistics. URL https: http///aaaaaattoy..../ wrg/
Samuel Lipping, Parthasaarathy Sudarsanam, Konstantinos Drossos, and Tuomas Virtanen. Clotho-aqa: A crowdsourced dataset for audio question answering. In 2022 30th European Signal Processing Conference (ELISIPCO), pp. 1140-1144. IEEE, 2022.
Shansong Liu, Atin Sakkeer Hussain, Chenshuo Sun, and Ying Shan. Music understanding llama: Advancing text-to-music generation with question answering and captioning. In ICASSP 2024-2024 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 286-290. IEEE, 2024.
Yunyi Liu, Craig Jin, and David Gunawan. Ddsp-sfx: coousticallygguidee sound effects generation with differentiable digital signal processing. arXiv preprint ar arXio:2309.0800 080600, 2023.
Duri Long, Aadarsh Padiyath, Anthony Teachey, and Brian Magerko, The role of collabo- ration, creativity, and embodiment in ai learning experiences. In Proceedings of the 13th Conference on Creativity and Cognition, pp- 1-10, 2021.
Peiling Lu, Xin Xu, Chenfei Kang, Botao Yu, Chengyi Xing, Xu Tan, and Jiang Bian. Musec- oco: Generating symbolic music from text. arXiv preprint ar Xrv::2306.0011 00110, 2023.
13

---

Published as a conference paper at COLM 2025
Ang Lv, Xu Tan, Peiling Lu, Wei Ye, Shikun Zhang, Jiang Bian, and Rui Yan. Getmusic: Generating any music tracks with a unified representation and diffusion framework. arxiv preprint ar arXi:23355.10841 2023.
Ziyang Ma, Zhuo Chen, Yuping Wang, Eng Stong Chng, and Xie Chen. Audio-cot: Exploring Chain-of-though reasoning in large audio language model. ar Xiv preprint ar Xiv::2501..07246 2025.
M Martinez Ramirez, Daniel Stoller, and David Moffat. A deep learning approach to intelligent drum mixing with the wave-u-net. Journal of the Audio Engineering Society, 69 (3):142, 2021.
Marco A Martinez-Ramfres Wei-Hsiang Liao, Giorgio Fabbro, Stefan Uhlich, Chihiro Nagashima, and Yuki Mitsufuji. Automatic music mixing with deep learning and out-of- domain data. ar preprint ar Xiv::220 11428, 2022.
Jon McCormack, Patrick Hutchings, Toby Gifford, Matthew Yee-King, Maria Teresa Llano, and Mark D'inverno. Design considerations for real-time collaboration with creative artificial intelligence. Organised Sound, 25(1):41-52, 2020.
Arthur Miller. The artist in the machine: The world of Al-powered creativity. Mit Press, 2019.
Dave Moffat and Mark Sandler. An automated approach to the application of reverberation. In Audio Engineering Society Convention 147. Audio Engineering Society, 2019a.
David Moffat. AT Music Mixing Systems. In Handbook of Artificial Intelligence for Music, pp- 345-375 Springer International Publishing, 2021.
David Moffat and Mark Sandler. Approaches in intelligent music production. Arts & health, 8(4):125, September 2019b.
Kumiyo Nakakoji. Meanings of tools, support, and uses for creative design processes. In International design research symposium, volume 6, pp. 156-165, 2006.
Steven Nicholls, Stuart Cunningham, and Richard Picking, Collaborative artificial intelli- gence in music production. In Proceedings of the Audio Mostly 2018 on Sound in Immersion and Emotion, pp. 1-4. 2018.
Erlinda C Palaganas and Joel C Estacio. Reflexivity and research methodology: A second glance. Philippine Journal of Nursing, 91(1):87-96, 2021.
Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. Bleu: a method for automatic evaluation of machine translation. In Pierre Isabelle, Eugene Charniak, and Dekang Lin (eds.), Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics, Pp. 311-318, Philadelphia, Pennsylvania, USA, July 2002. Association for Computational Linguistics. doi: 00011//107308.11033111 URL https: /atp::////cllahollogy org/P02-1040/.
Rohit Prabhavalkar, Takaaki Hori, Tara N Sainath, Ralf Schlüter, and Shinji Watanabe. End-to-end speech recognition: A survey. IEEE/ACM Transactions on Audio, Speech, and Language Processing, 32:325-351, 2023.
Alec Radford, Jong Wook Kim, Tao Xu, Greg Brockman, Christine McLeavey, and Ilya Sutskever. Robust speech recognition via large-scale weak supervision. In International conference on machine learning, pp. 28492-2888 28518. PMLRR 2023.
Marco A Martínez Ramírez, Oliver Wang, Paris Smaragdis, and Nicholas J Bryan. Dif- ferentiable signal processing with black-box audio effects. In ICASSP 2021-2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 66-70. IEEE, 2021.
Kalliopi Rigopouli, Dimitrios Kotsifakos, and Yannis Psaromiligkos. Vygotsky's creativity options and ideas in 21st-century technology-enhanced learning design. Education Sciences, 15(2):257, 2025.
14

---

Published as a conference paper at COLM 2025
Ananya B Sai, Akash Kumar Mohankumar, and Mitesh M Khapra. A survey of evaluation metrics used for nlg systems. ACM Computing Surveys (CSUR), 55(2):1-39, 2022.
John W Schaffer and Deron McGee. Knowledge-bnsed programming for music research, vol-
ume 13. AR Editions, Inc., 1997.
Ben Shneiderman. Creativity support tools. Communications of the ACM, 45(10):116-100, 2002.
Jonathan A Smith. Qualitative psychology: A practical guide to research methods. 2024.
Christian J Steinmetz, Jordi Pons, Santiago Pascual, and Joan Serrà. Automatic multitrack mixing with differentiable mixing console of neural audio effects. ICASSP 2021-2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 71-75, IEEE, 2021.
Sruti Subramanian, Katrien De Moor, Markus Fiedler, Kamil Koniuch, and Lucjan Janowski. Towards enhancing ecological validity in user studies: a systematic review of guidelines and implications for qoe research. Quality and User Experience, 8(1):6, 2023.
Changli Tang, Wenyi Yu, Guangzhi Sun, Xianzhao Chen, Tian Tan, Wei Li, Lu Lu, Zejun Ma, and Chao Zhang. Salmonn: Towards generic hearing abilities for large language models. ar arxiv preprint ar Xit:::2310.11 132899 2023.
Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, Aurelien Rodriguez, Armand Joulin, Edouard Grave, and Guillaume Lample. Llama: Open and efficient foundation language models, 2023. URL https://arxiv. http://aaxxv....//bbbb wrw.abs.///....o..oooo........./ httgg/abs/23003 13971.
Augoustinos Tsiros and Alessandro Palladini. Towards human-centric design framework for ai assisted music production. In Proceedings of the International Conference on New Interfaces for Musical Expression, pp. 399-404, 2020.
Maarten Van Someren, Yvonne F Barnard, and Sandberg. The think aloud method: practical approach to modelling cognitive. London: AcademicPress, 11(6), 1994.
Soumya Sai Vanka, Maryam Safi, lean-Baptiste Rolland, and George Fazekas. Adoption of Al Technology in the Music Mixing Workflow: An Investigation. 2023.
Soumya Sai Vanka, Christian Steinmetz, Jean-Baptiste Rolland, Joshua Reiss, and George
Fazekas. Diff-mst: Differentiable mixing style transfer. arXiv preprint arXiv:2407.088. 2024.
Victor Wang, Michael JQ Zhang, and Eunsol Choi. Improving llm-as-a-judge inference with the judgment distribution. arviv preprint ar Xrx::25003. 03064, 2025.
Hui Wei, Shenghua He, Tian Xia, Andy Wong, Jingyang Lin, and Mei Han. Systematic evaluation of llm-as-a-judge in Ilm alignment tasks: Explainable metrics and diverse prompt templates. Xrx preprint ftt::4088110066 2024.
Zhifei Xie, Mingbao Lin, Zihang Liu, Pengcheng Wu, Shuicheng Yan, and Chunyan Miao. Audio-reasoner: Improving reasoning capability in large audio language models. Xiv preprint ar Xri::2503.00311, 2025.
Mingliang Zeng, Xu Tan, Rui Wang, Zeqian Ju, Tao Qin, and Tie-Yan Liu. Musicbert: Sym- bolic music understanding with large-scale pre-training, arXiv preprint arXiv:2106.05630, 2021.
Tianyi Zhang, Varsha Kishore, Felix Wu, Kilian Q Weinberger, and Yoav Artzi. Bertscore: Evaluating text generation with bert. arXiv preprint ar Xie::904.006655, 2019.
Lianmin Zheng, Wei-Lin Chiang, Ying Sheng, Siyuan Zhuang, Zhanghao Wu, Yonghao Zhuang, Zi Lin, Zhuohan Li, Dacheng Li, Eric Xing, et al. Judging llm-as-a-judge with mt-bench and chatbot arena. Advances in Neural Information Processing Systems, 36:46595- 46623, 2023.
15

---

Published as a conference paper at COLM 2025
Jingming Zhuo, Songyang Zhang, Xinyu Fang, Haodong Duan, Dahua Lin, and Kai Chen. Prosa: Assessing and understanding the prompt sensitivity of Ilms. arXio preprint XI1::2410.12405,, 2024.
16

---

Published as a conference paper at COLM 2025
A Extended Related Works
This section provides a more detailed overview of literature related to the topics discussed in Section 2.
Audio-Language Models (ALMs). The field of ALMs has evolved rapidly from models learning joint audio-text representations via contrastive learning (e.g,, CLAP; Elizalde et al., 2023), which excel at zero-shot classification and retrieval but lack generative capa- bilities, towards integrating audio perception with the reasoning and generation power of LLMs (Deshmukh et al., 2023). ALMs released in the past few years including LTU (Gong et al., 2023), Qwen-Audio (Chu et al., 2023), SALMONN (Tang et al., 2023), and Audio Flamingo 2 (Ghosh et al., 2025)- - employ stronger LLMs (e.g, LLaMA-7B; Touvron et al., 2023), stronger audio encoders (e.g., AST; Gong et al., 2021), and instruction-finttuiin datasets (e.g,, OpenAQA-5M; Gong et al., 2023).
Recognizing the unique properties of music, specialized models have also emerged. Mi- crosoft's Muzic project explores Al music understanding and generation, including Mu- sicBERT (Zeng et al., 2021) which uses large-scale pre-training for symbolic music under- standing. While much of Muzic focuses on generation or symbolic data (Lv al., 2023; Lu et 2023), it highlights the importance of music-specific pre- pre-rainingg. Bridging acoustic mu- sic representation with LLMs, MERT (Li et al., 2023) employs sele-suee learning using both acoustic (RVQ-VAE) and musical (CQT) teachers. Building on this, MU-LLaMA (Liu et al., 2024) connects the MERT encoder to LLaMA for music-specific QA and captioning. Addressing similar goals, LLARK (Gardner et al., 2023) specifically targets multimodal instruction-folloinng for music understanding. The model integrates a pre-trained genera- tive music audio encoder (Jukebox (Dhariwal et al., 2020)) with Llama 2, fine-tuning on unified dataset created by augmenting open-source music datasets with estimated musical features (key, tempo, chords, beats) and LLM-generated instruction pairs covering music understanding, captioning, and reasoning. Our work selects and MU-LLaMA for fine-tuning. Qwen offers robust general audio capabilities and poten- tial for multi-audio input scenarios, LTU provides a strong foundation in general audio reasoning, and MU-LLLaMA offers specialized music understanding via MERT features, potentially capturing finer nuances relevant to mixing.
Audio-Instruct-1 LTU,
AI Music Mixing. Al approaches to music mixing have traditionally focused on either mimicking expert knowledge automating specific processes (Moffat & Sandler, 2019b; Moffat, 2021). Khowledge-basse and expert systems attempted to codify mixing rules, while optimization techniques sought ideal parameters based on defined objectives like target loudness or minimal masking (Moffat & Sandler, 2019a). Data-driven methods, leverag- ing machine learning, learn mixing functions from examples, predicting parameters for effects like EQ, compression, or even performing end-to-end mixing for specific stems like drums (Martinez Ramirez et al., 2021; Steinmetz et al., 2021; Martinez-Ramírez Koo et al., 2022; ?). Differentiable Digital Signal Processing has enabled novel approaches where neural networks learn to directly control or emulate audio effects (Liu et al., 2023; Ramírez et al., 2021). This allows for style transfer of effects and mixing styles (Vanka et al., 2024; Koo et al., 2023), or generating parameters from natural language, as seen in Text2FX (Chu et al., 2025). Graph neural networks are also being explored, for instance, to model audio effect chains (Lee et al., 2024). However, most existing AI mixing research applying effects, often lacking integration with conver-
al., 2022;
focuses on predicting parameters sational LLMs. User studies like those by (Vanka et al., 2023) highlight that professionals, in particular, desire assistive and co-creative tools that offer control and explanation, rather than black-box automation. They value tools that can handle tedious tasks but also facili- tate learning and creative exploration. In addition, much of the data used in developing commercial or academic mixing tools remains proprietary.
Text-and-Audio Datasets. Progress in ALMs and intelligent audio tools heavily relies on suitable datasets. For general audio, resources range from classification/ tagging datasets (e.g,, AudioSet (Gemmeke et al., 2017), FSD50K (Fonseca et al., 2021)) and captioning collections (e.g., AudioCaps (Kim et al., 2019), Clotho (Drossos et al., 2020)) to single-
17

---

Published as a conference paper at COLM 2025
turn QA benchmarks (e.g., ClothoAQA (Lipping et al., 2022) ) and large-scale, diverse QA datasets like OpenAQA-5M (Gong et al., 2023). Specific tasks like audio difference explanation also have emerging datasets (e.g,, ADIFF (Deshmukh et al., 2025)). For music, datasets such as MusicCaps (Agostinelli et al., 2023), DagnaTTagATuss data (Gardner et al., 2023) provide text annotations (captions, tags, QA pairs). Datasets like the Mix Evaluation Dataset offer valuable parameters alongside perceptual ratings, obtaining fine-grained parameter settings for individual tracks often requires further effort; our own work extends such data by annotating detailed DAW parameters, as described further in Appendix H.
(Law et al., 2009),
MusicQA (Liu et al., 2024), and LLARK's derived instruction-tunde
B Details of the Survey Gauging Interest in Al-Assisted Music Mixing
An online pilot survey was conducted prior to the main data collection phase to gauge inter- est in Al-assisted music mixing explanation tools, understand user needs and preferences, and recruit participants for the co-creative mixing sessions that generated the MIX ASSIST dataset. The survey was completed by 5 respondents with varying music production experience.
B.1 Survey Questions
The survey included the following questions:
Q1 Which of the following best describes your level of expertise in music production? (Options: Novice Hobbyist, Aspiring Professional, Professional, Expert)
Q4 Have you used any software or tools that include Artificial Intelligence (AI) features in your music production? If so, which ones? (Examples may include automatic mixing/ maattrrin suggestions, intelligent EQ adjustments, sample generation, etc.) you have not used any AT tools in your music production, please leave this section blank. (Options for Other)
Q5 How often do you find yourself struggling with a specific aspect of music mixing (e-g., EQing, compression, etc.)? (Options: Never, Rarely, Occasionally, Often, Always)
Q6 What information sources do you currently use to learn about music mixing tech- niques? (Select all that apply) (Options: YouTube/Onling Tutorials, Books, Music Production Forums, Online Courses, Social Media, Other)
Q7 How satisfied are you with the current methods you use to learn about music mixing? (Scale: Extremely dissatisfied to Extremely satisfied)
Q8 Would you be interested in receiving explanations about music mixing techniques from creative agent (e.g., an Al co-creative assistant)? (Options: Yes, No, Maybe)
Q9 Imagine creative agent that can explain music mixing techniques in real-time as you're working on a song. How helpful do you think such an agent would be? (Scale: Not at all useful to Extremely useful)
Q10 Which type of explanations would you find most helpful from a creative agent? (Select all that apply) (Options: Conceptual Explanations, Personalized Sugges- tions, Comparative lixamples, Step-by-Step Walk-Throughs, Visual Representations, Other)
Q11 Do you learn better from technical explanations, audio examples, or visual demon- strations while learning music production concepts?
Q12 Would you prefer the creative agent's explanations to be delivered in text, audio, visual or a combination of all three?
Q15 What features would you like most to see in a co-creative music mixing Al agent? (Select all that apply) (Options: Alternative perspectives, Targeted sound sculpting, Skill progression, Reference material integration, In-depth explanations, Other)
Compatibility,
A Adaptability, Real-time feedback,
18

---

Published as a conference paper at COLM 2025
Q13 In your words, describe how explanations from creative agent could be helpful for your music mixing process. (Open Text)
Q14 Are there any specific concerns you might have about using a creative agent for explanations during music mixing? (Open Text)
Q16 [Recruitment text for follow-up study with email collection field]
B.2 Survey Results Summary
The initial survey provided valuable insights into producer needs. Key findings from the 5 respondents (S1-55) who provided detailed answers are summarized below.
The respondents included Professionals (2/5), Aspiring Pro- fessionals (2/5), and Hobbyist (1/5). A majority reported struggling with mixing often (2/5) or always (1/5), with the remaining facing occasional challenges (2/5). This suggests mixing remains significant hurdle across different experience levels.
Expertise and Challenges:
Desired Features and Explanations: Top desired features among these 5 respondents included 'In-depth explanations' (3/5), 'Adaptability' (2/5), 'Rea--imme feedback' (3/5), and "Targeted sound sculpting' (4/5). Preferred explanation types spanned 'Personalized Suggestions' (3/5), "Comparative Examples' (3/5), 'Step-by-Step Walk- Dolk--Throughs' (2/5), and "Visual Representations' (2/5). A strong preference (4/5 5) emerged for receiving explanations via a combination of text, audio, and visual modalities.
Area
Summary Finding (N=5)
Experience
Professional, 2 Aspiring Prof., Hobbyist
Challenges
3/5 struggle Often Always, 2/5 Occasionally
Interest
3/5 Yes; 4/5 interested rated helpfulness Very / Extremely High
Top Features
Targeted sculpting (4), In-depth explanations (3), Real-time feed- back (3)
Explanation Pref.
Personalized (3), Comparative (3); Combination modality (4/5)
Table 4: Key Survey Findings Summary
Detailed responses, such as those from P1 and P4 mentioned in Section 3.1, highlighted desires for guidance (e.g,, learning from famous mixes, having someone to bounce ideas off) and specific concerns regarding adaptability and ethics (e.g., scraping techniques).
C Details of the Semi-Structured Interview on Future Design
Following the co-creative mixing sessions described in Section 3.2, semi-structured inter- views were conducted with participants to gain deeper insights into their experiences and perspectives on the potential role of Al in music mixing workflows.
C.1 Semi-Structure Interview Question Guide
How would you describe the ideal workflow when using co-creative music mixing agent?
Would you like to have it be more of a teacher or a creative partner?
- Would you use it to learn standard industry techniques or primarily for sug- gestions or recommendations based on your own musical tastes?
How do you envision the agent providing feedback suggestions during the mixing process?
what ways could co-creative music-mixing agents help overcome common chal- lenges limitations in the music-mixing process?
19

---

Published as a conference paper at COLM 2025
Do you have any concerns about using co-creative music mixing agent in terms of originality or artistic expression? If yes, what are these concerns?
If the co-creative agent produces recommendations that are heavily influenced by specific artists or work in the training data, how do you believe credit and attribution be handled? Should there be a system in place to acknowledge and compensate the original creators? What are your thoughts?
C.2 Thematic Analysis
Following the co-creative mixing experiment, semi-structured interviews were conducted to understand participant perspectives on integrating Al into the music mixing process. We coded the transcripts using thematic analysis (Smith, 2024) and affinity diagrams (Harboe et al., 2012) to identify recurring patterns and group related concepts regarding producers' desired workflows, interaction preferences, and concerns. One of the authors conducted and transcribed all interviews and constructed the first round of affinity diagrams. This author also extracted additional codes, refined emerging themes, and performed iterations until all responses were captured within overarching theme. Recognizing that the analysis was conducted by a single author, reflexive approach was integrated throughout the coding and theme development process to enhance trustworthiness (Palaganas & Estacio, 2021; Finlay, 2002). This involved ongoing critical reflection on potential biases, assumptions, and the researcher's positionality in shaping the interpretation of the data, aligning with recommended practices for rigorous qualitative analysis (Braun & Clarke, 2023).
C.2.1 Preferred Role of AI: Teacher and Technical Assistant
dominant theme emerging from the interviews was the preference for Al agents to function primarily as educational tools technical assistants, rather than fully autonomous creative partners. Participants frequently expressed a desire for AI to explain industry standards, demystify complex processes, and offer guidance grounded in established techniques, particularly when navigating unfamiliar genres or technical challenges. This aligns with research exploring Al as a tool to scaffold learning and augment human capabilities in creative domains (Rigopouli et al., 2025; Long et al., 2021). As E7 stated, the ideal Al would feel more like guide for technical execution:
"I picture more of, yeah, a teacher, right? 'Hey, this is how you do this, or, 'Hey, this needs compressor," but this is how you achieve that." E7
This sentiment was echoed by others who valued explanations of fundamental concepts like compression (A3) or sought confirmation on genre conventions, like A1 who found it helpful when E1 affirmed decisions about drum prominence in rock music. Producers desired Al to clarify the "why" behind techniques or offer standard approaches, acting as an accessible knowledge base. A7 wanted recommendations focused on technical balance rather than creative direction: "...it'd be in the techniques because what I like is my creativity... So it d be nice to know like... there's 's gotta be a balance of high and low." " A7. While some saw potential for co-creation (E4), the emphasis was often the AI supporting the producer's 's existing creative vision or providing technical validation (E5), rather than dictating creative choices.
C.2.2 Ideal Feedback Mechanisms and Interaction
Participants strongly favored feedback mechanisms that were explanatory, visually inte- grated, and controllable. There was a clear preference for understanding why an AT agent suggests certain action, aligning with principles of Explainable Al (XAI) which aim to make Al decision-making transparent to users (Bryan-Kinns et al., 2024). A5 explicitly desired this educational aspect: "..hhinn an Al partner who shows me how to do everything, and then when I have questions, being able to ask it, and it shows me, just like you said, with like a visualization and paragraph..."
Visual feedback integrated directly within the Digital Audio Workstation (DAW) was seen as highly beneficial. Rather than just text, producers wanted the AI agent to highlight
20

---

Published as a conference paper at COLM 2025
specific frequencies an EQ, demonstrate parameter changes visually, point to specific parts of the interface, making the guidance immediately actionable. A7 envisioned this interactive guidance:
"...I'd want like an example box to pop up right around it and show like your computer mouse like dragging, moving it, dragging, moving it. Like this is what would sound good." A7
Control over the interaction was also crucial. Many participants preferred the Al to remain passive unless explicitly asked for help (A2, E2), comparing the ideal interaction to a helpful assistant that doesn't 't intrude unless needed, like Microsoft's ' less intrusive "Clippy" or an agent that gets "out of my way until I do something that is kind of out of the norm." (A5). The ability to turn explanations or suggestions on/ off was also mentioned (A2).
C.2.3 Efficiency, Workflow Automation, and Scaffolding
Producers identified significant potential for Al to enhance workflow efficiency by automat- ing tedious organizational tasks and providing useful starting points. Handling mundane setup processes was seen as a way to preserve creative energy. As E4 noted, "Mixing is so tedious and meticulous that it's kinda draining, and so to be able to speed up that process with the mundane would be really nice." E4. Specific examples included automatically analyzing, classifying, grouping, and color-coding tracks based on instrument type (E1, A6).
Beyond organization, AI was seen as helpful for establishing a baseline mix or providing initial settings, akin to the presets templates found in tools like iZotope Ozone's 's mastering assistant (A1). This scaffolding allows producers to bypass some initial setup and move more quickly creative refinement. A1 articulated this desire for adaptable assistance depending on the task:
"...if you could sort of tell your Al partner how much help you're re wanting with this session or workflow, if you want really help like kind of be doing background balancing and stuff at all times so that you can really just focus on your creative vision." A1
This aligns with the goals of many Creativity Support Tools and AT workflow enhancements designed to handle repetitive tasks, potentially boosting productivity and making the creative process more enjoyable (Shneiderman, 2002; Nakakoji, 2006).
C.2.4 Concerns: Creativity, Originality, and Adaptability
Despite the enthusiasm for Al assistance, producers consistently raised concerns about its potential impact creativity and originality. A major worry was that relying too heavily on AI, especially one biased towards "industry standards," " could lead to homogenization and stifle the unique artistic voice or the serendipitous "happy accidents" that often fuel creative breakthroughs (E2). E3 worried that an Al strictly enforcing standards could hinder creativity, suggesting that "if you have something that's just going to help you achieve industry standard, it might as well just be preset." E3.
There was apprehension about AI learning a user's s style too well, potentially limiting exploration into new genres or sounds. E4 expressed this need for balance: "I do like the idea of adaptive, but the algorithm I think would still need to be somewhat flexible.. if it is too adaptive... then want to do something completely different and that adaptation just doesn't work with it." E4. Similarly, E7 explicitly stated, "I definitely don't want to learn me... Yeah, definitely want to test my assumptions... E7. These concerns mirror broader discussions about Al's role in creative fields - whether it acts as a tool, a collaborator, or a force that could diminish human artistic expression (Miller, 2019). The ideal AI, for many, would need to balance offering guidance with preserving the user't 's agency and potential for creative deviation (A3).
21

---

Published as a conference paper at COLM 2025
C.2.5 Ethics: Attribution, Compensation, and Data Provenance
Ethical considerations surrounding the data used train Al mixing agents were a significant and recurring theme. Producers emphasized the importance of transparency, attribution, and fair compensation for the original creators whose work, techniques, or styles inform the Al's knowledge and recommendations. This reflects growing concerns across AI develop- ment regarding data governance, privacy, and fairness (de Berardinis et al., 2025).
There was a strong consensus that creators whose specific styles or copyrighted material are used, especially if explicitly referenced (e.g, "make it sound like Flume" (A1)), should be acknowledged and likely compensated. A2 suggested royalty- system:
"I feel like people who contribute should get some sort of compensation or royallies off of the program that they made. Similar to album sales, everyone involved gets some sort of piece of AA2
Transparency about data sources was also crucial. E5 valued Al systems that cite their sources, even if imperfectly, to build trust. E2 favored "learn more" button linking suggestions back to original sources or creators. The need for consent from artists to use their work in training data was also highlighted (E1). While acknowledging the complexity, particularly distinguishing learning from copying (A1), participants felt existing frameworks for copyright and intellectual property should be adapted for AI, ensuring human creators remain central and are treated fairly (A1). As A7 stated, using someone else's 's hard-earned style without consent felt wrong: "...to be able to steal, to sound like someone else so much, is, yeah, defintiely don't t like it without consent." "
D LLM as Judge
D.1 Methodology Rationale
Evaluating the quality of outputs from generative language models, particularly for complex and subjective tasks like conversational music mixing advice, poses significant challenges. While human evaluation by domain experts provides high-quality assessments, it often faces limitations in scalability, cost, and consistency (Li et al., 2024a). Consequently, using LLMs to approximate human preferences (Ludee)-)) has emerged as scalable and increasingly standard evaluation framework (Wang et al., 2025; Zheng et al., 2023; Gu et al., 2024). This approach utilizes the advanced reasoning and contextual understand- ing capabilities of LLMs to approximate human preferences and judgments in a scalable, cost-effective, and often consistent manner (Li et al., 2024a). Studies have shown strong agreement between LLM judges and human evaluations across various domains and criteria (Wang et al., 2025; Zheng et al., 2023), although limitations such as potential biases and prompt sensitivity must be carefully managed. Common methodologies for L.MM-as-a-judee include pointwise scoring (evaluating a single output), pairwise comparison (choosing the better of two outputs), and listwise ranking (ordering multiple outputs) (Wang et al., 2025). Pairwise comparison grounds the judgment by directly comparing two items, often leading to better human agreement and calibration. However, fully ranking multiple items requires multiple pairwise comparisons, increasing cost and potentially suffering from intransitivity or position bias effects if not handled carefully (Wang et al., 2025). Listwise ranking, where the judge orders multiple items simultaneously, offers an alternative that provides maximal context to the judge for comparison (Wang et al., 2025). While potentially more cognitively demanding for the judge and susceptible to its own biases, asking for direct rank order can be efficient. For evaluating the nuanced quality of conversational music mixing advice-
from our three fine-tuned models (Qwen, LTU, MU-LLaMA) and in comparing our best model to a zero-shot prompted baseline-we needed a method that forces a clear differentiation. We opted to ask the judge (human or LLM) to produce a full rank order for the model outputs generated for each input prompt. This specific form of listwise ranking was chosen because it compels the judge to consider all responses within the same context, forcing a comparative judgment and yielding a complete preference order in single evaluation step, which aligns with our goal of clearly identifying the best
22

---

Published as a conference paper at COLM 2025
performing model among the candidates. Therefore, to rigorously evaluate the models, we adopted an LLM-as-a-judge strategy using this ranking approach. Our panel of LLM judges included several strong open-source models: gemma3, qwen3, 11ama3. and o3 03-mini. We utilized this framework for two key evaluations: (1) three-way ranking of our primary fine-tuned models (Table 5), and (2) direct comparison between best fine-tuned model and zero-shot prompting baseline to validate our approach (Table 6). To the best of our knowledge, this work represents the first application of the LLM-as-a-judge methodology specifically for evaluating conversational agents in the specialized domain of music mixing,
Table 5: LLM-as-a-Judge ranking results for baseline models. We report the percentage of times each model was ranked #1 by various judges and the average rank (lower better).
Qwen
LTU
MU-LLaMA
Judge
% Ranked #1
Avg. Rank
% Ranked #1
Avg. Rank
% Ranked #1
Avg. Rank
o3-mini
50.4%
1.59
44.4%
1.70
5.2%
2.71
qwen3:8b
50.0%
1.62
40.4%
1.79
9.6%
2.59
:bmma3:4b
45.2%
1.74
39.2%
1.87
15.2%
2.38
1amm3.1188
38.0%
1.89
34.8%
1.91
27.2%
2.20
Table 6: Comparison of Fine-Tuned (FT) vs. Zero-Shot Prompted (Base) Qwen Model.
Qwen_Fi
Qwen_Bass Base
Judge % Preferred
Avg. Rank
% Preferred
Avg. Rank
o3-mini
57.6%
1.42
42.4%
1.58
qwen3:8b
62.4%
1.38
37.6%
1.62
gemma:4b
55.2%
1.45
44.8%
1.57
1lam
53.2%
1.47
46.8%
1.53
D.2 Human Rater Validation
Before conducting the full evaluation, we performed a validation step to gauge the alignment between the LLM judge and human expert preferences within the music mixing domain. We randomly selected 25 prompts from our test set and generated responses from the fine-tuned versions Qwen, LTU, and MU-LLaMA for each. Three music producers were then asked to rank the three responses for each prompt from best (Rank 1) to worst (Rank 3), and were provided the same instructions as the LLM. Author 1 of this paper also ranked these responses in accordance with the instructions.
The prompt used for both human raters and the LLM judge is as follows:
Listing 1: LLM-as-a-Judge Prompt
You are an impartial judge evaluating how closely each response aligns with the way an expert producer would assist an amateur during a co-creative music mixing session.
Your PRIMARY goal to rank three given responses based on the following evaluation criteria:
CRITICAL EVALUATION CRITERIA (in order of importance):
1. TECHNICAL ACCURACY Information should be accurate within music production context
2. HELPFULNESS Addresses the issue at hand with practical, actionable advice or suitable explanation given the users behavior
3. CONVERSATION FLUENCY Response is natural, concise, and decisive
EXAMPLES:
This is an example of explanation with GOOD technical accuracy: "By pulling down the levels in the 400 600 Hz range and adding some
23

---

Published as a conference paper at COLM 2025
air around 1.3k 2k, you're creating room for the guitar to breathe and stand out in the mix."
Using the standard ranges of frequencies for different instruments while suggesting an EQ is helpful to amateurs in understanding both the technical implementation techniques are useful. This explanation provides correct ranges technically as well as providing a reasoning for what adding these elements will do for the mix.
well the reasoning behind why these
This is example of explanation with BAD technical accuracy: "I don understand what you mean by ppan the crash and ride'."
Panning is technique to help increase the stereo image of the mix by moving different instruments around the placement of the stereo field. This explanation does not understand this concept panning, which is foundation in mixing. For this reason, this explanation is bad in terms of technical accuracy.
This is example of explanation with GOOD helpfulness: "To adjust compression, first, locate compressor on your DAW (Digital Audio Workstation). It usually has a icon. You can drag the Wet control to the left or right to make the compression determines how much
Compressor
or Clipping
less pronounced. The wet control signal being compressed."
This explanation describes to the amateur music producer exactly what to do without questioning their creative decisions and provides greater depth of detail for what compressor does such that the amateur can then use this knowledge within their exploration the sonic space.
This is another example of GOOD helpfulness: "I need more information before I can respond. Please elaborate."
In some cases, this can be viewed as helpful such that the expert is asking clarifying questions opposed to assuming what the amateur meant and providing
for something that may not be correct.
This is an example of an explanation with BAD helpfulness: "You can 't just say, I want really low reverb, because you can t just turn it off."
This is another example of BAD helpfulness: "It sounds like you are trying to make the drum loop tighter and more focused, while also reducing its energy or intensity. This could be achieved by adjusting the tempo, adding effects such compression EQ, using different drum sample with different sound."
While this explanation may appear helpful on the surface, as opposed to asking a follow-up question before providing an answer, this explanation assumes what the means instead of providing clarifying question to resolve the discrepancy before continuing.
This kind of explanation belittles the user and undermines their learning potential throughout this process of mixing. This explanation does not help the user understand what may be wrong with their explanation of the reverb and instead does not encourage learning.
This is example of explanation with GOOD naturalness: "You 're gonna need a little bit of compression on that."
Slang terms such as gonna, gotta, sick, fire, or many other terms help promote sense of naturalness within the conversation overall.
24

---

Published as a conference paper at COLM 2025
These terms and other music production jargon may provide of ease and naturalness for the amateur who is receiving instructions. This wording helps provide a sense of naturalness throughout the conversation flow.
This is example of explanation with BAD naturalness: "I understand, thank you for letting me know. Is there anything else can help with? If not, have great day and let me know if you need any assistance in the future."
This explanation halts the communication between the expert and the amateur entirely and may also deter the amateur from asking follow-up questions due to the cold way in which the explanation is stated.
This is an example of an explanation with GOOD conciseness: "Yes, a compressor could help in controlling the dynamic range and making sure the volume levels are even across different parts of the song."
This explanation provides accurate information without being verbose and encourages follow-up questions from the amateur. Specifics such as a compressor and what it does are addressed well without being overly wordy.
This is an example of an explanation with BAD conciseness: "Yes, think they could a bit quieter to better balance with the guitar and vocals in the mix. It might also help to adjust the levels of each instrument to create more clarity and separation between them. Additionally, experimenting with different drum samples effects can depth and complexity to the overall sound."
This explanation wanders around the main argument and keeps adding sentences after initially providing suggestion at the beginning. The addition of these sentences might distract or confuse the amateur with too many additions to the central concept.
This is example of explanation with GOOD decisiveness: "Yeah, would put the high pass guitar three then." "
This explanation is succinct and provides clear and actionable path forward for the amateur when implementing this suggestion.
This is an example of an explanation with BAD decisiveness: "That good question. I think it's more of matter of what you can with the tube screamer.""
This explanation does not provide the amateur with for why something was selected and instead provides a generic response overall and turns the onus back toward the amateur for what they can do with the effect.
of certainty
You MUST rank these explanations, even if the difference is slight.
Output ONLY in JSON format: ('rank_1's (A, c), 'rank_2's (A,
C}; 'rank33's (A, B, or C}); with explanation.
Following this instruction block, the original user prompt and the three model responses (labeled A, B, C in randomized order) were provided to the raters/judge.
25

---

Published as a conference paper at COLM 2025
The aggregated results from the four human raters are summarized in Table 7. Qwen clearly preferred, receiving the most Rank votes, followed by LTU, and then MU-LLaMA.
Model
Rank Count
Rank Count
Rank Count
Qwen LTU
52
30
18
29
39
32
MU-LLaMA
19
31
50
Table 7: Aggregated Human Producer Rankings (25 Samples)
Nofe: Counts aggregated across 4 human raters for 25 prompts (100 total rankings per rank).
D.3 LLM Judge Validation
We then used the 03-mini LLM judge to evaluate the same 25 sets of responses using the identical prompt structure shown above, requesting 1st, 2nd, and 3rd place ranking in JSON format. As established in prior work, careful prompt design is crucial for effective LLM-as-a-judge evaluation, as performance can be highly sensitive to the phrasing, structure, clarity of criteria, and included examples (Zhuo et al., 2024; Gu et al., 2024; Wei et al., 2024). The prompt used in this study was iteratively refined to clearly define the evaluation criteria piiiiia--riiiiiinngggg technical accuracy, helpfulness, and conversational fluency for the music mixing co-creative context-and provide illustrative positive and negative examples, aiming to mitigate ambiguity and improve judge consistency (Gu et al., 2024; Li et al., 2024a).
The LLM judge's rankings for these 25 samples largely mirrored the human preferences, as shown in Table 8, Qwen was ranked first most often, further validating the LLM judge's capability reflect expert preferences in this domain for our task using the refined prompt.
Model
Rank Count
Rank 2 Count
Rank 3 Count
Qwen LTU
15
7
3
9
13
3
MU-LLaMA
1
5
19
Table 8: LLM Judge (o3-mini) Rankings (25 Samples)
D.4 Full LLM Judge Evaluation Results
Confident in the LLM judge's alignment for this task, we proceeded with a full evaluation across 250 test samples derived from the MIXASSIST dataset interactions. The o3 o3-mini judge produced 1st, 2nd, and 3rd place ranking for the three models (Qwen, LTU, MU-LLaMA) for every sample prompt. The aggregated performance metrics derived from these rankings are presented in Table 2.
The results indicate that the fine-tuned Qwen model consistently outperformed the other models, achieving the highest number of Rank 1 placements and the best average rank. LTU performed competitively, securing the second position, while MU-LLaMA significantly lagged behind, being ranked last in 76.0% of the samples.
D.5 Automated Text Generation Metrics
While human-centric evaluations (LLM-as-a-judge, user studies) were prioritized for as- sessing the quality of conversational mixing advice, we also computed standard automated text generation metrics comparing the fine-tuned models' outputs against the ground-truth human expert responses in the test set. These metrics include BLEU (Papineni et al., 2002), METEOR (Banerjee & Lavie, 2005), ROUGE-L (Lin, 2004), and BERTScore (Zhang et al., 2019). The results are presented in Table 9. It is worth noting that these metrics, particularly
26

---

Published as a conference paper at COLM 2025
those based on n-gram overlap, often show limited correlation with human judgments of quality for dialogue and instructional tasks.
Model
BLEU
BLEU-1
BLEU-2
BLEU-3
BLEU-4
METEOR
ROUGE-L
BERT-S
Qwen
0.0631
0.1274
0.0556
0.0363
0.0333
0.1504
0.1071
0.8482
LTU
0.0240
0.0820
0.0120
0.0021
0.0000
0.1015
0.0895
0.8349
MU-LLaMA
0.0274
0.0911
0.0157
0.0022
0.0004
0.0932
0.0830
0.8503
Table 9: Comparison of fine-tuned models using automated text generation metrics against human references.
D.6 Topic-Specific Performance
We further analyzed the LLM judge's preferences based the primary topic of the user's prompt (e.g,, focusing on drums, vocals, overall mix). Table 10 shows the percentage of times each model was ranked #1 for different topics, indicating potential strengths or weaknesses of the models concerning specific musical elements.
Topic
# Samples
Qwen (% Best)
LTU (% Best)
MU-LLaMA (% Best)
Overall Mix
53 (21.2%)
66.0%
28.3%
5.7%
Drums
101 (40.4%)
39.6%
54.5%
5.9%
Bass
19 (7 (7.6%)
57.9%
36.8%
5.3%
Guitars
45 (18.0%)
48.9%
44.4%
6.7%
Vocals
28 (11 11.2%)
57.1%
42.9%
0.0%
Keys
4 (1.6%)
50.0%
50.0%
0.0%
Table 10: LLM Judge (o3-mini) Rank #1 Percentage by Topic (250 Samples)
While Qwen performed best across most topics, including the 'Overall Mix' category, LTU showed notable strength in prompts specifically concerning drums. This suggests LTU might have captured nuances related to drum mixing particularly well during fine-tuning, However, Qwen's strong performance in general mix advice and other specific instruments solidified its position as the top-performing model overall for music mixing tasks.
Based on both human expert validation and LLM-as-a-judge evaluation using ranking, Qwen demonstrated a higher performance in generating relevant, accurate, and helpful conversational mixing advice compared LTU and MU-LLaMA when fine-tuned on the ASXISS dataset. Therefore, Qwen was selected as the base model for subsequent experiments and system development in our work.
E
Human Evaluation Study Details
E.1 Methodology
To complement the automated LLM-as-a-judge evaluation, we conducted a human eval- uation study focusing the perceived quality of the best-performing model's (Qwen) generated responses compared directly against the ground-truth human expert responses from the MIXASSIST dataset. This comparative evaluation was performed to help under- stand user preferences and the practical utility the Al assistant's responses in a music mixing context.
We randomly selected 100 prompts from the test set. For each prompt, we paired the ground-truth human expert response with the response generated by the fine-tuned Qwen model. These pairs were presented to participants in randomized order, with the human and AT responses randomly assigned to RRssponse A' or 'Response B' to mitigate order bias.
27

---

Published as a conference paper at COLM 2025
A total of 10 music producers were recruited for the study. Participants were compensated with a $5 Amazon Gift Card for their time. Each participant completed 10 comparison rounds. In each round, they were presented with the conversational prompt, the audio segment associated with the query, and the two responses (A and B).
Participants were asked to evaluate the responses based which they believed to be the best answer given the prompt and context, selecting one of four options for each pair: "Response A is better," "Response is better," "Both are good," " or "Both are bad."
E.2 Discussion of Preference Results
The aggregate results across 100 comparisons (10 participants evaluating 10 utterances each) are shown in Table 3. The generated Qwen responses were preferred 40% of the time, while the original human expert responses were preferred 33% of the time. In 12% of cases, participants found both responses good, and in 15%, they found both bad.
Expertise
N
Generated
Human
Both Good
Both Bad
Amateur
2
Intermediate
7
29
24
4
13
Expert
2
5
6
Table 11: Human Preference by Expertise (Counts)
The aggregate results presented in Table 3 indicate a slight preference among human evaluators for the generated responses from the fine-tuned Qwen model (40.0%) compared to the original human expert responses (33.0%). This finding, while based on a limited sample of 100 comparisons, warrants closer examination as to why generated responses were rated slightly higher.
A qualitative review of the individual comparison instances where generated responses were preferred suggests potential reasons for this trend:
Greater Detail and Explanation: some cases, the generated response provided more comprehensive background step-by-step guidance. For instance, when one user expressed general confusion about compression (Item ID 41), the generated response explained the concept ("Compression is technique used... It makes quiet parts louder...) and gave starting ratio suggestions, whereas the human response pivoted specific EQ advice for vocals. Similarly, in Item ID 21, the generated response explained the rationale for general drum compression, while the human response abruptly changed topic to vocals.
Structured Direct Responses: Generated responses sometimes offered more structured or directly relevant answers. In Item ID 45, the generated response recapped previous steps ("The first thing we did was adjust the panning..."" ") before asking for clarification, perceived as more helpful than the human's 's slightly vague instruction. In Item ID 74, faced with a question about separate drum reverb, the generated response was direct "I think we could do separate drum reverb," while the human expert provided a lengthy, multi-faceted anecdote about personal reverb techniques.
Consistent Focus: Generated responses generally addressed the immediate prompt directly, potentially being perceived as more focused than human responses that might anticipate future steps or change topic.
Conversely, the human expert responses were often preferred when they demonstrated strengths potentially harder for the current models to replicate:
Leveraging Implicit Context: Humans could often interpret ambiguous or under- specified user input. In Item ID 47, the user asked "What are you thinking, 0.5?", which the human expert implicitly understood and confirmed ("Yeah, I think that's
28

---

Published as a conference paper at COLM 2025
about good."), while the generated response asked for clarification ("I don't under- stand what you mean by "0.5"").
Conversational Fluidity and Naturalness: Human responses sometimes involved quick, natural affirmations or suggestions that felt more fluid. For example, in Item 1D 77, when the user described "rough" panning approach, the human expert responded encouragingly with "Yeah, just get real random with it," while the generated response was a simple "Okay, that sounds fine."
Actionable Guidance over Generic Analysis: In cases where the user input was just "Please analyze this audio segment" (Item ID 37, 59), the human expert provided immediate, actionable mixing suggestions ("All right, yeah, bring that way down... solo different things..."). "), whereas the generated response offered a generic, factual analysis of the audio's 's musical features ("This audio contains fast-paced rock piece...").
It is plausible that the structured, detailed, and consistently on-topic nature of the generated responses was perceived as more helpful in certain evaluation contexts, particularly when users sought explicit explanation or guidance. However, the preference is not overwhelming, and the strengths of human interaction conteetuul understanding, fluidity, and providing immediately relevant remain evident.
These observations should be considered preliminary due to the sample size (N=100 com- parisons across 10 evaluators). Further analysis, potentially involving qualitative coding of responses or larger-scale evaluations, would be needed to draw more definitive conclusions about the factors driving user preference in this co-creative, instructional context.
Real-Time Interaction Study Details
F1 Rationale and Methodology
While offline evaluations like LLM-as-a-judge (Appendix D) and pairwise human preference studies (Appendíx E) provide valuable insights into the quality of generated responses in isolation, they do not fully capture the user experience of interacting with the Al agent dynamically within a task context. Research HCI and AI evaluation emphasizes the im- portance of studying systems in settings that reflect real-world usage to achieve ecological validity (Subramanian et al., 2023; Kumar et al., 2024). Evaluating co-creative systems, in par- ticular, benefits from observing the interaction patterns, user satisfaction, and collaborative dynamics as they unfold (Karimi et al., 2018).
This study aimed to assess user experience and perceived utility when interacting with the co-creative mixing agent in real time. We deployed the fine-tuned Qwen model via a chat interface built HuggingFar Spaces, which allowed users to upload audio file and engage in a multi-turn conversation with the agent.
10 music producers (2 expert, 5 intermediate, 3 amateur) participants were recruited. Partic- ipants were compensated with a $2.50 Amazon Gift Card for their time. Each producer was provided with link to the HuggingFaac Space. They were instructed interact with the agent by uploading their audio of choice and using the chat interface to get recommenda- tions and guidance on mixing the track. Participants were asked to continue the interaction iteratively until they felt satisfied with the agent's 's recommendations or until approximately minutes had passed.
Immediately following the interaction, participants completed survey designed to capture their subjective experience regarding conversational quality, creative contribution, and overall usability.
E2 Survey Questions
The post-interaction survey included the following questions.
29

---

Published as a conference paper at COLM 2025
Q1: What is your level of music production expertise? (e.g., Amateur, Intermediate, Expert)
Q2: How natural did the conversation with the agent feel? (e.g., on scale of 1-4, where 1=Very Unnatural, 4=Very Natural)
Q3: How much did the agent contribute to the creative process, compared to your own input? (e.g,, on scale of 1-4, where 1-Significantly less than my own input, =Signifficantly more than my own input)
Q4: Did the agent suggest musical ideas that you wouldn't have thought of yourself? (e.g,, on a scale of 1-4, where 1=Not at all, 4=Definitely)
Q5: Are there any other insights you noticed during your interaction with the agent? (Open-ended text response)
E3
Quantitative Survey Results
Participants rated their experience several Likert scales (1-4). The aggregated results (N=10) are summarized below:
Response
Count
%
Response
Count
%
Very natural
40.0
Definitely
0.0
Somewhat natural
3
30.0
Somewhat
5
50.0
Somewhat unnatural
3
30.0
A little
20.0
Very unnatural
0
0.0
Not at all
3
30.0
(a) Conversation Naturalness (Q2)
(b) Novelty of Ideas (Q4)
Table 12: Real-Time Interaction Survey Results (N=10): Conversation Naturalness and
Novelty of Ideas.
Response (Agent vs. Own Input)
Count
Percentage (%)
Significantly more
0
0.0%
Slightly more Slightly less
40.0%
5
50.0%
Significantly less
10.0%
Table 13: Real-Time Interaction Survey Results: Agent Contribution (Q9)
Overall, the quantitative feedback suggests users found the agent reasonably conversational and capable of contributing some novel ideas, but perceived its creative contribution as generally less than their own input.
E4 Quantitative Analysis of Audio Understanding Guidance
The qualitative feedback from the real-time study highlighted limitations in the model's ability to provide guidance based on audio context. To explore this further and to better quantify the impact of fine-tuning on MIXASSIST, we conducted manual analysis of generated responses. One of the authors developed a coding scheme and labeled 100 generated conversations from our fine-tuned Qwen model and 100 from the base Qwen model (without fine-tuning).
This analysis revealed that fine-tuning MIXAsSIST significantly enhances the model's ability to provide substantive, task-relevant guidance. Our fine-tuned model identified specific audio issue in 36% of responses and proposed concrete, actionable solution in 41 % of responses. In contrast, the base model identified specific audio issue in only 26% of responses and proposed actionable solution in just 22% of responses.
30

---

Published as a conference paper at COLM 2025
Additionally, targeted analysis was conducted to evaluate the correctness of the guid- ancethaat is, whether the model's suggestion aligned with expert expectations for the given context. This evaluation found that the fine-tuned Qwen model proposed correct actionable guidance 35% of instances. The base Qwen model, without fine-tuning, proposed correct actionable guidance in only 14% of responses.
This direct comparison confirms that while the broader challenge of deep audio understand- ing and a model's ability to truly act a teacher remains unsolved (Xie et al., 2025; Gong et al., 2023; Ma et al., 2025), fine-tuning on MIXAssIE demonstrably improves a model's capacity engage with the music mixing task in meaningful, relevant, and correct way.
E5 Qualitative Feedback Analysis (Q11)
The open-ended question, "Are there any other insights you noticed during your interaction with the agent?", elicited responses from the 10 participants that were thematically analyzed. The following key themes emerged.
F.5.1 Audio Analysis Capability (Mixed/Limited)
Throughout this section, we use Participant IDs (e-g,, Participantl (P1)) to refer to the producers who are quoted. The number following the letter refers the group in which the completed the experiment. A core theme involved the agent's handling of audio. Several participants highlighted limitations or inconsistencies. P5 explicitly noted the agent's inability to gain insight from the track:
"It would have been more helpful had it been able to analyze the audio. It said that it didn't have any insight into the track itself." P5 (Intermediate)
P1 described varying levels of analysis depth, sometimes feeling superficial:
"...the agent wasn't always super clear about how it was analyzing the audio... sometimes it felt more like the agent was reading a description of the audio off a wiki page..." P1 (Intermediate)
Specific failures included misidentifying chord progressions (P6) or struggling with tempo detection (P7). Conversely, P9 experienced moment of insight:
"I noticed the agent quickdy identified my tendency to over boost the hass and gently suggested a more balanced EQ curve." " PP (Expert)
F55.2 Conversational Interaction (Mostly Positive)
The conversational aspect was generally well-received. P6 found it "very natural", P2 found it a "pleasure to use", and P1 appreciated its ability to adapt to non-technical language:
"I especially liked how when used non-specific, non-technical terms to describe a technical problem, the agent matched me by using those same terms in its responses." " P1 (Intermediate)
However, some negative experiences occurred, such as P5 feeling responses were "sort of canned" P4 noting occasional "confusion", and P7 encountering irrelevant suggestions:
"Strange responses as the conversation went on. Such as telling me go for a walk." " P7 (Amateur)
F.5.3 Quality and Utility of Suggestions
The advice provided was often considered useful, particularly for less experienced users (P6) as starting point (P3). The agent demonstrated adaptability in some cases:
31

---

Published as a conference paper at COLM 2025
"It adapted to my preference for a punchy drum sound, offering tailored compression settings." -P10 (Amateur)
Yet, there was a desire for greater depth or creativity. P5 wanted "more precise or little more out-of-the-box" ideas, and P3 expected different technical advice. A potential technical inaccuracy was also noted by P2 regarding reverb parameters.
F5.44 Minor Quirks/Bugs
Small technical glitches were observed, such as P2 finding a "stray non-English character" and P4 needing to repeat questions when the agent got confused.
F555 Desire for Future Capabilities
Participants suggested some possible future directions. P1 wished for assistance in learning terminology:
"I would love if in the future, it could also help me identify technical terms for problems can't t quite identify by name." " P1 (Intermediate)
P8 expressed interest generative music capabilities:
"It would' ve been dope to see how it generates beats and suggests melodies." " PS (Amateur)
The qualitative feedback from the 10 participants indicates that while the agent shows promise in conversational interaction and providing baseline guidance, significant im- provements are needed in its audio analysis capabilities and the depth/creativity of its suggestions.
G MIXASSIST Dataset Details
This section provides a detailed description of the MIXAsSIST dataset, including its con- struction, processing, statistical properties, and analysis. MIXASSIST designed facilitate research into conversational, co-creative AI for music mixing instruction by capturing audio-grounded, multi-turn dialogue between expert and amateur music producers.
G.1 Dataset Construction
The foundation of MIXASSIST lies in seven hour-long co-creative mixing sessions conducted with 12 music producers (7 experts, 7 amateurs, forming 7 distinct pairs). Participants were recruited following initial survey gauging interest in AT mixing assistance (see Appendix B). Participants were compensated for their time in the co-creative mixing sessions, with experts receiving $36 Amazon Gift Card and amateurs receiving $18 Amazon Gift Card. This study was IRB approved. During the sessions, amateurs used their preferred Digital Audio Workstation (DAW) to mix multitrack audio recordings provided for the study.
These multitracks were sourced from The Mix Evaluation Dataset (De Man & Reiss, 2017), publicly available collection designed specifically for research music mixing perception and practices (De Man & Reiss, 2017). This dataset was chosen because it aggregates multitrack sessions from various established sources, including professional recordings released under Creative Commons licenses (allowing reuse for research) and tracks provided by educational resources like Weathervane Music's "Shaking Through" series and Mike Senior's 's "Mixing Secrets" Multitrack Library. This compilation offers valuable mix of genres (e.g,, rock, pop, funk, country, soul) and recording contexts, providing ecologically valid material essential for the mixing task within our study. Table 15 details the specific song, artist, genre, and expert/ amateur participant pair for each of the seven sessions conducted.
32

---

Published as a conference paper at COLM 2025
Session ID (Genre)
Relevant Expert Turns
1 (ROCK)
50
2 (POP ROCK)
93
3 (FUNK)
55
(COUNTRY)
57
(HARD ROCK)
29
(SOUL)
45
7 (INDIE ROCK)
102
Table 14: Number of Relevant Expert Turns per Mixing Session
Note: Genres correspond to Table 15. Session IDs correspond to the expert/amateur pair number (egg,, Session ID 1 involves E1/A1).
Song Title
Artist
Genre
Expert (E)
Amateur (A)
Good Time
Louis Cressy Band Filthybird
ROCK
E1
A1
I'd Like to Know In the Meantime
POP ROCK
E2
A2
Fredy V
FUNK
E3
A3
Lead Me
The Done Fors
COUNTRY
E4
A4
Lolita
Purling Hiss
HARD ROCK
E5
A5
Not Alone
Fredy V
SOUL
E6
A6
Old Tree
Creepoid
INDIE ROCK
A7
Table 15: Details of the Seven Co-Creative Mixing Sessions
Note: Songs and artist information sourced from The Mix Evaluation Dataset (De Man Reiss, 2017), Participant IDs (e.g., E1, A1) correspond to the expert and amateur producers in each session.
During the sessions, amateurs employed a think-aloud (Van Someren et al., 1994) protocol while receiving guidance from the expert upon request. This setup aimed to capture realistic pedagogical interactions grounded in the audio context. Direct parameter logging was avoided to maintain ecological validity and allow participants their preferred DAW; nuances related to parameters are captured within the dialogue itself.
The approximately hours of session recordings underwent several processing steps to create the final dataset:
Transcription: Initial dialogue transcription was performed using Whisper (Radford et al., 2023).
Manual Cleaning & Correction: One author manually reviewed and corrected the transcripts. Filler words ("uh", "um") were removed, while conversational acknowledgmeents ("mm-hmm") were retained. Ellipses (...) were used to indicate pauses or sentence fragments characteristic of spontaneous speech. Utterances re- lated solely to experimental timing, researcher questions, or specific DAW interface elements (not mixing practice) were removed.
PII Filtering: During the manual cleaning phase, specific Personally Identifiable Information (PII) mentioned incidentally during the conversations, such partic- ipant names (beyond assigned IDs), specific school or work affiliations, or other unique identifying details, were removed redacted to protect participant privacy.
Speaker Segmentation:
Transcripts were split into amateur and expert utterances.
Relevant music-only audio segments corresponding to the discussion were extracted. This involved capturing the audio played back from the DAW before a conversation turn initiated or the relevant audio context for the dialogue. Care was taken to align audio semantically with the dialogue, though this process has inherent subjectivity. Voices were removed from these segments to ensure only music remained. The average duration of these audio segments is 19.44
Audio Segmentation:
33

---

Published as a conference paper at COLM 2025
seconds indicating the average length of audio playback prior to amateur asking a question.
Placeholder Augmentation: To handle conversational deviations (e.g., unsolicited expert advice, unanswered amateur questions), standardized placeholders were inserted. "Please analyze this audio segment" was added for the amateur when the expert provided unsolicited feedback on audio. "I need more information before I can respond. Please elaborate." was added for the expert if an amateur's 's question received no response. This maintained a consistent question-answer structure suitable for model training.
Topic Segmentation: Conversations were segmented into meaningful sub- conversations based on mixing focus (e.g. overall mix, drums, guitars, keys, vocals). This was achieved using instrument mentions, context, and conversation history, requiring topic switches to span more than one turn and semantically relevant.
Data Filtering for Content: To ensure the dataset focused on pedagogically useful interactions for training an assistive agent, a filtering stage was applied based on expert response content. A binary metadata tag, 'has content', was initially added to expert utterances to indicate if the response was relevant, addressed the amateur's 's query, and provided actionable mixing guidance or explanation. This labeling, while inherently subjective, was performed by one author utilizing reflexive practices to maintain consistency (Palaganas & Estacio, 2021; Finlay, 2002). Utterances consist- ing only of simple affirmations (e.g., "yeah", " "cool") or non-instructive filler were deemed detrimental for training helpful agent and marked as 'has content'=False The conversational pairs where the expert response was marked 'has content'TTru constitute the final MIXASSIST dataset (N=431), representing 67.34% of the original expert turns. All statistics and analyses, unless otherwise noted, pertain to this filtered dataset.
Contextual Augmentation (Topic Boundaries): To improve continuity between topic segments, opt-40-mini-2024-00-2 was used to generate contextual transi- tions within the first amateur utterance following topic change. This synthesized historical context from previous turns while aiming to maintain the amateur's 's voice, bridging potential gaps created by segmentation. (See Subsection G.3 for the prompt structure). These synthetic augmentations are primarily for model training coherence and are excluded from linguistic analyses reported elsewhere throughout this section.
The complete, unprocessed audio recordings containing both dialogue and DAW playback are also provided to support further research.
G.2 Experiment Participant Instructions
This section details the guidelines provided to the expert and amateur participants, outlining their roles and the mixing task for the co-creative sessions described in Appendix G.
G.2.1 Instructions for Amateur Participants
Task Overview Your task to create a complete mix of a song from raw multitrack audio files. You will work alongside more experienced music producer who acts as a mentor. However, they will only offer assistance if you directly request it. While the goal is to achieve a final mix aligned with a specific target genre using your own style, the primary objective is learn and improve your mixing skills through this interaction.
Materials Provided
Raw multitrack audio files (via Zoom).
A list of allowed audio effects (Gain, Pan, Equalization, Reverb, Compression, Gate, Limiter, Delay, Phaser, Flanger, Chorus). You may use some or all of these.
Your preferred Digital Audio Workstation (DAW) software (e.g., Reaper, Logic, Ableton) is required.
34

---

Published as a conference paper at COLM 2025
Session Procedure
1. Import and Setup: note the target genre for the mix (to be specified).
Open your DAW, import the provided raw audio tracks, and
2. Screen Sharing: Initiate screen sharing of your DAW window the researcher and expert can view your workspace.
3. Mixing Process: Begin mixing the tracks. You have full creative control over your approach.
4. Think Aloud Protocol: decisions. Specifically explain:
While mixing, continuously verbalize your thoughts and
Why you are making adjustments (e.g., "The snare feels lost, so I'm increasing its gain.").
Which parameters you are changing (e.g., " "Adjusting EQ on the bass to boost lows.")
Why you chose specific values (e.g,, "Setting compressor attack to 5ms let transients through.").
5. Requesting Assistance: If you encounter challenges need guidance, strongly encourage you actively ask the experienced producer for help, clarification, or their opinion. Engage in follow-up discussion until you reach understanding.
Important Considerations
The expert acts as an assistor/menttr and provides help only upon your request.
Experimentation with different mixing techniques is encouraged.
Focus on the primary goal: learning and improving mixing skills by leveraging the expert's guidance.
The session duration approximately 75 minutes.
G.2.2
Instructions for Expert Participants
Role Overview Your role is to guide and support less experienced music producer during their mixing task. Act as a mentor, offering expertise and insights while empowering them to lead the process and make decisions.
Initial Setup Checks
View Screen Share: Ensure you can clearly see the amateur participant's shared DAW screen.
Confirm Raw Tracks: Verify the DAW session contains only the raw audio files, without prior edits, to ensure fresh start.
Note Target Genre: Be aware of the specific target genre for the mix.
Mentoring Guidelines
Encouragement: Offer help and support as the amateur sets up their DAW session.
Active Listening: Pay close attention to the amateur's 's actions within the DAW and their verbalized thought process (think-aloud).
Responding Questions: Provide detailed, actionable advice when asked. Be specific about techniques, effects, and parameter settings/vai values where applicable.
For Specific Questions:
Example: Instead of just "Try compressor," suggest: "Consider adding a compressor on the snare with a 4:1 ratio and 20ms attack. This helps control dynamics for consistency." Aim for specificity.
35

---

Published as a conference paper at COLM 2025
For General Unnerrtainty: Offer guiding suggestions but encourage the amateur's ss choice.
Example: Instead of "Start with drums," " suggest: "Often, balancing drums and bass first builds solid foundation, but feel free explore other starting points. I'm here to help when you need guidance."
Explaining Rationale: Always explain the reasoning behind your recommendations to facilitate learning-
Example: Instead of "Use reverb on pop vocals," " explain: "In pop, reverb on vocals adds space. small room reverb with short decay can achieve this without cluttering the mix."
Positive Reinforcement: Maintain a supportive and encouraging tone. Acknowledge suc- cesses and frame mistakes learning opportunities.
Important Considerations
The primary goal is to facilitate the amateur's learning and growth, not necessarily to achieve a flawless mix.
Approach the interaction with patience, understanding, and flexibility.
Foster collaborative and positive learning environment.
The session duration approximately 75 minutes.
G.3 Context Generation Prompt Structure
To generate the introductory context sentence(s) for amateur utterances topic bound- aries (as described in the Contextual Augmentation step above), the following Python f-string based prompt structure was used with GPT-4o. The variables ("current.fopic" "all.previous.tums"" fuurrentfirst.tuur were populated dynamically for each transition point.
prompt You the amateur music producer continuing conversation.
The current topic is (current__topic).
Previous conversations content (most recent 10 turns): [json. dumps (all_
indent=2)]
Your next message will be: "[current_first_turn)
Before that m briefly summarize what previously discussed that led this point.
Write 1-2 sentences in a casual, first-person style as if you 're
+ the amateur producer recapping
what you were just talking about. Focus technical details and made.
decisions that The summary should flow naturally into your next message. wnn
Listing Python f-string prompt structure for context generation.
G.4 Dataset Statistics and Splits
The resulting AIXISSIST dataset comprises 431 substantive, audio-grounded expert re- sponses derived from original mixing sessions. Expert utterances average 36.57 tokens, while the preceding amateur utterances average 25.39 tokens (based on all original amateur turns). The dataset was partitioned into training, development, and test sets using the topic-based sub-conversations (of which there are 77 containing substantive content) as units, rather than splitting individual turns, to preserve conversational integrity and evalu- ate generalization across interaction dynamics. The splitting strategy aimed to rigorously
36

---

Published as a conference paper at COLM 2025
evaluate model generalization across different interaction styles and musical contexts (see Table 14 for session turn counts). Specifically, the complete sub-conversations from the shortest (Session ID 5, Hard Rock) and the second longest (Session ID 2, Pop Rock) overall sessions were allocated exclusively to the test set. This preserves the longest session (ID 7, Indie Rock) alloowig its potentially extended dialogues and varied sub-conversations to be represented across all splits while holding out the distinct genres of Pop Rock and Hard Rock provides diverse test cases against the remaining pool (Funk, Country, Soul, Rock, Indie Rock). Sub-conversations from the remaining five session groups were distributed across the training, development, and test sets. This approach ensures that the test contains examples from all seven producer pairs and song contexts, while the training and development sets contain data primarily from five pairs. Consequently, models are frequently evaluated on data from producer pairs and potentially genres unseen dur- ing fine-tuning, providing more robust assessment generalization capability. The resulting distribution across the splits is follows:
Train Set: 241 examples across 28 conversations.
Development Set: 34 examples across 11 conversations.
Test Set: 156 examples across 38 conversations.
This distribution results in varying average turns per conversation across splits (Train: 8.61, Dev: 3.09, Test: 4.11), reflecting the natural variability in topic discussion length and the inclusion of full short/long conversations in the test set.
G.5 Dataset Instance Examples
This section provides illustrative examples of the data instances contained within the MIXASSIST dataset. Each instance includes context (conversation ID, topic, turn ID), the input audio segment, the preceding conversational history (including generated summary and recent turns), the current user utterance, and the target expert assistant utterance.
Example 1: Discussing Keys
Conversation ID: group4.conv10
Topic:
keys
Turn ID:
Input Audio File: uutto//sgments//rrup//uuuut 37. wav
Input History: (Summaryy): We were discussing how approach mixing track with quiet vocals and many guitar tracks. The expert suggested starting with vocals and drums, and then using a gentle glue compressor on the drum bus to add presence and saturation. I think I'll keep balancing out this. (Assistant); Would you solo the keys and the accordion together?
User Utterance: Yeah! The keys are quiet.
Assistant (Target) Utterance: I think would put compressor bus on both of those. And turn that limiter on on the compressor. Maybe turn the threshold down little bit.
Example 2: Checking Overall Mix Level
Conversation ID:
group6_conv4 overall mix
Topic:
Turn ID:
Input Audio File: audlo/seegeents//rruuu////... wav
Input History: (Summary): We've been working on adjusting the EQ and compression to give the guitar more space in the mix, making some tweaks in the frequency ranges and considering adding reverb and delay for more
37

---

Published as a conference paper at COLM 2025
character. Now, I'm about to listen to an audio segment to see how these changes sound. Please analyze this audio segment.
(Assistant): Can we bump the volume a little bit? Just, we don't want it to be too far into red, but even just terms of like, feel like, still can't hear it over the, the others, so I think we have to pull down some of these percussion.
User Utterance: Little peaking here.
Assistant (Target) Utterance: Yeah, think it's okay, It's not clipping.
G.6
Conversational and Linguistic Analysis
Topic Flow: The original mixing sessions captured show an average of 10.0 topic switches per session. A common pattern observed is "zoom in, zoom out" approach, where participants focus on individual elements (like drums) before returning to assess the overall mix, mirroring professional workflows (Emil, 2024). Within the 431 substantive expert turns comprising MIXASSIST Drums are the most frequent topic (40.4%), followed by overall mix (25.3%), guitars (15.1%) 1)), vocals (8.6%), bass (6.5%), and keys (4.2%).
Linguistic Differences (Expert vs. Amateur): Analysis reveals nuanced linguistic patterns. Technical term usage, percentage of tokens, is remarkably similar between experts (in their content turns) and amateurs (across their turns) (Expert Content: 5.93%, Amateur All: 6.00%). Experts exhibit broader unique vocabulary (Shared Vocabulary Size: 660 lemmas, Expert-only; 504, Amateur-only: 360) and produce text with moderately higher complexity (Avg FKGL: Expert Content = 4.40, Amateur All 2.80). Additionally, experts utilize metaphorical language more frequently than amateurs (Avg Expert Metaphors per content turn: 0.715 vs. Avg Amateur per turn: 0.499, approx. 1.43x rate).
Instructional Patterns: Conversations exhibit clear question--xplanation structures, with amateurs often asking questions and experts providing explanations. Distinct step-by-step instructional sequences from experts, followed by amateur implementation attempts and feedback cycles, form recognizable pedagogical patterns. %(This paragraph remains valid)
Temporal Dynamics: Analysis over session time reveals strong evidence of active learning, Experts introduce technical terms at an average rate of 2.17 per relevant turn. Crucially, amateurs demonstrate significant positive growth in their technical terminology usage, increasing by an average of +42.31% from the first to the last third of their interactions. This indicates active vocabulary acquisition occurring alongside practical skill development during the collaborative mixing process.
G.7 Availability
The MIXASSIST dataset, including the processed conversational data, associated audio segments, and the raw session recordings, is publicly available to facilitate further research.
H MIXPARAMS Dataset Details
This appendix details the MIXPARAMS dataset, a resource created as an extension The Mix Evaluation Dataset (De Man & Reiss, 2017) and serving as a complementary dataset to the primary MIX ASSIST conversational corpus presented in this work. While MIXAsSIS focuses on the audio-grounde instructional dialogue (the "why"), MIXPARAMS provides detailed technical parameters (the "how") derived from corresponding source material.
H.1 Motivation and Purpose
MIXPARAMS was created to help advance research in intelligent music production, specifi- cally by providing detailed, ethically sourced, and interpretable data about music mixing parameters within Digital Audio Workstations (DAWs). A primary motivation was to fill a gap created by the scarcity of publicly available, detailed mix session data, as such
38

---

Published as a conference paper at COLM 2025
data is often proprietary. The dataset facilitates the exploration of tasks such predicting context-appropriate audio effect parameters (e.g,, EQ, compression, reverb settings), po- tentially linking these technical settings to the high-level perceptual evaluations available in the original Mix Evaluation Dataset or the instructional dialogue captured in M--As- SIST. Furthermore, the creation of this structured, tabular dataset enables investigation into whether interpretable machine learning models (e.g,, regression, multi-label classifica- tion) can achieve strong performance on parameter prediction tasks, offering an alternative to less transparent deep learning approaches like DDSP or graph neural networks often necessitated by lack of such data.
H.2 Dataset Content and Format
MIXPARAMS consists of detailed annotations for 114 individual mixes. These mixes were derived from selection of the non-copyrighted DAW session files originally collected for The Mix Evaluation Dataset. The specific songs included in MIXPARAMS are listed in Table 16. Crucially, the dataset uses the same source multitrack audio stems as The Mix Evaluation Dataset (and thus the same source songs as used in the MIXASSIST collection sessions), primarily sourced from The Open Multitrack Testbed, Weathervane Music's "Shaking Through", and Mike Senior's "Mixing Secrets" library.
Artist
Song Title
The Done Fors
Lead Me
Fredy V
In The Meantime
Fredy V
Not Alone
Broken Crank
Red To Blue
The Done Fors
Under A Covered Sky
The Done Fors
Pouring Room
Filthybird
I'd Like To Know
The Districts
Vermont
Creepoid
Old Tree
Purling Hiss
Lolita
Louis Cressy Band
Good Time
Table 16: Songs from The Mix Evaluation Dataset Included in MIXPARAMS
Note: These songs were selected from De Man et al. (De Man & Reiss, 2017) based on their non-copyrighted status, allowing for parameter annotation and distribution in MIXPARAMS.
Each instance in the dataset corresponds to a single track within one of the 114 annotated mixes. The annotations for each track include:
Metadata: Basic information including mix_name, song. name, artist_name name, and genre.
Track Information:
Details about the specific track within the mix, such as track name, track_instrument.sbbyy..
track_typa (e.g., audio, aux), and channel_mode (mono/stereo).
Audio File Reference:
Information linking to the source audío, including track audio path (path the original stem file), track audio sample rate, and measured track_audio_lufs (loudness).
Parameter Data: A structured representation (field named parameters) detailing the audio effects applied to the track and their settings. This includes common effects like Gain, Pan, EQ, Compression, and Reverb, along with their specific parameters (e.g., EQ frequency, gain, Q Compressor threshold, ratio, attack, release; Pan position).
The dataset focuses on common mixing parameters from native plugin; non-native plugins encountered in the original sessions were omitted during annotation. While the dataset provides parameter settings, it relies on the external availability of the original audio from their respective sources (Open Multitrack Testbed, Weathervane Music, Mixing Secrets).
39

---

Published as a conference paper at COLM 2025
H.3 Data Collection and Preprocessing
The parameter data was collected manually by an author of this work. The process involved opening each original DAW session file (Logic Pro Pro Tools) corresponding to the selected mixes (Table 16) provided by The Mix Evaluation Dataset and meticulously annotating the settings for audio effects applied to each track.
A significant challenge was that some plugin interfaces visually obscure their precise numerical parameter values. To address this, a custom software tool was developed and used to estimate these parameter values based on their visual representation within the plugin GUI. This tool is publicly available. If parameter's 's value could not be reliably determined or estimated, it was omitted from the dataset.
is important note that while automation (time-varying parameter changes) is crucial aspect of mixing, the dynamic automation data itself was not captured in MIXPARAMS due to the complexity of annotation. Instead, a parameter was automated, the value recorded in the dataset typically represents the parameter's 's state at the end point of the automation relevant to the mix section or snapshot being annotated. For instance, if track's volume fader was automated from -4dB to -2dB over section, the value annotated for that track's gain /volume would be -2dB. This captures the final intended level but not the dynamic transition.
Handling of Automation:
The raw DAW session files themselves contain the full automation data but are not part of the distributed MIXPARAMS dataset; researchers needing the original session files should refer to The Mix Evaluation Dataset. No other significant preprocessing was applied beyond the described annotation and estimation process.
H.4
Usage Notes
No canonical data splits (train /test) are mandated for MIXPARAMS, and users should define splits appropriate for their specific task. However, for reproducibility or comparison purposes, the dataset hosting includes metadata allowing for partitioning. One possible partitioning, based on the internal structure where each row represents an annotated track, yields 1.55k rows for training, 300 rows for development, and 500 rows for testing.
is noted that not all mixes included in The Mix Evaluation Dataset (and thus potentially annotated in MIXPARAMS) have corresponding perceptual evaluation data (preference ratings or comments). Mixes originating from Mike Senior's 's "Mixing Secrets" collection, for example, lack these perceptual ratings. Researchers intending to link parameter settings with perceptual data should filter MIXPARAMS accordingly.
H.5 Ethical Considerations
MIXPARAMS was created with the intention of providing an ethically sourced repository for music mixing research. It derives from publicly available data and session files originally collected by De Man et al. (De Man & Reiss, 2017). As this work focuses on annotating the technical artifacts (parameter settings) from those sessions rather than collecting new data directly from individuals, the original mix engineers were not re-contacted, and an additional iRB review was unnecessary for this specific annotation effort. The dataset contains no direct PII, sensitive, confidential, offensive information.
40


Reviews:
[{"review_id": "54eBPEoTUF", "reviewer": "Reviewer_ThtW", "summary": "This work presents MixAssist an audio-language dataset capturing the natural and nuanced multi-turn dialog between an expert and an amateur music producer during collaborative music mixing sessions. This work aims to aid the development of collaborative AI-assisted tools that can help amateur music producers seeking to learn by doing. The dataset consists of 7 hour-long sessions involving 12 producers in total. Specifically, a sample in the dataset contains summary of relevant historical context, the dialogues in the conversation (and sub-conversations), and the associated music-only audio segment. They also performed experiments where they benchmarked finetuned versions of different audio language models (ALMs) (using LoRA) such as Qwen-Audio, LTU, and MU-LLaMA on the proposed MixAssist dataset and evaluated the feedback given by these models using o3-mini as a judge, with Qwen outperforming other models, and human evaluation showed preference of the generated feedback over human annotations, which the authors speculate might be due more detailed explanations (longer response) provided by the model. The paper also includes results for real-time interaction of artists and the system where they majorly found the system to be both natural and helpful.\n\nOverall, this is a very well-written paper where each aspect is carefully thought of and executed, from the process to creating the dataset to taking decisions on how to do it. This paper would without a doubt be very valuable to the community.", "questions": "", "limitations": "", "rating": 9, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "- I really appreciate that the authors chose to work on the collaborative and pedagogical aspect of music creation/mixing rather than developing yet another method to automate this creative process.\n- This paper presents deep insights about what works with LLMs on this task and what doesn't -- certainly rare these days in papers. An in general, every section is well-motivated and eloquently written.\n\nIn my opinion, this paper ticks every check-box on what a good research paper should look like. It was a joy to read this paper and its discussions, so I just want to thank the authors for that.", "reasons_to_reject": "- If I had to point out something, please add results for a strong open-source model as the judge -- we now have Qwen3 and even Nvidia's nemotron Llama 3.1 235B (which is better than LLaMA 4 Behemoth on some tasks and on par with Deepseek R1).", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "sxWKI2I10D", "reviewer": "Reviewer_UNoT", "summary": "The paper presents a dataset of collaborative mixing sessions involving dialog between an expert and amateur mixing engineer. The dataset is used to finetune and evaluate LLMs on their ability to act as such an expert instructor, primarily for pedagogical reasons.", "questions": "", "limitations": "", "rating": 8, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "- The paper takes a very innotative approach to both the task and methodology of AI-aided mixing. Notwithstanding the motivation of training amateur engineers via dialogs, the data collection methodology of recording reqests and actions in the form of \"think aloud\" dialogs circumvent the known challenge in collecting audio engineering data.\n- One potential that the paper does not emphasize enough is the \"what's next\" after the expert explains about a concrete action after the amateur describes a desired change. In reality, this can not only help amateurs learn but also directly enable users to mix, even without the model interfacing with the DAW. For example, if an LLM can generate an expert-utterance such as \"I would cut 500Hz\", the user can just do this and see if it works. For most producers with little experience in mixing, figuring out \"I would cut 500Hz\" is the hardest part. \n- The survey of real practitioners in the industry described in 3.1 is very appreciated.\n- The data collection protocol in 3.2 is clearly described.", "reasons_to_reject": "- The dataset does not come with the submission. I would like to hear the sessions to ensure the validity of the data.\n- In Section 4, it is unclear what the input (which utterances are included as the conversation history?) and output of the model (does the model outputs an utterance? even when the atterance is not actionable?) is. \n- While the qualitative analysis in 4.3 and 4.4 is helpful, qualitative analyis is lacking. For example, \"has conten tag indicating substantive, actionable guidance\" previously described can enable quantitative evaluation of models (e.g., did it correct predict that it should EQ the bass? did it correctly identify the offending frequency to cut?).\n- \"Challenge 1: Audio Understanding\" deserves much more discussion, as one could doubt if the models actually did any thing useful based on that section. Is AI simply serving as a chatbot here, if it cannot accurately translate requests to parameters? Can AI actually identify issues in a mix given audio (I don't think this has been thoroughly studied)? If AI cannot make most of the mixing decisions correctly, how can it possibly act as a teacher? To me, this is the **most important weakness of the paper**.", "questions_to_authors": "- For commercialization, I imagine the most natural setting is a customer (instead of an amateur engineer) interacting with an expert (which could be an AI). The customer may also say things like (as Fig1) \"I want the drums to blend better\" but also vague things like \"It sounds so empty.\" How do you think the customer's utterances (e.g., request to revise when hiring a mixing engineer) would differ from amateurs'? What is the outlook of applying it in a customer-facing application?\n- While prompting is not experimented with in 4.1, I think there is still value in trying that just to understand how bad the models are and what the gaps are. Finetuning inevitably overfits to the bias of the dataset, and decreases outlook for generalization.", "extra_scores": {}}, {"review_id": "HiMzAUzAzJ", "reviewer": "Reviewer_jqzw", "summary": "The paper \"MixAssist: An Audio-Language Dataset for Co-Creative AI Assistance in Music Mixing\" introduces MIXASSIST, a novel dataset designed to support the development of AI systems that can provide co-creative assistance in music mixing. The dataset captures the dynamic, multi-turn dialogues between expert and amateur music producers during live mixing sessions, focusing on the instructional and conversational aspects of the mixing process. The authors conducted seven in-depth mixing sessions involving 12 producers, resulting in 431 audio-grounded conversational turns. The dataset is unique in its focus on capturing the \"why\" behind mixing decisions, providing a rich resource for training AI models to understand and participate in real-world music production dialogues. The paper also evaluates several state-of-the-art audio-language models (ALMs) fine-tuned on MIXASSIST, demonstrating promising results in generating contextually relevant mixing advice.", "questions": "", "limitations": "", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "The introduction of the MIXASSIST dataset represents a significant contribution to the field of AI in music production. The dataset is meticulously constructed, capturing the nuances of real-world music mixing dialogues, which makes it a valuable resource for training and evaluating AI models. The focus on co-creative and instructional interactions aligns with the growing interest in AI systems that can provide educational and collaborative support, rather than just automation. The paper provides a comprehensive evaluation of several ALMs, demonstrating the potential of fine-tuning models like Qwen-Audio on MIXASSIST to generate high-quality, contextually relevant mixing advice. The findings highlight the strengths and limitations of current ALMs in this domain, providing valuable insights for future research. The detailed methodology, including participant instructions, data processing steps, and evaluation metrics, ensures the reproducibility and reliability of the study. The release of the dataset, along with the processed data and raw recordings, encourages further research and development in this emerging area.", "reasons_to_reject": "One potential concern is the limited diversity of the dataset in terms of musical genres and producer backgrounds. While the dataset includes a variety of genres, the overrepresentation of certain genres (e.g., rock, pop) and the use of a specific set of multitrack recordings from The Mix Evaluation Dataset could limit the generalizability of the findings. The authors could benefit from discussing strategies to expand the dataset to include a broader range of musical styles and production contexts. Another issue is the reliance on a single model (Qwen-Audio) as the primary baseline, which may not fully capture the capabilities and limitations of different ALMs in this domain. Including a more diverse set of models in the evaluation could provide a more comprehensive understanding of the challenges and opportunities in developing AI assistants for music mixing. The paper could also benefit from a more detailed discussion on the ethical implications of using AI in creative domains, particularly regarding data provenance, attribution, and the potential impact on human creativity and originality.", "questions_to_authors": "How did the authors ensure the diversity and representativeness of the musical genres and producer backgrounds in the MIXASSIST dataset?", "extra_scores": {}}]

Author rebuttal:

Meta-review: This paper introduced a new audio-language dataset for natural and nuanced multi-turn dialog between an expert and an amateur music producer during collaborative setting. All the three reviewers have agreed on the novelty and valuable contribution of this dataset. While one reviewer (#jqzw) has raised ethic and baseline concern, the authors have carefully provided feedback to address them. I thus recommend Accept as is.
Decision: Accept