Automatic Stage Lighting Control: Is it a Rule-Driven Process or Generative Task?
Authors: Zijian Zhao
Dian Jin
Zijing Zhou
Xiaoyu Zhang
Venue: ICLR 2026 Poster (2026)
Keywords: Automatic Stage Lighting Control
Music Information Retrieval
Multi-Modal
Primary area: applications to computer vision, audio, language, and other modalities

Abstract: Stage lighting is a vital component in live music performances, shaping an engaging experience for both musicians and audiences. In recent years, Automatic Stage Lighting Control (ASLC) has attracted growing interest due to the high costs of hiring or training professional lighting engineers. However, most existing ASLC solutions only classify music into limited categories and map them to predefined light patterns, resulting in formulaic and monotonous outcomes that lack rationality. To address this gap, this paper presents Skip-BART, an end-to-end model that directly learns from experienced lighting engineers and predict vivid, human-like stage lighting. To the best of our knowledge, this is the first work to conceptualize ASLC as a generative task rather than merely a classification problem. Our method adapts the BART model to take audio music as input and produce light hue and value (intensity) as output, incorporating a novel skip connection mechanism to enhance the relationship between music and light within the frame grid. To address the lack of available datasets, we create the first stage lighting dataset, along with several pre-training and transfer learning techniques to improve model training with limited data. We validate our method through both quantitative analysis and an human evaluation, demonstrating that Skip-BART outperforms conventional rule-based methods across all evaluation metrics and shows only a limited gap compared to real lighting engineers. The self-collected dataset, code, and trained model parameters of this paper are provided at https://github.com/RS2002/Skip-BART .

Published as a conference paper at ICLR 2026
AUTOMATIC STAGE LIGHTING CONTROL: Is IT A RULE-DRIVEN PROCESS OR GENERATIVE TASK?
Zijian Zhao', Dian Jin Zijing Zhou?, Xiaoyu Zhang
'The Hong Kong University of Science and Technology 'The Hong Kong Polytechnic University The University of Hong Kong *City University of Hong Kong
ABSTRACT
Stage lighting is vital component in live music performances, shaping an gaging experience for both musicians and audiences. In recent years, Automatic Stage Lighting Control (ASLC) has attracted growing interest due the high costs of hiring or training professional lighting engineers. However, most exist- ing ASLC solutions only classify music into limited categories and map them predefined light patterns, resulting in formulaic and monotonous outcomes that lack rationality. To address this gap, this paper presents Skip-BART, an end-to- end model that directly learns from experienced lighting engineers and predict vivid, human-like stage lighting. To the best of our knowledge, this the first work to conceptualize ASLC as a generative task rather than merely a classifi- cation problem. Our method adapts the BART model to take audio music as in- put and produce light hue and value (intensity) output, incorporating a novel skip connection mechanism to enhance the relationship between music and light within the frame grid. To address the lack of available datasets, we create the first stage lighting dataset, along with several pre-training and transfer learning techniques to improve model training with limited data. We validate our method through both quantitative analysis and an human evaluation, demonstrating that Skip-BART outperforms conventional rule-based methods across all evaluation metrics and shows only a limited gap compared to real lighting engineers. The self-collected dataset, code, and trained model parameters of this paper are pro- vided at Pattps://qithie com RS2002 /ttp/ssiii-ir--e
INTRODUCTION
Recently, multi-modal machine learning has significantly advanced various fields in Music Infor- mation Retrieval (MIR), including video-to-music generation (Tian et al. [024),, music video (MV) generation (Chen al., 2025), and lyrics generation (Ma et al. 2024). Among these techniques, Automatic Stage Lighting Control (ASI.) shows potential live performances by shaping the atmosphere and influencing the emotions of both musicians and audiences, while offering an afford- able cost.
Currently, most existing ASLC methods are based on predefined rules that classify music into finite categories with deterministic patterns. The style-based (Stanescu et al. 2018) and emotion-bass (Hsiao et al. 2017) classification methods are two mainstream approaches. However, these meth- ods have several limitations: First, due to limited datasets and labeling difficulty, the accuracy of classification models is often insufficient, and the defined categories are typically coarse-graine grained.. Besides the biases caused by misclassification, this coarse-grained labeling directly impacts model performance. For example, the atmosphere black metal and folk black metal are entirely different, yet they may be grouped with thrash metal under the metal category, even combined with punk in the broader rock category. Second, as lighting is viewed as an amalgam of technology and art Gillette & McNamara 2019), it is debatable whether can simply use certain patterns define it. Additionally, McDonald et al. (2022) have demonstrated that limited relationship between hue
"Corresponding Author: Xiaoyu Zhang (xiaoyu.zhang canryuuhhann@citu...cooo @cittu.cdu.kk)
"o avoid confusion, in this paper, "lighting control" and "lighting generation" used interchangeahly.

---

Published as a conference paper at ICLR 2026
of light and emotion, raising questions about the appropriateness of mapping emotions solely to the light.
To address this research gap, we propose to treat ASLC as content generation task rather than merely a mechanized mapping process. Inspired by the success of generative models in MIR, we develop end-to-end deep learning methodology that trains network directly on ground truth data from professional lighting engineers. Initially, we introduce an automatic label generation method from video data and create a self-collected dataset titled Rock, Punk, Metal, and Core Livehouse Lighting (RPMC-L3) to address the scarcity of training datasets in this field. Subsequently, we present carefully designed neural network called Biddreiiiiettionall ers (akip-BART) for stage lighting generation from music. We also incorporate transfer learning and a pre-training method enable the model to learn effectively from limited data. We evaluate our methodology through both quantitative analysis and human evaluation. The results indicate that our method outperforms conventional rule-based approaches across all metrics and achieves perfor- mance comparable to that of human lighting engineers. In summary, the main contributions of this paper are as follows:
Legr-Regressive Transform-
To the best of our knowledge, we are the first to frame ASLC as a generation task rather than rule-driven mapping process, based the intuition that ASLC is essentially art creation process for humans. To support our claim, we study the offline primary ASLC task, which is modeled by taking a music sequence as input and generating single light hue and value as output. We further evaluate this model on real-world dataset, demonstrating the superiority of generative methods over previous rule-based approaches. Our novel perspective on generation provides a strong foundation for future research in ASLC.
We develop Skip-BART, a novel end-to-end deep learning framework to address the aforemen- tioned generative task for ASLC. Building on BART, our approach incorporates adapted embed- ding and head layers to support both music and light modalities, well as novel skip-connection module to enhance the relationship between music and light within a fine-grained context. Ad- ditionally, introduce pre-training and transfer learning mechanisms to improve model per- formance under limited training data. During inference, we implement a Restricted Stochastic Temperature- Conerrolll (RSTC) sampling method to ensure both diversity and stability in the generated results.
To validate our method, we present the first stage lighting dataset, RPMC-L2, and train Skip- BART on it. Through quantitative analysis and human evaluation, demonstrate that our method outperforms conventional rule-based approaches across all metrics. Specifically, our method yields significant difference (p 0.001) compared conventional rule-based methods, while showing no significant difference compared to human lighting engineers (p = 0.72) in the human evaluation. This suggests that our proposed method closely matches human lighting engineering performance. Furthermore, our method achieves promising results in cross-domain scenarios with other music styles, significantly outperforming previous rule-based methods.
RELATED WORK
Although ASLC is not a widely explored topic in MIR, it has received some attention in previous works. Most prior methods can be summarized as a paradigm that first classifies music into prede- fined categories, such emotion and style, and then defines corresponding light pattern for each category. For instance, works (Stanescu et al. 2018 Lei et al. 2021 Kanno & Fukuhara 2022] Liao et al., (023) utilize style and theme for classification based on audio or lyrics, respectively, while Mabpa et al. (2021) classify music using chords. However, these methods exhibit significant limitations. Firstly, there is insufficient theoretical support for mapping these categories to light. Ad- ditionally, coarse- voarse-grained grained classification severely impacts model performance. For example, Mabpa ctt al. (2021) classify chords into only 24 major and minor chords, neglecting more nuanced chords, such augmented, seventh, and suspended chords, which possess distinct colors and functions. Furthermore, many pieces of music lack lyrics or contain lyrics that are difficult to recognize (such as screaming in heavy music), rendering the method of Liao et al. 2023) ineffective.
Additionally, a series of works focus on emotion classification (Bonde et al., 2018 Hsiao et al. 2017 Moon al. 2015), employing various machine learning methods, including Support Vector
2

---

Published as a conference paper at ICLR 2026
Pre-trained PianoBART Backbone
LoRA
Encoder
Decoder
Skip Connection
Music Embedding
n-1
Light Embedding
Embedding Layer
Continuaus Music Tokens
Discrete Light Tokens
Shift Right
Pre-trained Open L3
Tokenization
gella Ja Syp
Music MLMS
HV Features
Extract Light Features
Extract Music
Live Video
Network Architecture: In the figure, 'ice" represents the frozen parameters, and 'fire' denotes the trainable ones.
Figure 1:
Machines (SVM), Support Vector Regression (SVR), and Multi-Layer Perceptrons (MLP). Vryzas al. (2017) extend this the field of drama. However, the relationship between music perfor- mance emotion and light remains controversial, with different studies presenting varying perspec- tives (Kanno & Fukuhara 2022 McDonald al., (022). We argue that simply mapping emotion to light is not a viable approach for two reasons: (i) Although Kanno & Fukuhara (2022) demonstrate that certain light colors are frequently associated with specific emotions by professional lighting engineers, this does not imply that these colors must be used when conveying those emotions. For example, in the dataset associated with this paper, we observe that the green hue appears groove metal associated with anger emotion, contradicting the color theory suggesting red should be used. (ii) Beyond the granularity issue mentioned earlier, the performance all rule-based methods de- pends the accuracy of the classifier. In some works (Zhaol (025), the emotion classification accuracies of various methods fall below 80%, raising doubts about the practicality of these meth- ods at their current stage.
Moreover, Tyroll et al.] (2020) present a different workflow, where a feature extractor is first trained via an autoencoder, and then the feature embeddings are directly mapped to light. However, the feature extraction capacity of autoencoders has been shown to be limited (Van Den Oord et al., 2017), and the mapping from feature embeddings light is also ambiguous. In contrast, pro- pose the first end-to-end method for stage lighting control, framing the task as generation problem rather than classification task. Compared to previous hierarchical mapping methods, our approach directly trains the model on real-world data from lighting engineers, resulting in improved perfor- mance.
METHODOLOGY
3.1
OVERVIEW
In this section, we introduce end-to-end model, Skip-BART, designed to generate single principal stage lighting based on music. Similar to previous works, we focus exclusively on offline primary single lighting generation. In the following content, describe our model structure, as depicted in Fig. [ We modify the embedding layers to accommodate our task's ' input, utilize pre-trained mod- els from other tasks for enhancement through transfer learning, and incorporate skip-connection
3

---

Published as a conference paper at ICLR 2026
(a) Normal Warm Light (b) Normal Cold Light
(c) Extreme Warm Light (d) Extreme Cold light
Each subfigure shows the input scene on the top, the result of the direct HSV extraction method in the bottom-left, and our extraction method in the bottom-right. (a)-(b) Both methods accurately extract the dominant hue. (c)-(d) Our method extracts colors closer to the original ap- pearance.
Figure 2:
mechanism to better capture the aligned relationship between music and light. Subsequently, we out- line the workflow of our method, which includes MLM-based pre-training, end-to-end fine-tuning, and inference using a RSTC sampling method.
Before delving into the details of our methodology, we first provide brief introduction to the data processing, with additional details left in Appendix C.3 To facilitate the analysis of light- ing conditions, the lighting information preprocessed in the HSV color space. The HSV color space separates the brightness (Value) information and the color (Hue) information, which makes the control and generation of lighting more intuitive, independent, and stable. Specifically, we set the Saturation (S) channel to constant value") of 255 (100%) to simulate the lighting effect and counteract environmental influences, ensuring that after factors such as air scattering, fog, or surface reflections reduce color purity, the light maintains vividness and minimizes distortions like fading or whitening. Compared to the light extraction approach that only considers directly extracting HSV features, data processing method enables more accurate color extraction under extreme intensity lighting conditions, as shown in Fig. 2
(x1, -2, FT extract the corresponding light sequence y = {y1, 1/2....,.,,.. y..). where T represents the number of frames. For each frame i, we define Y: = Th., Ty] E ZZ, where ni and u, denote the hue and value extracted from the HSV space.
Then, for each music sequence
3.2 NETWORK ARCHITECTURE
The model structure is illustrated in Fig. 1 In our task, the model takes entire piece of music as input and generates the corresponding lighting sequence. We believe that BART (Lewis) 2019) is well-suited for this task, as the bi-directional encoder effectively extracts contextual information simultaneously. while the unidirectional decoder generates outputs in an auto-regressive manner, thereby maintaining correct contextual correlations. To adapt BART for our task, we first modify its embedding layers to support the music and light input. To align the dimensions between Open 1.3 and BART,
MLP to further extract features from the embeddings produced by Open L3. The hue and value obtained from Appendix C.3 are then embedded using respective embedding layers and subsequently combined, allowing them to be input into the decoder of the backbone. As noted in (Zhao et al., 2024), the tokenization process may lead to some information loss. However, directly processing hue data with an MLP presents challenges due to the cyclical nature of hue values: the minimum and maximum hue values can be quite similar, complicating the learning process. In contrast, the embedding layer effectively captures the relationships between different tokens, making it more suitable for handling cyclic data like hue.
When applying BART for the lighting generation task, we notice that it can be challenging for the model to learn the rhythm, meaning that the light frequency and pattern changes do not always align with the music. One reason for this is that the attention mechanism in the decoder struggles to determine which light frame corresponds to which music frame. Although it is intuitive for humans to recognize the one-to-one correspondence (since the music input x and light output have the same length), the model finds it difficult to learn this relationship directly from complex and noisy
2To avoid ambiguity, we Value or V to refer the V component in HSV, and value to represent its numerical value here.
4

---

Published as a conference paper at ICLR 2026
1. Pre-train
Videos
Audio Music
Discrtminator
input
train
Light
2, Fine-tune
output
Regressor
orginx
3. Infer
Skip
Encoder
Decoder
Classifter
Classifier
Hoe
How
Skip BART als Encoder
Eacoder
17B
0.2%
Hie
Randomn MASK
Shift Right
Value 0
0.0%%
Restriction
Volue 234
3/5
Auto
Open L3
Open
Tokenization
Open
Regression
Stochaetic Temperature
Music MILSS
Music iMLMS
28
Light-HV
2
Music MLMS
Controlied
Sampling
Figure 3: Workflow of Skip-BART
data in end-to-end manner. To address this, propose skip mechanism that directly combines the music embedding and light embedding before inputting them to the decoder, thereby explicitly indicating the correspondence relationship. For example, the embedding of the light token y: will be combined with the embedding of the music token 2;-1 (noting that there is one-position shift, as all inputs to the decoder should shifted right by one position).
Additionally, transfer learning has demonstrated the benefits of using trained parameters from other domains (Pan 22044). Given limited datasets MIR, we aim introduce more external knowl- edge to enhance model performance. Consequently, adopt the backbone of PianoBART (Liang) jet al., (202),, a large-scale pre-trained model for symbolic music understanding and generation, in Skip-BART framework. Additionally, to embed much knowledge possible, utilize the Drop And Rescale (DARE) (Yu et al. 2024) method to combine the fine-tuned parameters of Pi- anoBART across all downstream tasks, including priming, melody extraction, velocity prediction, composer classification, and emotion classification. The combined model parameters are as follows:
0 0 Opre + + E(0 Opre)
0 DARE =Opre + (0i 0pre) ② mm
1-p
m. Bernoulli()) E {0, 1)
where 0 represents the combined model parameters, 6pre denotes the pre-trained model parameters, 0' represents the fine-tuned model parameters for the 5th downstream task, d is the dimension of the parameters, p is the drop rate, À is the scaling term, and O represents element-wise multiplication. We then employ LoRA (Hu et al., 2022) for efficient tuning in our tasks. Due to page limitations, further details can found (Yu et al.) 2024). Finally, for both pre-training and fine-tuning, we employ different heads for various tasks, which will be detailed in the next section.
In summary, our proposed network is based on the BART backbone, utilizing OpenL3 to extract mu- sic audio features and tokenization mechanism with embedding layer to extract light features, effectively addressing the cyclic problem of hue. A novel tiip-connection mechanism introduced enhance the relationship between music and light within the frame grid. Additionally, the back- bone directly incorporates pre- rreitrained parameters from PianoBART, providing solid starting point for subsequent training, while using LoRA for efficient tuning.
5

---

Published as a conference paper at ICLR 2026
3.3 WORKFLOW
3.3.1 MLM-BASED PRE-TRAINING
To fully utilize the training data, we first employ an unsupervised pre-training method that encour- ages the model to learn the underlying data structure. To prevent information leakage, we do not use light data during this phase. Instead, we input audio music into both the encoder and decoder. As noted in transfer learning, the parameters of the middle layers the model are generally sim- ilar across different modalities tasks. This similarity aids the decoder in initializing appropriate parameters, allowing knowledge to be transferred from music to light during fine-tuning.
For pre-training the model music data, we opt for the MLM task, following the approaches in (Zhao et al. 2024 Zhu et al., 2021 Lewis] 2019). A MLP is used after the decoder's 's output to map the output hidden state music sequence. We first utilize Open L3 to embed the audio music, resulting in an embedding sequence = {er, e2, e.,..., en
E Raxh, where / is the hidden dimension. We then randomly replace k% of the tokens with [MASK] tokens and train the model to recover them. Since the music tokens are continuous, we design the [MASK] token as a random variable sampled from a normal distribution, based on (Zhao al. 2024):
[MASK] = [M1 M2, M. RR M; ~ Nu', (u',o'),
where rrpressets transposition, ju and o² denote the mean and standard deviation of sequence in the 2thh feature dimension. this manner, the model not only aims to recover the obscured tokens but also identify which tokens have been replaced by [MASK], further assisting the model in learning the structure the music sequence. To enhance the realism of the recovered output, we also employ GAN-based discriminator distinguish between the original music and the reconstructed music, as described by (Zhao et al. 2024):
min max V) R
D
min max - -
D
where D and R represent the discriminator and recoverer, respectively. For the discriminator, adopt the same architecture as the sequence classifier in [Chu et al. (202I). It employs a modified self-attention layer capture the global information from the embedding sequence e, followed by linear layer that classifies the extracted feature as True or False
Here, to enhance parameter efficiency, allow the discriminator and recoverer
shared
Skip-BART backbone while employing separate heads, as suggested by (Zhao et al. 2024). Thus, can express the pre-training loss function of Skip-BART follows:
pre 014 + a2l2 azl3.
4 MSE(é, e)
(4)
22 MSE as (é,, ex)
43 = CrossEntropy (D()), 1),
where M represents the token index set of those replaced by [MASK], é denote the outputs of Skip- BART. 4 is the reconstruction loss resembling that of an autoencoder, l focuses the recovery loss for the masked tokens, 1a is the realism loss to enhance the believability of the recovered output, and the parameters an, 022, 0g are weights. Additionally, the loss function for the discriminator can be defined as:
Lais CrossEntropy (D()), 0) (rossEEntropy(((e).)))
(5)
Due to page limitations, we have included additional details regarding pre-training in Appendix A
3.3.2 AND-TO-END FINE-TUNING
During fine-tuning, utilize the Language Model (LM) task (Radford et al. 2018) (next token prediction) to supervise the model in generating the corresponding lighting sequences. We frame this task as classification problem where the model needs to classify the hue and value at each token

---

Published as a conference paper at ICLR 2026
position. The classification vocabularies defined by the binning scheme used in the histograms Da.e and De,s, as described Eq. IT Consequently, we employ two MLPs to map the decoder output to the classification probabilitæes for hue and value, respectively. The loss function can be expressed as:
Latf = CrossEntropy(hh. (h, h) + 82 (rosssEntropy((,
(6)
where i and i denote the model outputs, while 31 and 82 are hyper-parameters. learning speeds for these two attributes not the same. To encourage the model to focus more the attributes that are learned more slowly, we use adaptive weights as proposed in (Zhao) 2025):
observe that the
B
0003
ROO
where B/ represents the weight in epoch j. and ace and acc; denote the model accuracy for hue and value, respectively, in the validation set of the jth epoch.
3.3.3 RSTC-BASED INFERENCE
During the inference phase, Skip-BART generates the lighting sequence in an auto-regressive man- ner. To prevent model degradation and to implement a stochastic lemperature-controlli sampling method (Hsiao al. 2021). we represent this as follows:
(kki-BARRT( (z, 1 D1 1))
9 softmax
(8)
where t represents the temperature, and 151....,9:-1 denotes the previously generated lighting sequence. The right term of Eq. [ consists of two vectors, representing the probabilities of each cat- egory for hue and value, respectively. The hue and value then sampled from these two probability vectors.
Additionally, avoid overshooting, we ensure that the successive two lighting tokens do not exceed a specified threshold:
{ Du (h, = min hh h 180 1hg - 14-1133 < di
(o, 04-1) = |0; 0-11 < d
where D(..) .) is the distance function d' is the threshold. To satisfy this restriction, we set the proba- bility of all categories this distance zero during stochastic comperature-controllee sampling. We refer to this method as Restricted Stochastic Teeperrauure-Contollle (RSTC).
4 EVALUATION
4.1 SETUP
To illustrate the effectiveness of our method, we compare it with other approaches through both quantitative analysis and an human evaluation. In the quantitative analysis, we evaluate the differ- ences among our method, a conventional rule-based method, and several ablation studies in relation to the ground truth. We assume that the closer the generated results are to the ground truth, the better the model performs, a hypothesis that will be validated through human studies. During the human evaluation, participants are asked to score various methods, including the proposed Skip-BART, the Ablation Study, the previous methods (referred to as the based Method), and the ground truth across different dimensions.
More details about the model configuration and training process can be found the Appendix B The experiments running Ubuntu 22.04.5 LTS, equipped with an Intel(R) Xeon(R) Gold 6133 CPU @ 2.50 GHz, two NVIDIA 4090 GPUs, and one NVIDIA A100 GPU.
conducted using the PyTorch framework (Paszke al. 1912) server
Moreover, as mentioned in introduction, there is currently no dataset available for the lighting gen- eration task, which is one reason previous methods have relied on rules. We address this gap by
7

---

Published as a conference paper at ICLR 2026
Table I: Quantitative Results:
The best result is indicated in bold, and the second best is indicated
by underline. This notation will remain consistent in the following tables.
RMSE
MAEL
corr( A ) [x10 f)f
Methods
Hue
Value
Hue
Value
Hue
Value
Rule-based
48.67
93.39
43.43
86.55
0.50
0.58
Skip-BART
36.13
60.74
28.72
51.27
0.88
2.94
w/o skip connection
36,89
68.33
29.44
58.34
1.15
0.30
w/o light embedding
51.04
67.25
41.50
54.87
0.80
0,70
train from scratch
36.63
67.49
28.83
57.22
0.69
0.53
pre-train w/o random [MASK]
49.97
64.45
42.07
52.63
0.54
1.11
pre-train w/o discriminator
50.40
68.09
41.52
56.54
0.46
1.13
proposing the first stage lighting dataset named 'RPMC-L2* (Rock, Punk, Metal, and Core Live- house Lighting). This dataset is highly diverse, collected by the authors over the past five years, including contributions from numerous bands, livehouses, and four different capturing devices, cov- ering a wide range of styles such as metalcore, alternative rock, and post-punk. We construct the videos using the proposed method in Appendix C.3 from raw footage. Our dataset includes 699 sam- ples, with lengths ranging from 20 seconds to around 5 minutes, making it a relatively medium-sized dataset in MIR. More details can be found in the Appendix C
4.2 QUANTITATIVE ANAYSSIS
In the quantitative experiment, we compare the generation results of different methods against the ground truth. Since our method utilizes random sampling during training, comparing loss and accu- racy has limited significance, similar most generative works (Liang al. (2024). Therefore, we directly compute four metrics between the generated results and the ground truth: (1) Root Mean Square Error (RMSE) and (2) Mean Absolute Error (MAE), both measuring absolute prediction er- rors. For the Hue channel, RMSE/MAE are computed on the circular angle differences, ensuring that values like 0' and 179º are treated as close rather than far apart, as shown in Eq 9 (3) corr([A) measures the consistency of change magnitudes by computing the Pearson correlation of the abso- lute first-order differences between predictions and ground truth. The final value is multiplied by 100. The results presented in Table [ where it is evident that method outperforms all abla- tion studies and the rule-based method in both hue and value, achieving the lowest error. Since the rule-based method does not directly learn from the ground truth, may introduce some unfairness. We provide further explanations in the human evaluation section.
4.3 HUMAN EVALUATION
To better assess how our results align with human preferences, we designed a questionnaire to col- lect participants' feedback the stage lighting effects created by each method. We recruited 38 respondents through social media platforms and live music venues, with the human evaluation span- ning half month. The participants included 20 males and 18 females aged 18 18-55 (predominantly 18-30), among whom 9 had professional experience in lighting design, music production, or stage art, and over half were frequent live music attendees. All participants reported normal hearing. normal or corrected-to-noret
vision, and no history of color blindness or color weakness. All par- ticipants are required evaluate three music pieces alongside the lights across six metrics (Erdmann et al. 2025), with each metric scored from I to 7 (the higher the score, the better the evaluation). The lighting conditions were derived from four sources: Ground Truth, Skip-BART, Rule-based Method, and 8kip-BART (w/o skip connection). For the ablation study, we specifically evaluate Skip-BART without the skip connection. We considered two main factors: first, the efficiency of these compo- nents has been widely demonstrated in other fields; second, increasing the number of comparative methods complicates data collection and may discourage user participation, as longer question- naires can lead to fatigue and confusion while viewing the same lighting video for particular piece of music. For simplicity, we will refer to Skip BART (wo skip connection) as the Ablation Study in this section and the following tables. Due to page limitations, we outline the primary experimental
8

---

Published as a conference paper at ICLR 2026
process and results below, while the Appendix contains the complete survey questionnaire and the analytical results.
We first calculated mean scores for each vidco across all three music backgrounds. These averaged values served as composite scores for subsequent statistical analysis. Then, we employed Repeated Measures ANOVA to examine the scores of four videos across overall and six individual metrics, while controlling for within-subject variability inherent in repeated measurements. To control Type I errors in multiple comparisons, we applied the Bonferroni correction. For method-wise differenti- ation, we conducted post hoc comparisons using Bonferroni-corrected paired t-tests.
As shown in Table 2 and Table 3 Ground Truth receives the highest overall score (M = 4.5, SD 0.9), closely followed by our Skip- Skip-BART(M=4.3. SD=0.9) and the Skip-BART (w/o skip con- 1e=4,, SD=0.8), both of which are near the GT level. All three methods significantly outperform the rule-based approach(M-2.7, SD=113.p=00055, a trend that holds for all the met- rics. The only metric where Ground Truth does not achieve the best score is in emotion matching, supporting the perspective that the emotional aspects of music may have limited connection to lighting control (McDonald et al. 2022). We also observe that for the rule-based method, the score of the emotion mc331, SD=1.5) is higher than that of other metrics, which validates our reproduction efforts.
Table 2: Human Evaluation Scores: The six metric scores and overall evaluations of the four objects are expressed as Mean + Standard Deviation (M + SD) The bold text represents the best result, while the underlined text indicates the second-best result.
Method
Emotion
Impact
Rhythm Smoothness
Atmosphere
Surprise
Overall
In-domain Evaluation
Ground Truth
4.50=0.93
4.4820.99
4.61
4.62+1.07
4.49:0.89
4.34=1.10
0.5
Skip-BART
4.69:0.87
4.39+0.95
4.50=1.06
4.32&1.12
4.32+0.93
4.35+0.87 3.83+1.06
Ablation Study
4.31=0.94
3.78+0.96
4.54#1.08
4.43+1.12
4.11+0.98
3,50=1.00
4.1 1+0.84
Rule-based
3.12+1.52
2.65#1.39
2.54
2.56+1.27
2.77+1.50
2.35+1.40
2.67+1.29
Cross-domain
Evaluation
Skip-BART
4.31
4.18
4.60
4.6120.92
4.41*1.09
3.91=1.17
4.34
Ablation Study
3.94+1.31
3.93+1.12
1.44+1.02
4.42+1.00
4.04+1.11
1.79
4. 40
Rule-based
3.61+1.31
3.04
3.21
3.88&1.15
3.34
2.96#1.41
3.34
Comparing our SkiP-BART with the Skip-BART without skip connection, from which we removed the skip connection, we find that our method consistently outperforms in all metrics except for rhythm(AM -0.035, SD=0.112, p=1.00) and Smoothness(AM minimal performance gaps. The small gap in rhythm be attributed to randomness. In terms of smoothness, the skip connection links each frame of music to light directly, which might cause significant changes in light during major transitions in the music. Overall, method greatly outper- forms conventional based methods and achieves performance comparable to that of professional lighting engineers.
00.11, SD=0.16, p=1.00), with
Table 3: Pairwise Comparisons of Overall Scores for the Four Objects: The symbols and ** denote that the significance level (p) for the difference in means (AM) is less than 0.05 0.01, respectively.
In-domain Evaluation
Cross-domain Evaluation AM
Comparison
AM
SD
SD
P
Ground Truth vs. Skip-BART
0.16
0.10
0.724
Ground Truth vs. Ablation Study
0.40
0.10
0.003**
Ground Truth vs. Rule-based
1.84
0.21
<0.001#a
Skip-BART vs. Ablation Study
0.23
0.10
0.152
0.24
0.12
0.167
Skip-BART Rule-based
1.68
0.19
<0.001 we
1.00
0.17
<0.001 *
Ablation Study vs. Rule-based
1.44
0.16
<0.001# #
0.76
0.15
<0.001
Additionally, to evaluate our model's performance in a cross- scenario, we compare different methods across various music styles. For fairness, utilize songs generated by Suno (Suno) instead

---

Published as a conference paper at ICLR 2026
of selecting them manually. This approach allows us to assess the model's performance on entirely novel songs. Furthermore, since no ground truth is available, we omit this object and ask 30 users to evaluate other three objects across the following music genres: folk, R&B, and jazz. The detailed Our proposed method consistently achieves the highest scores across all metries, demonstrating significant differences compared the rule-based method.
results are presented in Tables 2
and g
5 LIMITATIONS AND FUTURE WORK
While our framework demonstrates positive results, there remain several limitations and opportuni- ties for further improvement. Below we highlight several key directions worth further exploration:
Generalizability Across Music Genres: In our experiment, we have demonstrated promising results from our method on both in-domain music genres (e.g., rock, punk, metal, and hardcore) and cross-domain genres (e.g.. folk, R&B, and jazz). However, further evaluation across wider range of styles could be explored in future research. In practice, our method is independent of the dataset, allowing users retrain fine-tune our model based their specific datasets. However, the scarcity of large, high-quality datasets has long posed a challenge in the MIR community. addition the pre-training and transfer learning mechanisms used in this paper, unsupervised domain alignment methods also present a potential paradigm, requiring only the music in the target domain without the need for light ground truth.
This paper introduces a generative model for lighting control, framing it creative task. However, discussed in Appendix D the model occasionally shows overly strong local fluctuations, which suggest that other popular generative paradigms and technologies deserve exploration. Additionally, emerging techniques such Re- inforcement Learning from Human Feedback (RLHF) (Ouyang et al. 2022) and transferring from general music foundation models (Suno: Yuan et
Alternative Generation Technology Exploration:
2025 may provide promising directions for
enhancing model capability.
Towards More Practical Application Scenarios: Similar most previous studies, our method currently supports only offline primary lighting generation. Future research could explore real- time and multi-light control. However, these upgrades face significant challenges. For multi-stage control, the heterogeneity among venues must be considered, as different livehouses often have varying light positions and quantities. Regarding online control, the primary challenge is the need to predict changes in music quickly and accurately, as light control decisions must be made within seconds. However, accurately predicting a song's 's progression difficult due to its artistic nature, where unexpected elements contribute to the charm of music.
Richer Modality Incorporation: In our model, is also possible to incorporate richer modalities, such as rhythm. musical sections (like verses and choruses), and lyrics. Models such as beat tracking (Chang & Su 2024) and melody extraction (section classification) (Zhao) 2025 can also be integrated to provide more informative signals.
CONCLUSION
In this paper, we propose the first end-to-end deep learning method, Skip-BART, for stage lighting generation. We modify the BART model enable the encoder extract music features in sin- gle pass and the decoder to generate light hue and intensity auto-regressively. framing ASLC as a generative task. A skip connection mechanism is introduced enhance the relationship between corresponding music and light frames. Additionally, we incorporate effective deep learning tech- niques to allow the model learn from limited data, including transfer learning, pre-training, and RSTC sampling. To support research in this field, we present dataset construction method based on raw video and release the first stage lighting dataset, RPMC-L2 Through quantitative analysis and human studies, we demonstrate the efficiency of our method and highlight the limitations of con- ventional rule-based approaches. Our Skip-BART shows performance closely matching that of real human lighting engineers. The results validate our perspective that music lighting control resembles an art generation task rather than a simple classification and mapping task based on predefined rules. This study aims to provide deeper insights and support for future research in this area.
10

---

Published as a conference paper at ICLR 2026
ACKNOWLEDGEMENT
We are grateful to the participants who volunteered for our human study and the anonymous review- ers who helped improve the quality of this paper. We also thank Yunzhong Luo for helping with proofreading the manuscript.
ETHICS STATEMENT Our study did not use any privacy data containing personally identifiable information; informa- tion from human evaluation participants was anonymized, and only essential background data were collected for statistical and analytical purposes. At the same time, we recognize that the generated lighting may involve high flicker frequencies, strong color contrasts, or rapid brightness changes, which could pose health or discomfort risks to individuals with photosensitive epilepsy or visual sensitivities.
REPRODUCIBILITY STATEMENT
The source code, trained parameters, and processed datasct are available at battps://githhe com Httmm//2002//kiipp-1
REFERENCES
Anna Alajanki, Yi-Hsuan Yang, and Mohammad Soleymani. Benchmarking music emotion recog- nition systems. PloS one, pp. 835-838 2016.
Esben Oxholm Bonde, Ellen Kathrine Hansen, and Georgios Triantafyllidis. Auditory and visual based intelligent lighting design for music concerts. Eai Endrosed Trasactions on Creative Tech- nologies, 5(15)=e2, 2018.
Chih-Cheng Chang and Li Su. Beast: Online joint beat and downbeat tracking based on streaming transformer. In ICASSP 2024-2024 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp.
IEEE, 2024.
Chuer Chen, Shengqi Dang, Yuqi Liu, Nanxuan Zhao, Yang Shi, and Nan Cao. Mv-crafter: An intelligent system for music-guided video generation. arXiv preprint arXiv:250..17267, 2025.
Tingwei Chen, Xiaoyang Li, Hang Li, and Guangxu Zhu. Deep learning- based fall detection using commodity wi-fi. Journal of Information and Intelligence, 2(4): -64,, 2024.
Yi-Hui Chou, Chen, Chin-Jui Chang, Joann Ching, Yi-Hsuan Yang, et al. Midibert-piano: Large- scale pre-training for symbolic music understanding. arXiv preprint arXiv:2/07.05222 2, 2021.
Aurora Linh Cramer, Ho-Hsiang Wu, Justin Salamon, and Juan Pablo Bello. Look, listen, and learn more: Design choices for deep audio embeddings. In ICASSP 2019-2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 1852-3855 IEEE, 2019.
Matthias Erdmann, Markus von Berg, and Jochen Steffens. Development and evaluation of a mixed reality music visualization for live performance based music information retrieval. Frontiers in Virtual Reality, 6: 1552321, 2025.
J Michael Gillette and Michael McNamara. Designing with light: an introduction to stage lighting. Routledge, 2019.
Sepp Hochreiter and Jürgen Schmidhuber. Long short-term memory. Neural computation, 9()): 1735-1780, 1997.
Shih- Wen Hsiao, Shih-Kat Kai Chen, and Chu-Hsuan Lee. Methodology for stage lighting control based music emotions. Information sciences, 412:14-35, 2017.

---

Published as a conference paper at ICLR 2026
Wen-Yi Yi Hsiao, Jen- Yu Liu, Yin-Chen Yeh, and Yi-Hsuan Yang. Compound word transformer: Learning to compose full-song music over dynamic directed hypergraphs. In Proceedings of the AAAI Conference Artificial Intelligence, volume 35, pp. 186, 2021.
Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu Yuanzhi Li, Shean Wang, Lu Wang, Weizhu Chen, et al. Lora: Low-rank adaptation of large language models. ICLR, 1(2):3, 2022.
Marina Kanno and Yoshihisa Fukuhara. Automatic stage illumination control system by impres- sion the lyrics and music tune. In 2022 13th International Congress on Advanced Applied Informatics Winter MAAA--AI-Wintery pp- 219-224, IEEE, 2022.
Jun Lei, Meng Chen, Zexi Wang. Ying Wang, and Juncong Lin. Music-driven lighting manipulation for stage performance visual design. Twelfth International Conference on Graphics and Image Processing (ICGIP 2020), volume 11720, pp. 646-655. SPIE, 2021.
Mike Lewis. Bart: Denoising sequence-to-sequene pre-training for natural language generation, 2019.
translation, and comprehension. arXiv preprint
Xiao Liang, Zijian Zhao, Weichao Zeng, Yutong He, Fupeng He, Yiyi Wang, and Chengying Gao. Pianobart: Symbolic piano music generation and understanding with large- scale pre-training. In 2024 IEEE International Conference on Multimedia and Expo (ICME), pp. 1-6. IEEE, 2024.
Yu-Ping Liao, Da-Chuan Chen, and Bo-Heng Chen. Automatic visual effect adjustment system. In 2023 International Automatic Control Conference (CACS), pp. 1-6. IEEE, 2023.
Xichu Ma, Varun Sharma, Min- Yen Kan, Wee Sun Lee, and Ye Wang. Keyrie: Unsupervised keywords extraction and expansion from music for coherent lyrics generation. ACM Transactions on Multimedia Computing, Communications and Applications, 21(1):1-28 2024.
Phattalceya Mabpa, Tirasak Sapaklom, Ekkachai Mujjalinvimut, Jakkrit Kunthong, and Piyasawat Navaratana Na Ayudhya. Automatic chord recognition technique for music visualizer applica- tion. In 202/ 9th International Electrical Engineering Congress (iEECON), pp. 416-419. IEEE, 2021.
James McDonald, Sergio Canazza, Anthony Chmiel, Giovanni De Poli, Ellouise Houbert, Mad- dalena Murari, Antonio Rodà, Emery Schubert, and Diana Zhang. Illuminating music: Impact of color hue for background lighting emotional arousal in piano performance videos. Frontiers in Psychology, 13:828699, 2022.
Chang Bae Moon, HyunSoo Kim, Dong Won Lee, and Byeong Man Kim. Mood lighting system reflecting music mood. Color Research & Application, 20(2):201-212, 2015.
Niels A Nijdam. Mapping emotion to color. Book Mapping emotion to color, pp. 2-9, 2009.
Shuteng Niu, Yongxin Liu, Jian Wang, and Houbing Song. A decade survey of transfer learning (2010-2020). IEEE Transactions on Artificial Intelligence, 1(2):151-166, 2021.
Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al. Training language models to fol- low instructions with human feedback. Advances neural information processing systems, 35: 27730-27744, 2022.
Sinno Jialin Pan. Transfer learning. In Data Classification, pp. 565-598. Chapman and Hal//CRC 2014.
Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, et al. Pytorch: An imperative style, high- performance deep learning library. arxiv 2019. arXiv preprini arXiv:1912.01/ 10, 1912.
Alec Radford, Karthik Narasimhan, Tim Salimans, Ilya Sutskever, et al. Improving language under- standing by generative pre-training. 2018.
12

---

Published as a conference paper at ICLR 2026
Julian- Dan Stanescu, Bogdan-Adrian Enache, George-Calin Seritan, Sorin-Dan Grigorescu, Florin- Ciprian Argatu, and Colis-Constnntin Adochiei. Automatic control system for stage lights. In 2018 International Symposium Fundamentals of Electrical Engineering (ISFEE), pp. 1-4.
IEEE, 2018.
Inc. Suno. Suno. URL https: suno. com
Zeyue Tian, Zhaoyang Liu, Ruibin Yuan, Jiahao Pan, Qifeng Liu, Xu Tan, Qifeng Chen, Wei Xue, and Yike Guo. Vidmuse: simple rideo-to-music generation framework with long-short-term modeling. arXiv preprint MKKi:22406.03221,, 2024.
Simon Borst Tyroll, Daniel Overholt, and George Palamas. Avai: A tool for expressive music visualization based on autoencoders and constant q transformation. In /7th Sound and Music Computing Conference, pp. 378-388 -888. Axea sas/SMC Network, 2020.
Aaron Van Den Oord, Oriol Vinyals, al. Neural discrete representation learning. Advances in neural information processing systems, 30, 2017.
N Vryzas, A Liatsou, Rigas Kotsakis, Charalampos Dimoulas, and George Kalliris. Augmenting drama: A speech emotion-controoledd stage lighting framework. Proceedings of the 12th inter- national audio mostly conference on augmented and participatory sound and music experiences, pp. 1-7, 2017.
Le Yu, Bowen Yu, Haiyang Yu, Fei Huang, and Yongbin Li. Language models are super mario: Ab- sorbing abilities from homologous models as free lunch. In Forty-first International Conference on Machine Learning, 2024.
Ruibin Yuan, Hanfeng Lin, Shuyue Guo, Ge Zhang, Jiahao Pan, Yongyi Zang, Haohe Liu, Yiming Liang, Wenye Ma, Xingjian Du, et Yue: Scaling open foundation models for long-form music 2025.
generation. arXiv preprint 2503vv2250..0055
Zijian Zhao. Let network decide what learn: Symbolic music understanding model based on large-scale adversarial pre-training. In Proceedings of the 2025 ACM International Conference Multimedia Retrieval, 2025.
Zijian Zhao, Fanyi Meng, Hang Li, Xiaoyang Li, and Guangxu Zhu Mining limited data suffi- ciently: A bert-inspired approach for esi time series application in wireless communication and sensing. arXiv preprint arXiv:2412.06866 2024.
Hongyuan Zhu, Ye Niu, Di Fu, and Hao Wang. Musicbert: self-supervised learning of music representation. In Proceedings of the 29th ACM International Conference Multimedia, pp- 3355-3963,25 2021.
13

---

Published as a conference paper at ICLR 2026
APPENDIX
Pre-training Details
15
Detailed Experiment Setup
15
Dataset Details
16
Overview
16
C.2
Audio Processing
16
Lighting Processing
17
C.4
Overall Dataset Settings
18
Multiple Hue Extraction
18
D Visualization Results
19
Human Study Details
20
Study Setup
20
E,2
Questionnaire Content
21
Data Aggregation
22
Significance Analysis
23
Cross Domain Evaluation
23
14

---

Published as a conference paper at ICLR 2026
A PRE-TRAINING DETAILS
In this section, we aim to provide additional details regarding the input and output, the neural network architecture, and the hyper-parameters involved in the pre-training phase, ensuring repro- ducibility.
During pre- vre-training, in order to avoid any potential information leakage, utilize only the music data without light. To provide the decoder with a robust starting point for fine-tuning, we adopt BART-based MLM approach. In this manner, the encoder encodes the masked music sequence, while the decoder receives the shift-right ground truth as input and attempts to accurately recover the original music sequence. Although there may be a gap between the music modality in pre- training and the light modality in fine-tuning, this pre-training method has been demonstrated to be effective, as supported by the theory of cross-modal transfer learning (Niu al. 2021). and it has been successfully applied in numerous domains (Chen et al., 2024).
Specifically. in each epoch, for a given embedding sequence {c1, e2. ,n. we first randomly mask some of the embeddings to generate the input for the encoder, denoted as ē (ě1, 02, ....... ,..) replaced with random [MASK] token. Otherwise, retains its original value
If é is chosen, e.. For the decoder, outputs the hidden features f = {f1, ...,............ }. These features are then fed into a regressor (a simple three-layer MLP with LeakyReLU input sequence, = {êg, é....., én }. Additionally, following the recommendations of Zhao et al. 2024), introduce discriminator to enhance the realism of the recovered music sequence. The discriminator takes the original music sequence e and the recovered sequence as input, attempting differentiate between them, while the decoder works to obfuscate this distinction. For the structure of the discriminator, we utilize sequence- classifier from MidiBERT (Chou et al. 2021). which employs a modified self-attention mechanism:
which the shift-right ground truth input, {[SOS], 61, e2..... (2....,e--1] cx-1).
the activation function) to generate the estimated
Maun = Softmax(LincaLLayer(( E R7xd
Flatten e) Rad
(10)
y Softmax (oftmax((nnrrLaaeer()) R
where Mann is the attention matrix, h represents the feature dimension of the input, d is the hidden dimension, and y yields a binary classification result.
Lastly, regarding the hyperparameters. we set the mask ratio as random number sampled from U5. 0.30) in each epoch. The weights $1. 32, and B3g in the loss function (Eq. 4 are set to 0.8, 0.2, and 0.1, respectively.
B DETAILED EXPERIMENT SETUP
The experimental subjects can be summarized as follows:
Ground Truth (GT): The ground truth in our dataset is derived from real lighting engineers. In the human evaluation, consider it specific method developed by humans.
Skip-BART: This refers to our proposed method.
Ablation Study: In comparison to conventional BART, our method's 's variants primarily focus on the skip connection module, the pre- pre-training and transfer learning mechanisms, and the embedding layer. In the ablation study, we assess the model's performance without these mechanisms. We denote them (i) Skip-BART (w/o skip connection): The model without the skip connection; (ii) Skip-BART (from scratch): The model trained from scratch, without any prior knowledge from transfer learning or pre-training; (iii) Skip-BART (w/o light embedding): The model that does not utilize the proposed discrete light embedding but instead employs MLP-based continuous embedding layer for light; (iv) Skip-BAR' (pre-train w/o random [MASK]): During pre- vraitrainnng, the [MASK] token is fixed instead of being randomly assigned as defined in Eq, 2 (v) Skip- BART (pre-train w/o discriminator): The model that does not utilize a discriminator during pre-training, i.e., 33 in Eq. is is set to 0.
Rule-based Method: In reproducing previous methods, we encountered several challenges, as most existing works do not release their code and parameters. Even when attempting implement
15

---

Published as a conference paper at ICLR 2026
them independently, the unreleased predefined light patterns made it impossible to map classifica- tion results to their corresponding lighting control outputs. We follow the core idea of Hsiao et al. (2017). which maps music to emotional categories and infers specific lighting patterns based embeddings in emotional space. For emotion classification, we adopt the method of Aljankii et al. (2016), training bidirectional Long Short- Term Memory (LSTM) model (Hochreiter & Schmid- huber 1997) on their published dataset, DEAM. For light mapping, we utilize the approach of Nijdam (2009). taking into account both hue and value.
For our Skip-BART model, the detailed configuration in our experiment are shown in Table with the backbone aligned with PianoBART (Liang et al. 2024).
Table Model Configurations: This table provides comprehensive overview of our Skip-BART settings used in experiments.
Configuration
Our Setting
Vocabulary Size for Hue and Value
[180. 256]
Lighting Embedding Dimension
512
Music Embedding Dimension
512
Input Length
1024
Number of Network Layers
Hidden Size
2048
Inner Linear Size
2048
Attention Heads
Dropout Rate
0.1
Total Number Parameters
240 M
Trainable Parameters
19 M
Optimizer
Adam W
Learning Rate
0.0001
Batch Size
GPU Memory Usage
18 GB
Pre-training Duration
15 hours
Fine-tuning Duration
1.5 hours
C DATASET DETAILS
C.I OVERVIEW
The dataset was collected from 35 live performances various commercial venues between Decem- ber 2020 and July 2024 by the authors. After data cleansing, videos shorter than 20 seconds were discarded, resulting
total of 699 retained samples. The samples primarily encompass music styles such as generalized rock, punk, metal, and core. To address copyright concerns and enhance usability, we provide extracted processed features instead of the original video files, resulting in approximately 40GB of HDF5 files.
C.2 AUDIO PROCESSING
Various methods were employed to process the audio data, including:
OpenL3 (Cramer et al. 2019): A deep neural network architecture for extracting high-level auditory embeddings through audio-visual correspondence learning.
Time-frequency representation employing triangular filter banks spaced ac- cording to the mel scale, approximating human auditory perception.
Mel Spectrogram:
Log-Mel Spectrogram: Decibel-scaled variant of the mel spectrogram enhancing the discernibil- ity of low-amplitude acoustic components through logarithmic compression.
16

---

Published as a conference paper at ICLR 2026
Constant-Q Transform (CQT):
Time freeufreyynncy analysis technique utilizing geometrically spaced frequency bins with constant quality factors, enabling multi-resolution spectral decom- position.
Short-Time Fourier Transform (STFT): Fundamental Lime-localized Fourier analysis employ- ing a sliding window function for stationary signal approximation.
Mel- hel-Frequency Cepstral Coefficients (MFCCs): Cepstral representation derived from mel-scale filter bank energies, optimized for speech recognition through deconvolution of spectral compo- nents.
Chroma STFT: Pitch-class energy profile computed from STFT magnitudes, representing har- monic content in the standard equal-tempered scale.
Chroma CQT: Enhanced chroma representation leveraging the frequency resolution advantages CQT for improved musical note analysis.
Chroma CENS: Energy- norgyl-nomalizee chroma variant incorporating temporal smoothing and loga-
rithmic compression for robust harmonic pattern recognition.
Spectral Centroid: Second-order spectral moment quantifying the center of spectral mass, corre- lating with perceptual brightness.
Spectral Bandwidth: Fourth-order spectral moment characterizing signal dispersion about the centroid, indicative of timbral richness.
Timbral descriptor quantifying the dynamic range between spectral peaks and troughs across sub-bands.
Spectral Contrast:
Spectral Rolloff: Cutoff frequency containing 95% of spectral energy distribution, serving harmonicity indicator.
Temporal-domain measure of signal oscillation rate, particularly effective for discriminating voiced and unvoiced speech segments and percussive transients.
Zero- Crossing Rate (ZCR):
The respective feature dimensions are detailed in Table 5
Table 5: Audio Features Dimension:L refers to the sequence length of audio.
Feature
Dimension
OpenL3
512, L)
Mel Spectrogram
128, L
Mel Spectrogram (dB)
128, L)
CQT
84, L)
STFT
(0255, L)
MFCC
128, L
Chroma Stft
12, L
Chroma Cqt
12, L)
Chroma Cens
12, L
Spectral Centroids
1, L)
Spectral Bandwidth
1,
Spectral Contrast
7, L
Sspectral Rolloff
1,
7ero Crossing Rate
1, T.)
C.3 LIGHTING PROCESSING
To facilitate the analysis of lighting conditions, the lighting information is preprocessed in the HSV color space. The HSV color space separates the brightness (Value) information and the color (Huc) information, which makes the control and generation of lighting more intuitive, independent, and stable. Specifically, we set the Saturation (S) channel to a constant value of 255 (100%) to simulate
TTo avoid ambiguity: we Value V to refer the V component in HSV, and value to represent its numerical value here.
17

---

Published as a conference paper at ICLR 2026
the lighting effect and counteract environmental influences commonly encountered in livehouse set- tings, ensuring that after factors such as air seattering, fog, or surface reflections reduce color purity, the light maintains vividness and minimizes distortions like fading or whitening. The Hue (H) and Value (V) channels are extracted frame-hy-frame from the video, with H ranging from [0, 179] (H is eyclic) and V ranging from [0, 255]. For each frame, compute the distributions (number of pixels for each value) of H and V. To avoid Hue distortion caused by low light intensity, H is extracted only from pixels where exceeds specified thresholds, set to values in the range {0, 30, 60, 90, 120, 150, 180, 210, 240}. Users can select the threshold values that best suit their specific applications.
For this study, in order obtain the audio sequences Sa- we segment it into overlapping frames {#1, 22,..., } with window length of second and a hop size of 0.1 second. To extract the pri- mary light from each image, we first convert each frame into the HSV color space. After flattening, each frame yields a set of (h, 8, tuples corresponding to its pixels. As described previously, we fix the saturation component 8, and therefore omit in subsequent description. We focus on the hue and value components and compute the histogram representations Hn, E N1800 and Hu E N256 for frame f as follows:
K
(Hn.)m Σ
1 2
1 = mm, m 0, 1779, 2.
k=1
(11)
K
2 (Hup)n = Σ
1102 =n), n= O,..., 255.
km
where 1{-} denotes the indicator function, h E {O..... 179} and of E {o..... ,255] are the hue
and value of pixel & in frame t, K is the number of pixels in frame t, and v' (O...., 255) is threshold used to exclude low-value pixels.
We define the tokenized hue the mode of the hue distribution, and the tokenized value as the weighted average of the value distribution for frame t:
h Mode (
y: = [he, F7 E Z2
where
M (n 1) (Hu,))
(12)
V7 Round
255 (H,,t)n
Here, we use the mean value to compute the tokenized intensity, and mode to compute the tokenized hue, as hue a circular variable. This results music sequence x = {ai, #2.... .T) and the corresponding light sequence {y1, ,2...., yr We consider the music sequence as input and the corresponding light sequence y as output.
C.4 OVERALI DATASET SETTINGS
As for the overall dataset setting for experiments, we set the sampling rate to 10 Hz, which means we obtain 10 frames for each second, and our model can process sequences up 102.4 seconds long. For samples exceeding this length, we randomly select a 102.4-second segment from the entire sequence. We split the dataset into training, validation, and testing sets in an 8:1:1 ratio. To avoid any potential information leakage, we ensure that all shows in the testing set are not included in the training and validation sets. For pre-training, we only use samples from the training and validation sets. In the evaluation, we compare the performance of different models on the testing set.
C.5 MULTIPLE HUE EXTRACTION
To support the next step multiple light control, we first propose multiple hue extraction method from single frame, which can provide ground truth for future research. We recommend using Von Mises Mixture Model (VMM) to extract the periodic structure from hue data, which naturally lies on the unit circle. The probability density function of the Von Mises distribution is defined as:
1
f(x k. =
cos(x - -1]],
(x)
18

---

Published as a conference paper at ICLR 2026
14
Raw distribution
Fitted VMM
12
Hue 1
Hue 2
10
Density
8
6
4
2
0
36
72
144
180
Figure 4: Visualization of VMM Fitted to Hue Data: The method enables the decomposition of mixed Hue distribution into several color components.
where μ is the mean direction, is the concentration parameter, and la denotes the modified Bessel function of the first kind of order zero.
A VMM models the hue distribution weighted sum of K such components:
p(x) Σ
F ( pks Kk),
where Ty are the mixture weights satisfying Σσ Fh 11
Given sampled values from the hue distribution input, the model outputs the parameters of cach Von Mises component. Parameter estimation is performed using the Expectation-Me PopotationnMMxxiizzaiion (EM) algorithm, analogous that used in Gaussian Mixture Models. In the absence of prior knowl- edge about the number of hue clusters, we evaluate a set of candidate models with varying compo- nent numbers and select the optimal one based the Bayesian Information Criterion (BIC).
sample frame from a real-world stage lighting sequence visualized in Fig. 4 It successfully decomposes the mixed hue distribution into two distinct color components, making it promising submodule for future studies.
D VISUALIZATION RESULTS
To provide an intuitive understanding of our model's performance on the stage lighting generation task, we visualize the output of the Skip-BART model, ground-truth and other two methods lighting control sequences, as shown in Fig. 5a and Fig. 5b Each example consists of the following four parts:
Ground Truth (GT): The ground-truth lighting sequence at each time frame. The color of each column directly represents the actual stage lighting color and brightness that moment.
Skip-BART: The lighting sequence predicted by the Skip-BART model.
19

---

Published as a conference paper at ICLR 2026
Ablation Study: The lighting sequence predicted by the BART model.
Rule Based: The lighting sequence predicted by rule based method.
For Ground Truth, the color of each column directly represents the actual stage lighting color and brightness at that moment. For other three models, the color of each column reflects the model's prediction of lighting color and brightness.
Noletion Stuby
Truch
Skip Ground
Ground
Rule Abletion
Rule Bosed
Skip BART
20
40
0
20
40
20
40
60
80
20
40
60
80
Frome
Frame
Frame
Frome
(a) Example
(b) Example 2
Figure 5: Visualization of lighting sequences generated by different methods. The top row shows the input Mel spectrogram, the middle row is the ground-truth lighting sequence, and the bottom row is the predicted sequence from the Skip-BART model. Each color represents unigue combination of lighting color and brightness over time. The red box (b) highlights a representative segment where Skip-BART closely matches the temporal lighting structure of the ground truth.
In addition, present a typical case in Fig. 5b where the red box highlights a segment transition that Skip-BART successfully identifies, aligning well with the ground truth. Following this transi- tion, both sequences exhibit brighter light than before. Despite these strengths, Skip-BART still has certain limitations. It occasionally exhibits overly strong local fluctuations and fails to maintain the long-tend rhythmic stability observed in the ground truth. This suggests that while the model handles short-term dynamics effectively, its global temporal structure modeling remains an area for improvement. The ablation variant, while visually distinct in color, still demonstrates some ability to follow the underlying temporal structure. In contrast, the rulc-based method, relying solely on emotion mapping, tends to produce slow, smooth transitions, typically from blue to cyan to green, which results in outputs that lack the realism and complexity of human-designed lighting.
Overall, Skip-BART built through transfer learning and pre-training so learned latent music representations. Also, it is supervised trained, which means it naturally learned the lighting patterns present in the data. Similar most generative models, also incorporates controllable randomness through the temperature parameter. This allows the model to move beyond a single deterministic output and introduce variation that may appear some "creativity".
E HUMAN STUDY DETAILS
E.
STUDY SETUP
We recruited 38 respondents through social media platforms and live music venues, with the human evaluation spanning half a month. The participants included 20 males and 18 females aged 18-55 (predominantly 18-30), among whom had professional experience in lighting design, music pro- duction, stage art, and over half were frequent live music attendees. All participants reported normal hearing, normal or corrected-to-normit vision, and no history of color blindness or color weakness.
The questionnaire was structured as follows:
3 music pieces: Each sampled randomly from the testing set, lasting 20 seconds.
4 objects per group: Ground Truth, Skip-BART, Skip- -kip-BART (w/o skip connection) (i.c., Ablation Study), and Rule-based.
20

---

Published as a conference paper at ICLR 2026
6 evaluation dimensions: Emotional Match Between Lighting and Music, Visual Impact, Rhyth- mic Synchronization Accuracy, Smoothness of Lighting Transitions, Immersive Atmosphere In- tensity, and Innovative Surprise (Erdmann et al. 2025
Each music piece constituted one group, including four videos corresponding to the four objects, with the music serving as background sound and a dynamic color block representing the light changes. Within each music group, the four videos were presented in a randomized order elimi- nate order effects. Participants were required to rate each video under each music group on the six dimensions using 7-point Likert scale (1 very dissatisfied, 7 very satisfied).
E.2 QUESTIONNAIRE CONTENT
Livehouse Stage Lighting Experience Questionnaire
Dear Respondent: We are researchers conducting study on Livehouse stage lighting experiences. Thank you for participating in this survey, which will take approximately 15 minutes to complete. Your responses will be kept strictly confidential and used solely for statistical analysis.
PART 1:
DEMOGRAPHICS
Q1. What is your age?
Under 18
0 18 24
D 25-30
31-40
41-55
51 and above
Q2. What is your gender?
Male
D
Female
Other
Q3. Current occupation:
Q4. Highest education level:
High school C Junior college
Bachelor's M Master's above
Q5. Professional expertise (multiple):
Lighting design
D Music production
Stage art None
PART 2: MUSIC BEHAVIORS
Q6. Frequently
listened genres (multiple):
Rock Electronic O Hip-hop
Pop
Metal
Jazz
Classical Other:
Q7. Livehouse attendance frequeney:
Weekly+ 0 2-3/month
O Monthly
Quarterly
O Biannually-
Q8. Openness new experiences:
1 (Very conservative)- - 2 4 4 6 (Very proactive)
Q9. Lighting impact perception:
1 (Very little) 2
6 (Very much)
PART 3: LIGHTING EVALUATION
Imagine you are Livehouse venue. Rate the following groups of lighting effects based on your simulated experience. Each group contains 4 videos (20 seconds each) with the same background music but different lighting models.
Instructions:
Q10-Q21.
(Group X, Music Track Y) Please rate the Livehouse stage lighting effects based on the following 6 dimensions:
21

---

Published as a conference paper at ICLR 2026
0b1893aQt
/ 0:20
Figure 6: Video Interface Example: The upper-left labels serve internal markers for video ran- domization to aid identification, with respondents unaware of their purpose.
Evaluation Dimension
Rating Scale (I=Lowest, 7-Highest)
1. Emotional music-light match
1
2
4
5
7
Visual impact intensity
1
2
4
5
Rhythmic synchronization
Transition smoothness
1
2
4
5
7
5. Immersive atmosphere
1
2
4
5
7
Innovation surprise
1
4
5
7
Here (1,2,3) represents the three music groups, and Y E f1, 4] represents the four objects, including Ground Truth (GT), Skip-BART (Ours), Skip-BART (w/o skip connection), and the Rule-based method. For simplicity, we refer to Skip-BART (w/o skip connection) as Ablation Study in the appendix. To ensure fairness, within each group, the different models are presented in random orders for each audience. Additionally, the structural annotations (e.g. demographics, group divisions, and technical model names) served as internal organizational markers and were not displayed to respondents.
PART 4: CONCLUSION AND ACKNOWLEDGMENT
Q22. Thank you for taking the time to participate this survey! Your feedback is invaluable to us. If you wish, please leave your contact information (e.g., email, phone number) for potential follow-up communication:
E.3 DATA AGGREGATION
The composite variables were systematically constructed through mean aggregation of multidimen- sional scores across experimental conditions. For each lighting pattern's 's dimensional evaluation, final composite scores were calculated by averaging metric performances from three distinct back- ground music groups. As exemplified in the main text, the notation (Ground Truth, Emotion) rep- resents the composite variable for the Ground Truth lighting pattern's emotional match between the lighting and music dimensions, derived through mean aggregation scores across all three background music conditions. Similarly, (Ground Truth, Overall) denotes the omnibus composite variable for Ground Truth, constructed by grand mean aggregation across all six evaluation dimen- sions and three musical conditions. This standardized mean- based composite variable construction methodology was uniformly applied to all experimental lighting patterns to ensure comparative con- sistency.
22

---

The results, shown in Table 5, demonstrated that the scores of the other three methods were sig- nificantly higher than those the Rule-based method. terms of specific pairwise comparisons, the score of Ground Truth was significantly higher than that of Ours (with or without skip connec- tion) (p = 0.003). Although the Ground Truth score was higher than that of Skip-BART, and the statistically Published as a conference paper at ICLR 2026
To refine descriptive statistical analysis, the medians and plurality of the scores for the four objects across the six metrics are reported in T
Table 6: Median and Plurality Scores:
The bold text represents the best result, while the underlined text indicates the second-best result. For the plurality, we retain the highest score when multiple pluralities exist.
Emotion
Impact
Rhythm
Smoothness
Atmosphere
Surprise
Overall
Median
Ground Truth
4.33
4.67
4.67
4.67
4.67
4.50
4.58
Skip-BART
4.67
4.50
4.50
4.33
4.33
4.17
4.42
Ablation Study
4.33
3.83
4.67
4.33
4.17
3.33
4.11
Rule-based
3.17
2.33
2.17
2.50
2.17
1.67
2.33
Plurality
Ground Truth
4.33
5.33
4.67
5.00
5.00
5.00
5.17
Skip-BART
4.67
3.67
5.33
4.33
5.67
4.33
4.72
Ablation Study
4.67
4.00
4.67
4.33
5.00
4.67
4.61
Rule-based
4.00
1.00
1.00
1.00
1.00
1.00
1.00
E.4
SIGNIFICANCE ANALYSIS
Given that the study involved repeated measurements of four methods across multiple metrics by the same group of subjects, Repeated Measures ANOVA selected for statistical analysis. This choice was made to effectively analyze the within-subject dependencies and to illustrate the advantages of the four methods across six metrics. To control for Type errors that can arise from multiple comparisons, the Bonferroni correction method was employed to adjust the significance level.
The overall scores of the four methods were used as the dependent variable. The Mauchly sphericity test indicated that Mauchly's W 0.32, p 0.001, necessitating the application the Greenhouse- Geisser correction. The results revealed significant main effect of the four methods, F(1.7, 63) = 61, p < 0.001, 172 = 0.62, suggesting significant differences the overall scores among the four types of videos. Here, W denotes the Mauchly test statistic for sphericity, while signifies the probability value. The stands for the F- ratio in ANOVA, with the figures in parentheses indicating the degrees of freedom (numerator, denominator). and 1?2 serves the measure of partial eta squared effect size. Subsequently, post hoc paired t-tests with Bonferroni correction were conducted.
Skip-BART score higher than that of Ablation Study, these differences significant.
Following the analysis of overall scores, Repeated Measures ANOVA was also conducted on the scores of the four video types under six metrics. Similar to the previous procedure, post hoc paired I-tests with Bonferroni correction were performed. The pairwise comparison results are summarized in Table
E.5 CROSS DOMAIN EVALUATION
In this section, we provide the detailed statistical results of the cross-domain evaluation, as shown in Tables which include three songs of different styles and evaluations from 30 users. Notably, our method consistently achieves higher scores compared to rule-based methods, demonstrating significant improvements.
23

---

Published as a conference paper at ICLR 2026
Table 7: Pairwise Comparisons among Ground Truth (GT), Skip-BART (SB), Ablation Study (AS), and Rule-Based (RB): The symbols * and *** denote that the significance level (p) for the difference in means (AM) is less than 0.05 or 0.001, respectively.
Metries
Comparison
AM
SD
Metries
Comparison
AM
SD
Emotion
GT SB
-0.19
0.15
1.000
Impact
GT vs SB
0.09
0.13
1,000
GT vs AS
0.19
0.11
0,598
GT vs AS
0.70
0.15
< 0.001
GT vs RB
1.38
0.25
0.0019*$
GT vs RB
1.83
0.23
< 0.001
SB vs
0.14
AS
0.14
0.001 see
SB vs RB
1.57
0.23
SB RB
1.75
0.23
0.001 1888
AS vs RB
1.18
0.22
0,001
AS vs RB
1.13
0.18
0.001 999
Rhythm
GT vs SB
0.11
0.12
1.000
Smoothness
GT vs
0.30
0.14
0.234
GT vs AS
0.08
0.14
1.000
GT vs AS
0.19
0.16
1,000
GT RB
2.07
0.26
0.001**9
vs RB
2.06
0,23
0.001
-0.04
0.12
1,000
SB AS
1,000
SB
1.96
0.21
0.001
SB RB
1.76
0.23
0.001 0.001 #88-
AS vs RB
1.99
0.25
0.001
AS vs RB
1.87
0.20
0.001
Atmosphere
GT vs SB
0.17
0.14
1.000
Surprise
GT vs
0.51
0.16
0.014
GT vs AS
0.38
0.15
0.088
GT vs AS
0.84
0.16
< 0.001
GT RB
1.72
0.25
0.001
GT vs RB
1.99
0,23
< 0.001
vs
0.13
AS
0.318
SB
1.55
0.21
0.001
SB RB
1.48
0.21
0.001 998
AS vs RB
1.34
0.18
0,001
AS vs RB
1.15
0.17
0.001
Table Median and Plurality Scores of Cross-Domain Evaluation
Emotion
Impact
Rhythm
Smoothness
Atmosphere
Surprise
Overall
Median
Skip-BART Ablation Study
4.25
4.11
4.80
4.72
4.52
4.07
4.30
3.73
3,89
4.59
4.58
4.00
3.71
4.13
Rule-based
3.50
2.83
3.05
3.89
3.14
2.72
3.08
Plurality
Skip-BART
4.00
3.00
3.33
5.67
4.67
4.33
3.94
Ablation Study
3.33
3.00
4.67
4.00
4.00
3.33
3.61
Rule-based
3.33
3.00
3.00
4.67
3.00
3.00
2.56
Table 9: Pairwise Comparisons of Cross-Domain Evaluation
Metries
Comparison
AM
SD
Metries
Comparison
AM
SD
p
Emotion
SB AS
0.37
0.20
0.249
Impact
SB vs AS
0.24
0.17
0.45
SB vs RB
0.70
0.21
0.007ss
SB vs RB
1.13
0.23
< 0.001
AS vs RB
0.33
0.22
0.435
AS vs RB
0.89
0.18
< 0.001
Rhythm
SB AS
0.16
0.16
1,00
Smoothness
SB vs AS
0.19
0.10
0.21
SB vs RB
1.39
0.25
0.001
SB vs RB
0.73
0.21
0.004
AS RB
0.21
0.001
vs
0.54
0.047
Atmosphere
SB AS
0.37
0.17
0.12
Surprise
vs AS
0.12
0.14
1.00
SB RB
0.21
0.001 aes
vs
0.96
< 99a
AS RB
0.70
0.17
0.001 eee
0.83
0.22
0.002**
24


Reviews:
[{"review_id": "TM6TrhHdiv", "reviewer": "Reviewer_1ux2", "summary": "This paper proposes Skip-BART, the first end-to-end deep learning framework for Automatic Stage Lighting Control (ASLC), reframing the task from a rule-driven classification problem into a generative modelling problem.\nThe authors adapt BART to generate lighting hue and intensity directly from audio input, introducing a skip-connection mechanism to align music and light on a frame-wise level. They also introduce transfer learning (leveraging PianoBART), pre-training via a masked language modelling objective, and a novel Restricted Stochastic Temperature-Controlled (RSTC) sampling for inference.\nA new dataset, RPMC-L2, containing 699 stage lighting samples from live performances, is constructed for training and evaluation. Quantitative metrics (RMSE, MAE, corr(|∆|)) and a human evaluation with 38 participants demonstrate that Skip-BART significantly outperforms rule-based baselines and achieves performance statistically indistinguishable from professional lighting engineers", "questions": "See Weakness", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 4, "strengths": "• The introduction of Skip-BART with skip connections for frame-level alignment is technically sound and novel.\n\n• The method is well-motivated and supported by solid engineering: effective dataset construction, comprehensive ablations, and careful pre-training/fine-tuning strategies.\n\n• The paper is clearly structured, with diagrams and methodological details that enhance understanding.\n\n• This work opens a new research direction in artistic multimodal generation, bridging music information retrieval and stage performance automation.", "weaknesses": "• The dataset is domain-specific (rock/punk/metal), limiting generalization to genres such as pop, jazz, or classical. Cross-domain evaluation shows promise but remains narrow in scope.\n\n• The pre-training details (e.g., MLM masking ratios, discriminator architecture) could be clarified further for full reproducibility.\n\n• Real-time or multi-light control is not addressed, which would be crucial for production-grade systems.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "1YoHTScYKf", "reviewer": "Reviewer_PdJ6", "summary": "This paper presents an engaging and well-articulated exploration of automatic stage lighting control, framing the problem not as a fixed, rule-driven process but as a generative task. The authors propose a deep learning model, Skip-BART, that translates audio into light parameters and seeks to bridge the gap between traditional handcrafted lighting rules and creative, data-driven systems. Alongside the model, the paper introduces a dataset and outlines a methodology that combines pre-training, fine-tuning, and sampling procedures.", "questions": "- Could the authors elaborate on whether Skip-BART is capable of generating new or unexpected lighting behaviors, or whether it primarily reproduces stylistic patterns learned from the training data? The paper strongly supports the generative framing, but most evidence focuses on similarity to ground truth and outperforming rule-based methods. Clarifying how “generative” this process truly is (e.g. in terms of diversity or novelty) would strengthen the conceptual contribution.\n\n- The human study is clearly designed and statistically analyzed, with 38 participants evaluating lighting across six perceptual dimensions. Still, a brief summary of participant demographics and evaluation setup in the main text (currently detailed in Appendix D) would make the study easier to follow and lend more transparency to the “human-level performance” claim.\n\n- ​The cross-domain evaluation (using Suno-generated folk, R&B, and jazz samples) is a nice addition, but it could be discussed more explicitly in terms of limitations. For instance, how robust is the model to genres or performance settings that differ substantially from those represented in the RPMC-L2 dataset? This reflection would clarify where the method generalizes well and where domain adaptation might be needed (paving the way and presenting a gap for further work).\n\n- ​While some constraints are mentioned in the discussion and appendices (e.g., rhythm\nstability and dataset scope), consolidating them in a concise section would help readers understand the boundaries of the approach. This could include points such as dataset bias toward rock/metal performances, limited exploration of multi-light setups, and future directions for broader aesthetic or cross-genre applications.", "limitations": "", "rating": 8, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "I find the work’s concept compelling. The authors do a good job explaining why rule-based and procedural systems struggle to capture the expressive and performative nature of lighting design. Framing lighting generation as a creative modeling problem rather than a deterministic optimization task opens promising ground between creative AI and live performance technology.\n\nThe interdisciplinary scope of the paper feels well balanced - it takes technical rigor seriously without losing sight of the artistic intent behind lighting. The proposed model and dataset are well described, which makes the work easy to follow and potentially reproducible. I also appreciate that the evaluation includes both quantitative metrics and human judgment, acknowledging that creative systems can’t be fully understood through quantitative metrics alone.\n\nThe results indicate that the model performs comparably to professional lighting engineers within the tested scenarios, providing convincing support for the paper’s overall argument.", "weaknesses": "Overall, the paper is strong, but I think the framing of its contribution could be clearer. The discussion of data limitations in prior work is well justified - the authors point out that existing methods rely on small, coarse-grained, or biased datasets, which constrain performance. However, this feels more like a practical limitation than a fundamental research gap. It would help if the paper made clearer whether its main contribution is technical (a new model that performs better) or conceptual (reframing stage lighting as a generative, creative process). It currently aims to do both, but the balance between these two dimensions isn’t always explicit.\n\nThe human evaluation adds a relevant perspective, and it’s great that the authors use both quantitative and perceptual measures. That said, readers would benefit from a brief summary of main participant details and evaluation criteria in the main text (the info is in Appendix D, but it would be better to have a glimpse of it earlier). The main section does report the sample size (n=38), number of pieces, and metrics used, but a concise overview of demographics and procedure would make the “human-level performance” claim easier to interpret at a glance.\n\nLastly, while the paper acknowledges some constraints in passing - such as occasional rhythm instability and the need for more long-term temporal coherence - these points are scattered across the main text and appendices. A short, dedicated Limitations and Future Work section would help consolidate them, making it clear where the approach currently struggles (e.g., dataset bias, dependence on specific musical genres, or limited cross-domain evaluation beyond music-driven contexts). At moments, the discussion leans heavily toward optimization details like sampling temperature and adaptive weighting. While technically relevant, this can “dilute” the creative framing that makes the generative perspective so compelling in the first place.\n\n\nOverall, I would recommend this work for acceptance once the points raised are addressed. The paper makes a meaningful contribution to the emerging conversation on generative modeling for creative control systems, presenting both a technical framework and an insightful reframing of stage lighting as a creative, data-driven process. Skip-BART is well-motivated and empirically supported and consistently surpasses rule-based baselines. With a few refinements (especially in clarifying the balance between the technical and conceptual contributions, summarizing key details of the human evaluation in the main text, and consolidating the discussion of limitations) I believe this work could serve as a strong reference point for future research on AI-assisted performance design and the broader intersection of generative models and creative practice.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "fhX2iGio8T", "reviewer": "Reviewer_JUmG", "summary": "This paper addresses the task of Automatic Stage Lighting Control (ASLC) by framing it as an end-to-end generative problem for the first time. The authors propose a novel end-to-end ASLC model, Skip-BART, and introduce a dedicated stage lighting dataset.", "questions": "- In the pre-training stage, is it the decoder or the regressor that is responsible for recovering the embeddings of the masked segments?\n- What are the targets of the decoder and the regressor during pre-training?\n- What is the architectural design of the regressor?", "limitations": "", "rating": 8, "confidence": 4, "soundness": 3, "presentation": 4, "contribution": 4, "strengths": "- The paper addresses an important task of the stage lighting generation.\n- The paper's contribution is significant by introducing the end-to-end ASLC task.\n- The paper is well written and easy to follow.\n- Thorough experiments showing the effectiveness of the proposed method.", "weaknesses": "- The description of the pre-training stage lacks sufficient detail (see the Questions section).\n- Some claims are not adequately supported by experimental evidence:\n    - Allowing the model to identify which tokens have been replaced by [MASK] assists the learning process.\n    - The role and impact of incorporating the discriminator\n- The method does not support real-time lighting generation. While not a critical limitation, real-time generation is a compelling use case for ASLC, particularly in live performances. The reliance on an encoder-decoder architecture restricts the model to offline processing, preventing it from handling streaming music input.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)