Photographic Lighting Design with Photographer-in-the-Loop
Bayesian Optimization
Kenta Yamamoto

Yuki Koyama

Yoichi Ochiai

University of Tsukuba
Japan

National Institute of Advanced
Industrial Science and Technology(AIST)Japan

University of Tsukuba,
R&D Center for Digital Nature,
Pixie Dust Technologies, Inc.
Japan

Figure 1: The proposed framework help the photographer find a good lighting configuration. Instead of manipulating lighting
parameters directly, the photographer needs only repeatedly selecting the best option from the ones suggested by the system,and providing a rough painting on the selected image to guide the search.

ABSTRACT
It is important for photographers to have the best possible lighting configuration at the time of shooting; otherwise, they needpost-processing on images, which may cause artifacts and deterioration. Thus, photographers often struggle to find the best possible
lighting configuration by manipulating lighting devices, including
light sources and modifiers, in a trial-and-error manner. In thispaper, we propose a novel computational framework to support
photographers. This framework assumes that every lighting device
is programmable; that is, its adjustable parameters (e.g., orientation,
intensity, and color temperature) can be set using a program. Using
our framework, photographers do not need to learn how the parameter values affect the resulting lighting, and even do not needto determine the strategy of the trial-and-error process; instead,photographers need only concentrate on evaluating which lighting
configuration is more desirable among options suggested by thesystem. The framework is enabled by our novel photographer-inthe-loop Bayesian optimization, which is sample-efficient (i.e., thenumber of required evaluation steps is small) and which can also
be guided by providing a rough painting of the desired lighting
configuration if any. We demonstrate how the framework works
in both simulated virtual environments and a physical environment, suggesting that it could find pleasing lighting configurations
quickly in around 10 iterations. Our user study suggests that theframework enables the photographer to concentrate on the lookof captured images rather than the parameters, compared with thetraditional manual lighting workflow.

CCS CONCEPTS• Computing methodologies → Graphics systems and interfaces.

KEYWORDSUIST ’22, October 29-November 2, 2022, Bend, OR, USA© 2022 Copyright held by the owner/author(s). Publication rights licensed to ACM.
This is the author’s version of the work. It is posted here for your personal use. Not forredistribution. The definitive Version of Record was published in The 35th Annual ACM
Symposium on User Interface Software and Technology (UIST ’22), October 29-November
2, 2022, Bend, OR, USA, https://doi.org/10.1145/3526113.3545690.

lighting design, human-in-the-loop optimization, Bayesian optimization, photography
ACM Reference Format:
Kenta Yamamoto, Yuki Koyama, and Yoichi Ochiai. 2022. Photographic
Lighting Design with Photographer-in-the-Loop Bayesian Optimization. In

UIST ’22, October 29-November 2, 2022, Bend, OR, USA

The 35th Annual ACM Symposium on User Interface Software and Technology(UIST ’22), October 29-November 2, 2022, Bend, OR, USA. ACM, New York,NY, USA, 11 pages. https://doi.org/10.1145/3526113.3545690

1

INTRODUCTION“A painting is complete when it has the shadows of agod.” — Rembrandt.

Considering a particular scene to capture, photographers make aconcerted effort to obtain an appropriate lighting configuration using various lighting devices to produce a pleasing and good-quality
photograph [18]. This effort is crucial, not only to best express
their artistic view, but also for their business success, particularly
if the image is for the commercial promotion of the subject (e.g.,a product or food). Furthermore, it is important to have the best
possible lighting configuration at the time of shooting; otherwise,
they need post-processing on images, which may cause artifacts
and deterioration.
However, lighting design at the time of shooting is not easy, especially for novice photographers. One reason is the complexity oflighting devices that they may need to use in combination (including
continuous and strobe lights, as well as modifiers such as reflectors,
umbrellas, barndoors, and softboxes); photographers need to be
familiar with the devices to understand how they might affect thelighting in their images. Another reason is the high dimensionality of the search space; each lighting device usually has several
adjustable parameters (e.g., position, orientation, and intensity),
and therefore the total dimensionality becomes intractably high fornovice photographers who do not have reasonable search strategies.
Moreover, photographers often do not have a specific lighting goalin their mind at first, making it necessary to experiment by trial
and error to understand the various lighting possibilities before
gradually forming a specific goal.
In this paper, we propose a novel computational framework
for lighting design in still-life photography (i.e., the shooting of
inanimate objects), as shown in Figure 1. This framework assumes
that every lighting device is programmable; that is, its adjustableparameters can be set using a program. Note that this situation has
already been popularized by DMX1 or IoT2 lights, and we believethat it will become even more common in studio photography
as programmed drones and robotic arms have increasingly beenutilized in cinematography. Using our framework, photographers
do not need to understand what parameters are available and how
these values affect the resulting lighting, and even do not needto determine how the trial-and-error process should go; instead,photographers need only concentrate on the appearance of theshooting result, evaluating which lighting configuration (as seen
through the camera lens) is more desirable.
This framework is enabled by a novel photographer-in-the-loop
optimization technique. Using this technique, the photographer
iteratively compares several lighting design options that the system
samples. Our technique is based on preferential Bayesian optimization (PBO) [4, 6, 13, 14], which is sample-efficient (i.e., the number ofrequired evaluation steps is small). Moreover, the sampling behavior
1 E.g., https://www.elationlighting.com/.
2 E.g., https://www.philips-hue.com/en-us.

Yamamoto et al.

can be guided by rough painting of a desirable lighting configuration, which can further accelerate the search. More specifically,
our technique is an extension of the one proposed by Chong et al.
[6], which was used in a similar workflow (i.e., the user iteratively
evaluates the options that the system samples based on PBO, andthe sampling behavior is guided by rough painting if necessary)and applied to deep generative image modeling. However, it is not
feasible to apply their technique to our lighting design scenario
directly because it requires the generation of many images withdifferent parameter sets (by using deep generative models in theirscenario and by manipulating lighting devices and shooting images
in our scenario) to determine the most effective next options ineach iteration. This is time-consuming in our case, and hence, wecannot use it interactively. Consequently, we propose integrating
a lighting prediction model into the PBO process to quickly inferapproximate lighting results.
We demonstrate how the framework operates in both simulated
virtual environments and a proof-of-concept physical environment,suggesting that it could find pleasing lighting configurations quickly
in only 10 iterations for problems ranging from 3 to 12 dimensions.
We also conducted a user experiment in which we asked participants to compare the proposed framework using a setup with lightsources and a reflector attached to robotic arms and the traditionalframework by manual adjustment, suggesting that our framework
could free photographers from being aware of raw parameters andmake them better concentrate on the look of captured images.
Our contributions are summarized as follows.
• We propose a novel photographic lighting design frameworkin which the system controls the lighting devices programmatically and allows the photographer to interactively finda good lighting configuration by focusing on the lit scenerather than directly manipulating the lighting parameters.
• We propose a novel photographer-in-the-loop lighting parameter search technique based on PBO, which is also capable of being guided by the photographer’s rough painting inputs. This technique is facilitated by extending the techniqueproposed by Chong et al. [6] such that it uses lighting prediction to avoid the time-consuming light-and-shoot process
when determining the next options, making our techniquesufficiently fast for interactive use.

2 RELATED WORK
2.1 Lighting Design
Lighting design systems have been explored for rendering computer
graphics scenes. Some systems enable lighting design via the direct
manipulation of highlights and shadows [24, 26]. Other systems
have painting interfaces for more intuitive operation; for example,
the position or intensity of the light source are derived from thepainting on a target scene [23, 27], and the environment map isgenerated from the painting input [21, 22]. Goal-based optimization
systems have also been proposed [7, 12]. Our work differs fromthese works in explicitly having the user in the search loop; ourframework allows photographers who do not have concrete goals
at first, to explore the design space by iteratively providing themwith reasonable options. Thus, we use the painting input as a guide,Photographic Lighting Design with Photographer-in-the-Loop Bayesian Optimization

Photographer Part

1. Selection

2. Painting

System Part

3. Search for Next Options

UIST ’22, October 29-November 2, 2022, Bend, OR, USAanswer absolute scores in perceptual evaluation scenarios [3, 33].
Our work is the first to apply PBO to photographic lighting design.
In particular, our technique is an extension of the technique proposed by Chong et al. [6]. Their technique is not directly applicableto our photographic lighting scenario in terms of the computational
speed; hence, we propose modifying their technique such that it
works reasonably fast in our scenario (see subsection 4.3).

2.3
Photographer-in-the-Loop

Figure 2: Workflow of the proposed framework. It consists ofthree steps: preference-based selection, painting, and search
for next options.

not an exact goal, to provide balanced options. Our work also differs
in its focus on real-world photography.
Researchers have also investigated lighting design methods forphotography in real scenes. Some studies have proposed new photographic lighting equipment by integrating lights with mechanically controlled devices, such as a steerable spotlight [17], a flashof whose direction is automatically controlled [19], and a lightattached drone [31]. A recent study proposed using 360-degree
cameras to guide lighting design [9]. Our work also uses advanced
lighting devices, but our contribution is suggesting how to usethem effectively in lighting design. Another popular approach isto synthesize desirable scene lighting in post-processing. Somestudies have proposed a method in which many images are taken
in advance under different lighting conditions and then synthesized [1, 2, 10]. Other studies have proposed a method in which a
deep neural network generates an image under arbitrary lighting
conditions when an original image and an environment map are
input to the system [32, 35]. This approach suffers from inevitableartifacts, and thus it is preferable to light the scene appropriately
at the shooting time to obtain artifact-free images.

2.2

Human-in-the-Loop Design Optimization

Human-in-the-loop optimization is an approach to solve optimization problems with “perceptual” objective functions. Typically, thesystem iteratively asks the human evaluator to perform a perceptual
task (e.g., pairwise comparison) until a good solution is found. For
example, Chiu et al. [5] used this approach for content generation
using deep generative models (i.e., finding an appropriate latentcode to generate desirable content).
Our photographer-in-the-loop technique is based on PBO, which
has been recently used in various design scenarios [3, 4, 6, 13, 14,36, 37]. PBO is an extension of Bayesian optimization (BO) [29],which is a global black-box optimization technique. Because of
its sample efficiency, BO has been used in optimization problems
with expensive-to-evaluate objective functions [8, 25, 34]. Unlike
BO, PBO uses relative evaluation among several options instead ofusing absolute evaluation of every individual design; this property
is important to us because it is difficult for evaluators to stably

Design Interfaces

Many interfaces for tweaking design parameters have been proposed thus far. Ours particularly follows the approach of exposing
the user to design candidates as opposed to design parameters, bywhich the user is free from the necessity of learning about the raw
parameters. The seminal work is done by Marks et al. [16], wheremultiple design options are visually presented to the user at once.
Later, this approach has been used in sequential settings [13, 20, 30],
where options are presented iteratively for refinement; we also takethis approach.

3 OVERVIEW
3.1 Lighting Devices
The proposed framework assumes that lighting devices are programmable. Lighting devices used in photography studios havevarious raw parameters (e.g., intensity, color temperature, position,and orientation), which have traditionally been adjusted manually. However, lighting devices that can adjust these parametersprogrammatically have become increasingly popular. For example,
stage lights can be programmed to adjust the colors and orientations1 . Using robotic arms, we can also programmatically adjust
the positions and orientations of lighting devices. Furthermore, researchers have proposed programmable light settings using drones[31] or servo motors [19] to programmatically control the lighting.
In our proof-of-concept lighting environment, a light source is
attached to the tip of a robotic arm so that the position and orientation can be adjusted (Figure 1). We also built simulated lighting
environments using Blender’s Cycles3 (a physically based pathtracer for production rendering), in which we could include morediverse lighting setups that could become more accessible in the
future to demonstrate the potential of our framework.

3.2

User Workflow

As shown in Figure 2, our framework constitutes an iterative processwith three steps: preference-based selection by the user, paint input
by the user, and search for the next options by the system. These
three steps are executed repeatedly until a satisfactory lighting
configuration is found.
(1) Selection. The system asks the user to select the most preferable option from the multiple ones that the system provides.
The options are visually displayed on the screen, so the comparison is easy. In our current implementation, we follow
the 4-gallery comparison strategy [3], that is, the system provides four options at once and asks the user to select thebest one. We choose this strategy for its simplicity in our
proof-of-concept implementation. Nonetheless, it is possible
3 https://www.cycles-renderer.org/

Yamamoto et al.

4 METHOD
4.1 Problem Formulation
Let the concatenation of all available lighting parameters in thetarget lighting environment beTx = 𝑥 1 · · · 𝑥𝑛 ∈ X,
(1)
where 𝑛 is the total number of the adjustable parameters. For example, a dimension of this vector may corresponds to the lightintensity of a lighting device, or the position of another lighting
device. For simplicity, we assume that the range of each parameter
is normalized into [0, 1]; that is, X = [0, 1]𝑛 .
Our goal is to find the optimal lighting parameter set that provides the best lighting configuration. Following the previous works[6, 13, 14], we assume the existence of a function that quantifies the“goodness” of a given parameter set (i.e., the goodness function). We
denote the goodness function as 𝑔 : X → R, where a larger value
indicates better. Thus, our goal is to find the optimal parameter set,x∗ , which is defined asx∗ = arg max 𝑔(x).

(2)

x∈X

Since the function 𝑔 is defined perceptually, we cannot directly
calculate its value. Thus, we need a human-in-the-loop approach
to solve this problem.

4.2

Preferential Bayesian Optimization

We use PBO with the photographer-in-the-loop setting to solve
the maximization problem described in Equation 2. In each iteration of PBO, the system provides the human evaluator a set of
options, and the human evaluator provides preferential feedback(i.e., which option is better than which) to the system. There havebeen proposed some variants of the form of options and preferential
feedback (e.g., [13, 14]), but we use 4-gallery comparison [3] becauseof its simplicity; the system always provides four options and thehuman evaluator selects the best one among the four options.
4.2.1 Without Paint-Based Guidance. We first describe our method
without taking the paint-based guidance into account. We will later
describe how the paints can be incorporated at subsubsection 4.2.2.
As for the strategy to determine the four options in each iteration,
we follow the method proposed by Brochu et al. [3]. That is, after

Ground Truth

to replace with more efficient strategies such as sequential
line or plane search methods (e.g., [13, 14]).
(2) Paint. Following previous works [23, 27], a paint interface isadopted to realize intuitive design input. In particular, theuser is asked to draw brush strokes with arbitrary colors
over the selected image. It is noteworthy that the paint canbe rough and incomplete because it is not an exact goal but
just a guide to bias the system-provided options.
(3) Search for Next Options. Based on the user’s selection andpaint, the system determines the next options. In our framework, the next four options are selected using the strategy
used by Brochu et al. [3]. That is, one is simply the optionthat the user selected last time, which is considered the “current best” design, and the other three are the new options
that the system carefully chooses by the PBO technique.

Prediction

UIST ’22, October 29-November 2, 2022, Bend, OR, USA

Figure 3: Examples of lighting prediction. Our method uses
predicted images (as fast approximations of shot images) todetermine the options for the next iteration. From left toright, the dimensionality is 4D, 5D, 8D, and 12D, respectively.
the 𝑖-th iteration, we choose one of the next options as the best
+ , andoption among those we have observed so far, denoted by x𝑖+1
we choose the other three next options as the ones that maximize
ac,1 ac,2 ac,3an acquisition function, denoted by x𝑖+1
, x𝑖+1 , x𝑖+1 .
Determining a single option. Before describing how PBO determines multiple options for the next iteration, for the purpose of
explanation, we first describe the case of involving only a single opac ) for thetion; that is, how PBO determines a single option (say, x𝑖+1
next iteration. Acquisition function is a function that evaluates how
effective an option is if it is observed in the next iteration (see [29]and our Supplemental Materials for details). Let 𝑎𝑖 : X → R be
the acquisition function at the 𝑖-th iteration, and let D𝑖 be the accumulated data so far provided by the human evaluator. The new
option for the next iteration is obtained by solving the following
maximization problem:ac
x𝑖+1= arg max 𝑎𝑖 (x) = arg max 𝑎(x; D𝑖 ).
x∈X

(3)

x∈X

Since this problem can have multiple local maxima, it is usually
solved by global optimization algorithms such as the DIRECT
method [11]. In our implementation, we combine the DIRECT
method with the L-BFGS method [15], which is a gradient-based
local optimization method, for computation efficiency. More specifically, we first perform the DIRECT method with 100 iterations to
obtain a reasonable solution, and then we use it as an initial solutionof the L-BFGS method to further refine it. In addition, while several
types of acquisition functions have been proposed so far [29], wechoose the expected improvement (EI) because it is one of the mostpopular choice (see Supplemental Materials for details).
Determining multiple options. Second, we explain how the aforementioned technique to determine a single option can be extended
to determine multiple options for the next query. We use Schonlau et al.’s technique [28] for this purpose. This technique greedily
ac,1determines the next options one by one. First, it determines x𝑖+1
ac,2
using Equation 3. Next, it determines x𝑖+1 by using Equation 3 butwith a slight modification to avoid sampling the same location as
ac,1
ac,3
x𝑖+1
. Finally, it determines x𝑖+1
by using Equation 3 again with aac,1slight modification to avoid sampling the same locations as x𝑖+1
ac,2
and x𝑖+1 . For more information, see our Supplemental Materials.

Photographic Lighting Design with Photographer-in-the-Loop Bayesian Optimization

Selection#Iteration 1

ai(x)G(I(x), Iiguide)

Paintaiguide(x;Iiguide)

Selectionai(x)#Iteration 2

G(I(x), Iiguide)

UIST ’22, October 29-November 2, 2022, Bend, OR, USA

Paintaiguide(x;Iiguide)

Selectionai(x)#Iteration 3

G(I(x), Iiguide)

Paintaiguide(x;Iiguide)

Figure 4: Result of a design session (2D) with visualization of the acquisition function values. The selected image for each
iteration is indicated by the orange borders. The map 𝑎𝑖 represents the original acquisition function used in standard BO. The
map 𝐺 represents the guidance function calculated by comparing the user’s painting input with the lighting prediction. The
guide
map 𝑎𝑖represents our acquisition function calculated using 𝑎𝑖 and 𝐺, and its maximizer is used for the next options’ search.
In each map, one blue dot is the parameter corresponding to the image selected by the user, and the three red dots are theparameter set to be presented next.
4.2.2 With Paint-Based Guidance. Following Chong et al. [6], we
incorporate paint-based guidance into the acquisition function. By
this, the photographer can roughly indicate their interest, and thesystem can consider the paint as a soft constraint in determining theoption in the next iteration. Thus, it is expected that less iterations
are necessary to find a preferable lighting.
guide
Let 𝐼𝑖be the painted guidance image that the photographer
provides after selecting the option at the 𝑖-th iteration. We replace
the acquisition function in Equation 3 with the following extended
acquisition function:
guide

𝑎𝑖

guide

(x; 𝐼𝑖guide) = 𝑎𝑖 (x) + 𝑤𝐺 (𝐼 (x), 𝐼𝑖

),

(4)

where 𝐺 is the guidance function that biases the acquisition function
towards the guidance image, 𝐼 is a function that takes a lighting
parameter set as input and returns an image shot with the specified
lighting configuration, and 𝑤 > 0 is a weight for controlling theeffect of the paint-based guidance. In our implementation, it is fixed
at 𝑤 = 1.0 throughout, which is an empirically successful value.
Similar to Chong et al.’s definition [6], we define 𝐺 as:Í guide(𝐼𝑖− 𝐼 (x)) 2 ∗ 𝑀
guide
𝐺 (𝐼 (x), 𝐼𝑖) = 1.0 −,(5)
𝑛 pixels
where 𝑀 is a binary mask for specifying painted pixels as ROI(region of interest) and excluding other pixels, ∗ is an element-wiseÍ
multiplication,means adding all the pixel values, and 𝑛 pixels
is the number of pixels in ROI (i.e., the summation of 𝑀 over all
elements). Note that we modified Chong et al.’s definition of 𝐺 [6]for making it independent from image resolution.

4.3

Lighting Prediction

The PBO method with paint-based guidance described in subsection 4.2 has a practical problem in that it takes a prohibitively longtime to search for the maximizer of the acquisition function (Equation 3) for each iteration. This is because each evaluation of theextended acquisition function (Equation 4) requires evaluating 𝐼 (·)
by physically manipulating lighting devices and shooting an image,which takes a few seconds even if these steps are fully automated.
Finding the maximizer with reasonable accuracy typically requires
a few thousand evaluations (depending on the dimensionality), andthus, it cannot be interactive.
To solve this problem, we propose replacing the actual lightingand-shooting step in the acquisition function evaluation with anapproximation using a lighting prediction. That is, we train a regressor beforehand using a set of shot images and use the image
predicted by the regressor in the acquisition function evaluation.
Thus, we need to shoot only three images per iteration to present
it to the photographer as new options. Note that the training datapreparation takes time but only once before the optimization process.
In our current implementation, we take 100 to 1000 photographs
by random sampling for the training dataset (e.g., 100 photos for3D and 1000 photos for 12D), and we apply the Gaussian process
regressor4 (GPR) in a pixel-wise manner. We choose GPR since
it can handle non-linearity, is available in the de-facto standard
library, and is differentiable (note that we need to calculate thegradient of the image prediction to efficiently solve Equation 3
using L-BFGS method). Figure 3 shows examples of the lighting
prediction results with the ground truth. In our observations, thissimple regression works reasonably well for our purpose; guiding
the sampling by rough paintings. For computational efficiency, oursystem generates predicted images in low resolution (100 × 100)and then calculates the distance to the rough painting, since higher
resolution is not necessary in this case.
The overall procedure of the proposed framework with the lighting prediction is described detailedly in Supplemental Materials.
Firstly, the system performs random sampling and training data
4 https://scikit-learn.org/1.0/modules/generated/sklearn.gaussian_process.

GaussianProcessRegressor.html

UIST ’22, October 29-November 2, 2022, Bend, OR, USA#Iteration 1

Paint

Selection#Iteration 2

Paint

Selection#Iteration 3

Paint(e) Physical
Robot Arms (8D)(d) 2 Lights Position
and Color (12D)(c) Position andIntensity (8D)(b) Reﬂector andLight Position (5D)(a) Position (3D)

Selection

Yamamoto et al.

Figure 5: Results of design sessions using our framework. We show scenes with various dimensions from 3D to 12D. The
selected image for each iteration is indicated by the orange borders. Paint guidances are provided in every iteration in this case,where not only the shades and highlights of the object but also the cast shadows are painted. (a, b, c, d) were performed insimulated environments using Blender, and (e) was performed in a physical environment with robotic arms, which was sameas the environment used in the user study.
acquisition to construct a regressor. Then, the iteration begins withrandomly chosen four options.

5function is constructed and its maximizer is used as the next options. The visualization of the acquisition function values suggeststhat the paint inputs successfully biased the acquisition function
shapes, making the next options close to the user’s intention.

RESULT

Visualization of Acquision Function. To effectively demonstrate
the behavior of the proposed framework, we first show a sequenceof lighting design in a simple two-dimensional simulated setting(the intensity of two light sources) with visualization of the acquisition function values. Figure 4 shows the result. The user is firstpresented with four random options, choosing one of them. Subsequently, the user adds paint strokes to the selected image. Basedon the PBO technique and the user’s paint input, the acquisition

Various Target Parameters. Next, we demonstrate the results ofapplying our framework for several problems with different target
parameter sets. We considered five lighting device conditions: (a) 3D(the light source position), (b) 5D (the position of the reflector andlight source), (c) 8D (the intensity and position of two light sources),(d) 12D (the position and color of the two light sources), and (e) 8D(eight axes of three robotic arms) parameter space. We used oursimulated environments for (a, b, c, d) and physical environment

Photographic Lighting Design with Photographer-in-the-Loop Bayesian Optimization(a) 3D (Light Position)

TargetPaint(b) 5D (Light and Reflector Position)

TargetPaint

UIST ’22, October 29-November 2, 2022, Bend, OR, USA(c) 8D (2 Lights Position and Power)

TargetPaint(d) 12D (2 Lights Position and Color)

Target

Paint

Figure 6: Results of the experiment. We tested on four different settings of lighting parameters. The top row shows thetransitions of the image gap values (i.e., the difference between the selected image and the target image) through the iteration.
We prepared two paint patterns for each setting: with a paint (green graph) and without a paint (navy graph). Additionally, amethod in which new options are randomly searched is also evaluated as a baseline method (gray graph). All graphs show themean and the colored areas show the standard deviation. The bottom row shows the target image (left) and the paint (right) for
each setting.
for (e). Figure 5 shows results with 3 iterations in five different
scenes. Note that, with the paint inputs, the user could specify thelightness and darkness of the object surface as well as the shadow
casted to the floor.

6

TECHNICAL EVALUATION

We conducted an experiment to evaluate how effectively our framework could search for desirable lighting configurations. An effective
search means that the difference between the target and the candidate images becomes small within less iterations. We define thedifference, or the image gap, as follows. We represent each image as a flattened vector in [0, 255]𝑊 ×𝐻 ×3 , where 𝑊 and 𝐻 are
the width and height of the image, respectively, then calculate theelement-wise absolute difference between the two images, and finally take the average over the elements. While this image gap canbe calculated by other methods (e.g., perceptual metrics), we adopt
pixel-wise and RGB-difference metrics for simplicity. The goal ofthis experiment is to evaluate how efficiently PBO works especiallywhen reasonable paint inputs are provided.
Four simulated scenes with different dimensions were used forthe experiment: 3D (the position of a light source), 5D (the positionof the reflector and light source), 8D (the intensity and positionof two light sources), and 12D (the position and color of the twolight sources) parameter spaces. To better understand the effect ofpaint inputs, we prepared two conditions for painting: with paint(colored strokes were carefully put) and without paint (no guidance
was used). For simplicity, a single set of paint strokes was unified
for each condition; that is, the same paint strokes were used forsynthesizing painted images in every iteration step. Additionally,
we also added a baseline of random sampling, where the new three

Target ImageNo Paint

Rough Paint

Precise Paint

Figure 7: Search results according to different painting patterns. The rough paint result is shown in orange. The meanand standard deviation of 20 trials are shown.

options were randomly chosen for each iteration step and one wasselected from four options. We performed 20 trials with random
initial pairs for each condition, and the mean and standard deviationof the image gap values were measured. Each trial continued for
20 iterations. In all conditions, we simulated the oracle of always
picking the option closest to the target from the given four options
in each iteration.
Figure 6 shows the results. Under the 3D, 5D, 8D, and 12D conditions, the difference from the target image is smaller when thepaint is provided than when it is not. Also, we could observe thatthe paint could make the search faster than without paint and therandom search.

UIST ’22, October 29-November 2, 2022, Bend, OR, USA

Dependency on paint guide accuracy. To evaluate how the optimization performance changes depending on the paint guide accuracy, we conducted an additional experiment using two paintguides with different levels of accuracy (namely, rough paint andprecise paint) in the same scene. Figure 7 shows the difference inthe optimization performance. Here, the parameter search of 12D(the position and color of the two lights) was set as in Figure 6 (d).
It can be seen that the precise paint is more efficient than the roughpaint, and if a more accurate guide is given, the desired lighting
design can be reached faster. Also, in the actual usage scene, by
adjusting the guide little by little for each iteration, the user may
be able to search efficiently even with a rough guide.

7(a) Proposed Framework

Orange Lightwith Robot Arm

Reflector
with Robot Arm

White Lightwith Robot Arm

Paint Interface

Shooting Objects

USER EVALUATION

To qualitatively understand the user experience in using our proposed framework, we conducted a user study using our physicalproof-of-concept environment with robotic arms. The specific goal
was to validate the concept that the proposed framework enables
the photographer to find the desired lighting configuration withfocusing on the illuminated scene rather than directly manipulating the lighting parameters. To better understand the benefits ofthe framework, we prepared a baseline for comparison: traditional
manual adjustment of physical lighting devices. The participants
were asked to perform lighting design tasks under both conditions
and answer the questionnaires regarding their experience.

7.1

Yamamoto et al.

(b) Traditional Framework

Orange Lightwith Tripod

White Lightwith Tripod
Captured image on
camera is displayed

Shooting Objects

Reflector
with Tripod

Study Setup

Participants. We recruited 12 participants (P1–P12) from amongst
our laboratory members. The average age of the participants was
23.4 with a standard deviation of 1.68 (8 male participants: 23.5(±1.91), 4 female participants: 23.4 (±1.69)). The participants werelimited to non-experts who had no work experience in photography
or video shooting and had little experience in handling lighting
equipment in studios. The pre-questionnaire regarding daily useof a camera revealed that 7 participants used only smartphones
to capture images, and 5 participants used single-lens cameras forphotography.
Apparatus. We have two setups: one for our framework andthe other for manual adjustment, as shown in Figure 8. For ourframework, we prepared three robotic arms (myCobot 280 M55 ),with two equipped with lights (white and orange 60W LED bulbs)
and the other equipped with a reflector (self-made, with diffuser
paper), as shown in Figure 8 (a). For the robotic arms with lights, theangles of three axes were used as parameters, and for the roboticarm with a reflector, the angles of two axes were used as parameters;
together, this became an 8-dimensional variable search problem.
For the manual lighting setup, we prepared the same lights andreflector, and installed those using tripods, as shown in Figure 8 (b).
The position and orientation of the camera and objects were fixed
for both conditions.
In our framework, the participants did not touch the robotic arms
and the lighting devices at all; instead, they operated only the paintinterface. As an input interface, a tablet (Wacom Cintiq 13HD6 )5 https://www.elephantrobotics.com/en/mycobot-en/
6 https://www.wacom.com/en-cn/products/pen-displays/cintiq-13-hd

Figure 8: Experimental environments. (a) The proposed
framework. Participants search for the desired lighting by
repeating selection and painting with the paint interface. (b)The traditional baseline framework. Participants search forthe desired lighting by manually moving the light sources
and reflector.

and a stylus pen were prepared. During the manual adjustment, theparticipants changed the position and orientation of the lighting
and the reflector physically. The images captured by the camera
were input to an external monitor in real-time.
Task. We prepared four objects to shoot: two food miniatures(one for the proposed framework and one for the traditional framework), an animal minifigure (for the proposed framework), andan insect figure (for the traditional framework), as shown in Figure 8. To counterbalance the experimental order, the sequence ofthe shooting of four objects was determined based on the Latin
square sampling.
First, the participants were given time to practice the proposed
framework and the manual framework. Then, the lighting design
task for each object was executed. In all lighting design tasks, participants were asked to freely explore the design they wanted. Aftereach shoot, the participants were asked to record their satisfaction
with the lighting results on a 7-point Likert scale. Upon completion

Photographic Lighting Design with Photographer-in-the-Loop Bayesian Optimization
Ours(a) Questionnaires
**

**

**

* p<0.05 ** p<0.01

Manual****(b) Satisfaction

UIST ’22, October 29-November 2, 2022, Bend, OR, USA

confirm that novice photographers can use our framework to findsatisfactory lighting configurations and have better experiences.
User Feedback. P12 commented that he had no concrete idea in
his mind beforehand but appreciated the interface (especially, thepresentation of the first four random suggestions) since it helped
him understand what he wanted to shoot. P2 appreciated the intuitiveness of the proposed framework and also could find unexpectedyet-good images that were different from his original intention during the iteration. On the other hand, P2, P3, and P9 commented thatthey felt stressed since they sometimes could not obtain desired
lighting results despite repeatedly specifying paintings. This could
be due to a device constraint; the desired lighting is not always
achievable by the available lighting equipments.

8
Figure 9: (a) Results of the questionnaires. The red and blue
bars show the obtained scores for the proposed frameworkand the traditional framework, respectively, and the black
error bars indicate the standard deviation. (b) Results of thesatisfaction scores for the lighting designs.

of the shoots, the participants were asked to complete the questionnaires regarding engagement, device operability, workflow easiness,
workflow efficiency, workflow stress, and workflow preferences ineach condition. These questions were answered on a 7-point Likert
scale from “strongly disagree” to “strongly agree” (from 1 to 7). The
specific questions were as follows.
Q1: When designing lighting, I was able to concentrate on thedesign without being aware of the parameters (e.g., position,orientation) of the lights and reflector.
Q2: The operation of the device was easy.
Q3: It was easy to design lighting with this workflow.
Q4: It was not a burden to design lighting with this workflow.
Q5: I didn’t feel any stress when designing lighting with thisworkflow.
Q6: I liked designing lighting with this workflow.
After answering the questionnaires, participants were also asked
to write free comments for the respective frameworks.

7.2

Results

Figure 9 (a) shows the results of the questionnaires regarding eachcondition. As a result of a paired 𝑡-test for all questionnaire items, asignificant difference of 𝑝 < 0.05 was confirmed, of which Q1–4 was
confirmed to have a significant difference of 𝑝 < 0.01. This provesthat our method was highly appreciated by the participants interms of user experience when compared to the traditional manualframework. In particular, the scores for Q1 validate our concept.
Figure 9 (b) shows the result of satisfaction scores with each design.
The mean value of satisfaction was higher when the proposed
framework was used, but no significant difference with the manual
condition was found. While it is not clear whether our framework
enables photographers to find better lighting designs, the results

DISCUSSIONS

Findings of user study. We have evaluated the qualitative aspectsof our proposed framework through the user study. The results of
the questionnaires confirmed that our framework could bring theconcentration on the shooting result to the user with releasing thenecessity of being familiarized with raw parameters. This validates
our key concept. The participants also appreciated the ease of theoperation and the entire workflow. Several participants commented
positively that the presentation of multiple candidates reduced thedesign task load, and that sometimes they could find unexpectedyet-good options. We consider that the exploration characteristics
of BO was important to provide such unexpected and inspirational
options.
Limitations of our user study. Our user study was for evaluating
the user experience, and there are several aspects that have notbeen evaluated. For example, it has not been evaluated whether ourframework can reduce the user time required for the lighting design
task execution. It is necessary to carefully design experimental conditions to evaluate such task performance; the task execution time
may be highly sensitive to the subject type and the number of available lighting equipment options, in both the proposed frameworkand traditional manual adjustment framework. It is also effective toperform a “targeted” design task with pre-defined goals (rather than
the free-from task starting with vague goals) for task performance
analysis. However, such “targeted” tasks are not our focus, and thus
we decided to use the free-form design task in our user study tovalidate the key concept, which is more important than analyzing
performance. In addition to the task performance evaluation, theexperience of expert users has not been evaluated yet. It is exciting
future work to extend our proof-of-concept implementation for
professional photographic studio settings.
Higher dimensionality. Our proposed framework can in theory be
used with many mechanically controlled lighting devices simultaneously. However, for supporting such high-dimensional cases, weneed to extend our framework in some ways; otherwise, the numberof necessary iterations could become intractably large. One possible approach is to allow users to search low-dimensional spaces
instead of selecting from limited number of options; Koyama et al.
[13, 14] demonstrated that this approach could drastically effective
to reduce the number of necessary iterations.

UIST ’22, October 29-November 2, 2022, Bend, OR, USA

Accuracy of Lighting Prediction. We used GPR as the regressor
and applied it to each pixel independently. We adopted this choicebecause of its simplicity and suitability to our problem setting. Its
accuracy is sufficient for biasing the sampling using rough painting.
Nonetheless, it is not accurate enough to predict complex changes
in lighting (e.g., highlights and shadows). One can adopt different
choices (e.g., deep neural networks) to implement the lighting prediction module in case a more accurate prediction is desirable. Also,
it is interesting future work to consider physical properties of lights(e.g., linearity) for better prediction with fewer training images.
Integration with expert knowledge and patterns. There are wellknown general rules (e.g., the three-point lighting method) and design patterns (e.g., the Rembrandt lighting) in photographic lighting
design. It is possible to incorporate such rules into our frameworkin the future by, for example, using as initial solutions. Moreover,a data-driven approach could be incorporated into the framework(i.e., imitating the lighting pattern of a well-known photographer).
Incorporating more parameters for photography. Although only
lighting device parameters were used as target parameters for the
search, our framework can be used with other related parameters
together such as the camera settings and shooting target parameters(e.g., the object’s orientation). Moreover, the use of some importantparameters for lighting design have not yet been demonstrated,such as strobes and diffusers. The proposed method could be applied
to such parameters as well because BO is domain-agnostic and has
no restrictions on the types of target parameters.
Extending feasible lighting designs. As physical proof-of-concept
equipment, we have prepared three robotic arms, two lights, andone reflector. While this equipment was sufficient to confirm our
key concept, it limits the range of feasible lighting designs (e.g., therelative position of the light and the reflector cannot be changed asshown in Figure 8). In fact, we received a few negative commentsfrom several participants in our user study that their design intent(paint guide) was not reflected in the lighting results in some scenes.
To expand the feasible lighting design area, it is necessary to develop
lighting devices that combine lights with drones or large XY plottersin the future.
Potential practical use case in photo-studios. In the photographic
studios, many non-photographers, such as art directors, advertisers, producers, and even the models who are the subjects of thephotographs, can now review and comment on the captured results in real-time. While manipulating the distribution of light inspace and the reflections onto objects requires a high level of professional skills, everyone can express opinions about the resulting
photographs. There is a need for a tool between the photographer
and the reviewer to share how they want to change the photographin a way that anyone can instruct, such as using a paint interface.
Towards virtual production and cinematography. While our framework focuses on photography, the core idea (i.e., finding appropriate parameters to arrange programmatically controlled lighting
devices) would be useful for cinematography in the future. In particular, virtual production studios already have such equipment. One
remaining future challenge would be how to design time-changing
lighting configurations effectively.

Yamamoto et al.

9

CONCLUSION

Lighting design is indispensable for good photography, even though
it is difficult for novice photographers. We proposed a novel photographic lighting design framework based on photographer-in-theloop Bayesian optimization. This framework frees photographersfrom manually adjusting the raw parameters of their lighting devices, allowing them to focus solely on the visual results of the shot.
Because our framework puts minimum assumptions on the typesof target parameters, it can be used with various lighting-related
equipment, such as high-brightness LED displays. Our proposed
framework opens the new possibility of combining many mechanically controlled lighting devices together to achieve more creativeand productive lighting environments for photographers.

ACKNOWLEDGMENTS
This research was supported by Pixie Dust Technologies, Inc. We
thank the members of Digital Nature Group for helping our research
work; Ippei Suzuki and Atsushi Shinoda for controlling robotic arms,
Ryo Iijima for developing the user interface software, and Kazushi
Nakamura for performing user study. The bunny model is provided
by the Stanford Computer Graphics Laboratory. The marble bustand clock models are provided by Poly Haven under CC0 license.

REFERENCES[1] Ivaylo Boyadzhiev, Jiawen Chen, Sylvain Paris, and Kavita Bala. 2016. Do-ityourself lighting design for product videography. In 2016 IEEE International
Conference on Computational Photography (ICCP). 1–9. https://doi.org/10.1109/
ICCPHOT.2016.7492878[2] Ivaylo Boyadzhiev, Sylvain Paris, and Kavita Bala. 2013. User-Assisted Image
Compositing for Photographic Lighting. ACM Trans. Graph. 32, 4, Article 36 (jul2013), 12 pages. https://doi.org/10.1145/2461912.2461973[3] Eric Brochu, Tyson Brochu, and Nando de Freitas. 2010. A Bayesian Interactive
Optimization Approach to Procedural Animation Design. In Proc. SCA ’10. 103–112. https://doi.org/10.2312/SCA/SCA10/103-112[4] Eric Brochu, Nando de Freitas, and Abhijeet Ghosh. 2007. Active Preference
Learning with Discrete Choice Data. In Proc. NIPS ’07. 409–416. http://papers.
nips.cc/paper/3219-active-preference-learning-with-discrete-choice-data[5] Chia-Hsing Chiu, Yuki Koyama, Yu-Chi Lai, Takeo Igarashi, and Yonghao Yue.
2020. Human-in-the-Loop Differential Subspace Search in High-Dimensional
Latent Space. ACM Trans. Graph. 39, 4 (July 2020), 85:1–85:15. https://doi.org/10.
1145/3386569.3392409[6] Toby Chong, I-Chao Shen, Issei Sato, and Takeo Igarashi. 2021. Interactive
Optimization of Generative Image Modelling using Sequential Subspace Searchand Content-based Guidance. Comput. Graph. Forum 40, 1 (February 2021),
279–292. https://doi.org/10.1111/cgf.14188[7] António Cardoso Costa, António Augusto Sousa, and Fernando Nunes Ferreira.
1999. Lighting Design: A Goal Based Approach Using Optimisation. In Rendering
Techniques’ 99, Dani Lischinski and Greg Ward Larson (Eds.). Springer Vienna,
Vienna, 317–328.
[8] John J. Dudley, Jason T. Jacques, and Per Ola Kristensson. 2019. Crowdsourcing
Interface Feature Design with Bayesian Optimization. In Proc. CHI ’19. 252:1–
252:12. https://doi.org/10.1145/3290605.3300482[9] Jane L. E, Ohad Fried, and Maneesh Agrawala. 2019. Optimizing Portrait Lighting
at Capture-Time Using a 360 Camera as a Light Probe. In Proceedings of the 32nd
Annual ACM Symposium on User Interface Software and Technology (New Orleans,
LA, USA) (UIST ’19). Association for Computing Machinery, New York, NY, USA,
221–232. https://doi.org/10.1145/3332165.3347893[10] Martin Fuchs, Volker Blanz, and Hans-Peter Seidel. 2005. Bayesian Relighting.
In Proceedings of the Sixteenth Eurographics Conference on Rendering Techniques(Konstanz, Germany) (EGSR ’05). Eurographics Association, Goslar, DEU, 157–
164.
[11] D. R. Jones, C. D. Perttunen, and B. E. Stuckman. 1993. Lipschitzian optimization
without the Lipschitz constant. Journal of Optimization Theory and Applications
79, 1 (01 Oct 1993), 157–181. https://doi.org/10.1007/BF00941892[12] John K. Kawai, James S. Painter, and Michael F. Cohen. 1993. Radioptimization:
Goal Based Rendering. In Proceedings of the 20th Annual Conference on Computer
Graphics and Interactive Techniques (Anaheim, CA) (SIGGRAPH ’93). Association

Photographic Lighting Design with Photographer-in-the-Loop Bayesian Optimization

UIST ’22, October 29-November 2, 2022, Bend, OR, USA

for Computing Machinery, New York, NY, USA, 147–154. https://doi.org/10.
1145/166117.166136[13] Yuki Koyama, Issei Sato, and Masataka Goto. 2020. Sequential Gallery for Interactive Visual Design Optimization. ACM Trans. Graph. 39, 4 (July 2020), 88:1–88:12.
https://doi.org/10.1145/3386569.3392444[14] Yuki Koyama, Issei Sato, Daisuke Sakamoto, and Takeo Igarashi. 2017. Sequential
Line Search for Efficient Visual Design Optimization by Crowds. ACM Trans.
Graph. 36, 4 (July 2017), 48:1–48:11. https://doi.org/10.1145/3072959.3073598[15] Dong C. Liu and Jorge Nocedal. 1989. On the Limited Memory BFGS Method forLarge Scale Optimization. Math. Program. 45, 1–3 (aug 1989), 503–528.
[16] Joe Marks, Brad Andalman, Paul A. Beardsley, William T. Freeman, Sarah F.
Gibson, Jessica K. Hodgins, Thomas Kang, Brian Mirtich, Hanspeter Pfister,
Wheeler Ruml, Kathy Ryall, Joshua E. Seims, and Stuart M. Shieber. 1997. Design
Galleries: A General Approach to Setting Parameters for Computer Graphics
and Animation. In Proc. SIGGRAPH ’97. 389–400. https://doi.org/10.1145/258734.
258887[17] Ankit Mohan, Jack Tumblin, Bobby Bodenheimer, Cindy Grimm, and Reynold
Bailey. 2006. Table-Top Computed Lighting for Practical Digital Photography. In ACM SIGGRAPH 2006 Courses (Boston, Massachusetts) (SIGGRAPH’06). Association for Computing Machinery, New York, NY, USA, 3–es. https://doi.org/10.1145/1185657.1185742[18] William Mortensen. 1935. Pictorial lighting. (1935).
[19] Lukas Murmann, Abe Davis, Jan Kautz, and Frédo Durand. 2016. Computational
Bounce Flash for Indoor Portraits. ACM Trans. Graph. 35, 6, Article 190 (nov
2016), 9 pages. https://doi.org/10.1145/2980179.2980219[20] Addy Ngan, Frédo Durand, and Wojciech Matusik. 2006. Image-driven Navigationof Analytical BRDF Models. In Proc. EGSR ’06. 399–407. https://doi.org/10.2312/
EGWR/EGSR06/399-407[21] Makoto Okabe, Yasuyuki Matsushita, Li Shen, and Takeo Igarashi. 2007. Illumination Brush: Interactive Design of All-Frequency Lighting. In Proceedings ofthe 15th Pacific Conference on Computer Graphics and Applications (PG ’07). IEEE
Computer Society, USA, 171–180.
[22] Fabio Pellacini. 2010. EnvyLight: An Interface for Editing Natural Illumination.
In ACM SIGGRAPH 2010 Papers (Los Angeles, California) (SIGGRAPH ’10). Association for Computing Machinery, New York, NY, USA, Article 34, 8 pages.
https://doi.org/10.1145/1833349.1778771[23] Fabio Pellacini, Frank Battaglia, R. Keith Morley, and Adam Finkelstein. 2007.
Lighting with Paint. ACM Trans. Graph. 26, 2 (jun 2007), 9–es. https://doi.org/
10.1145/1243980.1243983[24] Fabio Pellacini, Parag Tole, and Donald P. Greenberg. 2002. A User Interface
for Interactive Cinematic Shadow Design. ACM Trans. Graph. 21, 3 (jul 2002),
563–566. https://doi.org/10.1145/566654.566617[25] Michal Piovarči, Danny M. Kaufman, David I. W. Levin, and Piotr Didyk. 2020.
Fabrication-in-the-Loop Co-Optimization of Surfaces and Styli for Drawing Haptics. ACM Trans. Graph. 39, 4 (jul 2020), 116:1–116:16. https://doi.org/10.1145/
3386569.3392467[26] Pierre Poulin and Alain Fournier. 1992. Lights from Highlights and Shadows.
In Proceedings of the 1992 Symposium on Interactive 3D Graphics (Cambridge,
Massachusetts, USA) (I3D ’92). Association for Computing Machinery, New York,NY, USA, 31–38. https://doi.org/10.1145/147156.147160[27] Chris Schoeneman, Julie Dorsey, Brian Smits, James Arvo, and Donald Greenberg.
1993. Painting with Light. In Proceedings of the 20th Annual Conference on
Computer Graphics and Interactive Techniques (Anaheim, CA) (SIGGRAPH ’93).
Association for Computing Machinery, New York, NY, USA, 143–146. https://doi.org/10.1145/166117.166135[28] Matthias Schonlau, William J. Welch, and Donald R. Jones. 1998. Global versus
Local Search in Constrained Optimization of Computer Models. Lecture NotesMonograph Series 34 (1998), 11–25. http://www.jstor.org/stable/4356058[29] Bobak Shahriari, Kevin Swersky, Ziyu Wang, Ryan P. Adams, and Nando deFreitas. 2016. Taking the Human Out of the Loop: A Review of Bayesian Optimization. Proc. IEEE 104, 1 (January 2016), 148–175. https://doi.org/10.1109/
JPROC.2015.2494218[30] Lior Shapira, Ariel Shamir, and Daniel Cohen-Or. 2009. Image Appearance
Exploration by Model-Based Navigation. Comput. Graph. Forum 28, 2 (2009),
629–638. https://doi.org/10.1111/j.1467-8659.2009.01403.x[31] Manohar Srikanth, Kavita Bala, and Frédo Durand. 2014. Computational Rim
Illumination with Aerial Robots. In Proceedings of the Workshop on Computational
Aesthetics (Vancouver, British Columbia, Canada) (CAe ’14). Association for
Computing Machinery, New York, NY, USA, 57–66. https://doi.org/10.1145/
2630099.2630105[32] Tiancheng Sun, Jonathan T. Barron, Yun-Ta Tsai, Zexiang Xu, Xueming Yu, Graham Fyffe, Christoph Rhemann, Jay Busch, Paul Debevec, and Ravi Ramamoorthi.
2019. Single Image Portrait Relighting. ACM Trans. Graph. 38, 4, Article 79 (jul2019), 12 pages. https://doi.org/10.1145/3306346.3323008[33] Kristi Tsukida and Maya R. Gupta. 2011. How to Analyze Paired Comparison Data.
Technical Report UWEETR-2011-0004. University of Washington, Department
of Electrical Engineering. https://vannevar.ece.uw.edu/techsite/papers/refer/UWEETR-2011-0004.html[34] Zhiqi Yin, Zeshi Yang, Michiel Van De Panne, and Kangkang Yin. 2021. Discovering Diverse Athletic Jumping Strategies. ACM Trans. Graph. 40, 4 (jul 2021),
91:1–91:17. https://doi.org/10.1145/3450626.3459817[35] Xiuming Zhang, Sean Fanello, Yun-Ta Tsai, Tiancheng Sun, Tianfan Xue, Rohit
Pandey, Sergio Orts-Escolano, Philip Davidson, Christoph Rhemann, Paul Debevec, Jonathan T. Barron, Ravi Ramamoorthi, and William T. Freeman. 2021.
Neural Light Transport for Relighting and View Synthesis. ACM Trans. Graph.
40, 1, Article 9 (jan 2021), 17 pages. https://doi.org/10.1145/3446328[36] Yijun Zhou, Yuki Koyama, Masataka Goto, and Takeo Igarashi. 2020. Generative
Melody Composition with Human-in-the-Loop Bayesian Optimization. In Proc.
CSMC-MuMe ’20. 21:1–21:10. https://boblsturm.github.io/aimusic2020/papers/
CSMC__MuMe_2020_paper_21.pdf[37] Yijun Zhou, Yuki Koyama, Masataka Goto, and Takeo Igarashi. 2021. Interactive
Exploration-Exploitation Balancing for Generative Melody Composition. In Proc.
IUI ’21. 43–47. https://doi.org/10.1145/3397481.3450663