A Meta-Bayesian Approach for Rapid Online
Parametric Optimization for Wrist-based Interactions
Yi-Chi Liao

Ruta Desai

Alec M. Pierce

yi-chi.liao@aalto.f
Aalto University
Helsinki, Finland
Saarland University
Saarbrücken, Germany

rutadesai@meta.com
Fundamental AI Research (FAIR),
Meta Inc
Redmond, Washington, USA

alec.m.pierce@gmail.com
Reality Labs Research, Meta Inc
Redmond, Washington, USA

Krista E. Taylor

Hrvoje Benko

Tanya R. Jonker

kristatmail@gmail.com
Reality Labs Research, Meta Inc
Redmond, Washington, USA

benko@meta.com
Reality Labs Research, Meta Inc
Redmond, Washington, USA

tanya.jonker@meta.com
Reality Labs Research, Meta Inc
Redmond, Washington, USA

Aakar Gupta
aakar.hci@gmail.com
Reality Labs Research, Meta Inc
Redmond, Washington, USA
Fujitsu Research America
Pittsburgh, Pennsylvania, USA

ABSTRACTCCS CONCEPTS

Wrist-based input often requires tuning parameter settings in correspondence to between-user and between-session diferences, suchas variations in hand anatomy, wearing position, posture, etc. Traditionally, users either work with predefned parameter values
not optimized for individuals or undergo time-consuming calibration processes. We propose an online Bayesian Optimization (BO)based method for rapidly determining the user-specifc optimal
settings of wrist-based pointing. Specifcally, we develop a metaBayesian optimization (meta-BO) method, difering from traditional
human-in-the-loop BO: By incorporating meta-learning of prior
optimization data from a user population with BO, meta-BO enables
rapid calibration of parameters for new users with a handful of
trials. We evaluate our method with two representative and distinct wrist-based interactions: absolute and relative pointing. On
a weighted-sum metric that consists of completion time, aiming
error, and trajectory quality, meta-BO improves absolute pointing
performance by 22.92% and 21.35% compared to BO and manual
calibration, and improves relative pointing performance by 25.43%
and 13.60%.

• Human-centered computing → Pointing devices; Interaction techniques; • Computing methodologies → Machinelearning.

This work is licensed under a Creative Commons Attribution International
4.0 License.
CHI ’24, May 11–16, 2024, Honolulu, HI, USA© 2024 Copyright held by the owner/author(s).
ACM ISBN 979-8-4007-0330-0/24/05
https://doi.org/10.1145/3613904.3642071

KEYWORDS
Bayesian optimization, human-in-the-loop optimization, metaBayesian optimization, meta-learning, wrist-based interaction, calibration, target selection, adaptive interface.
ACM Reference Format:
Yi-Chi Liao, Ruta Desai, Alec M. Pierce, Krista E. Taylor, Hrvoje Benko,
Tanya R. Jonker, and Aakar Gupta. 2024. A Meta-Bayesian Approach for
Rapid Online Parametric Optimization for Wrist-based Interactions. In Proceedings of the CHI Conference on Human Factors in Computing Systems(CHI ’24), May 11–16, 2024, Honolulu, HI, USA. ACM, New York, NY, USA,38 pages. https://doi.org/10.1145/3613904.3642071

1

INTRODUCTION

In addition to its ubiquity in the HCI literature [2, 10, 29, 43, 63, 64],
wrist-based input has been posited in the industry as one of the key
candidates for driving future freehand interactions in AR1 . Akin toconventional interaction devices such as a mouse, which are characterized by parameters like transfer function [52] and input flter
variables [12], wrist-based devices also possess various parameters
for setting transfer functions or device calibration or other input
functionalities [11]. The parameter settings for the interactions
with these input devices signifcantly infuence the user experience and performance [13, 101]. However, the one-design-fts-all
parameter setting strategy of traditional input devices (e.g., mouse,
1 https://about.fb.com/news/2021/03/inside-facebook-reality-labs-wrist-based-interaction-for-the-next-computing-platform/

CHI ’24, May 11–16, 2024, Honolulu, HI, USA

Y.-C. Liao, et al.

Figure 1: Our absolute pointing interaction using a wrist-worn device. The sensed yaw (green) and pitch (red) values map to thecursor’s x and y coordinates.

f(v) · sin θs

f(vw)

θc

f(vw)

θs

f(vw)
θs

θs

f(v) · cos θsyy Ideal yvw
θsy Ideal y

vw
θs

x

θs

Ideal x

x
θ’

a

vw

b

x
Ideal xθ’cdFigure 2: Our relative pointing interaction. (a) Our study participants performed relative pointing in an arms-down position. (b)An ideal situation where the sensed wrist angle �� is aligned with the user’s intention. (c) A more realistic scenario where thedevice is worn with a subtle rotational diference � ’, hence the sensed direction �� is less than the user’s intended direction. (d)We use an angular correction parameter �� to compensate for such rotational diference. As described in Equation 8, �� is added
upon �� .

keyboard, touch-button, etc.) does not sufce for wrist-based devices for several reasons. Unlike other interaction techniques, agood parameter setting for wrist-based interactions varies significantly across diferent users owing to their physical, behavioral,
and preference-based diferences [29]. For example, some users
prefer to perform wrist motions with a wider range, while others
prefer narrower wrist motions. Further complicating matters, theoptimal parameter setting for a user may change based on how thedevice is worn, hand positions, and environmental factors. A user
may wear the device in slightly diferent positions each time, necessitating unique parameter settings for each use. To identify the bestuser-specifc and session-specifc parameter values, users typicallyundergo a manual calibration process to determine a reasonablesetting [28, 29]. However, calibration requires extra dedicated timefrom users, delaying the intended interaction [1, 29, 32, 44, 96], andis often designed manually by developers which does not guarantee
the optimal outcome.

Human-in-the-loop (HitL) optimization [31, 85] has the potential
to automatically identify optimal parametric settings while users
engage in the intended interaction [15, 45, 104]. Among various
optimization methods, Bayesian optimization (BO) has gained popularity for HitL applications due to its versatility and efectiveness[8, 17, 31, 49, 103]. Although BO has proven to be more sampleefcient than many other algorithms [8], it still requires many
iterations and a long duration to converge. For instance, Chan et al.
[15] spent an hour optimizing a pointing interaction using BO.
How can we enhance the efciency of BO for its applicationin online and rapid human-in-the-loop applications? Our solution
augments BO with prior experience, enabling it to proactively explore parameter areas with the most potential for promising user
performance. We adopt our solution from meta-Bayesian optimization (meta-BO) [3, 23, 54, 91, 93, 100], an emerging paradigm ofA Meta-Bayesian Approach for Rapid Online Parametric Optimization for Wrist-based Interactions

BO-based methods for rapid optimization by utilizing datasets gathered from similar optimization tasks2 . By leveraging the experienceof optimizing the parameter settings for a group of users in advance, meta-BO can efciently optimize for new users for the sameinteraction. While meta-BO has gained growing attention in machine learning research, its potential utility in HCI remains largely
unexplored. We extend a specifc meta-BO method called Transfer
Acquisition Function (TAF) [100], which builds independent population models using the past optimization data and then combinesthese population models with incoming data for rapid optimizationof interaction parameters in online settings. Our approach, which
we term Transfer Acquisition Function+ (TAF+ ) extends TAF forparametric optimization in interaction applications by enabling
designers to control weighting both in balancing an application’s
multiple objectives (e.g. speed and accuracy in pointing) and between prior population data and new incoming data in real-time
deployments. TAF+ thus enables design fexibility for achieving
diverse objectives and efcient adaptation.
We demonstrate the efcacy of our TAF+ -powered workfow for
rapid parametric optimization of two common and distinct wristbased pointing interactions: absolute pointing (Figure 1) and relative
pointing (Figure 2). Absolute pointing involves moving the cursor
using forearm motion sensed by a wrist inertial measurement unit
(IMU) similar to prior work [34, 65]. Relative pointing involves moving the cursor using relative wrist rotation [28, 43, 77]. Pointing is aparticularly relevant challenge for AR interfaces, and absolute andrelative pointing represent the two fundamental types of pointing.
Combined with the signifcance of wrist-based input for future AR
interfaces, wrist-based absolute and relative pointing interactions
are a timely and relevant problem to solve.
We conduct a study that evaluates our meta-BO method TAF+
along with two baseline procedures we mentioned above: manual calibration and standard Bayesian optimization. The results
show that meta-BO led to signifcantly better performance on aweighted-sum metric that consisted of completion time, aiming error, and trajectory quality. Specifcally, meta-BO improves absolute
pointing performance by 22.92% and 21.35% compared to BO andmanual calibration respectively, and improves relative pointing performance by 25.43% and 13.60% than BO and manual respectively.
In summary, we make the following key contributions:• Introducing a novel meta-BO method called Transfer Acquisition Function+ (TAF+ ) as an online, sample-efcient parametric
optimization approach in HCI : TAF+ extends TAF by enabling
designer control of (a) weighting multiple task objectives,and (b) tuning the importance between prior population andcurrent real-time user’s data.
• Demonstrating the efcacy of TAF+ to identify user-specifc,
optimal parametric settings for two distinct forms of wristbased pointing: TAF+ outperformed established baselines manual calibration and standard BO.

2 RELATED WORK
2.1 Wrist-based interactions
Inertial measurement units (IMUs) are used commonly to detect
hand motion. Dipietro et al. [20], Perng et al. [72] proposed various
glove-mounted IMU systems, and later research proposed wristworn form factors [2, 10, 43, 63, 64]. Previous works have deployed
IMUs for detecting human activity [2, 86], projecting raycastingwith the detected motion [33, 34, 65, 71], and gesture recognition[51, 59]. Our absolute pointing approach is similar to Nancel et al.
[65] where forearm movements sensed by an IMU control a cursor’s
position in a 2D interface. Our relative pointing approach uses wrist
rotations to control a cursor’s relative motion. Wrist rotation tracking via the wristband has typically used outside-in or inside-out
sensing. Outside-in uses sensors such as EMG [40, 79], electrical
impedance tomography [109], and capacitive sensing [75] for inferring wrist angles or gestures. Inside-out sensing uses wrist-worn
cameras [29, 43, 106]. For our relative pointing, we use a similar device and method as RotoWrist [77], which consists of a wrist-worn
IR sensor array that tracks the wrist’s relative angles.

2.2

optimizing for “a specifc user”.

Calibration

In HCI, calibration is a procedure for setting up system parameters
so the device interaction can work properly, such as tuning a sensor’s internal values [25, 74, 105] or setting parameters based onuser-dependent features [16, 53]. Calibration has been widely applied, such as for touchscreen interactions [53], gaze input [73], andwearable devices [58, 102]. However, efcient calibration for thetransfer function of input devices remains a signifcant challenge
in HCI due to the vast parameter space [52].
Given our focus, we review procedures pertaining to wristband
device calibration and pointing transfer function calibration. Wristbased pointing often needs to identify a function that maps sensor
values to cursor position. WristWhirl [29] requires users to movetheir wrists to the maximum along two axes, and then defnes amapping accordingly. Similarly, WristText [28] requires several
wrist rotations to capture the maximum sensor values. Our manual
calibration baseline for absolute pointing uses a similar approach
where the users identify their preferred operation ranges via forearm motion. While absolute pointing involves a one-to-one mapping
between input and output (cursor) displacement, relative pointing(e.g. mouse pointing or video game sensitivity) involves a CD-gainbased transfer function that varies cursor motion speed based on input motion speed [11, 52]. This function is typically pre-determinedusing trial-and-error [11] or heuristic iteration [52, 108] and is uniform across all users, with an option for the user to fnetune itthemselves (as in Windows, Mac, and Linux devices). Our manual
calibration baseline for relative pointing uses a similar approach
where we instruct the users to fnetune the velocity transfer function starting with a fxed predetermined setting.

2.3
2 Note that in the context of HitL optimization, “one optimization task” refers to

CHI ’24, May 11–16, 2024, Honolulu, HI, USA

Human-in-the-Loop optimization

Since such calibration procedures do not necessarily result in optimal settings, Human-in-the-Loop (HitL) optimization approaches
have been proposed. HitL optimization is a general framework in

CHI ’24, May 11–16, 2024, Honolulu, HI, USA

which users serve as the evaluation function, and a mathematical parametric optimization procedure aims to efciently identify
the optimal parameter setting [17, 31, 49, 103]. Among many optimization methods, Bayesian optimization (BO) is a computational
method that has been developed over decades [62, 84]. Prior work
has used BO as an HitL method [39, 47, 76] for minimizing temporalerror [57], increasing animation realism [9], game engagement [41],haptic distinguishability [56], hearing aids [68, 69], and optimizing
pointing transfer functions [15]. However, BO is mainly seen as
an ofine design tool [48, 56, 104] since it requires a large number of iterations resulting in long-duration user sessions. For a 3D
target-selection transfer function optimization Chan et al. [15], BO
required participants to spend 60-90 mins. Consequently, no priorwork has employed BO for real-time target selection interactions
with end-users. Another recent HitL approach, AutoGain [52], proposed updating the transfer function based on submovement errors.
However, AutoGain requires a dedicated session of 30 minutes toconverge and is constrained to the single objective of minimizing
aiming error. Our meta-BO method is aimed at overcoming theseissues and performing rapid, online multi-objective optimization.

2.4

Meta-learning for BO

Meta-learning is a general concept that aims to improve the learning
speed of a system by leveraging the prior experience of similartasks [46, 82, 92, 94]. Successful meta-learning implementations
have been proposed for recognition and reinforcement-learning
tasks using deep neural nets [6, 21, 26, 67, 78, 81].
In the context of BO, meta-Bayesian optimization (meta-BO) is amachine-learning paradigm consisting of diferent implementations
that use prior optimization data to improve the speed of ongoing
optimization [3, 18, 54, 95]. Owing to its recency, meta-BO has
not been employed yet to solve HCI problems. Given its promiseof speeding up standard BO, we employ the use of meta-BO for
our problem. There are multiple ways to apply meta-BO and ourgoal was to select one that would ft for HitL tasks like ours. The
frst approach is to ft all the prior data into a unifed GaussianProcess (GP) model [5, 7, 91, 107]. However, the model-ftting increases cubically (O (� 3 )) with the number of observations making
the computation time for suggesting the next design impracticallylong. Incorporating the sparse GP potentially allows for better scalability by only retaining a smaller representative dataset [88, 97].
However, new challenges and uncertainties arise from constructing
such complex techniques; for instance, determining the appropriate
number of datapoints for the sparse GP, tuning hyperparameters
efectively, managing increased model complexity, and balancing
computational efciency with accurate uncertainty estimates. The
second approach is to replace elements in BO with neural networks
trained on prior data [36, 89, 90, 93, 99]. However, it potentiallyrequires a relatively larger amount of data to pre-train the networks.
Moreover, it generally does not ofer explanability, which is crucial
for HitL applications. The fnal approach, the one that we adopt,is the weighting-based solution [55, 80, 100] which stores separate
GP models, each model being derived from the data of a previous
task (in our case, a participant session). The next design suggestion
is decided based on a weighted aggregation of all the previously
gathered GPs’ information. This approach has low computational

Y.-C. Liao, et al.

complexity, can work with small amounts of prior data, and ofers
higher explainability to the users with the opportunity of observing
the result generated by each model. Among several implementations along this line of research [24, 37, 55, 80], our method extends
Wistuba et al. [100]’s TAF approach. They demonstrate TAF witha single-objective and a naive test function. Our TAF+ approach
extends this to our multi-objective HitL scenario which presents
new challenges.

3

PRELIMINARIES: EXISTING METHODS

To appropriately explain TAF+ , we frst introduce the key concepts
for BO and meta-BO in this section. Given that TAF+ is built upon
an existing meta-BO method called Transfer Acquisition Function(TAF), we also provide an overview of TAF and its limitations.

3.1

BO using Expected Improvement as theacquisition function

BO identifes the optimal parameter setting that maximizes or minimizes an objective function � (e.g., completion time) over iterations.
In each iteration, BO selects a parameter setting (denoted as �) to beevaluated (� (�)), resulting in an objective function value � = � (�).
BO has two key elements: the acquisition function determines which� should be evaluated in each iteration, and the surrogate model ofthe true objective function � .
Since each evaluation of � is expensive e.g., through user interaction, BO relies on acquisition functions, which is cheaper to evaluate[38], to determine the next setting to be evaluated. In each iteration,
BO samples many parameter settings and calculates their acquisition values (its “worth value”) with the acquisition function. The �
with the highest acquisition value is picked for the actual evaluation. To generate the acquisition value of a given �, the acquisition
function relies on BO’s another element — the surrogate model.
This surrogate model is typically a Gaussian Process regression(GP) [70, 83], which generates the predicted objective value (�ˆ) and
its associated uncertainty (i.e., variance)3 for a given �. After theactual evaluation of � , each BO iteration results in an observationof (�, �) pair; all the observations are then ft into the surrogate
model (i.e., GP). As BO accumulates more data through iterations,
its GP becomes more accurate to the true function � , enabling theacquisition function to make better predictions. For more details
on BO, please see [27].
Among common acquisition functions [27, 98], we select Expected Improvement (EI) as the base acquisition function since it isused in the TAF paper, upon which we develop our TAF+ algorithm.
An intuitive way to understand Expected Improvement (�� (�)) isthat it calculates the amount of potential improvement in the objective function from the current best observation. A formal defnitionis:�� (�) ≡ E � (� ) {max[�ˆ(�) − � +, 0] | H},(1)
where �� (�) is the acquisition value at � for the current iteration�, �ˆ = �ˆ(�) is the predicted objective value at � based on the
GP model, � + is the best-observed performance thus far over the
3 Note: when dealing with human-in-the-loop tasks, it is recommended to incorporate

the GP with the inferred noise levels such as the Botorch’s single-task GP implementation: https://botorch.org/docs/models.

A Meta-Bayesian Approach for Rapid Online Parametric Optimization for Wrist-based Interactions

whole optimization history H, consisting of all previous datapoints{(� 1, �1 ) . . . (�� −1, �� −1 ))}.

3.2

Meta-BO

Meta-learning is a paradigm of machine learning, focused on achieving fast adaptation in a given task by leveraging prior data of similartasks [35]. In the context of HitL BO, each task is to identify the
optimal parameter setting of a user using BO. Thus, the goal ofmeta-learning for BO (i.e., meta-BO) is to leverage the optimization
data of previous users to enhance the efciency of BO for the newuser(s). There are generally two phases in meta-BO (Figure 3): The
frst phase is population modeling which involves gathering datafrom a set of users. We run HitL BO on each user, resulting in one
GP model (i.e., surrogate model) per user. We defne these models
as population models. In the second phase, adaptation, meta-BO is
deployed on the new users. In particular, a new GP is constructedby ftting the observations of the new user, while leveraging the
population models when possible. We call this new GP model adaptation model since it aims to “adapt” the previous GP models to the
new user.

3.3

Transfer Acquisition Function (TAF): a
meta-BO method

BO needs to search randomly in the initial iterations since its surrogate GP model does not have information to guide a meaningful
search. Transfer Acquisition Function (TAF) is a specifc meta-BO
method that addresses this limitation by utilizing previous population models as informative prior for guiding the search. Specifcally,
TAF is an acquisition function that considers both the adaptation
model built upon the observations of the current user and the previously gathered population models. Thus, even when the adaptation
model has no (or limited) information, population models can still
guide the optimizer in selecting a setting (�) that is likely to leadto good performances (�). Similar to the purpose of the regularacquisition functions in the BO process, the � with the highest TAF
value will then be evaluated by the new user in each iteration. The
gathered observation will ft into only the adaptation model, further
improving its predictions in the later iterations.
TAF aggregates the Expected Improvement (�� ) value from thecurrent adaptation model as well as from all the population models
for a given parameter setting �. Here we defne the �� calculated
from the population models as Population Expected Improvement(��� ) to diferentiate from the �� calculated from the current adaptation model. ��� is calculated similarly to �� (see Equation 1, andrefer to section 7 of the original paper [100] for more details). This
leads to one �� value and � ��� values for a given �, where � isthe number of population models. TAF then computes a weighted
combination of these values to obtain the acquisition function valueat �:Í�� �� (�) =� =1 � � ��� � (�) + � � +1 �� (�),Í�+1� =1 � �(2)where �� (�) is the Expected Improvement from the adaptation
model, while ��� � (�) is the Population Expected Improvement

CHI ’24, May 11–16, 2024, Honolulu, HI, USA

based on the �-th population model. � ∈ 1, ..., � denotes the index of the population models. Finally, � � are weights on each ��� ,and � �+1 is the weight assigned on the �� . We defne this summary across diferent models as “between-model combination”(Figure 3).
3.3.1 Model weights. TAF computes a weighted combination ofthe �� and ��� values using weights � � . A higher weight meansthat this model’s �� (or ��� ) value is more valuable or reliable. Here,
we defne such weights on models as “model weights”. Following
Wistuba et al. [100], we use a variance-based method for determining these weights. An intuitive explanation is that the weightof a model is based on the confdence of its prediction at �. Low
variance from a particular model (see the red area of each modelin Figure 3) indicates higher confdence, so its resulting weight ishigher in TAF computation (Equation 2). On the contrary, whenthe variance is large, the model has less confdence in its prediction,
so the corresponding weight should be lower4 .
The population models generally do not have a large variancesince they are already ftted with data from the previous optimization processes. However, at the beginning of an adaptation, theadaptation model has none or very few datapoints, so the varianceis overall high, leading to its low model weight. Hence, TAF initiallyrelies more on the population models. Figure 3 shows an example.
As the adaptation model is ftted with more observations from thecurrent user, its variances decrease over iterations. Consequently,
TAF gradually increases the adaptation model’s weight after ample
iterations, achieving personalization.
3.3.2 Limitations of TAF. Prior works have evaluated the performance of TAF with various single-objective testing functions. TAF
signifcantly outperformed the standard BO while being computationally lightweight [93, 100]. However, TAF has two major limitations, making it unsuitable for realistic HitL problems. It was
designed to handle a single objective. Yet, a realistic interaction
usually involves multiple objectives, which are unclear as to how toaddress with TAF. Secondly, TAF does not allow proactively shifting
weights from population models to the adaptation model. Although
TAF gradually increases the weights of the adaptation model withmore observations, there are scenarios in which we hope TAF relieson the adaptation model in earlier iterations. For example, when
the new user exhibits behaviors diferent from all the population
models, shifting the importance to the adaptation model allows fora more efcient user-specifc adaptation. However, TAF does nothave a mechanism to support that.

4

TRANSFER ACQUISITION FUNCTION+(TAF+ ) AND ITS ACCOMPANYING
WORKFLOW

We develop our Transfer Acquisition Function+ (TAF+ ) algorithm
by leveraging TAF. Accompanying the algorithm is its workfow(see Figure 5). The frst three steps are ofine steps to be performedby the developer before deploying the method on the device. The
4 Please refer to Section 6.1 of the original paper [100] for details, in which this particularweighting method was named TAF-M. The same implementation was replicated in
Volpp et al. [93], where it was referred to as TAF-ME.

CHI ’24, May 11–16, 2024, Honolulu, HI, USA

Y.-C. Liao, et al.

PHASE 2: Adaptation

PHASE 1: Population modeling
Population
user #2

Population
user #1

y

Mean

0.7

No data yet(or limited data)

Data

Data
Population
model #1

New userxy

PEI1(0.7): model’s acquisition value (eq. 1)
w1 : model’s weight based on
the variance (red area)

Mean

Population
model #2

x

PEI2 (0.7)
w2

0.7

Adaptation
model

y

Mean

0.7

x

EI (0.7)
w3

Between-model Combination(Summarizing acquisition values
across all models, eq. 2)

Value of
TAF(0.7)

Adaptation model
Population models

Figure 3: Overview of Transfer Acquisition Function (TAF) with an example of 2 population models: TAF is a weighted sumof several acquisition values generated by the currently constructed model and the models gathered in advance (i.e., �� or��� � where � is the index of the models). In PHASE 1, Population models are constructed per user using optimization data.
Each model predicts the user performance �ˆ (red line), the corresponding acquisition value (��� ), and the uncertainty ofthis prediction (the red area) of a specifc �. In PHASE 2, a new Adaptation model is created for the new user. To derive theacquisition value, TAF computes between-model combination values across all models (including adaptation model’s �� andpopulation models’ ����) based on the model weights. Model weights are denoted as � � , and they are computed based on thevariance (width of the red area) of each prediction. The �� (or ��� ) with a higher uncertainty have lower weights. The example
is computing the TAF at � = 0.7, where the adaptation model has very high uncertainty in early iterations, so the TAF value
is majorly determined by the population models. As the adaptation model gains more observations, TAF will gradually bedominated by the Adaptation model, leading to the user-specifc optimal result.

fnal step is the online adaptation that the end-users experience
while performing the pointing interactions with the device.
Here, we frst introduce TAF+ and then provide an overview of
each step of the workfow. Lastly, we compare the TAF+ workfow
with other baseline methods.

4.1

Transfer Acquisition Function+ (TAF+ )The main idea of TAF+ is to mitigate the limitations of TAF withtwo crucial extensions: dynamically handling multiple objectives
and proactively balancing the weights of the population models andthe adaptation model.
4.1.1 Extension 1: Dynamically handling multiple objectives. In realistic interactions, there is usually more than one objective function,
which limits the direct application of TAF. A naive solution is aweighted-sum approach that transforms multiple objectives intoa single objective, where a set of weights on each objective must
be predefned. The resultant weighted-sum objective can then be
used for both population modeling and adaptation phases, thereby
enabling the application of TAF to multi-objective settings. Despite
its simplicity, this approach has various limitations for realistic
tasks. The weight assignment needs to be arbitrarily done by thedesigner beforehand, and there is no fexibility to tune the weights
later. In practice, there is a high potential that the designer would
need to adjust the weights since the predefned weights may not
be ideal.

As shown in Figure 4, our TAF+ takes a diferent approach. Instead of predetermining a fxed set of weights for objective functions, our population modeling is performed in a multi-objective
manner. That is, our population models generate multiple acquisition values, each for one objective, instead of only a single value,
for a given �. Then, in the adaptation phase, TAF+ dynamicallycombines these acquisition values into one value per model according to the weights assigned to the objective functions. Such weights
can be adjusted whenever needed.
We denote the acquisition values on diferent objectives as �� �or ��� � , where � is the index of the objectives. During the adaptation phase, for each model, TAF+ frst combines these values ofdiferent objectives into a single acquisition value (�� + or ��� + ,where the + sign indicates this is an aggregated value) based on
the weights of objectives. We refer to this combination as “withinmodel combination” to diferentiate from the between-model
combination (Figure 3), and we defne these weights of the objectives as “objective weights” to diferentiate them from theweights on models. This feature enables a designer to dynamicallyadjust the objective weights at any time, even after the population modeling. Designers can even dynamically tailor the objective
weights for diferent users or contexts. In subsection 4.2, we illustrate a workfow that allows designers to identify the optimal
objective weights based on the users’ subjective ratings (see step
2 in Figure 5). Subsequently, TAF+ combines all the acquisition
values across diferent models into a fnal value based on the modelA Meta-Bayesian Approach for Rapid Online Parametric Optimization for Wrist-based Interactions

CHI ’24, May 11–16, 2024, Honolulu, HI, USA

PHASE 1: Population modeling

PHASE 2: Adaptation
Population
user #2

Population
user #1

New user

No data yet(or limited data)

Data

Data
Population
model #1

Mean of obj1
y1

Mean of obj2
y2
Population
model #2

0.7

x

0.7

x

1

PEI 1 (0.7)
w 11

0.7

Adaptation
model

x
2

1

2

PEI 1 (0.7)
w 21(Summarizing across multi obj.)+

PEI 1 (0.7): summarized model’s acquisition value (eq. 4)
w+1 : summarized model’s weight (eq. 5 & 6)

Between-model
combination, eq. 3

Within-model
combination++

PEI2 (0.7)

EI (0.7)
w+3

+

w2

Value of+TAF (0.7)

EI (0.7)
w 23

EI (0.7)
w 13

Within-model
combination

Within-model combination

2

1

PEI 2 (0.7)
w 22

PEI 2 (0.7)
w 12Within-model combination
Between-model combination

Figure 4: Transfer Acquisition Function+ (TAF+ ) in an example task of two objectives: Both the Adaptation model and thePopulation models can generate predicted performances of two objectives (red/blue lines), the corresponding acquisition values(�� � or ��� �� , where � is the index of the objectives and � is the index of the models), and the uncertainty of this prediction (shownas the red/blue are). The two major phases (population modeling and adaptation) are identical to TAF (Figure 3). For handling
multiple objectives, TAF+ frst performs a within-model combination: Within a model �, it summarizes acquisition values (�� ��or ��� �� ) of diferent objectives into a single weighted-sum value (�� �+ or ��� �+ ) and summarizes weights of diferent objectives(� �� ) into a single weight (� +� ) based on the objective weights. Once every model has a summed acquisition value (�� + or ��� �+ )and a model weight (� +� ), TAF+ then performs a between-model combination, similarly to TAF, deriving the fnal TAF+ value.

weights. Such a between-model combination is identical to thatin TAF. To this point, TAF+ can be formally written as:++++� =1 � � ��� � (�) + � � +1 �� (�),Í�+1 +
�=1 � �Í�+� �� (�) =(3)

where �� + and ��� + are weighted sums of �� and ��� values respectively from several objectives. � ∈ 1, ..., � denote the index of
the population models. Assuming � diferent objectives, we candenote �� + and ��� + as:��� �+ (�) =

�
∑
�=1

� � · ��� �� (�),�� + =�∑� � · �� � (�),index of the objectives. During the within-model combination in
TAF+ , similar to the process of deriving �� + or ��� + , the objective
weights � � are utilized to combine multiple weights � � into one
single weight � +� :

(4)

�=1

where � ∈ {1, ..., �} denotes the index of objectives, and � � is theobjective weight of the �-th objective function. Furthermore, incontrast to TAF, where each population or adaptation model associates with a single objective and has one model weight, TAF+
considers multiple objectives. Consequently, each model has multiple weights – each is associated with one objective for each EIor PEI. Formally, we denote these weights as � � where � is the� +� = � (�) ·

�
∑
�=1

� � · � �� (�),

+
� �+1
=

�
∑

�
�� · ��
+1 (�),(5)

�=1where � � is the same objective weight shared with Equation 4, and� �� (�) is the �-th model’s weight on the �-th objective function. Each� �� (�) value is calculated in a manner similar to TAF, as describedin subsubsection 3.3.1. � (�) is a decay factor applied only to the
population models, which will be elaborated in subsubsection 4.1.2
and Equation 6.
4.1.2 Extension 2: Proactively balancing the weights of population
models and the adaptation model. The second extension is meant
to cope with the potentially high user diversity. � (�) in Equation 5denotes the decay of the weights on the population models. This
decay is a scalar value ranging [0, 1], which is directly multiplied by
the original weight values � �� (�). � (�) = 1 means there is no decay,

CHI ’24, May 11–16, 2024, Honolulu, HI, USAand � (�) = 0 means the adaptation fully relying on the adaptation
model. An ideal decay should update over iterations – it may not beefective in the early iterations since the adaptation model has very
limited information but should increase over iterations as more data
becomes available from the current user. The decay � (�) is thus
an iteration-dependent function with two hyperparameters: � 1 theiteration number after which the decay kicks into efect, and theother � 2 determines the rate of the decay. We can formally describe� (�) as:
1,if � ≤ � 1� (�) = 1 − (� − � 1 ) · � 2, if � 1 < � ≤ � 1 + �1
(6)
2


 0,
otherwise
where � is the count of the iteration, � 1 is a positive integer (or 0),
and � 2 ∈ (0, 1]. There is no decay when the iteration count is lessthan � 1 . After � 1 -th iteration, the decay starts with the rate of � 2 ;i.e., every iteration, the scalar � (�) decreases by � 2 . Once the scalarreaches 0, it stays at 0, so � +� stays 0 as well, and hence adaptation
starts fully relying on the adaptation model.
With this extension, TAF+ allows actively determining how the
population models should decrease their importance. For instance,
in a population where all the users exhibit high similarity, theoptimal design for the new user is likely to be highly similar to the
population models. Hence, the designer can set � 2 as 0, allowing the
population models to consistently guide the current optimization.
On the other hand, when there is a higher diversity between users,
a designer can leverage the population models in the initial steps
and let the adaptation quickly develop based on the current data.
In such a case, the designer can set � 1 and � 2 to properly decay toachieve faster adaptation. Our workfow has an additional step toderive the optimal � 1 and � 2 (see step 3 in Figure 5). On the contrary,
TAF does not have a similar mechanism, potentially not suitablefor interactions requiring fast adaptation.
4.1.3 Potential generalizability of TAF+ for HitL optimization: TAF+
shares the foundational principles of BO, a versatile parametric optimization method with minimal assumptions of the task [27, 84]
and has proven its generalizability over a wide range of HCI applications [9, 15, 17, 47, 57, 87]. Extended from BO, the only essential
assumption of our TAF+ is that while users have individual differences, there exist parameter ranges that generally lead to gooduser performance. This is a common assumption in HCI and design,where a certain range of parameters or designs are considered efective for the broader user base despite individual disparities. Thus,
TAF+ can be used as a general approach for other HitL problems as
well. In rare occasions where there are completely no overlapping
traits between the population models, designers can use the TAF+
workfow to forecast this outcome (see subsection 4.5) and use othermethods instead. We demonstrate the potential generalizability of
TAF+ with a series of simulations, as presented in subsection 4.6.

4.2

Overview of the TAF+ workfow

Figure 5 provides the overview of TAF+ workfow, which contains
three steps to prepare the population models followed by the deployment in the adaptation phase. The frst step of TAF+ workfow
entails building population models that generate multi-objective

Y.-C. Liao, et al.

predictions. This step eliminates the need to predefne the objective weights for multiple objectives in advance and enables the
fexibility of setting them later. The second step focuses on identifying the optimal objective weights � corresponding to the maximum subjective ratings. By optimizing the objective weights, thedesigners can actively steer the optimization to explore the parts of
the Pareto-frontier that maximize the user’s feedback. In the third
step, the optimal decay hyperparameter settings (Equation 6) for
the gathered population models are identifed using grid search insimulations. Diferent decay hyperparameter settings are tested insimulation to identify which setting leads to the optimal simulated
adaptation performance. Finally, we deploy meta-BO, our TAF+ ,in adaptation on the end-users, where the population models,
optimal objective weights, and the decay hyperparameters are
utilized. We detail each step below.

4.3

Step 1: Population modeling

We performed a data collection where users went through HitL
optimization guided by multi-objective BO. The data of each user isthen used to construct a GP model, which predicts the performanceof multiple objective functions when given a �. Note that this step
does not incur any additional costs for end-users because the endusers only experience the adaptation phase (see Figure 5).

4.4

Step 2: Objective weight � optimization

TAF+ transforms the optimization problem from multiple objectives
into a single objective based on objective weights. The selected
three objectives involve intrinsic trade-ofs, the same as other input
devices. To identify the optimal weights, the subjective ratings ofeach user’s Pareto-optimal designs are obtained and used to identify
the weight set that results in the designs with the highest subjective
rating. This step is a “population-level” weight optimization because
its goal is to identify the weight setting that captures the highest
ratings across all users.
4.4.1 Objective weight optimization for a single user: Consider asimplifed single-user scenario for a better understanding of our
method (Table 1). Our procedure involves sampling a list of possible
weight sets (e.g, [0.1, 0.1, 0.8], [0.1, 0.2, 0.7], ... for a problem withthree objectives). Each weight set is applied to all the Pareto-optimal
designs’ objective values of this user to identify the design among
the Pareto frontier with the highest weighted-sum objective value.
We then record the user’s rating for this particular design as thescore of this weight. By trying out all the weights and comparing
their corresponding ratings, we can identify the optimal objective
weights that leads to the highest user rating.
With a single user, we could conclude the objective weightassignment by assigning weights in accordance with the highest
user rating. However, at the population level, diferent users may
favor the objectives diferently. We therefore need to fnd the objective weight assignment that is the best across all users in the
population. For instance, for the user presented in Table 1, design B
has the highest user rating, and its third objective function has thehighest value. Intuitively, it suggests that this user favors the third
objective; then assigning the objective weights as [0.1, 0.1, 0.8] isa reasonable and straightforward solution. However, at the population level, diferent users potentially favor diferent objectives, so it

A Meta-Bayesian Approach for Rapid Online Parametric Optimization for Wrist-based Interactions

CHI ’24, May 11–16, 2024, Honolulu, HI, USA

A new
user

Population
Subjective ratings on
Pareto-optimal designs

Multi-objective
optimization
STEP 1:
Population
modeling

Multi-objective
population models

STEP 2:
Objective
weightoptimization

Optimalobjective weights(α)

STEP 3:Decay
hyperparameter
optimization

Optimalhyperparametersetting (d1, d2)

Rapid, online
optimization
Adaptation+(Deploy TAF )Optimal setting
for the new user

Figure 5: TAF+ workfow: subsection 4.2 provides the details and explanations of each step. Similar to other meta-learning
workfows, the frst step in ours is population modeling, and fnally, TAF+ is deployed on the end-users (adaptation). TAF+
workfow has two additional steps (steps 2 and 3) for deriving the optimal objective weights and the decay hyperparameters.

Pareto-optimal
designs/settings

Original
objective values

Weighted-sum
objective value(weights=[0.7, 0.2, 0.1])

Weighted-sum
objective value(weights=[0.2, 0.3, 0.5])

Weighted-sum
objective value(weights=[0.2, 0.6, 0.2])

User ratingA
B
C[7, 2, 5][5, 4, 8][3, 9, 2]

5.8
5.1
4.1

5.5
6.2
4.3

3.6
5
6.4

75
100
1

Table 1: The table shows a set of Pareto-optimal designs in a three-objective optimization problem. We demonstrate how to
obtain the optimal objective weights leveraging the user ratings. The second column shows that designs A, B, and C havediferent objective value sets. Columns 3 - 5 demonstrate we can calculate the weighted-sum objective values when a set ofobjective weights are given. Diferent objective weights lead to diferent optimal designs. For instance, under weights= [0.7, 0.2, 0.1], design A is the best design. However, under weights = [0.2, 0.3, 0.5], design B is the optimal design. Since wehave gathered the user’s ratings on each design, we can compare which objective weight setting leads to the design that
corresponds to a higher user rating. In this example, the second weight ([0.2, 0.3, 0.5]) is the most preferred weight among thethree weight settings because it leads to B as the fnal design, whose user rating is the highest, 100.

is unlikely to intuitively identify a single objective that leads to thehighest ratings for all. Therefore, a grid search is a more thoroughsolution. More details are below.
4.4.2 Objective weight optimization across all users: To obtain
the best weight setting across all users, we leverage the processdescribed in subsubsection 4.4.1 for each user in the population. In
particular, we obtain the weighted-sum optimal designs among thePareto-optimal designs of all users by running the aforementioned
process for a given weight set (e.g, [0.1, 0.1, 0.8], [0.1, 0.2, 0.7], [0.1,0.3, 0.6], ...). Then, all the users’ subjective ratings corresponding
to the resultant optimal designs are combined as a fnal score forthat objective weight set. Finally, the fnal scores of all weightsets are compared, and the weight set that leads to the highest net
user rating is identifed. Appendix A.1 elaborates on the needfor searching for the best objective weight confguration in thismanner through a small example scenario. Appendix A.2 also
provides the detailed algorithm that is explained here.

4.5

Step 3: Decay hyperparameter optimization

TAF+ has a set of hyperparameters that decreases the weights ofthe population models (Equation 6). The step aims to identify the
optimal hyperparameter setting through simulations. In these simulations, we take one population model at a time and treat it as a
new user. Meanwhile, we treat the remaining models as population

models to conduct TAF+ . We then performed a grid search overdiferent sets of < � 1, � 2 > values to identify the < � 1, � 2 > pair thatyielded the best performance at the population level. This optimal
hyperparameter setting can then be used for subsequent TAF+ runs.
This simulation can be an efective pre-check step before deploying meta-BO. Future practitioners can utilize such simulations toforesee the potential efcacy of meta-BO for their own tasks. In
rare cases, users may behave completely diferently for a particularinteraction. This step informs the practitioners that TAF+ would
not outperform BO regardless of the hyperparameter setting, sothey can deploy a standard BO instead. Furthermore, practitioners
can utilize the simulations to observe the performance of meta-BO
with diferent numbers of population models and learn that larger
user groups may be needed in certain cases.

4.6

Evaluating TAF+ ’s viability via simulations
on synthetic test functions

Before applying TAF+ onto the target interactions, we present synthetic simulations that evaluate TAF+ in multi-objective problems
with common test functions, such as Sphere5 , Branin6 , and Hartmann 3D7 . Our simulations use the same objective functions and5 See https://www.sfu.ca/~ssurjano/spheref.html.
6 See http://www.sfu.ca/~ssurjano/branin.html.
7 See https://www.sfu.ca/~ssurjano/hart3.html.

CHI ’24, May 11–16, 2024, Honolulu, HI, USA

number of parameters as relative pointing. We evaluated TAF+ ’s
performance under 6 diferent objective weight confgurations,highlighting its advantage of ofering high fexibility for the designers to fne-tune the objective weight when needed. We further
evaluated TAF+ ’s performance with fve levels of user group similarity, showing its efectiveness even when users exhibit high differences. We also evaluated the potential generalizability of TAF+
with four diferent functions and with fve population model sizes.
Our simulation analysis shows that TAF+ always converged to
global optimality and outperformed BO in various conditions. This
provides evidence for its potential in a wide range of interactions.
Appendix B presents the detailed procedure and results of oursimulations.

4.7

Summary

To summarize, TAF+ improves over TAF so as to handle HitL scenarios in two aspects: it can fexibly handle multiple objectives
and it can proactively decay the weights of prior models as
the user progresses in the adaptation phase. Further, TAF+ achieves
its goal of converging faster than standard BO in the synthetic
simulations.

5

INTERACTIONS

We study two distinct and representative wrist-based pointing interactions: absolute pointing and relative pointing. While these twointeractions share the same task and objective functions, they use
diferent hardware (IMU v.s. infrared sensors), diferent body parts(forearm v.s. wrist angular motion), diferent device parameters(forearm yaw-pitch v.s. wrist angle), diferent transfer functions(linear v.s. sigmoid) and diferent parameter counts (2 v.s. 4). We
purposely chose these two cases to demonstrate the efectivenessand the potential generalizability of the meta-BO approach. Below,
we frst describe the shared details and then the two interactions.

5.1

Task and software interface

5.1.1 Task: Both interactions involved 2D target selection (Figure 6), where participants were asked to move the cursor to thetarget and select it by performing a double pinch. The pinches
were detected using the highly accurate active electrical sensing
approach, the same technique used in ElectroRing [42]. Participants were asked to select the targets “as quickly and accurately as
possible”.
5.1.2 Interface: Our goal is to deploy meta-BO online while theuser performs pointing in a real-world interface. We thus design
our study interface with varying target sizes and distances in a gridto resemble a real-world interface (see Figure 6 a). The targets are
circular, arranged in an 8 × 4 grid. The diameter of each circle isuniformly sampled from a range of [20, 35] mm. The default color of
all the circles is light grey, and the target circle is highlighted in blue.
Upon selection, the target turns red. The system then randomly
samples a diferent circle as the next target. The cursor does notreset to the origin between selections. We apply a one-euro flteron the cursor position to overcome jitter [12].

Y.-C. Liao, et al.

5.2

Objective functions

Two interactions share the same objective functions, which we aim
to optimize. To prevent bias toward any objective function duringmulti-objective optimization, these objective functions were further
linearly normalized into the range of [−1, 1]. We converted the
problem into a maximization problem, so 1 is the best performanceand −1 is the worst8 . Similar normalization was also done in priorwork [15].
1. Normalized completion time (��� ): Completion time (CT) is theduration from the moment that the cursor leaves the previous target
to when the selection is complete, a typical way to assess the input
efciency. Our targets varied in size and distance; a standard way
to counterbalance the efect of these variances is to average theperformance over many selections (e.g., [15]). However, since weaimed to use the minimal number of selections for the optimization
process, we normalized the completion time with the Index ofDifculty (ID) [61], and thus ��� = ���� .
2. Trajectory aiming error (� ��): TAE is a crucial metric for evaluating pointing accuracy [15, 52]. We follow AutoGain [52] that used
the Persistence1D algorithm [50] to segment a full trajectory intosub-movements and then calculated the aiming error of each ballistic sub-movement. For simplicity, we assumed the implicit aiming
point to always be the target position. After excluding the unaimed,interrupted, and non-ballistic movements, we then calculate theaiming error (overshoot and undershoot) of each sub-movement
as the distance between the cursor position at the local minimum
speed and the closest edge of the target (Figure 6 a). Summingup all the aiming errors (��� , where � ∈ [1..�] is the �-th ballisticÍ
sub-movement), � �� = ��=1 ��� .
3. Trajectory Travel Distance (�� �): TTD measures the amountof detour of a selection trajectory. A selection could be fast andaccurate, but if the cursor travel distance from the previous to thecurrent target is longer than the shortest distance, it indicates apotentially skewed transfer function. This metric is expressed as�� � = �� /�� , where �� is the measured travel distance and ��is the ideal distance. In absolute pointing, if the transfer function
values misalign with the user’s assumed ratio, the user would takeextra distance to move the cursor along the intended direction. In
relative pointing, a rotational misalignment in the x-y mapping may
result in deviations in motion. While TTD may be correlated withthe previous objective functions, the correlation is partial. TTD
captures additional useful information. Note that there are intrinsic
trade-ofs in the selected objectives.

5.3

Optimization iteration

From a 4-person pilot test, we decided to have 6 target selections
in each optimization iteration. The frst two selections were seen
as “practice” because the users may still be adapting to the new
setting. We took the average value of the three objective functions(see subsection 5.2) of the last four selections as the fnal objective
values of that iteration.
8We conducted a 4-participant pilot study to derive the parameters for the linear

normalization which ensures three objective functions have similar mean values andranges.

CHI ’24, May 11–16, 2024, Honolulu, HI, USA

Cursor velocity

A Meta-Bayesian Approach for Rapid Online Parametric Optimization for Wrist-based Interactions(c3=0.5)(c2 = 2)ab

Wrist velocitycFigure 6: Our study interface: (a) The main study interface. The smaller black dot is the cursor, the red circle is the previous
target, and the blue circle is the current target. (b) The interface for calibrating for the relative pointing. The cursor only moves
on the y-axis during calibration when the participant performs radial-ulnar deviation motions. (c) Example transfer functions
for relative pointing.

Table 2: Design parameterization of absolute pointing. Bothscalars have been normalized to the [0, 1] range.

Table 3: Design parameterization of relative pointing.
Design Parameter

5.4

Design Parameter

Range

�� :
�� :

[0, 1][0, 1]

Scalar factor maps ��� to �
Scalar factor maps ����ℎ to �

angular correction parameter
Sigmoid function (overall shape)
Sigmoid function (center � position)
Sigmoid function (overall scale in � axis)[ −10◦ , 50◦ ][0.5, 2.5]
[0.5, 2.5]
[0.5, 2]Absolute pointing

Our absolute pointing interaction utilizes an IMU on the wrist to
detect the absolute position of the forearm (Figure 1). The challenge
here is to identify the ideal function that maps the forearm positions
to the cursor’s 2D positions.
5.4.1 Device and interaction: The interaction linearly maps the
IMU < ���, ����ℎ > to < �, � > coordinates on the interface.
Initially, the < ���, ����ℎ > corresponding to the user’s preferredcentral forearm position in the air is mapped to < 0, 0 >. The systemthen linearly updates the cursor’s � & � positions based on ��� &����ℎ respectively. Ideally, each < �, � > coordinate corresponded
to a specifc < ���, ����ℎ > pair.
5.4.2 Design parameters: The transfer functions for determining
cursor position are:�� (�) = �� (0) + (��� (�) − ��� (0)) · �� ,(7)
� � (�) = � � (0) + (����ℎ(�) − ����ℎ(0)) · � � ,where �� (0) and � � (0) denote the cursor’s centered � and � positions in the scene. �� (�) and � � (�) stand for the cursor’s � and �
positions at time �. ��� (0) and ����ℎ(0) are the centered ��� and����ℎ values. ��� (�) and ����ℎ(�) are the ��� and ����ℎ value attimestamp �. The two design parameters that need to be determined
are the transfer scalars �� and � � (Table 2). For diferent users, different motion ranges may be optimal (which may also depend on
the task), thus requiring per-user parameter optimization.

5.5

�� :
�1:
�2:
�3:

Range

Relative pointing

Our relative pointing (Figure 2) interaction is similar to mousepointing where users move the cursor by clutching [66]. The cursor’s direction is determined by the sensed wrist movement’s angle.

The cursor’s moving velocity is determined by a velocity functionthat takes the wrist’s relative velocity as input. The user pinches toinitiate cursor motion, rotates the wrist to move the cursor in thedesired direction, and releases the pinch to end cursor motion. The
user clutches and performs this repeatedly to reach the target.
5.5.1 Device and sensing: We used the same approach as
Salemi Parizi et al. [77] to detect wrist angles via infrared (IR)sensors on a wristband. The detected wrist fexion/extension andradial/ulnar deviation at each timestamp were mapped to the wrist
plane’s x and y coordinates, respectively (see Figure 2b). Withmeasurements at two consecutive timestamps, we derive the wrist
velocity (�� ), and the detected wrist movement’s angle (�� ).
5.5.2 Determining cursor’s moving direction: One important factor
is the wearing position of the device. Ideally, the device should beplaced exactly perpendicular to the body (Figure 2b). Then, moving
the cursor along �� would match the user’s intention. Yet, users
wear the sensor slightly diferently. The slight angular diference
between the ideal and worn positions can cause the cursor to movein unintended angles (see Figure 2c).
We introduced an angular correction parameter �� , which is avalue that directly adds to the sensed wrist angle such that � � =�� + �� , where � � is the corrected angle, �� is the sensed angle,
and �� is the correction parameter. After the correction, the cursor
moves as intended � � . Figure 2d shows an efective correction.
Thus, �� is an important design parameter that directly impacts�� �. Here, we defne our relative transfer function:�� (� ) = �� (� − 1) + � (�� ) · cos � � ,� � (� ) = � � (� − 1) + � (�� ) · sin � �(8)where �� (�), � � (� ) are the cursor �, � at timestamp �. � (�� ) determines the cursor velocity based on wrist velocity �� .

CHI ’24, May 11–16, 2024, Honolulu, HI, USA

5.5.3 Determining cursor’s velocity: As in [65], our velocity function � (�� ) is a sigmoid function, and consists of 3 parameters(� 1 , � 2 , � 3 ), which defne the sigmoid curve properties (detailed inTable 3):1
· � 3,(9)
1 + � −� 1 × (�� −� 2 )In total, there are 4 design parameters to be optimized: the aforementioned angular correction, �� , and 3 parameters in the velocity
function. From an optimization perspective, relative pointing ismore complex than absolute. In addition to diferent users having
diferent wrist rotation ranges, here tiny variations in wearability
may lead to large sensing variations due to how IR sensors work.
Additionally, since only the relative wrist motion matters, the user’s
arm is positioned downwards here (e.g., [60]), which enables relaxeduse (see Figure 2a).
� (�� ) =6

IMPLEMENTING TAF+ WORKFLOW FOR
OUR WRIST-BASED INTERACTIONSIn section 4, we introduced the details of each step in the TAF+
workfow. Here, we report the process, data collection, and resultsof each step with our target interactions.

6.1

Step 1: Population modeling via a user datacollection

For each interaction, participants performed the task while theparameters were changed every iteration based on multi-objective
BO using Expected Hypervolume Improvement [19] and BoTorch[4]9 . The objective functions and design parameters were detailed
in section 5. We ran 25 and 40 iterations for absolute and relative
pointing, respectively. These numbers refect the complexity of thetasks. These collected data, i.e., 25 and 40 pairs of (�, �), were used
to construct the population models.
6.1.1 Procedure. We recruited 14 participants, 5 male, 8 female, and1 non-binary, aged 22 − 57 (������ = 28.5). None of them have any
experience with VR. The data collection took 2 hours. Because we
aim to analyze the two interactions independently, all participants
went through relative pointing frst and then absolute pointing,rather than mixing the order of the interactions. To familiarize
participants with the task, we provided 10 “practice iterations”,
in which we uniformly sampled design instances from the entire
design space. There was a 1-minute break every 10 iterations and a
5-minute break between the two interactions.

Y.-C. Liao, et al.

objective functions, and there is not a single optimal design thatcan maximize the three performance metrics.

6.2

We determine the ideal values of the objective weights throughoptimization as described below.
6.2.1 Gathering user ratings. We compare the quality of the Paretooptimal weight settings based on the participants’ subjective ratings.
For each participant, after they fnished the 25 and 40 iterations in
the previous step, we extracted all the Pareto-optimal parametersettings among these samples. Then, we asked the participants toperform pointing with these settings and rate them subjectively.
The instruction was: “For each of the following designs, please rate
how much you agree with this statement: [This design is easy to use].
Please rate from 1 to 100; 1 stands for strongly disagree, and 100 meansstrongly agree.” Participants interacted with each Pareto-optimal
design twice in a randomized order. We took the average of thetwo ratings on the same parameter setting as its fnal rating. We
normalized each user’s ratings such that the lowest and highest
ratings of a user would be 1 and 100. If the rating scale is scarce(e.g., only 5 or 7 levels), we may end up with many optimal weightsthat reach identical results when running the objective weight
optimization. We therefore provided a 1-100 scale to gain the most
granular information even if the users are not able to be exact intheir assessments. To avoid a scenario where a user who rates all
designs highly dominates the optimization process, we normalizedeach user’s rating to a fxed range.
6.2.2 Results of objective weight optimization: We observed that
5 users see the 1-100 range as 10 levels (only rated at tens digits),
4 users see it as 20 levels (only rated at every fve digits), and wefurther found 5 participants provided more fne-grained ratings(such as 73 or 92). This shows that the 1-100 scale allows eachuser to be fexible about the granularity they want to use for theirscores. Following the procedures described in subsection 4.4, we
sampled all the weight combinations to the frst decimal and thenperformed the population-level objective weight optimization.
The resulting optimal weight settings of [NCT, TAE, TTD] for
absolute and relative pointing were [0.4, 0.3, 0.3] and [0.5, 0.3, 0.2],
respectively. The resulting objective weights highlighted theconsistent importance of speed (indicated by ��� ) in pointing
interactions.

6.3
6.1.2 Deriving population models. We derived 14 population models for each interaction. Appendix C shows the plots of the hypervolume increase of two interactions at each iteration. On average,
there were 4.8 (�.�. = 0.78) and 5.79 (�.�. = 1.12) Pareto-frontiersettings for absolute pointing and relative pointing, respectively.
This showed there are intrinsic trade-ofs between the selected
9 Hyperparameter settings: We used a single-task GP with Matern 5/2 kernel. The

single-task GP in our implementation infers a homoskedastic noise level of the observed
data (https://botorch.org/docs/models). Similar to Chan et al. [15], we set � = 1. We
further set 10 optimization restarts for the optimization of the acquisition function,
1024 restart candidates for the acquisition function optimization, and 512 Monte Carlosamples to approximate the acquisition function. A similar example can be found athttps://botorch.org/tutorials/multi_objective_bo.

Step 2: Objective weight � optimization
based on user ratings

Step 3: Decay hyperparameter optimization
via simulations

We simulated the TAF+ with diferent < � 1, � 2 > values with the
population models and observed which value pair results in thebest performance. We set the �1 values to be [1, 2, 3, 4, 5, 6, 7, 8, 9]
and the �2 values to be [0.1, 0.2, 0.3]. We created a list of < � 1, � 2 >pairs of all the possible combinations. Additionally, we included twobaselines in the simulation. The frst was a standard single-objective
Bayesian optimization where the objective was the weighted-sum
objective. The second baseline was the TAF that handled multiple
objectives using Equation 4 and no decay on the population model.
For each < � 1, � 2 > pair, we ran 14 simulations. In each simulation,

A Meta-Bayesian Approach for Rapid Online Parametric Optimization for Wrist-based Interactions

CHI ’24, May 11–16, 2024, Honolulu, HI, USA

Relative pointing

Iteration number

Iteration number

Net performance overmultiple objectives

Absolute pointing

Figure 7: Result of the decay hyperparameter optimization. We simulated the user performance over 10 iterations with many
combinations of hyperparameter values. Note that the TAF (multi-objective) condition handled multiple objectives with
Equation 4. We derived the global maximum from a grid search of the GP prediction. The results showed that TAF+ with proper
decay setting led to the best performance. An extended version of this simulation is presented in Appendix D.

we singled out a population model and treated it as the new test
user. We then excluded this model and utilized the remaining 13
population models to work with TAF+ . We set the optimization
iteration to 10. Thus, each < � 1, � 2 > pair resulted in 14 (users) ×10(iterations) datapoints.
6.3.1 Results of the simulation. The simulation results are shown in
Figure 7. With �1 = 2, �2 = 0.3, TAF+ achieved the best performancefor both interactions. Since there were too many combinations, weonly show the performance of the best setting and two important
baselines. TAF+ with the optimal hyperparameter outperformed
the standard BO. Additionally, TAF (multi-objective) had benefts
in the early iterations but struggled to improve quickly, indicating
directly deploying TAF without a mechanism to balance population
models and the adaptation model may hinder adaptation efciency.
This issue would not exist if all the users had high similarities;
the new user’s optimal design would be highly aligned with the
population. Yet, this assumption may not hold in certain scenarios,e.g., our interactions.

6.4

Summary

Through this workfow, we derived (1) 14 population models, (2)optimal objective weights based on subjective ratings, and (3)optimal setting of hyperparameters (�1 and �2), which will be
employed in the next adaptation phase.

7

ADAPTATION: EVALUATING META-BO ON
NEW USERS

We evaluated the efcacy of our meta-BO (TAF+ ) algorithm andworkfow with 11 new participants for both pointing interactions.
We evaluated against two baseline procedures: standard BO andmanual calibration.

7.1 Experimental design
11 entirely new participants were recruited for the evaluation study:
6 males and 5 females, aged 23 − 42 (������ = 29.5). None of them
have any experience pointing in VR or using VR. We conducted a
within-subjects study with 2 independent variables: optimization
procedure (meta-BO vs. standard BO vs. manual calibration) anditeration (10 iterations) for each of the pointing interactions. The
procedures were counterbalanced using a Latin square.

7.2

Adaptation procedures (meta-BO, BO, andmanual)7.2.1 Meta-BO (TAF+ ). We set up TAF+ with 14 population models, optimized objective weights, and decay hyperparameters as
described earlier10 .
7.2.2 Standard BO. For the BO procedure, we had 5 initial random
samplings and 5 optimization iterations. These values were determined from a 4-participant pilot test. To ensure a fair comparison,
BO optimizes for the same weighted-sum objective obtained in Step2, which TAF+ also optimizes for. Other hyperparameter settings
are the same as in population modeling.
7.2.3 Manual calibration. Diferent from meta-BO and BO, theusers adjust the weight settings based on their perception unrelated
to the weighted-sum objective. For absolute pointing, an established
way to calibrate for similar interactions is to ask the users to indicate
their preferred operational range, and then map the maximum
detected values to the boundary of the interface (E.g. [28, 29]). We
followed the same, mapping the user’s comfortable horizontal (yaw)

10 The hyperparameter settings of the adaptation model followed the ones used in

population modeling: a single-task GP with Matern 5/2 kernel and inferred noise levels.
For computation efciency, at each iteration, we generated 1024 parameter settings ascandidates using a Sobol grid.

CHI ’24, May 11–16, 2024, Honolulu, HI, USA

and vertical (pitch) forearm motion range to the scene boundaries
along �, �.
For relative pointing, we frst determined �� (correction parameter) for each user by using the radial-ulnar deviation of the user
(corresponding to vertical cursor motion) since it has a smaller
range than fexion-extension. The users were asked to perform
radial-ulnar deviation to move the cursor back and forth between
a top and a bottom target (Figure 6b). The diference between therecorded direction and the ideal vertical direction determined �� .
Next, we calibrated the velocity function. Since � 3 controls theoverall scale of the transfer function, we viewed this parameter as
the most critical parameter. Users did 12 random target selections
during which they were allowed to adjust � 3 by using a slider as
many times as they wanted, similar to the sensitivity tuning of acomputer mouse. As for � 1 and � 2 , we used TAF+ to propose the� 1 and � 2 values assuming there is no adaptation model; i.e., these
are the best suggestions based on the population data. The values
are 0.914 (� 1 ) and 1.438 (� 2 ). Unlike meta-BO and BO, manual calibration occurs before the data gathering. Once manual calibration
was complete, the parameter values were fxed and did not change
throughout the iterations.

7.3

Study procedure

Similar to step 1, participants performed relative pointing frst,
followed by absolute pointing. Instructions were the same as before. Diferent from step 1, each procedure only had 10 iterations.
Since there were 3 procedures (meta-BO, BO, and Manual) and 10
iterations for each procedure, every pointing interaction had 30
iterations in total. After each procedure, we asked the participants
to fll out the NASA-TLX questionnaire to assess their subjective
workload. Afterward, we conducted an open-ended interview tounderstand the participants’ experience.

7.4

Results

The weighted-sum performances at each iteration for both pointing
interactions are shown in Figure 8.
7.4.1 Absolute pointing: There were two independent variables:
the adaptation procedures (meta-BO, BO, Manual) and the iterations (1 - 10). A 2-way repeated-measures ANOVA with Greenhouse–Geisser correction showed no statistically signifcant interaction between the two variables (� (1.651, 21.729) = 1.684, � = 0.194).
A simple main efects analysis found a signifcant diference between the procedures (� (2, 30) = 1.652, � = 0.027). Pairwise comparisons showed signifcant diferences between meta-BO and BO,
and between meta-BO and Manual (both � < 0.05), which indicated that meta-BO resulted in higher overall performances thanthe other two procedures. Overall, meta-BO enables performancesthat are on average 22.92% and 21.35% higher than BO and manual calibration across the 10 iterations. The simple main efects
analysis also showed a signifcant diference between the iterations(� (9, 100) = 2.171, � < 0.001).
Another pairwise comparison between the iterations within eachprocedure found that for the meta-BO procedure, the performancedid not signifcantly improve beyond iteration 6. On the other hand,
BO still made signifcant improvements up to iteration 9. This indicates that meta-BO converges faster to optimal performance. We

Y.-C. Liao, et al.

conducted a pairwise comparison between procedures for each
iteration (Figure 8) which shows that meta-BO consistently outperforms BO and Manual in several iterations. The detailed numbers
for Figure 8 are provided in Appendix E.
7.4.2 Relative pointing: A 2-way repeated-measures ANOVA
showed no statistically signifcant interaction between procedureand iteration (� (1.922, 29.403)= 1.623, � = 0.188). A simple main effects analysis found a signifcant diference between the procedures(� (2, 30) = 7.231, � = 0.005). Pairwise comparisons showed signifcant diferences between the meta-BO procedure and BO procedure(� = 0.006) and between the meta-BO procedure and the Manual
procedure (� = 0.033), which indicated that meta-BO resulted inhigher overall performances than the other two procedures. Averaging the performance of 10 iterations, meta-BO enables 25.43%
and 13.60% better performances than BO and manual in relative
pointing. A simple main efects analysis also showed a signifcant
diference between the iterations (� (9, 100) = 44.795, � < 0.001).
Pairwise comparisons between the iterations within each procedure
showed that for meta-BO, the performance did not signifcantly
improve after iteration 6. Meanwhile, BO improved up to iteration
7. Similar to absolute pointing, meta-BO is faster to converge to optimal performance. We conducted a pairwise comparison between
procedures for each iteration (Figure 8) which shows meta-BO
consistently outperforms BO and Manual in several iterations.
7.4.3 Other analyses: We found only one signifcant diference in
the NASA-TLX questions: For absolute pointing, both Meta-BO andBO led to signifcantly lower frustration than the Manual procedure,
indicating Meta-BO delivers better or comparable user experiences.
This is mainly because the users needed to invest more efort in thecalibration process but it did not lead to a better experience. Fromthe interviews, we further learned that the participants’ experienceis highly infuenced by the Index of Difculty of targets, than by thetransfer function. More detailed analyses on perceived workload
and user experience are presented in Appendix F. We further
analyzed the individual metrics derived from the three proceduresfor both interactions, and found that meta-BO generally led tocomparable or signifcantly better performances than the baselines.
Please refer to Appendix G for more details and the plots of theindividual metrics. Finally, each user’s performance is separately
presented in Appendix H, and we visualized the objective functionof absolute pointing in Appendix I.

7.5

Findings and discussion

Overall, we found that meta-BO allowed for signifcantly better
performances than standard BO and manual calibration. For absolute pointing, standard BO started with a lower initial performance
which is not surprising since it was not informed by prior population models. Standard BO further took more time to converge inboth absolute and relative pointing. Even though meta-BO continued to improve until iteration 6, we can see that it reached near-peak
performance by iteration 3, which translates to just 18 selections.
Thus, meta-BO overcomes the slow start and slow convergenceissues of standard BO for our tasks.
With manual calibration, since the participant calibrates it in
the beginning according to their preference, the expectation would

A Meta-Bayesian Approach for Rapid Online Parametric Optimization for Wrist-based Interactions

CHI ’24, May 11–16, 2024, Honolulu, HI, USA

Weighted-sum objective value
over multiple objectives

Absolute pointing

*

**

**

**

**

*

Relative pointing

**

*
*

Iteration number

**

**

**

**

**

**

*

*

Iteration number

Figure 8: The performance (weighted-sum objective value over multiple objective functions) in 10 iterations. We have normalized
all the objective functions into the range of [0, 1]. The error bar shows a 95% confdence interval. The red ∗ (meta-BO andBO), the black ∗ (meta-BO and Manual), and the blue ∗ (Manual and BO) signs denote a signifcant diference between two
procedure at that iteration. Note that we showed “the best” performance reached from the beginning to each iteration. Since
we are comparing the optimal performance given the same amount of iteration, this is the conventional way of showing andcomparing the performance. Also note we have converted it to a maximization problem (see subsection 5.3). The detailed
numbers of this fgure are presented in Appendix E.

be that it will perform well at least in the beginning. While there
are a few indications that the manual calibration procedure may
have marginally better performance than BO, the diference is not
signifcant. Thus, manual calibration does not guarantee an excellent result. This could be because users may not fne-tune to theextent of fnding the optimal design. Even though the manual calibration parameters did not change, its performance improved overtime, presumably because the participants were adapting over time.
However, despite user adaptation, meta-BO consistently outperforms manual calibration for iterations 3-8. Given that meta-BO
as an online procedure can match explicit manual calibration in
the beginning and outperform it in intermediary trials, it is a viable candidate to replace explicit manual calibration procedures inreal-world deployments.

8

DISCUSSION

In this work, we propose a novel HitL technique for rapid, online
personalized parametric optimization for wrist-based input. It is
known that adapting or optimizing transfer functions is challenging;wrist-based interaction adds further difculties due to wearability and posture factors. We tackled the two most representativeand distinct wrist-based input interactions. With just 14 users for
population modeling, meta-BO outperforms the existing manual
calibration and standard BO approaches for new users. This demonstrates the specifc utility of meta-BO for interactions that beneft
from personalized parametric settings. Meta-BO can eliminate dedicated calibration routines for wearable device interactions andhelp rapid attainment of optimal settings for each user. Given the
results from two diferent pointing applications and a series of simulations and the fact that meta-BO does not have any overbearing

assumptions (subsubsection 4.1.3), this work provides a meta-BO
workfow that HCI researchers and practitioners can further apply
to their applications and other problem contexts. We also showed
how TAF+ extended the TAF approach for HitL applications. This
involved the derivation of optimal objective weights based onusers’ subjective ratings and confguring decay hyperparameters
through simulations to balance the population and adaptation models. The optimal weight derivation occurs after population modelingin our workfow, allowing higher fexibility to tune the objective
weights when needed. There are multiple open questions andlimitations that future work can address.

Encountering new users: Meta-BO utilized the similarities between
prior users and new users to converge to optimal settings faster. To
account for a new user that is drastically diferent and the potentialof negative transfer, we introduced decay parameters to let the
optimization proactively rely on the observations from the newuser. Our results show that with only 14 population models, metaBO converges faster on new users on average. Further, we analyzed
the individual users in Appendix H and found P5 (Table 14) as one
example of a user who has drastically diferent optimal parametersfrom others. As we see in P5’s performance plotted in Figure 25,
meta-BO is still able to converge to a performance comparable tothe baselines. It also shows the benefts of having a diverse initialuser set. Of course, when a user behaves even more extreme, such as
wearing the device completely wrongly or their optimal parametersetting is beyond the parameter range, TAF+ can not adapt forsuch cases. To address this challenge, future work could consider
developing mechanisms to diagnose a new user’s performance inreal-time and switch to standard BO or manual optimization.

CHI ’24, May 11–16, 2024, Honolulu, HI, USA

Enhancing the eficiency, scalability, and determining the objectvie
weights: To enable more efcient calibration of wrist-based interactions, it is worth exploring more advanced normalization techniques to determine the quality of a setting with fewer selections. In
addition, although TAF/TAF+ is relatively lightweight, its computation cost is linearly increased by the number of models, introducing
difculties when scaling up the population models. This issue could
be mitigated by calculating the �� and ���� in parallel. Further, thecurrent approach to deriving optimal objective weights basedon subjective ratings of the population may not be suitable for all
new users. Future research could investigate deriving the individualuser’s optimal objective weights by user feedback during adaptation; preferential Bayesian optimization [30] may be incorporated
in this direction.
Potential co-adaptation: In the evaluation of TAF+ , there were only a
few observations in each adaptation procedure, and each parametersetting was evaluated only once, making it hard to detect user
learning. Future work should consider developing methods that takethe user to revisit certain parameter settings for better-estimatinguser learning. Also, more advanced computational methods are alsoneeded to infer the user’s learning.

Y.-C. Liao, et al.

procedure is challenging and often specifc to a particular device orinteraction. Moreover, our study indicated that manual calibration
does not always yield optimal results. Our success in wrist-based
pointing demonstrates that meta-BO holds signifcant potential as a
general calibration method across various interactions and devices,eliminating the difculties associated with designing calibration
procedures and achieving a better user experience. We encourage
future research to explore the application of meta-BO for optimizing parameter settings across diverse applications, both within HCI
and beyond. We anticipate this work will pave the way for more
personalized and adaptive user interfaces.

ACKNOWLEDGMENTS
We would like to thank Dr. Kashyap Todi for his suggestions on
interaction metrics and discussions throughout the project. We
extend our gratitude to our colleagues in Reality Labs Research fortheir unwavering support. Additionally, we wholeheartedly thankDr. Hee-Seung Moon, Dr. John J. Dudley, and Prof. Antti Oulasvirta
for their valuable comments, which signifcantly enhanced thequality of the paper.

REFERENCES
Multi-objective TAF:. Meta-BO is an emerging topic with many
open research questions and opportunities. The feld of HCI canbeneft highly from it. In addition to applying TAF+ to other applications, future research can investigate other meta-BO methodsfor HitL optimization. One potential direction is extending TAF
for multi-objective tasks by changing the base acquisition function
to Expected Hypervolume Increase (EHVI). We ran a simulation
with this approach using the population models, and the results
showed that TAF outperformed the standard multi-objective BO,
as plotted in Appendix J. However, multi-objective TAF would
require the end-users to engage with extreme designs that heavily
prioritize one objective while neglecting others. Further, one would
need a heuristic to select one design from the Pareto-frontier which
may or may not be the one preferred by the user. Alternatively,
the user will need to manually determine one fnal setting among
the Pareto-frontier through trials, which will introduce more efort
to the users. Finally, the time required for computing the multiobjective TAF to yield the next parameter setting during adaptation
is massive for each iteration because the computation complexityis � (the total number of models) multiplied by the complexity forcomputing EHVI from a model. Thus, it would be unsuitable foronline adaptations which require a fast turn-around time.

9

CONCLUSION

In this paper, we present an online, fast-converging parametric optimization procedure through meta-Bayesian optimization. We introduce a novel meta-BO algorithm, TAF+ , and a tailored workfow tomeet the unique requirements of human-in-the-loop problems. We
apply TAF+ and its workfow to two distinct wrist-based interactions. The positive result and the outcome of each step showcase theefectiveness and efciency of meta-BO compared to conventional
calibration procedures and state-of-the-art BO. Calibration is a common practice among HCI practitioners and researchers, typicallycreated by developers or designers. Crafting an efective calibration[1] Karan Ahuja, Sven Mayer, Mayank Goel, and Chris Harrison. 2021. Pose-onthe-Go: Approximating User Pose with Smartphone Sensor Fusion and Inverse
Kinematics. In Proceedings of the 2021 CHI Conference on Human Factors in
Computing Systems (Yokohama, Japan) (CHI ’21). Association for Computing
Machinery, New York, NY, USA, Article 9, 12 pages. https://doi.org/10.1145/
3411764.3445582[2] Ahmed Ayman, Omneya Attalah, and Heba Shaban. 2019. An Efcient Human
Activity Recognition Framework Based on Wearable IMU Wrist Sensors. In
2019 IEEE International Conference on Imaging Systems and Techniques (IST). 1–5.
https://doi.org/10.1109/IST48021.2019.9010115[3] Tianyi Bai, Yang Li, Yu Shen, Xinyi Zhang, Wentao Zhang, and Bin Cui.
2023. Transfer Learning for Bayesian Optimization: A Survey. arXiv preprint
arXiv:2302.05927 (2023).
[4] Maximilian Balandat, Brian Karrer, Daniel Jiang, Samuel Daulton, Ben Letham,
Andrew G Wilson, and Eytan Bakshy. 2020. BoTorch: a framework for efcient
Monte-Carlo Bayesian optimization. Advances in neural information processing
systems 33 (2020), 21524–21538.
[5] Rémi Bardenet, Mátyás Brendel, Balázs Kégl, and Michèle Sebag. 2013. Collaborative Hyperparameter Tuning. In Proceedings of the 30th International Conferenceon International Conference on Machine Learning - Volume 28 (Atlanta, GA, USA)(ICML’13). JMLR.org, II–199–II–207.
[6] Yoshua Bengio, Samy Bengio, and Jocelyn Cloutier. 1990. Learning a synaptic
learning rule. Citeseer.
[7] Edwin V Bonilla, Kian Chai, and Christopher Williams. 2007. Multi-task Gaussian process prediction. Advances in neural information processing systems 20(2007).
[8] Ali Borji and Laurent Itti. 2013. Bayesian optimization explains humanactive search. In Advances in Neural Information Processing Systems, C.J.
Burges, L. Bottou, M. Welling, Z. Ghahramani, and K.Q. Weinberger (Eds.),
Vol. 26. Curran Associates, Inc. https://proceedings.neurips.cc/paper/2013/fle/
a3f390d88e4c41f2747bfa2f1b5f87db-Paper.pdf[9] Eric Brochu, Tyson Brochu, and Nando de Freitas. 2010. A Bayesian Interactive
Optimization Approach to Procedural Animation Design. In Proceedings of the2010 ACM SIGGRAPH/Eurographics Symposium on Computer Animation (Madrid,
Spain) (SCA ’10). Eurographics Association, Goslar, DEU, 103–112.
[10] Grigore C Burdea and Philippe Coifet. 2003. Virtual reality technology. John
Wiley & Sons.
[11] Géry Casiez and Nicolas Roussel. 2011. No More Bricolage! Methods andTools to Characterize, Replicate and Compare Pointing Transfer Functions.
In Proceedings of the 24th Annual ACM Symposium on User Interface Software
and Technology (Santa Barbara, California, USA) (UIST ’11). Association for
Computing Machinery, New York, NY, USA, 603–614. https://doi.org/10.1145/
2047196.2047276[12] Géry Casiez, Nicolas Roussel, and Daniel Vogel. 2012. 1€ Filter: A Simple Speedbased Low-pass Filter for Noisy Input in Interactive Systems. In CHI’12, the30th Conference on Human Factors in Computing Systems. ACM, Austin, UnitedStates, 2527–2530. https://doi.org/10.1145/2207676.2208639

A Meta-Bayesian Approach for Rapid Online Parametric Optimization for Wrist-based Interactions[13] Géry Casiez, Daniel Vogel, Ravin Balakrishnan, and Andy Cockburn. 2008.
The Impact of Control-Display Gain on User Performance in Pointing Tasks.
Human–Computer Interaction 23,
3 (2008), 215–250.
https://doi.org/10.1080/07370020802278163
arXiv:https://www.tandfonline.com/doi/pdf/10.1080/07370020802278163[14] Géry Casiez, Daniel Vogel, Ravin Balakrishnan, and Andy Cockburn. 2008. The
impact of control-display gain on user performance in pointing tasks. Human–
computer interaction 23, 3 (2008), 215–250.
[15] Liwei Chan, Yi-Chi Liao, George B Mo, John J Dudley, Chun-Lien Cheng,
Per Ola Kristensson, and Antti Oulasvirta. 2022. Investigating Positive and
Negative Qualities of Human-in-the-Loop Optimization for Designing Interaction Techniques. In Proceedings of the 2022 CHI Conference on Human
Factors in Computing Systems (New Orleans, LA, USA) (CHI ’22). Association for Computing Machinery, New York, NY, USA, Article 112, 14 pages.
https://doi.org/10.1145/3491102.3501850[16] Siyuan Chen, Julien Epps, Natalie Ruiz, and Fang Chen. 2011. Eye Activity
as a Measure of Human Mental Efort in HCI. In Proceedings of the 16th International Conference on Intelligent User Interfaces (Palo Alto, CA, USA) (IUI
’11). Association for Computing Machinery, New York, NY, USA, 315–318.
https://doi.org/10.1145/1943403.1943454[17] Chia-Hsing Chiu, Yuki Koyama, Yu-Chi Lai, Takeo Igarashi, and Yonghao Yue.
2020. Human-in-the-Loop Diferential Subspace Search in High-Dimensional
Latent Space. ACM Trans. Graph. 39, 4, Article 85 (aug 2020), 15 pages. https://doi.org/10.1145/3386569.3392409[18] Zhongxiang Dai, Yizhou Chen, Haibin Yu, Bryan Kian Hsiang Low, and Patrick
Jaillet. 2022. On provably robust meta-Bayesian optimization. In Proceedings of
the Thirty-Eighth Conference on Uncertainty in Artifcial Intelligence (Proceedingsof Machine Learning Research, Vol. 180), James Cussens and Kun Zhang (Eds.).
PMLR, 475–485. https://proceedings.mlr.press/v180/dai22a.html[19] Samuel Daulton, Maximilian Balandat, and Eytan Bakshy. 2021. Parallel bayesian
optimization of multiple noisy objectives with expected hypervolume improvement. Advances in Neural Information Processing Systems 34 (2021), 2187–2200.
[20] Laura Dipietro, Angelo M. Sabatini, and Paolo Dario. 2008. A Survey of GloveBased Systems and Their Applications. IEEE Transactions on Systems, Man,
and Cybernetics, Part C (Applications and Reviews) 38, 4 (2008), 461–482. https://doi.org/10.1109/TSMCC.2008.923862[21] Yan Duan, John Schulman, Xi Chen, Peter L Bartlett, Ilya Sutskever, and Pieter
Abbeel. 2016. Rl2: Fast reinforcement learning via slow reinforcement learning.
arXiv preprint arXiv:1611.02779 (2016).
[22] Michael Emmerich and Jan-willem Klinkenberg. 2008. The computation of the
expected improvement in dominated hypervolume of Pareto front approximations. Rapport technique, Leiden University 34 (2008), 7–3.
[23] Matthias Feurer, Benjamin Letham, and Eytan Bakshy. 2018. Scalable metalearning for bayesian optimization using ranking-weighted gaussian processensembles. In AutoML Workshop at ICML, Vol. 7.
[24] Matthias Feurer, Benjamin Letham, Frank Hutter, and Eytan Bakshy. 2018. Practical transfer learning for Bayesian optimization. arXiv preprint arXiv:1802.02219(2018).
[25] Richard S Figliola and Donald E Beasley. 2020. Theory and design for mechanicalmeasurements. John Wiley & Sons.
[26] Chelsea Finn, Pieter Abbeel, and Sergey Levine. 2017. Model-agnostic metalearning for fast adaptation of deep networks. In International conference on
machine learning. PMLR, 1126–1135.
[27] Peter I Frazier. 2018. A tutorial on Bayesian optimization. arXiv preprint
arXiv:1807.02811 (2018).
[28] Jun Gong, Zheer Xu, Qifan Guo, Teddy Seyed, Xiang ’Anthony’ Chen, Xiaojun Bi,
and Xing-Dong Yang. 2018. WrisText: One-Handed Text Entry on Smartwatch
Using Wrist Gestures. In Proceedings of the 2018 CHI Conference on Human
Factors in Computing Systems (Montreal QC, Canada) (CHI ’18). Association
for Computing Machinery, New York, NY, USA, 1–14. https://doi.org/10.1145/
3173574.3173755[29] Jun Gong, Xing-Dong Yang, and Pourang Irani. 2016. WristWhirl: One-Handed
Continuous Smartwatch Input Using Wrist Gestures. In Proceedings of the 29th
Annual Symposium on User Interface Software and Technology (Tokyo, Japan)(UIST ’16). Association for Computing Machinery, New York, NY, USA, 861–872.
https://doi.org/10.1145/2984511.2984563[30] Javier González, Zhenwen Dai, Andreas Damianou, and Neil D Lawrence. 2017.
Preferential bayesian optimization. In International Conference on Machine Learning. PMLR, 1282–1291.
[31] Deepak Gopinath, Siddarth Jain, and Brenna D Argall. 2016. Human-in-theloop optimization of shared autonomy in assistive robotics. IEEE Robotics andAutomation Letters 2, 1 (2016), 247–254.
[32] Aakar Gupta and Ravin Balakrishnan. 2016. DualKey: Miniature Screen Text
Entry via Finger Identifcation. In Proceedings of the 2016 CHI Conference on
Human Factors in Computing Systems (San Jose, California, USA) (CHI ’16).
Association for Computing Machinery, New York, NY, USA, 59–70. https://doi.org/10.1145/2858036.2858052

CHI ’24, May 11–16, 2024, Honolulu, HI, USA[33] Faizan Haque, Mathieu Nancel, and Daniel Vogel. 2015. Myopoint: Pointing
and Clicking Using Forearm Mounted Electromyography and Inertial Motion
Sensors. In Proceedings of the 33rd Annual ACM Conference on Human Factors
in Computing Systems (Seoul, Republic of Korea) (CHI ’15). Association for
Computing Machinery, New York, NY, USA, 3653–3656. https://doi.org/10.
1145/2702123.2702133[34] Juan David Hincapié-Ramos, Kasim Özacar, Pourang P. Irani, and Yoshifumi
Kitamura. 2016. GyroWand: An Approach to IMU-Based Raycasting for Augmented Reality. IEEE Computer Graphics and Applications 36, 2 (2016), 90–96.
https://doi.org/10.1109/MCG.2016.21[35] Timothy Hospedales, Antreas Antoniou, Paul Micaelli, and Amos Storkey. 2021.
Meta-learning in neural networks: A survey. IEEE transactions on pattern analysis
and machine intelligence 44, 9 (2021), 5149–5169.
[36] Bing-Jing Hsieh, Ping-Chun Hsieh, and Xi Liu. 2021. Reinforced few-shot
acquisition function learning for bayesian optimization. Advances in Neural
Information Processing Systems 34 (2021), 7718–7731.
[37] Carl Hvarfner, Danny Stoll, Artur Souza, Marius Lindauer, Frank Hutter, andLuigi Nardi. 2022. � BO: Augmenting Acquisition Functions with User Beliefs
for Bayesian Optimization. arXiv preprint arXiv:2204.11051 (2022).
[38] Donald R Jones, Matthias Schonlau, and William J Welch. 1998. Efcient global
optimization of expensive black-box functions. Journal of Global optimization
13 (1998), 455–492.
[39] Florian Kadner, Yannik Keller, and Constantin Rothkopf. 2021. Adaptifont:
Increasing individuals’ reading speed with a generative font model and bayesian
optimization. In Proceedings of the 2021 CHI Conference on Human Factors in
Computing Systems. 1–11.
[40] Frederic Kerber, Pascal Lessel, and Antonio Krüger. 2015. Same-Side Hand
Interactions with Arm-Placed Devices Using EMG. In Proceedings of the 33rd
Annual ACM Conference Extended Abstracts on Human Factors in Computing
Systems (Seoul, Republic of Korea) (CHI EA ’15). Association for Computing
Machinery, New York, NY, USA, 1367–1372. https://doi.org/10.1145/2702613.
2732895[41] Mohammad M. Khajah, Brett D. Roads, Robert V. Lindsey, Yun-En Liu, andMichael C. Mozer. 2016. Designing Engaging Games Using Bayesian Optimization. In Proceedings of the 2016 CHI Conference on Human Factors in Computing
Systems (San Jose, California, USA) (CHI ’16). Association for Computing Machinery, New York, NY, USA, 5571–5582. https://doi.org/10.1145/2858036.2858253[42] Wolf Kienzle, Eric Whitmire, Chris Rittaler, and Hrvoje Benko. 2021. ElectroRing:
Subtle Pinch and Touch Detection with a Ring. In Proceedings of the 2021 CHI
Conference on Human Factors in Computing Systems (Yokohama, Japan) (CHI ’21).
Association for Computing Machinery, New York, NY, USA, Article 3, 12 pages.
https://doi.org/10.1145/3411764.3445094[43] David Kim, Otmar Hilliges, Shahram Izadi, Alex D. Butler, Jiawen Chen, Iason Oikonomidis, and Patrick Olivier. 2012. Digits: Freehand 3D Interactions
Anywhere Using a Wrist-Worn Gloveless Sensor. In Proceedings of the 25th
Annual ACM Symposium on User Interface Software and Technology (Cambridge,
Massachusetts, USA) (UIST ’12). Association for Computing Machinery, New
York, NY, USA, 167–176. https://doi.org/10.1145/2380116.2380139[44] Jiwan Kim and Ian Oakley. 2022. SonarID: Using Sonar to Identify Fingers on a Smartwatch. In Proceedings of the 2022 CHI Conference on Human
Factors in Computing Systems (New Orleans, LA, USA) (CHI ’22). Association for Computing Machinery, New York, NY, USA, Article 287, 10 pages.
https://doi.org/10.1145/3491102.3501935[45] Myunghee Kim, Ye Ding, Philippe Malcolm, Jozefen Speeckaert, Christoper J.
Siviy, Conor J. Walsh, and Scott Kuindersma. 2017. Human-in-the-loop Bayesian
optimization of wearable device parameters. PLOS ONE 12, 9 (09 2017), 1–15.
https://doi.org/10.1371/journal.pone.0184054[46] Alice Y Kolb and David A Kolb. 2009. The learning way: Meta-cognitive aspectsof experiential learning. Simulation & gaming 40, 3 (2009), 297–327.
[47] Yuki Koyama and Masataka Goto. 2022. BO as Assistant: Using Bayesian Optimization for Asynchronously Generating Design Suggestions. In Proceedingsof the 35th Annual ACM Symposium on User Interface Software and Technology(Bend, OR, USA) (UIST ’22). Association for Computing Machinery, New York,NY, USA, Article 77, 14 pages. https://doi.org/10.1145/3526113.3545664[48] Yuki Koyama, Issei Sato, and Masataka Goto. 2020. Sequential Gallery forInteractive Visual Design Optimization. ACM Trans. Graph. 39, 4, Article 88(aug 2020), 12 pages. https://doi.org/10.1145/3386569.3392444[49] Yuki Koyama, Issei Sato, Daisuke Sakamoto, and Takeo Igarashi. 2017. Sequential
line search for efcient visual design optimization by crowds. ACM Transactions
on Graphics (TOG) 36, 4 (2017), 1–11.
[50] Yeara Kozlov and Tino Weinkauf. 2013. Persistence1D. https://github.com/
yeara/Persistence1D.
[51] Ananda Sankar Kundu, Oishee Mazumder, Prasanna Kumar Lenka, and Subhasis
Bhaumik. 2018. Hand gesture recognition based omnidirectional wheelchair
control using IMU and EMG sensors. Journal of Intelligent & Robotic Systems 91(2018), 529–541.
[52] Byungjoo Lee, Mathieu Nancel, Sunjun Kim, and Antti Oulasvirta. 2020. AutoGain: Gain Function Adaptation with Submovement Efciency Optimization. In

CHI ’24, May 11–16, 2024, Honolulu, HI, USA

Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems(Honolulu, HI, USA) (CHI ’20). Association for Computing Machinery, New York,NY, USA, 1–12. https://doi.org/10.1145/3313831.3376244[53] Lik Hang Lee, Ngo Yan Yeung, Tristan Braud, Tong Li, Xiang Su, and Pan
Hui. 2020. Force9: Force-Assisted Miniature Keyboard on Smart Wearables.
In Proceedings of the 2020 International Conference on Multimodal Interaction(Virtual Event, Netherlands) (ICMI ’20). Association for Computing Machinery,
New York, NY, USA, 232–241. https://doi.org/10.1145/3382507.3418827[54] Wendi Li, Ting Wang, and Wing W. Y. Ng. 2021. Population-Based Hyperparameter Tuning With Multitask Collaboration. IEEE Transactions on Neural Networks
and Learning Systems (2021), 1–13. https://doi.org/10.1109/TNNLS.2021.3130896[55] Yang Li, Yu Shen, Huaijun Jiang, Wentao Zhang, Zhi Yang, Ce Zhang, and Bin
Cui. 2022. TransBO: Hyperparameter Optimization via Two-Phase Transfer
Learning. In Proceedings of the 28th ACM SIGKDD Conference on Knowledge
Discovery and Data Mining. 956–966.
[56] Yi-Chi Liao, John J. Dudley, George B. Mo, Chun-Lien Cheng, Liwei Chan, Antti
Oulasvirta, and Per Ola Kristensson. 2023. Interaction Design With Multiobjective Bayesian Optimization. IEEE Pervasive Computing (2023), 1–10. https://doi.org/10.1109/MPRV.2022.3230597[57] Yi-Chi Liao, Sunjun Kim, Byungjoo Lee, and Antti Oulasvirta. 2020. Button Simulation and Design via FDVV Models. In Proceedings of the 2020
CHI Conference on Human Factors in Computing Systems (Honolulu, HI, USA)(CHI ’20). Association for Computing Machinery, New York, NY, USA, 1–14.
https://doi.org/10.1145/3313831.3376262[58] Kieran Little, Chris W. Antuvan, Michele Xiloyannis, Bernardo A.P.S. de
Noronha, Yongtae G. Kim, Lorenzo Masia, and Dino Accoto. 2019. IMUbased assistance modulation in upper limb soft wearable exosuits. In 2019 IEEE
16th International Conference on Rehabilitation Robotics (ICORR). 1197–1202.
https://doi.org/10.1109/ICORR.2019.8779362[59] Jiayang Liu, Lin Zhong, Jehan Wickramasuriya, and Venu Vasudevan. 2009.
uWave: Accelerometer-based personalized gesture recognition and its applications. Pervasive and Mobile Computing 5, 6 (2009), 657–675. https://doi.org/10.1016/j.pmcj.2009.07.007 PerCom 2009.
[60] Mingyu Liu, Mathieu Nancel, and Daniel Vogel. 2015. Gunslinger: Subtle Armsdown Mid-Air Interaction (UIST ’15). Association for Computing Machinery,
New York, NY, USA, 63–71. https://doi.org/10.1145/2807442.2807489[61] I. Scott MacKenzie. 1992. Fitts’ Law as a Research and Design Tool in HumanComputer Interaction. Hum.-Comput. Interact. 7, 1 (mar 1992), 91–139. https://doi.org/10.1207/s15327051hci0701_3[62] J. Mockus. 1975. On the Bayes Methods for Seeking the Extremal Point. IFAC
Proceedings Volumes 8, 1, Part 1 (1975), 428–431. https://doi.org/10.1016/S14746670(17)67769-3 6th IFAC World Congress (IFAC 1975) - Part 1: Theory,
Boston/Cambridge, MA, USA, August 24-30, 1975.
[63] Alessandra Moschetti, Laura Fiorini, Dario Esposito, Paolo Dario, and Filippo
Cavallo. 2016. Recognition of Daily Gestures with Wearable Inertial Rings andBracelets. Sensors 16, 8 (2016). https://doi.org/10.3390/s16081341[64] Alessandra Moschetti, Laura Fiorini, Dario Esposito, Paolo Dario, and Filippo
Cavallo. 2017. Daily activity recognition with inertial ring and bracelet: An
unsupervised approach. In 2017 IEEE International Conference on Robotics andAutomation (ICRA). 3250–3255. https://doi.org/10.1109/ICRA.2017.7989370[65] Mathieu Nancel, Emmanuel Pietriga, Olivier Chapuis, and Michel BeaudouinLafon. 2015. Mid-Air Pointing on Ultra-Walls. ACM Trans. Comput.-Hum.
Interact. 22, 5, Article 21 (aug 2015), 62 pages. https://doi.org/10.1145/2766448[66] Mathieu Nancel, Daniel Vogel, and Edward Lank. 2015. Clutching Is Not(Necessarily) the Enemy. In Proceedings of the 33rd Annual ACM Conferenceon Human Factors in Computing Systems (Seoul, Republic of Korea) (CHI
’15). Association for Computing Machinery, New York, NY, USA, 4199–4202.
https://doi.org/10.1145/2702123.2702134[67] Alex Nichol and John Schulman. 2018. Reptile: a scalable metalearning algorithm.
arXiv preprint arXiv:1803.02999 2, 3 (2018), 4.
[68] Jens Brehm Nielsen, Jakob Nielsen, Bjørn Sand Jensen, and Jan Larsen. 2013.
Hearing aid personalization. (2013).
[69] Jens Brehm Bagger Nielsen, Jakob Nielsen, and Jan Larsen. 2015. PerceptionBased Personalization of Hearing Aids Using Gaussian Processes and Active
Learning. IEEE/ACM Transactions on Audio, Speech, and Language Processing 23,
1 (2015), 162–173. https://doi.org/10.1109/TASLP.2014.2377581[70] Michael A Osborne, Roman Garnett, and Stephen J Roberts. 2009. Gaussian
processes for global optimization. In 3rd international conference on learningand intelligent optimization (LION3). Springer-Verlag Berlin, Germany, 1–15.
[71] Kasım Özacar, Juan David Hincapié-Ramos, Kazuki Takashima, and Yoshifumi
Kitamura. 2017. 3d selection techniques for mobile augmented reality headmounted displays. Interacting with Computers 29, 4 (2017), 579–591.
[72] J.K. Perng, B. Fisher, S. Hollar, and K.S.J. Pister. 1999. Acceleration sensing
glove (ASG). In Digest of Papers. Third International Symposium on Wearable
Computers. 178–180. https://doi.org/10.1109/ISWC.1999.806717[73] Ken Pfeufer, Melodie Vidal, Jayson Turner, Andreas Bulling, and Hans Gellersen.
2013. Pursuit Calibration: Making Gaze Calibration Less Tedious and More
Flexible. In Proceedings of the 26th Annual ACM Symposium on User Interface

Y.-C. Liao, et al.

Software and Technology (St. Andrews, Scotland, United Kingdom) (UIST ’13).
Association for Computing Machinery, New York, NY, USA, 261–270. https://doi.org/10.1145/2501988.2501998[74] Iman Prayudi and Doik Kim. 2012. Design and implementation of IMU-based
human arm motion capture system. In 2012 IEEE International Conference on
Mechatronics and Automation. 670–675. https://doi.org/10.1109/ICMA.2012.
6283221[75] J. Rekimoto. 2001. GestureWrist and GesturePad: unobtrusive wearable interaction devices. In Proceedings Fifth International Symposium on Wearable
Computers. 21–27. https://doi.org/10.1109/ISWC.2001.962092[76] Yi Ren and Panos Y Papalambros. 2011. A design preference elicitation queryas an optimization process. Journal of Mechanical Design 133, 11 (2011).
[77] Farshid Salemi Parizi, Wolf Kienzle, Eric Whitmire, Aakar Gupta, and Hrvoje
Benko. 2021. RotoWrist: Continuous Infrared Wrist Angle Tracking Usinga Wristband. In Proceedings of the 27th ACM Symposium on Virtual Reality
Software and Technology (Osaka, Japan) (VRST ’21). Association for Computing
Machinery, New York, NY, USA, Article 26, 11 pages. https://doi.org/10.1145/
3489849.3489886[78] Adam Santoro, Sergey Bartunov, Matthew Botvinick, Daan Wierstra, and Timothy Lillicrap. 2016. Meta-Learning with Memory-Augmented Neural Networks. In Proceedings of The 33rd International Conference on Machine Learning(Proceedings of Machine Learning Research, Vol. 48), Maria Florina Balcan and
Kilian Q. Weinberger (Eds.). PMLR, New York, New York, USA, 1842–1850.
https://proceedings.mlr.press/v48/santoro16.html[79] T. Scott Saponas, Desney S. Tan, Dan Morris, Ravin Balakrishnan, Jim Turner,
and James A. Landay. 2009. Enabling Always-Available Input with MuscleComputer Interfaces. In Proceedings of the 22nd Annual ACM Symposium on User
Interface Software and Technology (Victoria, BC, Canada) (UIST ’09). Association
for Computing Machinery, New York, NY, USA, 167–176. https://doi.org/10.
1145/1622176.1622208[80] Nicolas Schilling, Martin Wistuba, and Lars Schmidt-Thieme. 2016. Scalable
Hyperparameter Optimization with Products of Gaussian Process Experts. In
Machine Learning and Knowledge Discovery in Databases, Paolo Frasconi, Niels
Landwehr, Giuseppe Manco, and Jilles Vreeken (Eds.). Springer International
Publishing, Cham, 33–48.
[81] Jürgen Schmidhuber. 1992. Learning to Control Fast-Weight Memories: An
Alternative to Dynamic Recurrent Networks. Neural Computation 4, 1 (1992),
131–139. https://doi.org/10.1162/neco.1992.4.1.131[82] Nicolas Schweighofer and Kenji Doya. 2003. Meta-learning in Reinforcement
Learning. Neural Networks 16, 1 (2003), 5–9. https://doi.org/10.1016/S08936080(02)00228-9[83] Matthias Seeger. 2004. Gaussian processes for machine learning. International
journal of neural systems 14, 02 (2004), 69–106.
[84] Bobak Shahriari, Kevin Swersky, Ziyu Wang, Ryan P Adams, and Nando De Freitas. 2015. Taking the human out of the loop: A review of Bayesian optimization.
Proc. IEEE 104, 1 (2015), 148–175.
[85] Bobak Shahriari, Kevin Swersky, Ziyu Wang, Ryan P. Adams, and Nando deFreitas. 2016. Taking the Human Out of the Loop: A Review of Bayesian
Optimization. Proc. IEEE 104, 1 (2016), 148–175. https://doi.org/10.1109/JPROC.
2015.2494218[86] Ahmed Sharshar, Ahmed Fayez, Yasser Ashraf, and Walid Gomaa. 2021. Activity
With Gender Recognition Using Accelerometer and Gyroscope. In 2021 15th
International Conference on Ubiquitous Information Management and Communication (IMCOM). 1–7. https://doi.org/10.1109/IMCOM51814.2021.9377388[87] Junxiao Shen, Jinghui Hu, John J. Dudley, and Per Ola Kristensson. 2022. Personalization of a Mid-Air Gesture Keyboard using Multi-Objective Bayesian
Optimization. In 2022 IEEE International Symposium on Mixed and Augmented
Reality (ISMAR). 702–710. https://doi.org/10.1109/ISMAR55827.2022.00088[88] Edward Snelson and Zoubin Ghahramani. 2005. Sparse Gaussian processes
using pseudo-inputs. Advances in neural information processing systems 18(2005).
[89] Jasper Snoek, Oren Rippel, Kevin Swersky, Ryan Kiros, Nadathur Satish,
Narayanan Sundaram, Md. Mostofa Ali Patwary, Prabhat Prabhat, and Ryan P.
Adams. 2015. Scalable Bayesian Optimization Using Deep Neural Networks. In
Proceedings of the 32nd International Conference on International Conference on
Machine Learning - Volume 37 (Lille, France) (ICML’15). JMLR.org, 2171–2180.
[90] Jost Tobias Springenberg, Aaron Klein, Stefan Falkner, and Frank Hutter. 2016.
Bayesian optimization with robust Bayesian neural networks. Advances inneural information processing systems 29 (2016).
[91] Kevin Swersky, Jasper Snoek, and Ryan P Adams. 2013. Multi-Task Bayesian
Optimization. In Advances in Neural Information Processing Systems, C.J.
Burges, L. Bottou, M. Welling, Z. Ghahramani, and K.Q. Weinberger (Eds.),
Vol. 26. Curran Associates, Inc. https://proceedings.neurips.cc/paper/2013/fle/
f33ba15efa5c10e873bf3842afb46a6-Paper.pdf[92] Sebastian Thrun and Lorien Pratt (Eds.). 1998. Learning to Learn. Kluwer
Academic Publishers, USA.
[93] Michael Volpp, Lukas P Fröhlich, Kirsten Fischer, Andreas Doerr, Stefan Falkner,
Frank Hutter, and Christian Daniel. 2019. Meta-learning acquisition functions

A Meta-Bayesian Approach for Rapid Online Parametric Optimization for Wrist-based Interactions

for transfer learning in bayesian optimization. arXiv preprint arXiv:1904.02642(2019).
[94] Jane X Wang. 2021. Meta-learning in natural and artifcial intelligence. Current
Opinion in Behavioral Sciences 38 (2021), 90–95. https://doi.org/10.1016/j.cobeha.
2021.01.002 Computational cognitive neuroscience.
[95] Xilu Wang, Yaochu Jin, Sebastian Schmitt, and Markus Olhofer. 2022. Recent
Advances in Bayesian Optimization. arXiv preprint arXiv:2206.03301 (2022).
[96] Yuntao Wang, Jiexin Ding, Ishan Chatterjee, Farshid Salemi Parizi, Yuzhou
Zhuang, Yukang Yan, Shwetak Patel, and Yuanchun Shi. 2022. FaceOri: Tracking
Head Position and Orientation Using Ultrasonic Ranging on Earphones. In
Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems(New Orleans, LA, USA) (CHI ’22). Association for Computing Machinery, New
York, NY, USA, Article 290, 12 pages. https://doi.org/10.1145/3491102.3517698[97] Yuyang Wang and Roni Khardon. 2012. Sparse Gaussian processes for multi-task
learning. In Machine Learning and Knowledge Discovery in Databases: European
Conference, ECML PKDD 2012, Bristol, UK, September 24-28, 2012. Proceedings,
Part I 23. Springer, 711–727.
[98] James Wilson, Frank Hutter, and Marc Deisenroth. 2018. Maximizing acquisition
functions for Bayesian optimization. Advances in neural information processing
systems 31 (2018).
[99] Martin Wistuba and Josif Grabocka. 2021. Few-shot Bayesian optimization withdeep kernel surrogates. arXiv preprint arXiv:2101.07667 (2021).
[100] Martin Wistuba, Nicolas Schilling, and Lars Schmidt-Thieme. 2018. Scalable
gaussian process-based transfer surrogates for hyperparameter optimization.
Machine Learning 107, 1 (2018), 43–78.
[101] Jacob O. Wobbrock, James Fogarty, Shih-Yen (Sean) Liu, Shunichi Kimuro, andSusumu Harada. 2009. The Angle Mouse: Target-Agnostic Dynamic Gain
Adjustment Based on Angular Deviation. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (Boston, MA, USA) (CHI’09). Association for Computing Machinery, New York, NY, USA, 1401–1410.
https://doi.org/10.1145/1518701.1518912[102] Charence Wong, Zhi-Qiang Zhang, Benny Lo, and Guang-Zhong Yang. 2015.
Wearable Sensing for Solid Biomechanics: A Review. IEEE Sensors Journal 15, 5(2015), 2747–2760. https://doi.org/10.1109/JSEN.2015.2393883[103] Doris Xin, Litian Ma, Jialin Liu, Stephen Macke, Shuchen Song, and Aditya
Parameswaran. 2018. Accelerating human-in-the-loop machine learning: Challenges and opportunities. In Proceedings of the second workshop on data management for end-to-end machine learning. 1–4.
[104] Kenta Yamamoto, Yuki Koyama, and Yoichi Ochiai. 2022. Photographic Lighting
Design with Photographer-in-the-Loop Bayesian Optimization. In Proceedingsof the 35th Annual ACM Symposium on User Interface Software and Technology(Bend, OR, USA) (UIST ’22). Association for Computing Machinery, New York,NY, USA, Article 92, 11 pages. https://doi.org/10.1145/3526113.3545690[105] Hui-Shyong Yeo, Wenxin Feng, and Michael Xuelin Huang. 2020. WATouCH:
Enabling Direct Input on Non-Touchscreen Using Smartwatch’s Photoplethysmogram and IMU Sensor Fusion. In Proceedings of the 2020 CHI Conference on
Human Factors in Computing Systems (Honolulu, HI, USA) (CHI ’20). Association
for Computing Machinery, New York, NY, USA, 1–10. https://doi.org/10.1145/
3313831.3376198[106] Hui-Shyong Yeo, Erwin Wu, Juyoung Lee, Aaron Quigley, and Hideki Koike.
2019. Opisthenar: Hand Poses and Finger Tapping Recognition by Observing
Back of Hand Using Embedded Wrist Camera. In Proceedings of the 32nd Annual
ACM Symposium on User Interface Software and Technology (New Orleans, LA,
USA) (UIST ’19). Association for Computing Machinery, New York, NY, USA,
963–971. https://doi.org/10.1145/3332165.3347867[107] Dani Yogatama and Gideon Mann. 2014. Efcient transfer learning method forautomatic hyperparameter tuning. In Artifcial intelligence and statistics. PMLR,
1077–1085.
[108] Jaesik Yun, Youn kyung Lim, Kee-Eung Kim, and Seokyoung Song. 2015. Interactivity Crafter: An Interactive Input-Output Transfer Function Design
Tool for Interaction Designers. Archives of Design Research 28 (2015), 21–37.
https://api.semanticscholar.org/CorpusID:64149473[109] Yang Zhang and Chris Harrison. 2015. Tomo: Wearable, Low-Cost Electrical
Impedance Tomography for Hand Gesture Recognition. In Proceedings of the
28th Annual ACM Symposium on User Interface Software (Charlotte, NC, USA)(UIST ’15). Association for Computing Machinery, New York, NY, USA, 167–173.
https://doi.org/10.1145/2807442.2807480

CHI ’24, May 11–16, 2024, Honolulu, HI, USA

CHI ’24, May 11–16, 2024, Honolulu, HI, USA

Y.-C. Liao, et al.

A

DETAILS OF OBJECTIVE WEIGHTS
OPTIMIZATION
A.1 A demonstrative example of objective
weights optimization with two users
To explain the need for the across-user objective weight optimization approach in step 2 of our workfow, we provide an example
where two users are involved. User A’s and User B’s Pareto-optimal
performances are presented in Table 4 and Table 5, respectively.
Only by examining User A’s profle (Table 4), design B has thehighest user rating, and its third objective has the highest value.
Therefore, setting weights as [0.1, 0.1, 0.8] is a straightforward confguration that leads to the highest user rating. On the other hand,examining only User B’s profle (Table 5), design A has the highest
user rating, and its frst objective has the highest value, so one could
conclude with weights [0.8, 0.1, 0.1]. However, looking at the grouplevel, setting weights as [0.1, 0.1, 0.8] or [0.8, 0.1, 0.1] to both users
at the same time would result in the other user ending up with asuboptimal design (i.e., the design not associated with the highest
user rating). On the contrary, with an appropriate search, setting
objective weights as [0.4, 0.2, 0.2] would allow both users to end
up having the designs with the highest ratings. With more users
taken into consideration, it becomes increasingly challenging todirectly see which objective weight confguration is optimal forthe group. Hence, a principled grid search is an easier and moreprincipled approach.

A.2

Algorithm for Objective Weight
Optimization

Altorighm 1 presents the details of objective weight optimization.

Pareto-optimal
designs/settings

Original
objective values

Weighted-sum
objective value(weights=[0.8, 0.1, 0.1])

Weighted-sum
objective value(weights=[0.1, 0.1, 0.8])

Weighted-sum
objective value(weights=[0.4, 0.2, 0.4])

User ratingA
B
C[7, 2, 5][5, 4, 8][3, 9, 2]

6.4
5.2
3.5

4.9
7.3
2.8

4.8
6.4
3.8

20
100
1

Table 4: The table shows a set of Pareto-optimal designs in a three-objective optimization problem of an example User A.
Intuitively, setting the objective weights as [0.1, 0.1, 0.8] would easily end up having design B, which has the highest user
rating (100). However, when taking the second user (presented in Table 5) into consideration, [0,1, 0.1, 0.8] would bring thesuboptimal design (i.e., not associated with the highest rating) to the other. However, setting objective weights as [0.4, 0.2,
0.4] leads to the design of the highest ratings for both users.

Pareto-optimal
designs/settings

Original
objective values

Weighted-sum
objective value(weights=[0.8, 0.1, 0.1])

Weighted-sum
objective value(weights=[0.1, 0.1, 0.8])

Weighted-sum
objective value(weights=[0.4, 0.2, 0.4])

User ratingA
B
C[8, 3, 5][2, 1, 7][4, 4, 3]

7.2
2.4
3.9

5.1
5.9
3.2

5.8
3.8
3.6

100
50
1

Table 5: The table shows a set of Pareto-optimal designs in a three-objective optimization problem with an example User
B. Intuitively, setting the objective weights as [0.8, 0.1, 0.1] would end up having design A, which has the highest user
rating (100). However, this weight setting is not optimal for the other user (presented in Table 4). While not intuitive, setting
objective weights as [0.4, 0.2, 0.4] leads to the designs with the highest ratings for both.

A Meta-Bayesian Approach for Rapid Online Parametric Optimization for Wrist-based Interactions

CHI ’24, May 11–16, 2024, Honolulu, HI, USA

Algorithm 1 Optimize objective weight confguration based on user ratings.
Inputs:� ∈ [1, � ] for there are � users in total; � stands for the �-th user among all users.
� � is the total number of Pareto-optimal designs for the �-th user. For instance, the 5th user
tries out 10 parameter settings, and 3 of them result in Pareto-optimal performance;then �5 = 3.
� ∈ [1, � � ] denotes the �-th user’s �-th Pareto-optimal design.
�� �� [1..�] is the �-th user’s �-th Pareto-optimal objective value set, which contains
� objective functions.
��� is the �-th user’s rating for the �-th Pareto-optimal designs.
2: Outputs:�������_����ℎ� [1..�], an optimal weight setting for � objectives.
Initialize:����ℎ�� ← All possible non-zero weight combinations to the frst decimal point.
(e.g., ����ℎ�� [0] = [0.8, 0.1, 0.1], ����ℎ�� [1] = [0.7, 0.2.0.1] ...)
scores ← []4: for � ∈ [1, � ] do���� ← max(� � )6:���� ← min(� � )for � ∈ [1, � � ] do

⊲ Normalize user ratings.
⊲ Get the maximum rating of user j.
⊲ Get the minimum rating of user j.

�� −����

���� ← 1 + 99 · ����−����end for
10: end forfor ����ℎ� ∈ ����ℎ�� do
12:score_of_this_weight = 0
for � ∈ � doÍ��
14:
index ← argmax( �=1 ����ℎ� [�] · �� � [�])8:⊲ Normalize the user rating to be in the range of [1, 100].

⊲ Start the actual optimization. Loop over all the weight settings.
⊲ Loop over every user.
⊲ Get the index of the highest weighted sum.

� ∈� �

score_of_this_weight ← score_of_this_weight +������
�16:
end forscores.append(score_of_this_weight)
18: end forfnal_index ← argmax(scores)
20: optimal_weight ← weights[fnal_index]
return optimal_weight⊲ Add the rating of the highest value to the score.

⊲ Get the index of the weight setting with the highest score.
⊲ Get the optimal weight setting.
⊲ Return the optimal weight setting.

CHI ’24, May 11–16, 2024, Honolulu, HI, USA

Y.-C. Liao, et al.

Parameters

Corresponding Objective Functions

Range

�1
�2
�3
�4

Sphere 1, center = (0.55, 0.4)
Sphere 1, center = (0.55, 0.4) & Sphere 2 (center = (0.6, 0.45)
Sphere 2, center = (0.6, 0.45) & Sphere 3, center = (0.65, 0.35)
Sphere 3, center = (0.65, 0.35)[0, 1][0, 1][0, 1][0, 1]

Table 6: The parameters of the base function used in our Simulation 1 and Simulation 2. Here we present the corresponding
objective functions of each parameter and the parameters’ ranges.

B

EVALUATING TAF+ ’S VIABILITY VIASIMULATIONS ON SYNTHETIC FUNCTIONS

We evaluate the performance of our proposed TAF+ in multiobjective tasks through simulations with commonly used testing
functions. For the evaluation of TAF in single-objective problems,
please refer to Wistuba et al. [100] and Volpp et al. [93]. Here, we
aim to answer four goals through these simulations:• Simulation 1: Evaluating the robustness of TAF+ under diverse objective weight confgurations.
• Simulation 2: Validating the efectiveness of TAF+ undervarying levels of user similarity.
• Simulation 3: Investigating the generalizability of TAF+
across diferent testing functions.
• Simulation 4: Investigating the performances of TAF+ withdiferent numbers of population models.

B.1

Base function for simulation 1 andsimulation 2

In Simulations 1 and 2, we utilize the same base function, which has
four design parameters and three objective functions. The numberof parameters and functions aligns with our relative pointing interaction. Specifcally, we have adopted the 2-dimensional Spherefunction11 as our choice for objective functions. To elaborate further, our Sphere function is mathematically defned as follows:Í� = 1 − �2=1 (�¤� − �� ) 2 × �. Here, �¤� represents the selected valueof a design parameter, and �� denotes the center of the squarefunction. This Sphere function has its maximum objective value(� = 1) when the selected �¤ is right at the center position; i.e.,(�¤1, �¤2 ) = (� 1, � 2 ). As the parameter values deviate from this central point, the objective values gradually decrease. We incorporate
an additional coefcient �, set to be 8, to accelerate the rate ofdecay in the function. We chose the Sphere function for two reasons. Firstly, it efectively simulates the real-world scenario wherea user’s performance gradually decreases as they deviate from theoptimal design parameter setting. Secondly, the Sphere function is
widely recognized and used for similar optimization evaluations.
We then utilize the Sphere function to construct the base function, which contains four parameters (� 1, � 2, � 3, � 4 ) and 3 objectivefunctions (�1, �2, �3 ). Details are provided in Table 6. We frst create
three distinct Sphere functions, each having a unique center position (� 1 , � 2 ). Consequently, there is no single parameter setting thatmaximizes all three Sphere functions simultaneously. Furthermore,
the frst two parameters in the base function (� 1, � 2 ) contribute to
the frst Sphere function, (� 2, � 3 ) contribute to the second Spherefunction, and (� 3, � 4 ) contribute to the third Sphere function. Parameters � 2 and � 3 are shared by two Sphere functions. This decision
was made to introduce trade-of scenarios: there exists no single � 2or � 3 value that can optimize both Sphere functions. Thus, when
performing multi-objective optimization, the outcome comprisesa series of Pareto-optimal values but not a single optimal value,
which simulates the trade-ofs in real-world design challenges. The
range of the parameters is [0, 1] and the range of the objective
values is roughly [−1, 1].
Finally, when obtaining the output (�) from the Sphere functions,
we introduce a noise value for mimicking humans’ noisy performance. The noise value is sampled from a Gaussian distribution,
denoted as ����� ∼ N (�, � 2 ), where � = 0 and � = 0.05.

B.2

Generating synthetic users and user groups

In our simulations, we shift and scale the base function to createa group of testing functions, which is a common approach [24]12 .
We also call these generated testing functions as user functions orsynthetic users. Each created user function can be seen as a uniqueuser as it has its own set of Pareto-optimal parameter settings andthe corresponding objective values. More specifcally, we shift every
parameter value in the base function (� 1 , � 2 , � 3 , � 4 ) by diferent
amounts. Figure 9(a) shows as an example. The magnitudes of theseshifts are sampled from a uniform distribution within given ranges.
We can formally denote this shifting as ��′ = �� + �� , where ��is the original parameter value, ��′ is the shifted value, � ∈ {1, 4}(each represents a parameter), and the shifting for each parameter�ℎ� � � _����� �ℎ� � � _�����
is �� ∼ � (−,).
2
2
In addition to shifting the base function, we also scale theobjective function values to further generate diversity. A scalar
is directly multiplied by the objective value; see Figure 9(b) as
an example. Such a scalar is drawn from a uniform distribution
where 1 is the center. We denote this function scalar as S ∼�����_�����
�����_�����
� (1 −,1 +). When the scalar (�) exceeds
22
1, the maximum value of the function will go beyond 1. Conversely,
if � is less than 1, it reduces the function output value. An example
is illustrated in Figure 9 (b). This scaling simulates the diferent
levels of performance exhibited by diferent users.
To simulate diferent scenarios where the user groups have various levels of similarities, we sample function shifting (�) andscalar (S) from diferent ranges (�ℎ� � �_����� and �����_�����).
In the below simulations, we deployed diferent �ℎ� � �_����� and�����_�����. A small range naturally generates a group of functions
with the highest similarities, simulating a highly similar user group.
An illustrated example is illustrated in Figure 9(c). Conversely, a
12 The frst fgure in https://botorch.org/tutorials/meta_learning_with_rgpe also

11 See https://www.sfu.ca/~ssurjano/spheref.html.

presents an example of shifting and scaling.

A Meta-Bayesian Approach for Rapid Online Parametric Optimization for Wrist-based Interactionsab

CHI ’24, May 11–16, 2024, Honolulu, HI, USAcdFigure 9: Here we use a typical Gaussian distribution to illustrate how we generate a group of testing functions by shiftingand scaling a base function: (a) demonstration of shifting a base function, (b) demonstration of scaling a base function, (c) an
example user group with a higher similarity, (d) an example user group with higher diversity.

large range leads to a group of highly diverse functions, mimicking
a group of very diversifed users. See Figure 9(d) for an illustrated
example.
In each of the following simulations, we randomly sample 20
shifting (�) and scaling (S) to create 20 distinct synthetic users, each
representing a user. Among these 20 synthetic users, 10 of them aredesignated as "population users." We perform population modelingon these users. The remaining 10 functions are considered "newusers." TAF+ , is subsequently applied to these new synthetic users.

B.3 Simulation 1: Validating the robustness of
TAF+ under various objective weight
settings
One important feature of our TAF+ is that it allows the designers or developers to dynamically assign the objective weights
when performing weighted-sum optimization in the adaptation
phase. Simulation 1 aims to investigate whether TAF+ can maintain stable performance across varying objective weights. Notethat the three Sphere functions in our base function have distinct
optimal parameter values, and two parameters (� 2, � 3 ) are sharedby two Sphere functions. Thus, altering the objective weights
in weighted-sum optimization results in diferent sets of optimal
parameter settings. Since our focus is on assessing the impact ofobjective weight assignments, we intentionally chose the smallest sample range to create the user group: both �ℎ� � � _����� and�����_����� are set to be 0.01.
Step 1: Population modeling via a user data collection.
Following our workfow (see subsection 4.2), we performed multiobjective BO to construct 10 population models13 . The progress isplotted in Figure 10.
Step 2 & 3: Objective weight and decay hyperparametersetting. To test the performance of TAF+ under a wide spectrumof objective weights, we deliberately test 6 sets of objective
weights ([1, 0, 0], [0, 1, 0], [0, 0, 1], [0.33, 0.33, 0.34], [0.5, 0.3, 0.2],
and [0.3, 0.5, 0.2]) when deploying TAF+ on the new users (functions). Since this group of users has high similarity, there is no
population model decay applied in this simulation (for details regarding such a decay, please see subsection 4.1).
Adaptation: Evaluating TAF+ on new users. We applied
TAF+14 on 10 new users (unseen user functions). We further applied
standard BO15 and random sampling as baseline conditions. The
resulting performances of TAF+ when objective weights are set

13 The hyperparameter setting of multi-objective BO is the same as introduced insubsection 6.1 with 20 initial random samples followed by 20 optimization iterations.
14 The hyperparameter settings are similar to subsection 7.2 with 256 random samplingwhen calculating TAF+ values.
15 The hyperparameter setting is as introduced in subsection 7.2 of the main paperwith 20 initial random sampling and 20 optimization iterations.

CHI ’24, May 11–16, 2024, Honolulu, HI, USA

Y.-C. Liao, et al.

1.0Hypervolume

0.8
0.6
0.4
0.2
0.0
Max
MOBO
Random

0.2
0.4

0

5

10

15

20

25

Iterations

30

35

40

Figure 10: The hypervolume per iteration during the population modeling in simulation 1. The random condition was to serve
as a baseline. The error bar denotes a 95% confdence interval. The global maximum hypervolume was determined by a gridsearch when there was no noise involved.

as [1, 0, 0], [0, 1, 0] (Figure 11), [0, 0, 1], [0.33, 0.33, 0.34] (Figure 12),
[0.5, 0.3, 0.2], and [0.3, 0.5, 0.2] (Figure 13) are plotted below.
Findings. Overall, TAF+ performs stably across diferent objective weight confgurations. It consistently converges to the global
maximum value with 3 or 4 iterations. This result highlights theadvantage of using multi-objective BO in population modeling: itefectively allows the designers to tune the objective weights
when needed while maintaining high efciency when deploying
TAF+ . Conversely, such fexibility would not exist if the population
modeling phase deployed weighted-sum single-objective BO.

B.4

Simulation 2: Validating the efectiveness of
TAF+ under diferent user group similarities

In the second simulation, we aim to understand the performanceof TAF+ when dealing with diferent levels of similarities in theuser group. To that end, we modify the sampling range of shifting(�ℎ� � �_�����) and scaling (�����_�����) when generating the testing functions. Diferent sampling ranges will result in functions ofdiferent levels of similarities. The sampling ranges in this simulation are 0.05, 0.1, 0.2, and 0.3. Particularly, a shifting range of 0.3
can create a huge diversity, given that the whole parameter range
is only 1. Similar to Simulation 1, we sampled 20 functions for eachsampling range; 10 serve as population users, and the remaining 10serve as new users.
Step 1: Population modeling via user data collection. Similar
to the previous simulation, for each sampling range, we performed
multi-objective BO to construct 10 population models16 . The resulting hypervolumes are plotted in Figure 14 and Figure 15. Despite
the increasingly higher variations between functions (indicated by
the error bar), multi-objective BO can efectively explore the Pareto
frontier for each group.
Step 2 & 3: Objective weight and decay hyperparameter setting. To focus on investigating the efect of user similarity,
we fxed the objective weight at [0.3, 0.5, 0.2] throughout thissimulation. We followed the workfow (see subsection 6.3) to optimize the decay hyperparameter settings (� 1 and � 2 ) For sampling
range 0.05, the optimal hyperparameter setting is (� 1, � 2 ) = (7, 0.1).
For sampling range 0.1, the optimal hyperparameter setting is
16 The hyperparameter setting of multi-objective BO is the same as introduced insubsection 6.1 with 20 initial random samples followed by 20 optimization iterations.

A Meta-Bayesian Approach for Rapid Online Parametric Optimization for Wrist-based Interactions

Objective weights = [1, 0, 0]

CHI ’24, May 11–16, 2024, Honolulu, HI, USA

Objective weights = [0, 1, 0]

Figure 11: The resulting performance of TAF+ when the objective weights are [1, 0, 0] and [0, 1, 0], and the population modelinghas � and S are sampled from range of 0.01. The error bar indicates a 95% confdence interval. The global maximum objective
value was determined by a grid search when there was no noise involved.

Objective weights = [0, 0, 1]

Objective weights = [0.33, 0.33, 0.34]

Figure 12: The resulting performance of TAF+ when the objective weights are [0, 0, 1] and [0.33, 0.33, 0.34], and the population
modeling has � and S are sampled from range of 0.01. The error bar indicates a 95% confdence interval.

(� 1, � 2 ) = (4, 0.1). For sampling range 0.2, the optimal hyperparameter setting is (� 1, � 2 ) = (4, 0.2). For sampling range 0.3, theoptimal hyperparameter setting is (� 1, � 2 ) = (3, 0.2). These optimal
decay settings show that as the user diversity increases, the decay
should happen earlier and faster to allow for better adaptation of
the new user’s characteristics.
Adaptation: Evaluating TAF+ on new users. We applied TAF+
on 10 new user functions. Standard BO and random sampling are
set as baselines. The resulting performances of TAF+ are presented
in Figure 16 and Figure 17.

Findings. TAF+ overall performs well across diferent sampling
ranges. Upon closer examination, we noted that greater diversity,
such as sampling from ranges of 0.2 or 0.3, results in a lower starting
performance for TAF+ . This outcome is natural, as higher sampling
ranges introduce potentially larger diferences between the population users and the new users, necessitating more iterations for
TAF+ to adapt to the new user functions. However, despite thelower starting point, TAF+ still converges to the maximum objective value within 10 iterations. This highlights TAF+ ’s potential
even when being applied to an interaction where users exhibit highdiversity in their preferences and performances.

CHI ’24, May 11–16, 2024, Honolulu, HI, USA

Objective weights = [0.5, 0.3, 0.2]

Y.-C. Liao, et al.

Objective weights = [0.3, 0.5, 0.2]

Figure 13: The resulting performance of TAF+ when the objective weights are [0.5, 0.3, 0.2] and [0.3, 0.5, 0.2], and the population
modeling has � and S are sampled from range of 0.01. The error bar indicates a 95% confdence interval.

Sampling range = 0.05

Sampling range = 0.1

Figure 14: The hypervolume per iteration during the population modeling in Simulation 2. The random condition was to serve
as a baseline. The error bar denotes a 95% confdence interval, and the maximum volume is determined by a thorough gridsearch.

B.5

Simulation 3: Validating the generalizabilityof TAF+ with diferent testing functions

From the previous two simulations, we demonstrated the consistent
performance of TAF+ when facing the three-sphere base function.
In the third simulation, we investigate the efcacy of TAF+ whenfacing other, potentially more challenging, base functions. These
diferent base functions are introduced, simulating diferent interactions. There are three base functions in this simulation:• Relocated Spheres: This base function is similar to the oneused in the previous simulations, which contains 3 Spherefunctions. However, the Spheres here have diferent andfurther apart center locations. See Table 7.
• Spheres + Branin: This base function has two Sphere functions
and a Branin17 . Branin is a commonly used function for
testing optimization and it is considered more challenging
than Sphere. Details are in Table 8.

17 Please see http://www.sfu.ca/~ssurjano/branin.html for more details. We normalized

the parameter space into [0, 1] range, and reversed the function so that it becomes a
maximization problem.

A Meta-Bayesian Approach for Rapid Online Parametric Optimization for Wrist-based Interactions

Sampling range = 0.2

CHI ’24, May 11–16, 2024, Honolulu, HI, USA

Sampling range = 0.3

Figure 15: The hypervolume per iteration during the population modeling in Simulation 2. The random condition was to serve
as a baseline. The error bar denotes a 95% confdence interval, and the maximum volume is determined by a thorough gridsearch.

Sampling range = 0.05

Sampling range = 0.1

Figure 16: The resulting performance of TAF+ when the functions’ shifting (�) and scaling (S) are sampled from range of 0.05
and 0.1. The error bar indicates a 95% confdence interval.

• Sphere + Branin + Hartmann 3D: This base function is further
complicated. It contains a Sphere, a Branin, and a Hartmann
3D18 . Hartmann 3D is a function that takes 3 parameters
as input, and it is also a widely recognized testing function.
Note that � 2 in this function is shared by three functions,which added more complexity to this task. Details of thisfunction are in Table 9.

18 Please see https://www.sfu.ca/~ssurjano/hart3.html for more details. We reversed

the function so that it becomes a maximization problem.

We set the sampling range as 0.1 for both function shifting (�)and scaling (S) when generating user functions. Same as in previous
simulations, we sampled 20 user functions; 10 serve as population
users, and the remaining 10 serve as new users.
Step 1: Population modeling via user data collection. We
followed the same procedure, using multi-objective BO, to constructpopulation models. The resulting hypervolume is plotted in Figure 18. Because Spheres + Branin and Sphere + Branin + Hartmann
3D functions are more challenging than the three-Sphere function,

CHI ’24, May 11–16, 2024, Honolulu, HI, USA

Y.-C. Liao, et al.

Sampling range = 0.2

Sampling range = 0.3

Figure 17: The resulting performance of TAF+ when the functions’ shifting (�) and scaling (S) are sampled from range of 0.2
and 0.3. The error bar indicates a 95% confdence interval.
Parameters

Corresponding Objective Functions

Range

�1
�2
�3
�4

Sphere 1, center = (0.55, 0.3)
Sphere 1, center = (0.55, 0.3) & Sphere 2 (center = (0.8, 0.35)
Sphere 2, center = (0.8, 0.35) & Sphere 3, center = (0.7, 0.4)
Sphere 3, center = (0.7, 0.4)

[0, 1][0, 1][0, 1][0, 1]

Table 7: The parameters of the Relocated Spheres base function.

Parameters

Corresponding Objective Functions

Range

�1
�2
�3
�4

Sphere 1, center = (0.55, 0.3)
Sphere 1, center = (0.55, 0.3) & Branin
Branin & Sphere 2, center = (0.7, 0.4)
Sphere 2, center = (0.7, 0.4)

[0, 1][0, 1][0, 1][0, 1]

Table 8: The parameters of the Spheres + Branin base function.

Parameters

Corresponding Objective Functions

Range

�1
�2
�3
�4

Sphere 1, center = (0.55, 0.3)
Sphere 1, center = (0.55, 0.3) & Branin & Hartmann 3DBranin & Hartmann 3D
Hartmann 3D[0, 1][0, 1][0, 1][0, 1]

Table 9: The parameters of the Sphere + Branin + Hartmann 3D base function.

we extended the optimization iterations to ensure the quality of the
population models.
Step 2 & 3: Objective weight and decay hyperparametersetting. Similar to Simulation 2, we set the objective weight at[0.3, 0.5, 0.2] in this simulation. We followed the workfow to optimize the decay hyperparameter settings, and the resulting optimal
setting is (� 1, � 2 ) = (4, 0.1).
Adaptation: Evaluating TAF+ on new users. We deployed
TAF+ on 10 new users. The resulting performances of TAF+ compared against BO and random sampling are presented in Figure 19.

Findings. The results highlight the promising performance of
TAF+ when facing diferent sets of functions, demonstrating thepotential generalizability of TAF+ for diferent scenarios and interactions. It is also worth noting that BO encounters challenges
in efciently reaching optimal objective values when applied tomore complex functions, such as mixing Branin, Sphere, and Hartmann 3D. However, with the aid of population models, TAF+ isable to leverage the prior experience to address such complex functions efciently; which further showcases the generalizability of
our approach when facing challenging optimization problems.

A Meta-Bayesian Approach for Rapid Online Parametric Optimization for Wrist-based Interactions

Relocated Spheres

Spheres + Branin

CHI ’24, May 11–16, 2024, Honolulu, HI, USA

Sphere + Branin + Hartmann 3DFigure 18: The resulting performance of TAF+ when the functions’ shifting (�) and scaling (S) are sampled from range of 0.05
and 0.1. The error bar indicates a 95% confdence interval.

Relocated Spheres

Spheres + Branin

Sphere + Branin + Hartmann 3DFigure 19: The resulting performance of TAF+ when the functions’ shifting (�) and scaling (S) are sampled from ranges of 0.2
and 0.3. The error bar indicates a 95% confdence interval.

B.6

Simulation 4: Investigating the performanceof TAF+ under diferent population modelsizes

In this simulation, we aim to evaluate the performance of TAF+
when it has various numbers of population models. We utilized the
3-Sphere base function, the same as Simulations 1 and 2 (see Table 6).
The sampling ranges (�ℎ� � �_����� and �����_�����) are set to be
0.2, and the results are presented in Figure 20. We do not present thedetails of population modeling and decay hyperparameter setting
as they are the same as in previous simulations 1 and 2.
Adaptation: Evaluating TAF+ with diferent numbers of
population models. The resulting performance over iterations
are shown in Figure 20. Overall, with a higher number of the population model, TAF+ potentially has a better starting performance.
However, regardless of the number of population models, with suffcient iterations and proper decay hyperparameter confguration,
TAF+ can still converge toward optimality.
Figure 20: The resulting performance of TAF+ when the number of population models varies.

CHI ’24, May 11–16, 2024, Honolulu, HI, USA

Y.-C. Liao, et al.

Relative pointing

Hypervolume

Hypervolume

Absolute pointing

Iteration number

Iteration number

Figure 21: The resulting hypervolume per iteration for both interactions during population modeling. The error bar denotes a
95% confdence interval.

B.7

Overall fndings of the simulations

The above simulations collectively demonstrate the robustness of
TAF+ under diverse conditions, highlighting its adaptability andgeneralizability.
Simulation 1 examines TAF+ under varying objective weights,
and the positive results highlight the fexibility of adjusting theobjective weights when deploying TAF+ . This fexibility is particularly useful when conducting weighted-sum optimization forreal-world interactions, where designers may need to fne-tune theweighting on objectives. Simulation 2 assesses TAF+ ’s performance
under diferent levels of user diversity, and the results showed
TAF+ ’s efectiveness under diversifed user groups, highlighting
TAF+ ’s ability to adapt to diferent user performances and preferences. Simulation 3 validates TAF+ ’s generalizability across avariety of base functions. Despite the increasing challenges of thefunctions, TAF+ retains promising performance whereas standard
BO may not be able to converge efciently. Simulation 4 shows theimpact of diferent numbers of population models. While the lesspopulation models may lead to a lower initial performance, TAF+
can converge more efciently than standard BO. To summarize, theresults and fndings provide evidence that TAF+ is adaptable andcapable of handling a wide spectrum of problems with diferent
levels of difculties and conditions.
Finally, to further assess the computation complexity, the frst
3 simulations with 10 population models were run on a ThinkPad
X1 Carbon, with Ubuntu 22.04.3 OS and an Intel i7-10150U CPU.
Throughout these simulations, TAF+ ’s computation time for each
iteration is 6.17 seconds (�.�. = 1.17).

C

HYPERVOLUME INCREASE IN
POPULATION MODELING

Figure 21 shows the hypervolume of both interactions at each
iteration during the population modeling.

D EXTENDED SIMULATION FOR THE DECAY
HYPERPARAMETER OPTIMIZATION
To validate the convergence of the optimization methods, we extended the decay hyperparameter optimization. Figure 22 shows theextended simulation for the decay hyperparameter optimization.
The result shows that TAF+, TAF, and BO all converge near the
global max given enough iterations.

A Meta-Bayesian Approach for Rapid Online Parametric Optimization for Wrist-based Interactions

CHI ’24, May 11–16, 2024, Honolulu, HI, USA

Relative pointing

Iteration number

Iteration number

Net performance overmultiple objectives

Absolute pointing

Figure 22: The resulting performance under simulation. The max performance was derived by a grid search.

E

Table 11: The mean and standard deviation of the weightedsum performance at each procedure in relative pointing.

DETAILS OF THE EVALUATION

Here, we provide the detailed numbers of the evaluation, which
complements Figure 8. Table 10 and Table 11 list all the mean andstandard deviation of weighted-sum performance in absolute pointing and relative pointing, respectively. We also found the overall
performance is better than the estimated performance in step 3(decay hyperparameter optimization, and see Figure 7). We examined the performance we gathered in step 1 (population modeling)
and compared it to the performance gathered in adaptation19 . Witht-test, we found a signifcant diference (� (23) = −3.576, � = 0.02)
between the population user group (mean = 72.87, �.�. = 0.114) andthe adaptation user group (mean = 0.87, �.�. = 0.11). The diferencein the performance levels may cause the overall diference in absolute pointing between Figure 7 and Figure 8. The positive result in
Figure 8 also validates the efcacy of TAF+ when the users havediferent levels of performance.
Table 10: The mean and standard deviation of the weightedsum performance at each procedure in absolute pointing.
# of Iteration

Meta-BO

BO

Manual

1
2
3
4
5
6
7
8
9
10

0.64 (s.d. = 0.09)
0.71 (0.08)
0.79 (0.05)
0.81 (0.03)
0.83 (0.03)
0.85 (0.03)
0.86 (0.03)
0.87 (0.02)
0.87 (0.02)
0.87 (0.02)

0.31 (s.d. = 0.08)
0.56 (0.07)
0.62 (0.06)
0.70 (0.05)
0.71 (0.05)
0.71 (0.05)
0.72 (0.05)
0.75 (0.05)
0.75 (0.05)
0.75 (0.05)

0.42 (s.d. = 0.1)
0.57 (0.08)
0.70 (0.07)
0.61 (0.07)
0.66 (0.07)
0.72 (0.06)
0.74 (0.06)
0.75 (0.05)
0.78 (0.05)
0.80 (0.04)

19 For each user, we frst derived the highest weighted-sum performance, and thenwe compared the optimal performance of the population users and the users in theadaptation phase.

# of Iteration

Meta-BO

BO

Manual

1
2
3
4
5
6
7
8
9
10

0.27 (s.d. = 0.07)
0.46 (0.05)
0.60 (0.03)
0.62 (0.04)
0.66 (0.03)
0.68 (0.02)
0.68 (0.02)
0.68 (0.02)
0.68 (0.02)
0.70 (0.02)

0.14 (s.d. = 0.05)
0.35 (0.07)
0.37 (0.06)
0.43 (0.05)
0.53 (0.04)
0.55 (0.04)
0.59 (0.04)
0.61 (0.03)
0.61 (0.03)
0.61 (0.03)

0.31 (s.d. = 0.05)
0.42 (0.06)
0.50 (0.05)
0.50 (0.05)
0.50 (0.05)
0.58 (0.03)
0.59 (0.03)
0.61 (0.03)
0.63 (0.03)
0.64 (0.04)

CHI ’24, May 11–16, 2024, Honolulu, HI, USA

Y.-C. Liao, et al.

Table 12: The mean and standard deviation of the raw NASA-TLX scores in absolute pointing. One-way repeated-measures
ANOVAs showed that the only signifcant diference was found in the sixth question where both Meta-BO and BO led tosignifcantly lower scores than the Manual procedure.
Questions
1. How mentally demanding was the task?
2. How physically demanding was the task?
3. How hurried or rushed was the pace of the task?
4. How successful were you in accomplishing what you were asked to do?
5. How hard did you have to work to accomplish your level of performance?
6. How insecure, discouraged, irritated, stressed, and annoyed were you?*F

ANALYSIS ON THE PERCEIVED WORKLOAD
AND USER EXPERIENCE

We present the detailed result of the NATA-TLX questionnaire in
Table 12 and Table 13 for absolute pointing and relative pointing,respectively. We performed one-way repeated measures ANOVA
with Greenhouse-Geisser correction on each question individually.
The only statistically signifcant diference was found in the 6thquestion, which is related to the perceived frustration (“How insecure, discouraged, irritated, stressed, and annoyed were you?”),in absolute pointing (� (1.26) = 6.735, � < 0.05). Pairwise comparison with Bonferroni correction showed that both Meta-BO andBO led to signifcantly lower frustration levels than Manual (both� < 0.05). In the interview, most participants refected on the factthat the manual calibration is time-consuming and not bringing
a better experience is frustrating and unwanted. We did not fnd
signifcant diferences for the rest questions in both interactions.
Overall, these results indicate that Meta-BO delivers a comparable
or better experience to the users than other procedures.

User experience

Based on the user interviews, we learned that most users felt both
TAF+ and BO gradually proposed improving designs. Interestingly,
six users felt BO proposed abrupt parameter settings. P1 mentioned,“[...] this round (BO) has more noticeable changes. It suddenly changedBO

Manual6.55 (s.d. = 4.76)
7.91 (4.93)
5.18 (4.69)
4.45 (3.46)
9.09 (4.91)
3.82 (2.67)

5.64 (s.d. = 3.67)
6.45 (2.73)
3.73 (3.32)
2.91 (1.97)
7.73 (3.04)
3.82 (3.16)

8.64 (s.d. = 5.46)
9.18 (4.83)
4.91 (3.67)
5.18 (4.14)
9 (5.98)
8 (5.67)

speeds (the transfer function) [...] It was a bit unexpected.” P5 madea similar remark, “It feels a bit more unstable than the previous
ones (TAF+ and Manual). Overall, it still improves, but sometimes it
behaves strangely.” Such abrupt changes were not mentioned with
the TAF+ and Manual procedures.
While the performance diferences between the techniques areclear, participants found it somewhat difcult to clearly state which
technique was better at adapting better and faster. Our experiment
involves selecting targets of diferent distances and sizes, further
adding to the challenge of diferentiating the techniques subjectively when the user is focused on the task and not on perceiving thediferences. Prior work that analyzed user experience in pointing
interactions had similar challenges. For example, Casiez and Roussel [11] investigated the impact of various transfer functions. Theysummarized that the users could not tell the diferences between
diferent transfer functions despite the signifcant diferences in performance. Casiez et al. [14] conducted a thorough investigation on
the impact of diferent types of transfer functions, focusing only on
performance metrics. Lee et al. [52] compared various transfer function adaptation techniques and found few signifcant diferences
when analyzing NASA-TLX scores.

G

F.1

Meta-BOPERFORMANCE OF EACH PERFORMANCE
METRIC

Figure 23 and Figure 24 show the individual performance metrics
throughout 10 iterations for absolute pointing and relative pointing,respectively. We ran one-way repeated-measures ANOVA on each
iteration across all metrics and found that the meta-BO procedure
led to either comparable or signifcantly better results than theother baselines.
For normalized completion time in absolute pointing: MetaBO outperformed BO (� < 0.01) at the third, sixth, and eighth
iterations; Meta-BO also outperformed Manual at the fourth iteration. For aiming error and trajectory travel distance in absolute
pointing: Meta-BO outperformed BO (� < 0.01) at the third iteration, and also outperformed Manual at the fourth iteration.
For normalized completion time in relative pointing: MetaBO outperformed BO (� < 0.01) at the third, sixth, and seventh
iterations; Meta-BO also outperformed Manual at the fourth iteration. For trajectory travel distance in relative pointing: Meta-BO
outperformed both BO and Manual (� < 0.01) at the third iteration. These signifcant diferences are also marked in Figure 23 andFigure 24.

A Meta-Bayesian Approach for Rapid Online Parametric Optimization for Wrist-based Interactions

CHI ’24, May 11–16, 2024, Honolulu, HI, USA

Table 13: The mean and standard deviation of the raw NASA-TLX scores in relative pointing. No signifcant diferences were
found across all questions.
Questions
1. How mentally demanding was the task?
2. How physically demanding was the task?
Absolute
3. How hurried or rushed was the pace of the task?
4. How successful were you in accomplishing what you were asked to do?
5. How hard did you have to work to accomplish your level of performance?
6. How insecure, discouraged, irritated, stressed, and annoyed were you?

*

*

*

Manual9.09 (s.d. = 5.52)
10.09 (5.89)
6.36 (4.90)
4.64 (4.27)
8.27 (4.00)
6.36 (4.41)

8.36 (s.d. = 4.65)
8.64 (4.25)
5.09 (3.81)
2.91 (1.97)
8.91 (4.48)
5.91 (3.36)

8.55 (s.d. = 4.39)
8.73 (3.41)
5.18 (3.22)
5.09 (3.70)
10 (2.76)
6.73 (4.54)

Iteration number

Trajectory travel distance

* *

*

Ratio

*

BO

Aiming error

System unit

Second / bits

Normalized completion timeMeta-BO*

Iteration number

Iteration number

relative

Figure 23: The performance of individual objective functions in absolute pointing. The error bar shows a 95% confdence
interval. The red ∗ (meta-BO and BO) and the black ∗ (meta-BO and Manual) signs denote a signifcant diference between thetwo procedures at that iteration.

*

Iteration number

Trajectory travel distance

Ratio

* **

Aiming error

System unit

Second / bits

Normalized completion time

Iteration number**

Iteration number

Figure 24: The performance of individual objective functions in relative pointing. The error bar shows a 95% confdence interval.
The red ∗ (meta-BO and BO) and the black ∗ (meta-BO and Manual) signs denote a signifcant diference between the twoprocedures at that iteration.

H

PERFORMANCES AND OPTIMAL
PARAMETER SETTINGS OF EACH USER

Figure 25 shows the performances of absolute pointing of each user,
and Table 14 presents all the users’ optimal parameter settingsof absolute pointing. 10 users start with better performance withmeta-BO than with other baselines. Only P5 had a lower initial
performance; still, the performance of Meta-BO quickly increased
after the frst 2 iterations for this user. Upon examining the optimal
parameter setting of each user (Table 14), we can see most users’
optimal parameter settings have relatively high �� and � � , whichis aligned with Figure 27. P5 is exceptional since their optimal parameter setting is drastically diferent from others, which explains
their lower initial performance in Figure 25 (also denoted as P5).
However, TAF+ can still identify an optimal setting which is diferent from other participants for P5, highlighting its capacity to deal
with drastically diferent new users.
Figure 26 shows the performances of relative pointing of eachuser, and Table 15 shows the optimal parameter settings for relative
pointing of each user. 4 users have the highest performance withmeta-BO than with other procedures in the frst iteration. Later atthe 3rd iteration, meta-BO results in the best performances for 8

Weighted-sum objective value
over multiple objectives

CHI ’24, May 11–16, 2024, Honolulu, HI, USA

Y.-C. Liao, et al.

P1

P2

P3

P4

P5

P6

P7

P8

P9

P10

Meta-BO
BO
Manual
P11

Iteration number
Figure 25: The performances of absolute pointing of each individual user. The � axis is the number of iterations, and the �axis is the weighted-sum fnal objective value. Note that we have converted the problem into a maximization problem bynormalizing the objective functions.

users, which highlights TAF+ ’S strength in faster adaptation. Table 15 further shows there exist good parameter ranges for mostusers. For instance, the best range of �� and � 1 is around 0.7 to
1. However, P4’s optimal setting has a particularly low � 1 , which
results in a slightly lower starting performance when using TAF+
(P4 in Figure 26). Again, despite the diversity, TAF+ is able to identify the unique optimal setting, showing its ability to handle thediversity within the user group.

Weighted-sum objective value
over multiple objectives

A Meta-Bayesian Approach for Rapid Online Parametric Optimization for Wrist-based Interactions

CHI ’24, May 11–16, 2024, Honolulu, HI, USA

P1

P2

P3

P4

P5

P6

P7

P8

P9

P10

Meta-BO
BO
Manual
P11

Iteration number
Figure 26: The performances of relative pointing of each individual user. The � axis is the number of iterations, and the � axis is
the weighted-sum fnal objective value. Note that we have converted the problem into a maximization problem by normalizing
the objective functions.

CHI ’24, May 11–16, 2024, Honolulu, HI, USA

Y.-C. Liao, et al.

User ID

��

��

P1
P2
P3
P4
P5
P6
P7
P8
P9
P10
P11

0.81
0.94
0.78
0.71
0.28
0.87
0.86
0.82
0.65
0.67
0.92

0.86
1.00
0.86
0.78
0.21
0.86
0.80
0.94
0.89
0.77
0.94

Table 14: Every user’s optimal parameter setting for absolute
pointing. Note that we normalized all the parameter values
into the [0, 1] range for easier interpretation.

User ID

��

�1

�2

�3

P1
P2
P3
P4
P5
P6
P7
P8
P9
P10
P11

0.58
0.86
0.89
0.99
0.69
0.67
0.66
0.73
0.81
0.81
0.79

0.73
0.95
1.00
0.02
0.73
0.78
1.00
0.75
0.66
0.82
0.04

0.22
0.47
0.56
0.40
0.25
0.26
0.32
0.38
0.37
0.25
0.20

0.49
0.34
0.38
0.53
0.40
0.45
0.26
0.55
0.13
0.26
0.62

Table 15: Every user’s optimal parameter setting for relative
pointing. Note that we normalized all the parameter values
into the [0, 1] range for easier interpretation.

A Meta-Bayesian Approach for Rapid Online Parametric Optimization for Wrist-based InteractionsI

VISUALIZING THE OBJECTIVE FUNCTION
OF ABSOLUTE POINTING INTERACTION

running Ubuntu 22.04.3 OS with an Intel i7-10150U CPU. The relatively long computation time highlights the potential challenges ofapplying multi-objective TAF to fast-paced interactions.

Predicted Mean

To visualize the objective function of absolute pointing, we ftted
all the observations (<parameter values, resulting objective value>),across all users and across both TAF+ and BO procedures into a GP.
Then, we query the GP’s predicted mean. The resulting objectivefunction values are plotted as a heatmap (Figure 27).
We can see that high �� and � � values generally allow for better
user performance. Also, when �� and � � have high diferences(indicating unbalanced transfer functions along two axes), the user
performance is generally poor.

Sy

Sx
Figure 27: The visualization of the objective function of theabsolute pointing.

J SIMULATING MULTI-OBJECTIVE TAF USINGEXPECTED HYPERVOLUME INCREASE
Figure 28 shows the hypervolume at each iteration of the simulation.
There are three optimization procedures: multi-objective TAF, standard multi-objective BO, and random samples. The multi-objective
TAF largely follows our TAF+ setting as described in the paper butchanged the base acquisition function from Expected Improvement
to Expected Hypervolume Improvement (EHVI) [19, 22]. With thischange, multi-objective TAF identifes the Pareto-optimal parameter settings. In contrast, TAF+ identifes one optimal parametersetting for a weighted-sum objective metric. The baseline multiobjective Bayesian optimization with EHVI was implemented with
BoTorch [4]20 . In this simulation, the computation time per iteration
ranges from 62 to 95 seconds, conducted on a ThinkPad X1 Carbon
20 Hyperparameter settings: We used a single-task GP with Matern 5/2 kernel. Similar

to Chan et al. [15], we set � = 1. We further set 10 optimization restarts for the
optimization of the acquisition function, 1024 restart candidates for the acquisition
function optimization, and 512 Monte Carlo samples to approximate the acquisition
function.

CHI ’24, May 11–16, 2024, Honolulu, HI, USA

CHI ’24, May 11–16, 2024, Honolulu, HI, USA

Y.-C. Liao, et al.

Relative pointing

Iteration number

Iteration number

Hypervolume

Absolute pointing

Figure 28: The hypervolume of three diferent procedures derived via a simulation.