SlowGuess/NeurIPS_2023_OCR huggingface dataset:
The kernel and hyperparameters with the lowest Brier score, averaged across all 9 other patients, were selected (Matérn 5/2). To verify that this kernel performed well, we also ran human-in-the-loop optimization (HILO) for 20 random simulated patients, using the best hyperparameters for each of the four previously mentioned kernels. The results are shown in Figure C.1. The Matérn 5/2 kernel performed slightly better than the Matérn 3/2 kernel, and significantly better than the ARD kernel. While performance was similar to the Squared Exponential kernel, we ultimately selected Matérn 5/2 due to its lower Brier score.
Figure C.1: Joint perceptual loss (y axis, log scale) plotted throughout HILO with different Gaussian process kernels. Error bars denote SEM.
C.2 Hyperparameter Optimality
The Gaussian process kernel hyperparameters selected with our transfer learning strategy performed well in our simulations, leading to higher-quality patient-specific stimulus encodings (Figure 4). This transfer learning approach was chosen to match a clinical setting, where limited human data availability. However, it is certainly possible that better performing hyperparameters exist. To investigate the optimality of our hyperparameters, we examine two other strategies: an ideal case where 'patient-optimal' hyperparameters are used for each patient, and an online strategy where hyperparameters are updated during optimization with each patient,
Patient-optimal Hyperparameter Selection In this strategy, hyperparameters were precomputed for each patient by simulating 200 random duels. Kernel parameters were again fit using type II maximum likelihood estimation [60]. These 'patient-optimal' hyperparameters were then used during HILO for the patient.
Figure C.2: Reconstruction error using Gaussian process kernel hyperparameters selected via transfer learning and 'patient-optimal' strategies throughout HILO.
Optimization results for 20 random patients are shown in Figure C.2. Both the transfer and the patient-optimal settings led to similar performance, and there was no significant difference between the final reconstruction errors  $(p > .05$ , two-sided paired t-test). Thus it seems the transfer learning strategy indeed selected hyperparameters that generalize well for new patients. Note that in general the transfer learning selection strategy is more practically applicable than the patient-optimal strategy since it does not require a long calibration period, but the patient-optimal strategy is an alternative that could be used for the first human subjects when no human data (only simulated data) is available.
Online hyperparameters selection While it is common to keep kernel hyperparameters constant during optimization [33, 61, 62], online optimization is an alternative, where kernel parameters are periodically re-fit to the patient data during optimization. We tested online optimization where hyperparameters were recalculated with an update period of 1, 5, 10, or 20 duels, or were never updated. In all settings, the initial hyperparameters were chosen using the transfer learning strategy.
The results for online hyperparameters optimization are shown in Figure C.3. All update periods eventually converged to similar performance as with the transfer learning strategy (i.e. never updating hyperparameters), but since recalculating hyperparameters is costly, online hyperparameter updates increased the optimization time required to reach a desired performance (Figure C.3, right). This suggests that just using the transfer learning hyperparameters is a better strategy than online updates.
Figure C.3: Reconstruction error (left) and Bayesian optimization time required to reach a specified reconstruction error (right) for HILO with online updates of GP hyperparameters.
C.3 Acquisition function
The acquisition function is responsible for choosing  $\phi_{1}$  and  $\phi_{2}$  for each duel, and must balance exploration of the search space with exploiting values of  $\phi$  that are expected to work well. The Maximally Uncertain Challenge (MUC) [42] acquisition presented in the main text was initially compared against 2 other top-ranking [42] acquisition functions: Bivariate Expected Improvement (Bivariate EI) [63], and Dueling Upper Credibility Bound (Dueling UCB) [64]. In addition, we also compare against a baseline acquisition, where  $\phi_{1}$  and  $\phi_{2}$  are chosen randomly for each duel. We ran HILO for the same 20 random patients for each acquisition.
The joint perceptual loss throughout optimization for each acquisition is presented in Figure C.4. All of the tested acquisition functions dramatically outperformed the random baseline, which converged to a value near the mean DSE without HILO. Although MUC and Dueling UCB performed similarly, we ultimately selected MUC due to its slightly lower final loss.
Figure C.4: Joint perceptual loss (y axis, log scale) plotted throughout HILO with different acquisition functions. Error bars denote SEM.
C.4 Batch Bayesian Optimization
While the proposed Bayesian optimization strategy of sequentially presenting the user with stimuli would likely not be prohibitively time consuming (about 17 minutes with 100 duels, 10 seconds per duel), it is possible that batch Bayesian optimization could speed up optimization. We consider two formulations of batch Bayesian optimizations: 1) the user selects their preference from a batch of stimuli in each iteration, and 2) each duel still has only two options, but batches of duels are precomputed to save on updated the Gaussian process posterior and the acquisition function time between duels.
Option 1 is theoretically more ideal, since more information acquired in each comparison would hopefully allow for fewer comparisons. However, this would require the patient to remember an entire batch of stimuli before making a comparison, and in practice phosphenes can be very difficult to discriminate [7]. It is difficult to evaluate the effect of this cognitive burden on simulated patients, and we thus leave it to future work to consider whether the parallelization of data acquisition would make up for the increased difficulty of the task.
Therefore, option 2 was tested by precomputing batches of 1 (i.e. original), 3, 6, or 10 duels at a time. We used a batch variant of the maximally-uncertain challenge acquisition function [42]. Results for 20 simulated patients are shown in Figure C.5. Precomputing batches reduced the optimization time required to reach a desired performance, but at the cost of requiring more duels. In practice, the optimal batch size could be determined by balancing the time required per duel with the time required for optimization. Faster acquisitions (e.g. KernelSelfSparring [65]) could further reduce optimization time.
Figure C.5: Reconstruction error (left) and Bayesian optimization time required to reach a specified reconstruction error (right) for HILO with different batch sizes.
C.5 Baselines and Robustness
In this section, we provide further details for the naive encoder we compared against, and the robustness experiments.
Naive Encoder The naive encoder is the encoding strategy currently used in commercial epiretinal prostheses [3]. This encoder operates under the assumption that each electrode can be thought of as a pixel in an image. The optimal stimulus under this assumption is therefore simply a downsampled version of the target image. The frequency and pulse duration are constant across all electrodes. This naive encoder has been previously shown to be suboptimal [14], but we still include it as a comparison to the currently used encoding strategy.
Robustness In section 4.4 we evaluate the robustness of human-in-the-loop optimization (HILO) to misspecifications in the forward model. Here, we provide specific details on the implementation of these misspecifications, and how we adapted the baseline encoders to the misspecified scenarios.
- Axon Trajectories: The simulated axon map from [47] has two parameters,  $\beta_{inf}$  and  $\beta_{sup}$ , which control the axon trajectories in the inferior and superior retina, respectively. [47] also reports the observed ranges for these parameters:  $\beta_{sup} \in [-2.5, -1.3]$  and  $\beta_{inf} \in [0.1, 1.3]$ . The unmodified model uses the centers of these ranges. Under misspecification, we randomly set both  $\beta_{sup}$  and  $\beta_{inf}$  to one of these bounds for each patient.
- Thresholds: Threshold is the amplitude at which a phosphene becomes visible to a patient  $50\%$  of the time. In epiretinal prostheses, thresholds are notoriously noisy, and vary significantly across electrodes, patients, and over time [48]. While some progress has been made towards predicting these thresholds [66], most state-of-the-art models assume that thresholds are known.
With this misspecification, the assumed threshold on each electrode was modified by a random but systematic amount. Specifically, the threshold on each electrode was randomly selected to be between  $\frac{1}{2}$  x and 2x its original value for the 100% condition and between  $\frac{1}{4}$  x and 4x its original value for the 300% condition.
- Out of Distribution: It is also possible that a new patient does not fall within our assumed ranges. Thus, we tested a variant where the true patient-specific parameters  $\phi$  were sampled from outside the ranges in Table 1. Specifically, each parameter was sampled to be  $0 - 50\%$  above or below the specified range (some parameters were clipped to stay within defined ranges, e.g.,  $\lambda$  cannot be outside of [0, 1)).
During HILO, the acquisition functions have specified bounds that constrain candidate  $\phi$ . We therefore tested two variants, one where PBO was allowed to expand the bounds, and another where it was confined to within its original bounds. The end results were similar in terms of DSE performance, so the variant with its original bounds is presented in the main text.
DSE- $\phi_{Guess}$  is our best approximation of what a DSE would guessed patient-specific parameters would perform, and is bounded by the performance of a DSE with mean  $\phi$ , and random  $\phi$  from the ranges in Table 1. For each of the misspecifications, the mean and random  $\phi$  baseline DSEs are still encoded with the same  $\phi$  as in the unchanged patient, but since the phosphene model for the patient is changed, the resulting loss is different. DSE- $\phi_{True}$  is still parameterized with the patients true  $\phi$ , however, the true  $\phi$  are no longer a perfect description of the misspecified patient. This is shown by the fact that HILO surpasses the true encoders performance: under the misspecified model, there exists some other  $\phi$  which leads to percepts with improved perceptual quality when decoded using the misspecified phosphene model, compared to those encoded with the true  $\phi$ . This highlights the robustness of optimization based on user preferences.