Original: More trustworthy Bayesian optimization of materials properties by adding humans into the loopAnonymous Author(s) Af ﬁ liation Address email
Revised: More trustworthy Bayesian optimization of materials properties by adding humans into the loop

Original: Abstract
Revised: Abstract

Original: Bayesian optimization (BO)  has proven to be  effective approach for guiding sampleef ﬁ cient exploration of materials domains and is increasingly being used in automated materials optimization set-ups.
Revised: Bayesian optimization (BO) is a popular sequential machine learning optimization strategy for black-box functions. BO has proven to be an effective approach for guiding sample-efficient exploration of materials domains and is increasingly being used in automated materials optimization set-ups.

Original: However, when exploring novel materials,sample quality may vary unexpectedly, which caneven invalidate the optimizationprocedure ifit remains undetected.
Revised: However, when exploring novel materials, sample quality may vary unexpectedly, which, in the worst case, can invalidate the optimization procedure if undetected.

Original: Thisissue limits the use of highly-automatedoptimization loops, especially in high-dimensional materials spaces with a lot ofsamples.
Revised: This limits the use of highlyautomated optimization loops, especially in high-dimensional materials spaces that require more samples.

Original: Sample quality may be hard to de ﬁ ne unequivocally for a machine buthuman scientists are usually good at judging sample quality, at least on a cursoryyet often suf ﬁ cient level.
Revised: Sample quality may be hard to define unequivocally for a machine but human scientists are usually good at quality assurance, at least on a cursory yet often sufficient level.

Original: In this work, we demonstrate that humans can be addedinto the BO loop as experts to comment on the sample quality, which results inmore trustworthy BO results.
Revised: In this work, we demonstrate that humans can be added into the BO loop as experts to comment on the sample quality, which results in more trustworthy BO results.

Original: We implemented human-in-the-loop BO via a datafusion approach and applied virtual BO cycles on experimental perovskite ﬁ lmstability data from literature.
Revised: We implement human-in-the-loop BO via a data fusion approach and simulate BO of experimental perovskite film stability (data from the literature).

Original: The human-in-the-loop approach facilitates automated materials design and characterization by reducing the occurrence of invalidoptimization results. 
Revised: Our human-in-the-loop approach facilitates automated materials design and characterization by reducing the occurrence of invalid optimization results. 36th Information Processing (NeurIPS 2022).

Original:  Introduction
Revised:  Introduction

Original: Bayesian optimization (BO) is an active learning method designed for globally optimizing a blackbox function with sequential acquisitions of samples [1].
Revised: Bayesian optimization (BO) is a machine learning method designed to globally optimize a black-box function with sequential acquisitions of samples [1-2].

Original: BO is intended especially for guidingthe acquisitions, which can be costly, towards the most promising areas of the search space [3].
Revised: BO is intended specifically for guiding the sample acquisitions – that can be costly – towards the most promising areas of the search space [3].

Original: Inparticular, BO utilizes a probabilistic surrogate model (typically, a Gaussian process, GP) that predictsthe target property across a ﬁ xed search space of input variables.
Revised: More precisely, BO utilizes a probabilistic surrogate model (typically, a Gaussian process (GP)) that predicts the target property across a fixed search space of input variables.

Original: The surrogate model is improvediteratively by  collecting new samples and the sampling strategy is determined by an acquisitionfunction.
Revised: The surrogate model is improved via active learning by iteratively collecting new samples. The sampling strategy is determined by an acquisition function that has access to the surrogate model.

Original: The acquisition function is based on the current surrogate model, and aims at balancingexploration of new areas or exploitation of  already sampled regions [3].
Revised: Acquisition functions balance the exploration of new areas against the exploitation of promising, already sampled regions [3].

Original: BOis a promising method to be used in self-driving materials laboratories because it can be combinedwith automation to form semi- or fully autonomous materials optimization loops.
Revised: BO can be combined with automation to form semi- or fully autonomous materials optimization loops in self-driving materials laboratories (the concept illustrated in Figure 1a step (i), in which compositions suggested by BO are synthesized in a laboratory and then measured).

Original: In materials science, BO has successfully been used for guiding  experimental or computational exploration of novelmaterials spaces(concept shown in step i in Figure 1a) and for determining  optimal combinations ofdevice settings [3-6].
Revised: In materials science, BO has successfully been used for guiding the experimental or computational exploration of novel materials spaces and for determining the optimal combinations of device settings [4-7].

Original: One of the factors limiting the broader use of BO in experimental materials science is the varyingquality of the experimentally prepared samples.
Revised: One of the factors limiting the broader use of BO in experimental materials science is the varying quality of the experimentally prepared samples.

Original: The samples should be of suf ﬁ cient quality toensure valid characterization results for the target property, which is a necessity in any experimentalwork with or without BO.
Revised: The samples should be of sufficient quality to ensure valid characterization results for the target property, which is a necessity in any experimental work.

Original: Especially when exploring materials compositions, tuning the samplepreparation conditions (e.g., annealing temperature) as a function of the composition may be requiredfor maintaining the good sample quality across the design space.
Revised: In material composition explorations, for example, the sample preparation conditions (e.g. annealing temperature) might have to be adjusted for specific compositions to maintain sufficient quality.

Original: Following the sample quality andadjusting procedures as necessary ispossible with smallsample sizes, or search spaces with existing domain knowledge on the nature and location of the possible sample quality issues.
Revised: Such adjustments are possible only when the number of required samples is small or when domain knowledge on the nature and location of potential sample quality issues is available.

Original: However, theincreasing level of laboratory automation facilitates a  shift toward higher-dimensional (i.e, moresamples needed) andmore novel search spaces (i.e., more unexpected events during the search).
Revised: The increasing level of laboratory automation facilitates a paradigm shift towards higher-dimensional (i.e., more samples needed) and novel search spaces (i.e., more unexpected events during the search).

Original: Human scientists can no longer manually check all  samples in-situ, else they would severely slowdown the otherwise accelerated optimization loop.
Revised: In that scenario, human scientists can no longer manually check all the samples in-situ, else, they would severely slow down the otherwise accelerated optimization loop.

Original: As another approach, it would be principallypossible to implement automated sample quality tracking via a series  added characterization methods,but this is in many applications prohibitively costly and dif ﬁ cult to maintain.
Revised: Alternatively, automated sample quality tracking via a series of added characterization methods could in principle be implemented, but this is in many applications prohibitively costly and difficult to maintain.

Original: Also, sample qualityis hard to de ﬁ ne precisely and unequivocally to a machine because there is a multitude of potentialsample quality issues, especially in novel search spaces in which unanticipated events frequentlyoccur.
Revised: Moreover, tracking sample quality fully automatically by a machine would require defining sample quality precisely and unequivocally before having prepared the samples. This is challenging especially in novel search spaces in which unanticipated events frequently occur.

Original: An experienced human scientist is more versatile  at detecting by quick inspection– and thencon ﬁ rming with a suitable characterization method when needed – sample quality issues, also theunexpected ones.
Revised: An experienced human scientist remains more versatile than a machine at suspecting sample quality issues after a quick inspection and – when needed – confirming them with a suitable characterization method.

Original: We therefore propose to add humans into the BO loop for evaluating sample quality but querying thehuman only when needed, thus preventing the humans from becoming a bottleneck.
Revised: We thus propose to add humans into the BO loop to evaluate sample quality, and query them only when the human opinion is needed to prevent the humans from becoming a bottleneck in the loop.

Original: Human-in-theloop approaches do exist in machine learning [7-11] but they are less frequent in materials science[12-13].
Revised: Human-in-the-loop approaches do exist in machine learning [8-12] but they are less frequent in materials science [13-14].

Original: In this contribution, we show via a simulated example that is based on real experimental data[3] that adding human into the BO loop can reduce the likelihood of converging to erroneous optimawith low-quality samples.
Revised: In this contribution, we show via a simulated example that is based on real experimental data [4] that adding humans into the BO loop reduces the likelihood of ending the search in erroneous optima.

Original: Thus, adding the human into the loop  increases trust in BO results with alimited amount of added effort.
Revised: Adding humans into the loop thus increases trust in BO optimization without significantly increasing the overall effort.

Original: By showcasing the human-in-the-loop approach, we hope to triggerfurther discussion onwhat are the roles of human scientists in  AI-guided design of new materials, andon how sample quality should be addressed in partially for fully self-driving materials laboratories.
Revised: By showcasing our human-in-the-loop approach, we hope to trigger discussion on the roles of human scientists in the AI-guided design of new materials, and on how sample quality should be addressed in partially for fully self-driving materials laboratories.

Original:  Results and discussion 
Revised:  Results and discussion 2.1 Framework

Original: 2.1 Setup

Original: We investigate the effects of the human-in-the-loop BO  in an example setting of compositionalstability optimization of perovskite ﬁ lm samples (Figure 1a, see the detailed methods description in Supplementary Methods).
Revised: We investigate the proposed human-in-the-loop BO methodology in an example setting of compositional stability optimization of perovskite film samples (Figure 1a, see the detailed methods description in Supplementary Methods Section A.1).

Original: We do not generate new experimental data  for this contribution.
Revised: New experimental data has not been generated for this contribution.

Original: Instead,we simulate BO loops utilizing experimental BO results from article [3] and sample from theenvironmental stability landscape(Figure 1b) produced in [3].
Revised: Instead, we simulate BO loops utilizing experimental BO results from [4] and sample from the environmental stability landscape produced in [4] (Figure 1b) to retrieve emulated experiment data.

Original: The composition of a ternary perovskite Cs x MA y FA z PbI 3 , wherex + y + z
Revised: The composition of a ternary perovskite Cs x MA y FA z PbI 3 , where

Original: = 1 , was in the previouswork [3] optimized to survive environmental stress from heat, humidity, and visible light illumination.
Revised: [Equation] was, in the previous work [4], optimized to survive environmental stress from heat, humidity and visible light illumination.

Original: At compositions suggested by BO, perovskite  thin ﬁ lm samples were prepared  and exposed to agingtests (detailed in Supplementary Methods).
Revised: The BO optimization variables were x , y and z , the compositional proportions of A site compounds in the perovskite ABX 3 crystal structure. Perovskite thin film samples were prepared with compositions suggested by BO and exposed to aging tests (detailed in Supplementary Section A.1.1).

Original: Since the investigated perovskite compositions undergocolor changes from dark to yellowﬁ lm when degrading [14], thechosen BO  targetvariable was thecolor change of the perovskite ﬁ lm integrated over time (during an aging test of ﬁ xed duration).
Revised: Since the investigated perovskite compositions undergo color changes from dark to yellow when the films degrade [14], the BO minimization target was taken to be the integrated color change of the perovskite film

Original: The BO input variables to be optimized were the compositional proportions of A site compounds x and y (and z = 1 − x − y ) in the perovskite ABX 3 crystal structure.
Revised: I c (cf Supplementary Section A.1.1).

Original: The loop of preparing samples withcompositions suggested by BO (a  batch-mode BO implementation with local penalization, expectedimprovement acquisition function, and GP surrogate model) was continued until the search converged.
Revised: The loop of preparing samples with compositions suggested by BO (a local penalization batch-mode BO implementation with expected improvement acquisition function and GP surrogate model with Matern kernel) was iterated until the search converged.

Original: The GP model of environmental stability  (Figure 1b) is reproduced from the converged BO search inarticle [3].
Revised: Here we adopt the same BO input space x, y, z and target variable I c to be minimized, sampling from the environmental stability GP model (Figure 1b) that we reproduced from the converged BO search in [4].

Original: It should be mentioned that in [3], the expected improvement acquisition function wasmodi ﬁ ed with a data fusion method to account for a secondary design requirement, thermodynamicphase stability of the compositions.
Revised: It should be mentioned that in [4], the expected improvement acquisition function was modified with a data fusion method to account for a secondary design requirement, thermodynamic phase stability of the compositions.

Original: In this work, we  utilize the  data fusion method for adding ahuman into the loop, but do not use the thermodynamic phase stability data.
Revised: In the current work, we do not use the thermodynamic phase stability data, but we utilize the same data fusion method for adding the humans into the loop. The data fusion method (inspired by the unknown constraint method [16]) uses the estimated probability of filling the secondary design requirement (here, sufficient sample quality evaluated by humans) as a down-weighting cost factor in the BO acquisition function.

Original: An experienced scientist can provide an initial evaluation of perovskite ﬁ lm quality based on theircolor and uniformity.
Revised: An experienced scientist can provide an initial evaluation of perovskite film quality based on its color and uniformity.

Original: All the samples producedwere initially photographed in [3].
Revised: All the samples produced in the earlier work [4] were photographed.

Original: Thus, in thiswork, we graded the sample quality on a scale of 0 (good quality) - 3 (low quality) post-hoc basedon the saved photographs (example pictures shown in Supplementary Figure 1).
Revised: Thus, in this work, we graded the sample quality on a scale of 0 (good quality) - 3 (low quality) post-hoc based on the saved photographs (example pictures shown in Supplementary Figure S2).

Original: We subsequentlytrained a GP model on this data (Figure 1c, see Supplementary Methods for details) that serves ashuman opinion in the present study.
Revised: We subsequently trained a GP model on this data (Figure 1c, see Supplementary Section A.1.2 for details) that serves as the human opinion in the present work.

Original: Interestingly, the resulting human opinion model (formed withno knowledge on the individual sample compositions) closely resembles the thermodynamic phasestability approximation in [3] (see Supplementary Figure S2).
Revised: Interestingly, the resulting human opinion model (formed with no knowledge on the individual sample compositions) identifies the same part of the search space as low quality than the thermodynamic phase stability approximation in [4] (see Supplementary Figure S3).

Original: This suggests  human evaluation onperovskite ﬁ lm samples indeed links to physically meaningful properties of the samples.
Revised: This suggests the human evaluation on perovskite film samples indeed links to physically meaningful properties of the samples.

Original: The comparison of the two GPs (Figure 1b-c) reveals a global optimum region (Region A) in thesearch space that has good quality samples and is environmentally stable.
Revised: The comparison of the two GP models (Figure 1b-c) reveals a global optimum region (Region A) in the search space that has good quality samples and is environmentally stable.

Original: There is also a falseoptimum region (Region B): it results in seemingly almost equally good environmental stabilitybut many of the samples are actually low quality, leading to invalid stability measurements in thisregion (the reasons discussed in Supplementary Section A.2).
Revised: There is also a false optimum region (Region B), which results in seemingly almost equally good environmental stability but many of the samples are actually low quality, leading to invalid stability measurements in this region (the reasons discussed in Supplementary Section A.3).

Original: Hence, in this work, the focus is oninvestigating if human input can stir the BO search away from Region B and facilitate convergencetowards the optimum in Region A.
Revised: Hence, in this work, we investigate if human input can steer the BO search away from Region B and facilitate convergence towards the optimum in Region A.

Original: 2.2 Human-in-the-loop implementation
Revised: 2.2 Human-in-the-loop implementation

Original: Human integration requires two steps: deciding when human opinion is required and integratingit into the BO loop (Figure 1a, details  in Supplementary Methods).
Revised: Human integration requires two steps: deciding when human opinion is required and integrating it into the BO loop (Figure 1a steps ii and iii, the details of our implementation in Supplementary Section A.1.3).

Original: For both steps, we have manychoices.
Revised: For both steps, we have many choices.

Original: This work is not a comprehensive evaluation of all  possible choices, but instead serves asa proof-of-concept human-in-the-loop con ﬁ guration that provides promising results for materialsoptimization loops.
Revised: This work is not a comprehensive evaluation of all the possible choices but instead, it serves as a proof-of-concept human-in-the-loop configuration that provides promising results for materials optimization. Here, the stability of all the samples is queried and human evaluation is requested on some of these samples at the time when the next samples are suggested.

Original: Here, we request human evaluation on the next samples suggested by BO only when there are noprevious human evaluations on the compositions nearby (the radius of the compositional exclusioncircle serving effectively as a hyperparameter) or when there are rapid changes in the environmentalstability as a function of composition (the limit value of the gradient serving as another hyperparameter).
Revised: Human evaluation is requested only when there are no previous human evaluations on the compositions nearby (the radius of the compositional exclusion circle, r excl , serving effectively as a hyperparameter) and there are rapid changes in the environmental stability as a function of composition (the limit value of the gradient, g excl , serving as another hyperparameter).

Original: The hyperparameter values are chosen based on experimental domain knowledge (see 
Revised: The hyperparameter values are chosen based on experimental domain knowledge (see Supplementary Section A.1.3): [Equation]

Original: Supplementary Methods section).
Revised: which is a tenth of the whole compositional range in this work, and [Equation]

Original:  The human evaluations are, in this work, integrated into the BO loop via  data fusion[3] (inspired bythe unknown constraint method [15]): the point-wise human opinions are approximated with a GPsurrogate model that spans the whole search space. 
Revised: where σ and l are the standard deviation and lengthscale of the environmental stability surrogate model, respectively. The human evaluations are, in this work, integrated into the BO loop via the previously mentioned data fusion method [4]: the point-wise human opinions are approximated with a GP surrogate model that spans the whole search space. The predictive mean from the human GP surrogate model, µ human , are scaled to a probability distribution (the scaling factor β serving effectively as a hyperparameter): [Equation]

Original: The surrogate model is scaled sigmoidally to aprobability distribution (the scaling factor serving effectively as a hyperparameter) and integratedinto the BO loop in the form of an added cost to the acquisition function.
Revised: Probabilities are integrated into the BO loop as an added cost to the acquisition function α : [Equation] which suppresses sampling from regions that are likely to have low-quality samples.

Original: 2.3 Convergence results
Revised: 2.3 Results

Original: We evaluate the success of the human integration based on three criteria: the optimum environmentalstability value found by BO, the compositional distance of this composition from Region A (evaluatedas the simple regret), and the number of queries made on the human (here, number of samples fromthe human evaluation model).
Revised: We evaluate the success of the human integration based on three criteria: the optimum environmental stability value found by BO, the compositional distance of this optimum to Region A (evaluated as the simple regret), and the number of queries made on the human (here, number of samples from the human evaluation model).

Original: The BO loop is initiated with two randomly selected samples and thereare 100 rounds in the whole BO cycle.
Revised: The BO cycle is initiated with two uniformly at random selected samples and iterated 100 rounds.

Original: To gather statistics, the BO cycle is repeated 25 times.
Revised: To gather statistics, the BO cycle is repeated 25 times.

Revised: The human-in-theloop BO (that queries human only when necessary, Figure 1a steps (i)-(iii)) is compared to vanilla

Original:  Our results  show that human integration leads to faster  convergence of the search to Region A (Figure2a): the human-aided version  converges in 15 rounds of sampling to a lower regret thanthe vanilla BO does during the whole cycle of 100 rounds.
Revised: BO (without human integration, Figure 1a step i) and a version that queries human for every sample (Figure 1a steps (i) and (iii)). Our results in Figure 2a show that vanilla BO frequently converges to other regions than Region A – namely Region B – because the regret remains high. Human integration leads to faster and more reliable convergence of the search to Region A: the human-aided version (orange line) converges in 15 rounds of sampling to a lower regret than vanilla BO in 100 iterations.

Original: These results are gained by showing on average 5%of the samples to the human (the development shown in Figure 2b). 
Revised: This is while having shown, on average, 5% of the samples to humans (Figure 2b). A traditional way of evaluating the convergence of BO is to track the optimum value found (here stability).

Original: It should be noted that whetherthe algorithm converges into Region A or Region B cannot be detected from the convergence plot only (Supplementary Figure S3). This isbecause both regions seem good based on the stabilityproxy – however, this result is invalid in Region B due to low quality samples, which is why humanopinions are valuable here.
Revised: All the compared methods perform similarly by this measure (Supplementary Figure S4) since Regions A and B have almost equal stability values. It is the human evaluation only that reveals that Region B has low quality samples and that the stability measurement values in this region can hence not be trusted.

Original: The ideal rate of showing samples to a human depends on the preferredbalance of increasing trust on the BO results (in which case a lot of samples should be shown to ahuman) or facilitating accelerated BO cycle (less samples should be shown to  a human).
Revised: The ideal rate of querying the human depends on the circumstances for including the human in the first place. For prioritizing increasing trust in the BO results, more samples should be shown to humans, while other factors like the desired acceleration of the optimization cycles sets a limit to how much the human role can be increased.

Original: In the lowextremum, no samples are shown to a human and the performance corresponds to the vanilla BO casein Figure 2.
Revised: In the low limit of the number of queries, no samples are shown to humans and the performance corresponds to the vanilla BO in Figure 2a.

Original: In the high extremum, showing all  the samples to a human results only in marginaladdedconvergence bene ﬁ ts (Figure 2b) with a lot more effort invested into the evaluations (Figure 2c).
Revised: In the high limit, the humans inspect all samples, but the benefits are marginal (Figure 2a), while the human effort is high (Figure 2b). Further work is required to determine an optimal querying frequency that is applicable to new materials science problems.

Original: Adding human  into the BO loop affects the acquisition locations, which indirectly introduces biasfrom human opinions into the resulting surrogate model of the target property, here environmentalstability.
Revised: In this context, it should be noted that our implementation does not stop the BO search if the requested human evaluations are not delivered in time (and they can still be provided in the later rounds), which is a choice made to avoid humans becoming a bottleneck. Adding humans into the BO loop affects the acquisition locations, which indirectly introduces bias from human opinions into the resulting surrogate model of the target property, here environmental stability.

Original: Therefore it is speci ﬁ cally important to analyzethe results of the human-in-the-loop BOin conjunction with the mean and variance predictions from the surrogate models of both the targetproperty and  human evaluation.
Revised: Therefore it is particularly important to analyze human-in-the-loop BO results in conjunction with the mean and variance predictions from the surrogate models of both the target property and the human evaluations.

Original: The predictions from the surrogate model are less reliable in regionsthat have been sampled sparsely, thus regions avoided due tothe sample quality issues need to beconsidered for future searches with re ﬁ ned sample preparation methods.
Revised: The predictions from the surrogate model are less reliable in regions that have been sampled sparsely (i.e., have high entropy). Thus regions avoided due to sample quality issues need to be considered for future searches with refined sample preparation methods.

Original: The human-in-the-loopimplementation in this work does not remove thesamples recognized as low-quality  from the targetproperty dataset.
Revised: The human-in-the-loop implementation in this work does not remove the low-quality samples from the dataset as they contribute to the variance of the stability surrogate model.

Original: This choice was made in order to make it easier to follow BO progress withaccumulating samples.
Revised: This choice also facilitates the BO result analysis – regions with low-quality samples but good proxy variable values can be quickly identified for refined investigation by comparing the two surrogate models.

Original: Future investigations are needed to evaluate, if this choice should be re ﬁ ned.
Revised: Future work will re-evaluate, if this choice should be refined to optimize the algorithmic performance.

Original: Future work will also investigate different human-in-the-loop techniques and implementations.
Revised: Our future work will also compare different human-in-the-loop techniques and implementations.

Original: Our demonstration is on perovskite thin ﬁ lm data, for which  visual inspection of sample quality wasa suitable way toquickly estimate sample quality.
Revised: Our demonstration is on perovskite thin film aging data, for which the visual inspection is a suitable way to estimate sample quality and can be done for few pre-defined samples while transferring the samples to the aging test device.

Original: Sample quality issues are present also in othertypes of materials samples and similar approaches for pre-screening sample quality by an experiencedhuman scientist can be found for many of them.
Revised: Sample quality issues are present also in other types of materials samples and similar approaches for pre-screening sample quality by an experienced human scientist can be found for many of them.

Original: Thus, the presented approach is applicable alsobeyond perovskite optimization tasks.
Revised: Thus, the presented approach is applicable beyond perovskite optimization tasks.

Original:  Conclusions
Revised:  Conclusions

Original: Our proof-of-concept human-in-the-loop Bayesian optimization  process demonstrates that addinghuman into experimental materials optimization loops for evaluating sample quality can facilitate theconvergence of Bayesian optimization into meaningful optima with samples of suf ﬁ cient quality (incontrast to wasting sampling to regions with low-quality samples and, consequently, invalid data). 
Revised: Our proof-of-concept human-in-the-loop Bayesian optimization (BO) process demonstrates that adding humans into experimental materials optimization loops for evaluating sample quality can facilitate the convergence of BO into meaningful optima with samples of sufficient quality (in contrast to wasting sampling in regions with low-quality samples and, consequently, invalid data). Here, we have shown an example case of perovskite composition optimization for environmental stability. Our work illustrates how human-in-the-loop methods could aid AI-guided materials design. Our work aims to facilitate high-dimensional and novel materials searches in partially self-driving laboratories — which are currently hindered by concerns on varying sample quality — and to trigger further discussions on how sample quality should be quantified and ensured in future materials searches.

Original: Here, we have shown an example case of perovskite composition optimization for environmentalstability.
Revised: Acknowledgments and Disclosure of Funding

Original: Our work exempli ﬁ es how the human-in-the-loop methods could be useful in AI-guideddesign of materials.
Revised: This work was supported by the Academy of Finland (Flagship program: Finnish Center for Artificial Intelligence FCAI) and the Academy of Finland through projects 334532 and 348180. Authors thank the reviewers for valuable comments during the review process.

Original: This work aims to facilitate the movement towards increasingly high-dimensionaland novel materials searches in today’s partially self-driving laboratories — which are currentlyhindered by concerns on varying quality of the samples — and triggering further discussions on howsample quality should in future be optimally ensured in materials searches.
Revised: Authors declare no competing interests. A Appendix A.1 Methods A.1.1 Perovskite film data No perovskite films were prepared as a part of this work but experimental data from literature [4] was used as the basis of simulations of this work. Perovskite film preparation and measurements are detailed in article [4].

Revised: As an overview of the experiments in literature [4], 112 Cs x MA y FA 1 − x − y PbI 3 perovskite film samples with varying compositions were prepared as a part of an experimental BO loop to find environmentally the most stable perovskite compositions.

Revised: The films were spin-coated inside a glovebox on top of glass substrates.

Revised: Solutions over- stoichiometric with PbI 2 and chlorobenzene antisolvent were used in the synthesis.

Revised: Following the spin-coating, the samples were annealed at 403K for 20 min in a glovebox.

Revised: Perovskite film samples were prepared in batches of 28 samples with varying compositions, and 4 rounds of aging tests with 28 perovskite films at a time were required for the BO loop to converge.

Revised: The perovskites were in each aging test degraded for 7,000 minutes under

Revised: 85 ± 2 ◦ C temperature, 85 ± 5% air humidity, and 0.15 ± 0.01Sun visible-only illumination, and photographed automatically with 5-minute intervals.

Revised: A proxy variable for the degradation of the films was defined as [4]:

Revised: [Equation]

Revised: (6) where c is the pixel-averaged color of the films, respectively, t is time

Revised: , T is aging test duration, and R , G , and B are the red, green, and blue RGB channels in the sample photos, respectively.

Revised: The smaller I c value, the more stable the film is.

Revised: A.1.2 Implementation of the ground truth models

Revised: In the simulated BO loops, the experimental environmental stability measurements and human evaluations of the samples were replaced by querying two models assumed to provide ground truth on environmental stability and human evaluations.

Revised: The simulated loops were chosen in order to be able to quantify the performance differences between the Bayesian optimization implementations that were compared.

Revised: The two ground truth models are Gaussian process (GP) regression models with a radial basis function kernel presented in Figure 1b-c – one for the estimated environmental stability of the perovskite films and another for the human evaluations of the film quality.

Revised: The fitted ground truth models are shared in the adjoining open code repository HPER (see Section A.1.5).

Revised: It should be noted that since these two models are fitted based on data resulting from a Bayesian optimization search (documented in literature [4]), they are inherently biased by the unevenly sampled data, and also by measurement variations.

Revised: They should thus be regarded as the estimates of the ground truth (not the exact ground truth).

Revised: However, for the purposes of the simulations performed in this work, these models provide a representative case of actual measurement results one could get on perovskite stability and sample quality in a laboratory.

Revised: For fitting the environmental stability model, the full dataset of environmental stability measurements of 112 perovskite samples from literature [4] was used.

Revised: For fitting the human evaluations model, 56 of these samples were evaluated by a human.

Revised: Only the first half of the samples queried in the BO search documented in [4] were evaluated because in the later stages of the search, the suggestions had already converged into a compositionally limited region.

Revised: The evaluations were made by the first author of this contribution but sample compositions were hidden from the evaluator at the time of the evaluations.

Revised: Mean predictions with Gaussian noise from the environmental stability GP model (Figure 1b) were used instead of environmental stability experiments and noiseless mean predictions from the human evaluation GP model (Figure 1c) were used instead of human opinions during the virtual BO loops in this work.

Revised: A.1.3 Implementation of human in the loop for Bayesian optimization

Revised: Human-in-the-loop implementation developed for this work is shared in an open code repository HPER (see Section A.1.5).

Revised: HPER was build on top of a Bayesian optimization Python implementation shared as a part of earlier work [4] in an open code repository SPProC

Revised: (see Section A.1.5).

Revised: To clarify the contributions in this work compared to SPProC: SPProC contains a modification to BO Python package GPyOpt for implementing the data fusion approach [4] for integrating additional data sources into the acquisition function of the BO algorithm.

Revised: In HPER, SPProC is modified by adding the human queries into the main BO loop and fed into the BO algorithm using the data fusion method.

Revised: Human is in this implementation queried only when the chances of getting low-quality samples are estimated to be high.

Revised: The estimation is done at the end of each BO round when the next samples to be suggested are compiled.

Revised: Only the samples that are suggested as a part of standard BO are subjected to a further evaluation on if the human opinions should be requested for these samples.

Revised: This choice was made because it may be dismotivating for humans to evaluate samples that had been prepared with a lot of effort but not exposed to the measurement of the actual target value of interest.

Revised: The chances of getting low-quality samples are assumed to be high when they are from new regions not having been sampled before, and when rapid changes in the values of the target variable are expected at the composition in question.

Revised: If these two criteria are met, a human evaluation on the sample is requested at the same time when new samples are suggested.

Revised: At the beginning of the next BO round, the BO algorithm receives both the target variable data and, separately, the human evaluations (assuming that humans have completed the requested evaluations – the BO will run normally even if the humans would be delayed in providing their opinions).

Revised: In the integration phase, the accumulated point-wise human evaluations are fed into BO with data fusion [4]: a GPR model is fitted on the human evaluated sample quality data points and the mean predictions of the model, µ human ( x, y, z ) , are scaled sigmoidally to a probability distribution of getting good samples:

Revised: [Equation]

Revised: where β is a fixed scale factor.

Revised: Finally, P human ( x, y, z ) is used as an added cost for the BO acquisition function A ( x, y, z ) to reduce sampling from the regions that are likely to produce low-quality samples:

Revised: [Equation]

Revised: The benefit of the data fusion approach is that the choice of the scale factor β can be used for describing the certainty of human evaluations on sample quality.

Revised: When conservative values of β are chosen, no region is fully removed from the BO search but only the tendency to sample from risky regions is reduced.

Revised: Thus, the likelihood of BO wasting bandwidth to loq-quality samples is reduced but if the risky region seems to contain extremely good samples, BO still samples some samples from there.

Revised: The presented approach for integrating human into the loop for BO requires setting three parameters that act effectively as hyperparameters: two parameters affecting when human is queried (the radius of the compositional exclusion zone inside of which human is not queried, r excl , and limit gradient of the environmental stability vs. composition below which human is not queried, g excl ) and one affecting the scaling of human observations into a probability distribution ( β ).

Revised: In this work, g excl was fixed to lengthscale and standard deviation of the environmental stability surrogate model (GP regression), l and σ , respectively:

Revised: [Equation]

Revised: thus allowing g excl to adapt to the surrogate model that develops round by round.

Revised: The maximum possible number of human queries during the BO is limited by the size of the axclusion zone.

Revised: Here, r excl was fixed to:

Revised: [Equation]

Revised: [Equation]

Revised: In practice, the exclusion zone should not be smaller than the experiment resolution (it is estimated in [4] that the resolution of being able to prepare samples with different compositions is 0.01).

Revised: Additionallt, since the composition range is up to 1, a reasonable r excl value in this application must fulfill 0 .

Revised: 01 < r excl > 0 .

Revised: The sigmoid scale factor β determines how conservative the model is towards human observations when translating them to a probability of getting good-quality samples.

Revised: In this work, we have chosen a conservative estimate β = 0 .

Revised: 1 illustrated in Figure S1.

Revised: Additionally, because the human evaluations ranged from 0 to 3, they were shifted by 1.5 before the sigmoid transformation for symmetry.

Revised: Generally, the parameters r excl , g excl , and β may need to be adjusted based on the application.

Revised: The decisions should be made considering the estimated burden for humans, the speed of the rest of the optimization cycle with respect to the speed of human queries, and the positive and in some cases possibly also negative effects of human bias brought into the optimization cycle.

Revised: Ways to automatically set the parameters will be investigated in future in order to simplify the use of human integration code.

Revised: A.1.4 Evaluating the performance of optimization algorithms

Revised: The human-in-the-loop BO described in the previous subsection was compared to two other approaches: vanilla BO that is the traditional BO without human integration, and a human-in-the-loop implementation with a simplistic query policy of asking human opinions on every sample.

Revised: All the three approaches used as much of identical settings as possible: the acquisition function was expected improvement (this served as the base acquisition function for the human-in-the-loop versions) with jitter of 0.01, the surrogate models were Gaussian process models with Matern kernel for the stability model and radial basis function kernel for the human model, and 100 rounds of sampling one sample on each round were performed.

Revised: These BO cycles were repeated 25 times to estimate the performance of the algorithm versions in terms of the current optimum I ∗ c ( t ) found by round t , the number of human queries made, and simple regret:

Revised: [Equation]

Revised: [Equation]

Revised: where ( x ∗ t , y ∗ t , z ∗ t ) is the composition of I ∗ c ( t ) and ( x opt , y opt , z opt ) is the composition of the true optimum.

Revised: A.1.5 Data and code availability

Revised: The perovskite data used for simulations was produced in work [4] and has been shared by the authors of [4] in a Github repository:

Revised: • https://github.com/PV-Lab/SPProC

Revised: The raw data produced in [4] has been shared by the corresponding authors of [4] in the following repositories:

Revised: • https://doi.org/10.6084/m9.

Revised: figshare

Revised: / • https://doi.org/10.6084/m9.figshare.

Revised: • https://doi.org/10.6084/m9.figshare.

Revised: The human-in-the-loop code implementation prepared in this work, together with the GP models used as the source of ground truth data in the simulations in this work, are shared in a Github repository:

Revised: • https://github.com/TadaSanar/HPER

Revised: A.2 Human evaluations as the basis for forming the ground truth model for human-evaluated sample quality in the example perovskite dataset

Revised: Human evaluations of perovskite film samples were performed post-hoc from photographs that had been taken from fresh samples before aging them in article [4].

Revised: Example evaluations are shown in Figure S1.

Revised: Human evaluation is mostly linked to the yellowness of the sample (yellow color of lead iodide perovskites indicates either non-photoective yellow phases being present in the film or the perovskite having decomposed even before the aging test, which means yellow films are low quality).

Revised: Two of the samples in Figure S1 have the same composition (FAPbI 3 ) but they still look different and thus, have received a different evaluation on sample quality.

Revised: This is an example of sample-to-sample variance that is present in the dataset.

Revised: A.3 Competing optima in the example perovskite search space

Revised: Figure 1b illustrates that in the example case of optimizing Cs x MA y FA 1 − x − y PbI 3 perovskite composition for environmental stability, there are two competing regions with seemingly high environmental stability: Region

Revised: A and Region B.

Revised: The model in Figure 1b has been fitted on the measurement results in the source article [4].

Revised: These measurements are not direct measurements of the environmental stability of the perovskite samples but measurements of a proxy variable that is easier to probe: the color change of the sample integrated over time during the aging test (more information in Supplementary Section A.1).

Revised: Both Regions A and B therefore have samples that do not change their color much during an aging test.

Revised: However, in Region B, the samples tend to suffer from phase de-mixing into minority phases already before the aging test starts [4], which is thus not captured by the integrated color change metric.

Revised: In extreme cases, the high-Cs films in Region B suffering from de-mixing may even look yellow even as fresh due to the presence of non-photoactive, yellow CsPbI 3 phase.

Revised: The phase de-mixing deteriorates the photovoltaic properties of the films in Region B, resulting in low-quality perovskite samples, and thus, Region B is actually an invalid optimum.

Revised: The phase de-mixing issue can be estimated via thermodynamical phase stability.

Revised: If Gibbs free energy of mixing, dG mix , is positive for a certain composition, it should de-mix into minority phases.

Revised: The estimated dG mix over the Cs x

Revised: [Equation]

Revised: perovskite search space in Figure S3 indeed suggests Region B suffers from phase de-mixing and the rest of the search space is phase-stable.

Revised: Interestingly, the comparison of Figure S3 and Figure 1c reveals that human evaluations on sample quality also highlight Region B as a region with low-quality samples.

Revised: This is likely because the presence of the highest proportions of non-photoactive yellow minority phases is visible even by the eye.

Revised: Thus, in this search space, human evaluations actually serve as an incomplete, low-fidelity option to more accurate but computationally costly density functional theory calculations.

Revised: regression model density functional theory

Revised: According to the estimate, all the compositions with positive energies should be thermodynamically instable.

Revised: The data used contains only binary perovskite compositions from the triangle edges (shown in the figure) and the model is thus less reliable with ternary compositions in the interior of the triangle.

Revised: The GP model was fitted using the same settings than are used in the source article [4]

Revised: A

Revised: 4 Convergence of the optimum stability found

Revised: Bayesian

Revised: In the test application case presented in this work, Regions A and B produce almost equally low I c values.

Revised: Thus, it cannot be detected based on the found optimum value only if the search has converged into Region A or B

Revised: The sampling from environmental stability model (Figure 1b) is in the virtual BO implemented with Gaussian noise to mimic real experiments.