In psychophysical experiments time and the limited goodwill of participants is usually a major constraint. This has been the main motivation behind the early development of adaptive methods for the measurements of psychometric thresholds. More recently methods have been developed to measure whole psychometric functions in an adaptive way. Here we describe a Bayesian method to measure adaptively any aspect of a psychophysical function, taking inspiration from Kontsevich and Tyler's (1999) optimal Bayesian measurement method. Our method is implemented in a complete and easy-to-use MATLAB package. Over the years a large number of methods have been developed with the goal to obtain the most information about an observer's behaviour in a limited amount of time. They range from simple empirical schemes such as the staircase, to more elaborate mechanisms with better theoretical foundations, such as the QUEST or ZEST methods (Watson & Pelli, 1983; King-Smith et al., 1994) . Some only estimate the threshold, while others aim to provide information about the slope of the psychometric function as well (Kontsevich & Tyler, 1999; Kujala & Lukka, 2006 ). Here we build on the latter to provide a very general framework for adaptive measurement and tting of psychometric functions. We provide the option of focusing on a just one aspect of a psychometric function (for example, threshold or slope alone), or on the whole psychometric function. By using the framework of Bayesian inference, we provide theoretically well-founded methods for estimating the parameters and obtaining maximal information about them. An additional concern has been with the development of fast algorithms so that waiting time between trials could be minimised. We provide an easy-to-use software package in MATLAB for researchers without the time, the required technical background, or the inclination to implement our algorithms. A psychometric function links the stimulus level (for example, contrast or signal-to-noise ratio) to a probability of response (often, the probability of a correct response). It can be dened by three parameters: one for the position along the stimulus level axis, one for the slope, and one for the lapse rate. The eects of the three are illustrated on gure 3. Thresholds are dened in the following way: rst pick a performance level, for example 75%. Then the corresponding threshold is given by the inverse of the psychometric function, ie, it is the stimulus level such that the probability of response is 0.75. The aim of a acquiring responses from an observer is to estimate some feature of the psychometric function, either its three parameters, or just its threshold or slope. The non-adaptive way to do this is for the experimenter is to spend some time in a dark room playing around with the stimulus and then choosing within a range a number of stimulus values they think appropriate to measure observer's performance. This is known as the method of constant stimuli. The usual result is that due to variability between observers, and due also to the fact that the experimenter, having spent countless hours doing the task, will overestimate everybody else's performance, thinking that 1% contrast should be easy enough for anyone. The method of constant stimuli thus leads to very wasteful data collection, with observers being tested at chance performance, or close to 100%. In the worst-case scenario, none of the datapoints are actually informative. Despite this aw, the method of constant stimuli enjoys the considerable resilience of a gold standard, notably because it results in ts that are easy to evaluate visually -but this is sometimes misleading, as we explain in section 2.9.7 below. The general class of alternatives to the method of constant stimuli is known in psychophysics as adaptive methods, and in other circles as active learning (Cohn et al., 1995) . Adaptive methods take advantage of the intuitive idea that if an observer has already answered 10 out of 10 times correctly at a given level, then there isn't much point in further interrogation. In Bayesian adaptive methods, the measurement process is viewed as the step-by-step update of a probability distribution over the object of interest. Imagine we want to measure the 75% threshold: we start out by expressing our prior information about potential thresholds as a prior distribution. Just as in the case of the method of constant stimuli, we use the information acquired in the hours spent ddling around and exploiting next-door colleagues to get some idea of what would be a range of realistic thresholds for other observers. This information will be udpated every time we collect a response from the observer: we update our beliefs according to whatever data we get. How exactly we need to update our beliefs is given by Bayes' theorem. The process is illustrated in gure 1 and 2. The uncertainty that remains about the threshold (or about any other set of parameters) is given by the entropy of the distribution. A distribution that has a single peak -indicating that we are extremely sure about the value of our parameter -will have minimum entropy, while a at distribution -indicating complete lack of condence -has maximum entropy. A decrease in entropy, then, is equivalent to an increase in condence. At each step we ask: what is the stimulus level so that the condence will increase maximally on average? 2 Figure 1 : An illustration of Bayesian belief updating. We start out with a prior for the 75% threshold centered around 3 (black solid curve). On the rst trial, we test the observer at stimulus level 6 and get an incorrect response. We update our beliefs according to Bayes' theorem, yielding the rst posterior distribution (blue dashed curve). The distribution agrees with intuitive reasoning: an incorrect response at level 6 would be unlikely if the real threshold were below 3, but more likely if it were above. On the 2d trial, the posterior from our rst trial becomes our prior, and we measure a correct response at level 3, which is relatively likely under most threshold values between 2 and 4, and the posterior probability (red dashed curve) is changed only slightly. Notice how the uncertainty decreases over time, rst fast, then more slowly. This is a general feature of Bayesian updating. 3 Obviously if we know that the probability that the observer's threshold is over 20% is 0.01, then there is no point in picking a stimulus level in the vicinity of 20%. The right stimulus level is given by a minimum in a cost function, which can be computed by the algorithms we detail below. For psychometric functions in forced-choice experiments we use the following form 1 : • x is the stimulus level • µ and σ describe the placement and slope of the psychometric function • γ, a constant, is the chance rate. In 2AFC designs, γ is set to 0.5. • Φ(x; µ, σ) is the cumulated Gaussian function. This choice of shape is on aesthetic grounds, choosing for example a logistic shape makes little to no dierence in practice. • Following the recommendations in Wichmann & Hill (2001a,b) we include a lapse term, λ. Here we assume that on every trial, the observer either lapses and makes a random choice (with probability of success γ) or behaves in a stimulus-dependent manner, with probability of success γ + (1 − γ)Φ(x; µ, σ). Including a lapse term is useful in order to insure robustness against random errors at high stimulus levels. One can measure psychometric functions with dierent goals in mind. If the psychometric function is the independent variable in an experiment, one will wish to know its parameters θ = (µ, σ, λ) as well as possible, using as few datapoints as possible. This is the aim of Kontsevich and Tyler's original method. In another scenario, one wishes only to measure thresholds, for example to set the stimulus level in an experiment so that the diculty is more or less the same for all observers, or simply because the threshold provides a useful summary of performance. Sometimes one may want to know more than just one threshold, for example both the .73 and the .65 thresholds could be of interest: we show in section 5.3 that this reduces to the Ψ method
An adaptive Bayesian method for estimating the parameters of a flexible psychometric function is described in REF .
A flexible Bayesian method for adaptive measurement in psychophysics
arXiv: Applications
Mathematics Computer Science