Repository: psychopy/psychopy

Issue:
QUEST intensity scaling should be removed
Currently, `data.QuestHandler.__init__()` takes the optional parameter `stepType`, which is later used to transform intensity values, as in:

``` python
        startVal = self._intensity2scale(startVal)
        startValSd = self._intensity2scale(startValSd)
```

where `_intensity2scale()` is defined as:

``` python
    def _intensity2scale(self, intensity):
        """returns the scaled intensity level based on value of self.stepType"""
        if self.stepType=='db':
            scaled_intensity = numpy.log10(intensity) * 20.0
        elif self.stepType=='log':
            scaled_intensity = numpy.log10(intensity)
        else:
            scaled_intensity = intensity
        return scaled_intensity
```

However, this interferes with the QUEST algorithm, which is implemented in `contrib.quest`, and already fully takes care of the stimulus selection. Jon also [mentioned](https://groups.google.com/d/msg/psychopy-users/5AsGnvLsPpU/wr1EPMnYJIYJ) something similar on the mailing list already.

The `stepType` argument should be removed. If a user wants to specify `startVal` and `startValSd` e.g. in terms of log intensities, they could easily do the calculation manually, i.e.,

``` python
QuestHandler(np.log10(startVal), np.log10(startValSd), ...)
```

QUEST will take care of the rest.

Also, currently `stopInterval`, `minVal` and `maxVal` are not transformed, so I assume the current implementation is incorrect for all but linear `stepType`s anyway.

QUEST intensity scaling should be removed
Currently, `data.QuestHandler.__init__()` takes the optional parameter `stepType`, which is later used to transform intensity values, as in:

``` python
        startVal = self._intensity2scale(startVal)
        startValSd = self._intensity2scale(startValSd)
```

where `_intensity2scale()` is defined as:

``` python
    def _intensity2scale(self, intensity):
        """returns the scaled intensity level based on value of self.stepType"""
        if self.stepType=='db':
            scaled_intensity = numpy.log10(intensity) * 20.0
        elif self.stepType=='log':
            scaled_intensity = numpy.log10(intensity)
        else:
            scaled_intensity = intensity
        return scaled_intensity
```

However, this interferes with the QUEST algorithm, which is implemented in `contrib.quest`, and already fully takes care of the stimulus selection. Jon also [mentioned](https://groups.google.com/d/msg/psychopy-users/5AsGnvLsPpU/wr1EPMnYJIYJ) something similar on the mailing list already.

The `stepType` argument should be removed. If a user wants to specify `startVal` and `startValSd` e.g. in terms of log intensities, they could easily do the calculation manually, i.e.,

``` python
QuestHandler(np.log10(startVal), np.log10(startValSd), ...)
```

QUEST will take care of the rest.

Also, currently `stopInterval`, `minVal` and `maxVal` are not transformed, so I assume the current implementation is incorrect for all but linear `stepType`s anyway.



Discussion:

Resolving patch:
diff --git a/psychopy/data.py b/psychopy/data.py
--- a/psychopy/data.py
+++ b/psychopy/data.py
@@ -2702,7 +2702,6 @@ def __init__(self,
                  nTrials=None,
                  stopInterval=None,
                  method='quantile',
-                 stepType='log',
                  beta=3.5,
                  delta=0.01,
                  gamma=0.5,
@@ -2751,10 +2750,6 @@ def __init__(self,
                 The method used to determine the next threshold to test. If you want to get a specific threshold
                 level at the end of your staircasing, please use the quantile, mean, and mode methods directly.
 
-            stepType: *'log'*, 'db', 'lin'
-                The type of steps that should be taken each time. 'db' and 'log' will transform your intensity levels
-                into decibels or log units and will move along the psychometric function with these values.
-
             beta: *3.5* or a number
                 Controls the steepness of the psychometric function.
 
@@ -2794,15 +2789,17 @@ def __init__(self,
         """
 
         # Initialize using parent class first
-        StairHandler.__init__(self, startVal, nTrials=nTrials, extraInfo=extraInfo, method=method,
-                                stepType=stepType, minVal=minVal, maxVal=maxVal, name=name, autoLog=autoLog)
+        StairHandler.__init__(
+                self, startVal, nTrials=nTrials, extraInfo=extraInfo,
+                method=method, stepType='lin', minVal=minVal,
+                maxVal=maxVal, name=name, autoLog=autoLog
+        )
 
         # Setup additional values
         self.stopInterval = stopInterval
 
-        # Transform startVal and startValSd based on stepType
-        startVal = self._intensity2scale(startVal)
-        startValSd = self._intensity2scale(startValSd)
+        startVal = startVal
+        startValSd = startValSd
         self._questNextIntensity = startVal
 
         # Create Quest object
@@ -2827,7 +2824,7 @@ def addResponse(self, result, intensity=None):
         if intensity is None:
             intensity = self._questNextIntensity
         else:
-            intensity = self._intensity2scale(intensity)
+            intensity = intensity
             # Update the intensity.
             #
             # During the first trial, self.intensities will be of length 0,
@@ -2863,7 +2860,7 @@ def calculateNextIntensity(self):
             self._nextIntensity = self.maxVal
         elif (self._nextIntensity < self.minVal) and self.minVal is not None:
             self._nextIntensity = self.minVal
-        self._questNextIntensity = self._intensity2scale(self._nextIntensity)
+        self._questNextIntensity = self._nextIntensity
     def _intensity(self):
         """assigns the next intensity level"""
         if self.method == 'mean':
@@ -2873,43 +2870,23 @@ def _intensity(self):
         elif self.method == 'quantile':
             self._questNextIntensity = self._quest.quantile()
         # else: maybe raise an error
-        self._nextIntensity = self._scale2intensity(self._questNextIntensity)
-
-    def _intensity2scale(self, intensity):
-        """returns the scaled intensity level based on value of self.stepType"""
-        if self.stepType=='db':
-            scaled_intensity = numpy.log10(intensity) * 20.0
-        elif self.stepType=='log':
-            scaled_intensity = numpy.log10(intensity)
-        else:
-            scaled_intensity = intensity
-        return scaled_intensity
-
-    def _scale2intensity(self, scaled_intensity):
-        """returns the unscaled intensity level based on value of self.stepType"""
-        if self.stepType=='db':
-            intensity = 10.0**(scaled_intensity/20.0)
-        elif self.stepType=='log':
-            intensity = 10.0**scaled_intensity
-        else:
-            intensity = scaled_intensity
-        return intensity
+        self._nextIntensity = self._questNextIntensity
 
     def mean(self):
         """mean of Quest posterior pdf"""
-        return self._scale2intensity(self._quest.mean())
+        return self._quest.mean()
 
     def sd(self):
         """standard deviation of Quest posterior pdf"""
-        return self._scale2intensity(self._quest.sd())
+        return self._quest.sd()
 
     def mode(self):
         """mode of Quest posterior pdf"""
-        return self._scale2intensity(self._quest.mode()[0])
+        return self._quest.mode()[0]
 
     def quantile(self, p=None):
         """quantile of Quest posterior pdf"""
-        return self._scale2intensity(self._quest.quantile(p))
+        return self._quest.quantile(p)
 
     def confInterval(self, getDifference=False):
         """give the range of the 5-95% confidence interval"""
@@ -3286,11 +3263,14 @@ def _createStairs(self):
                     stepType=stepType, minVal=minVal, maxVal=maxVal)
             elif self.type in ['QUEST','quest']:
                 # see above
-                thisStair = QuestHandler(startVal, startValSd=condition['startValSd'],
-                    pThreshold=pThreshold, nTrials=nTrials, stopInterval=stopInterval,
-                    method=method, stepType=stepType, beta=beta, delta=delta,
-                    gamma=gamma, grain=grain, range=range, extraInfo=extraInfo,
-                    minVal=minVal, maxVal=maxVal, staircase=staircase)
+                thisStair = QuestHandler(
+                        startVal, startValSd=condition['startValSd'],
+                        pThreshold=pThreshold, nTrials=nTrials,
+                        stopInterval=stopInterval, method=method,
+                        beta=beta, delta=delta, gamma=gamma, grain=grain,
+                        range=range, extraInfo=extraInfo, minVal=minVal,
+                        maxVal=maxVal, staircase=staircase
+                )
             thisStair.condition = condition#this isn't normally part of handler
             #and finally, add it to the list
             self.staircases.append(thisStair)


Test patch:
diff --git a/psychopy/tests/test_data/test_stairs.py b/psychopy/tests/test_data/test_stairs.py
--- a/psychopy/tests/test_data/test_stairs.py
+++ b/psychopy/tests/test_data/test_stairs.py
@@ -1,20 +1,18 @@
 """Test StairHandler"""
-from __future__ import print_function
+from __future__ import division, print_function
 from psychopy import data, logging
-from psychopy.tests import utils
 import numpy as np
-import os, glob, shutil
-logging.console.setLevel(logging.DEBUG)
-
+import shutil
 from tempfile import mkdtemp
 
+logging.console.setLevel(logging.DEBUG)
 DEBUG=False
 
+np.random.seed(1000)
 
-class TestStairHandlers(object):
-    """test staircase but using the experiment handler attached as well
-    """
-    def setup(self):#setup is run for each test within the class
+
+class _BaseTestStairHandler(object):
+    def setup(self):
         self.tmpFile = mkdtemp(prefix='psychopy-tests-testStaircase')
         self.exp = data.ExperimentHandler(name='testExp',
                         savePickle=True,
@@ -24,63 +22,245 @@ def setup(self):#setup is run for each test within the class
             print(self.tmpFile)
 
     def teardown(self):
-        #    remove the tmp files
         shutil.rmtree(self.tmpFile)
 
-    def doTrials(self, stairs, responses, levels=None, reversals=None):
-        """run the trials and check that the levels were correct given the responses
+    def simulate(self, stairs, responses, intensities,
+                 reversalsPoints=None, reversalIntensities=None):
+        """
+        Simulate a staircase run.
+
+        :Paramters:
+
+        stairs : StairHandler
+            A StairHandler instance.
+        responses : array-like
+            Responses of the simulated observer.
+            For example `[0, 1`] for a correct, followed by an incorrect
+            response.
+        intensities : array-like
+            Intensity levels as calculated by the staircase procedure,
+            based on the simulated observer's `responses`.
+        reversalPoints : array-like, optional
+            The trial numbers at which reversals occurred.
+        reversalPoints : array-like, optional
+            The intensity levels at which reversals occurred.
+
         """
+        # Simulate an experimental run.
         self.exp.addLoop(stairs)
-        for trialN, thisLevel in enumerate(stairs):
+        for trialN, intensityN in enumerate(stairs):
             stairs.addResponse(responses[trialN])
-            stairs.addOtherData('rt', thisLevel*0.1)
-            if DEBUG:
-                if levels:
-                    print(trialN, thisLevel, responses[trialN], stairs.stepSizeCurrent, levels[trialN])#the latter is the expected level)
-                else:
-                    print(trialN, thisLevel, responses[trialN])# there was no expected level given
+            stairs.addOtherData('RT', 0.1 + 2*np.random.random_sample(1))
             self.exp.nextEntry()
-        assert stairs.data == responses
-        if hasattr(stairs, 'minVal'):
-            assert min(responses)>=stairs.minVal
-            assert max(responses)>=stairs.maxVal#assume it also has a maxVal
-        if levels:
-            assert np.allclose(stairs.intensities, levels), "staircase levels not as expected"
-        if reversals:
-            assert np.allclose(stairs.reversalIntensities, reversals)
-
-    def test_staircaseLinear(self):
+
+        assert stairs.finished  # Staircase terminated normally.
+        assert stairs.data == responses  # Responses were stored correctly.
+        # Trial count starts at zero.
+        assert stairs.thisTrialN == len(stairs.data) - 1
+
+        # Intensity values are sane.
+        assert np.min(stairs.intensities) >= stairs.minVal
+        assert np.max(stairs.intensities) <= stairs.maxVal
+        assert np.allclose(stairs.intensities, intensities)
+
+        # Reversal values are sane.
+        if stairs.nReversals is not None:
+            # The first trial can never be a reversal.
+            assert stairs.nReversals <= len(stairs.data) - 1
+
+            assert stairs.nReversals == len(stairs.reversalPoints)
+            assert stairs.nReversals == len(stairs.reversalIntensities)
+
+        if stairs.reversalPoints:
+            assert (len(stairs.reversalPoints) ==
+                    len(stairs.reversalIntensities))
+
+        if reversalIntensities:
+            assert np.allclose(stairs.reversalIntensities,
+                               reversalIntensities)
+
+
+class TestStairHandler(_BaseTestStairHandler):
+    """Test StairHandler, but using the ExperimentHandler attached as well.
+    """
+    def test_StairHandlerLinear(self):
+        nTrials = 20
+        startVal, minVal, maxVal = 0.8, 0, 1
+        stepSizes = [0.1, 0.01, 0.001]
+        nUp, nDown = 1, 3
+        nReversals = 4
+        stepType='lin'
+
+        stairs = data.StairHandler(
+            startVal=startVal, nUp=nUp, nDown=nDown, minVal=minVal,
+            maxVal=maxVal, nReversals=nReversals, stepSizes=stepSizes,
+            nTrials=nTrials, stepType=stepType
+        )
+
+        responses = makeBasicResponseCycles(
+            cycles=3, nCorrect=4, nIncorrect=4, length=20
+        )
+
+        intensities = [
+            0.8, 0.7, 0.6, 0.5, 0.4, 0.41, 0.42, 0.43, 0.44, 0.44, 0.44,
+            0.439, 0.439, 0.44, 0.441, 0.442, 0.443, 0.443, 0.443, 0.442
+        ]
+
+        reversalIntensities = [0.4, 0.44, 0.439, 0.443]
+
+        self.simulate(stairs, responses, intensities,
+                      reversalIntensities=reversalIntensities)
+
+    def test_StairHandlerLog(self):
         nTrials = 20
-        stairs = data.StairHandler(startVal=0.8, nUp=1, nDown=3, minVal=0, maxVal=1,
-            nReversals=4, stepSizes=[0.1,0.01,0.001], nTrials=nTrials,
-            stepType='lin')
-        responsesToMake=makeBasicResponseCycles()
-        levels = [0.8, 0.7, 0.6, 0.5, 0.4, 0.41, 0.42, 0.43, 0.44, 0.44, 0.44,
-                  0.439, 0.439, 0.44, 0.441, 0.442, 0.443, 0.443, 0.443, 0.442]
-        reversals = [0.4, 0.44, 0.439, 0.443]
-        self.doTrials(stairs, responsesToMake[:nTrials], levels=levels, reversals=reversals)
-
-    def test_staircaseDB(self):
+        startVal, minVal, maxVal = 0.8, 0, 1
+        # We try to reproduce the values from test_StairHandlerDb().
+        stepSizes = [0.4/20, 0.2/20, 0.2/20, 0.1/20]
+        nUp, nDown = 1, 3
+        nReversals = 4
+        stepType='log'
+
+        stairs = data.StairHandler(
+            startVal=startVal, nUp=nUp, nDown=nDown, minVal=minVal,
+            maxVal=maxVal, nReversals=nReversals, stepSizes=stepSizes,
+            nTrials=nTrials, stepType=stepType
+        )
+
+        responses = makeBasicResponseCycles(
+            cycles=3, nCorrect=4, nIncorrect=4, length=20
+        )
+
+        intensities = [
+            0.8, 0.763994069, 0.729608671, 0.696770872, 0.665411017,
+            0.680910431, 0.696770872, 0.713000751, 0.729608671,
+            0.729608671, 0.729608671, 0.713000751, 0.713000751,
+            0.72125691, 0.729608671, 0.738057142, 0.746603441,
+            0.746603441, 0.746603441, 0.738057142
+        ]
+
+        reversalIntensities = [
+            0.665411017, 0.729608671, 0.713000751, 0.746603441
+        ]
+
+        self.simulate(stairs, responses, intensities,
+                      reversalIntensities=reversalIntensities)
+
+    def test_StairHandlerDb(self):
         nTrials = 20
-        stairs = data.StairHandler(startVal=0.8, nUp=1, nDown=3, minVal=0, maxVal=1,
-            nReversals=4, stepSizes=[0.4,0.2,0.2,0.1], nTrials=nTrials,
-            stepType='db')
-        responsesToMake=makeBasicResponseCycles()
-        levels=[0.8, 0.763994069, 0.729608671, 0.696770872, 0.665411017, 0.680910431,
-                0.696770872, 0.713000751, 0.729608671, 0.729608671, 0.729608671, 0.713000751,
-                0.713000751, 0.72125691, 0.729608671, 0.738057142, 0.746603441, 0.746603441,
-                0.746603441, 0.738057142]
-        reversals = [0.665411017, 0.729608671, 0.713000751, 0.746603441]
-        self.doTrials(stairs, responsesToMake[:nTrials], levels=levels)
-
-
-def makeBasicResponseCycles():
-    """helper function to make basic set of responses
+        startVal, minVal, maxVal = 0.8, 0, 1
+        stepSizes = [0.4, 0.2, 0.2, 0.1]
+        nUp, nDown = 1, 3
+        nReversals = 4
+        stepType = 'db'
+
+        stairs = data.StairHandler(
+            startVal=startVal, nUp=nUp, nDown=nDown, minVal=minVal,
+            maxVal=maxVal, nReversals=nReversals, stepSizes=stepSizes,
+            nTrials=nTrials, stepType=stepType
+        )
+
+        responses = makeBasicResponseCycles(
+            cycles=3, nCorrect=4, nIncorrect=4, length=20
+        )
+
+        intensities = [
+            0.8, 0.763994069, 0.729608671, 0.696770872, 0.665411017,
+            0.680910431, 0.696770872, 0.713000751, 0.729608671,
+            0.729608671, 0.729608671, 0.713000751, 0.713000751,
+            0.72125691, 0.729608671, 0.738057142, 0.746603441,
+            0.746603441, 0.746603441, 0.738057142
+        ]
+
+        reversalIntensities = [
+            0.665411017, 0.729608671, 0.713000751, 0.746603441
+        ]
+
+        self.simulate(stairs, responses, intensities,
+                      reversalIntensities=reversalIntensities)
+
+
+class TestQuestHandler(_BaseTestStairHandler):
+    """Test QuestHandler, but using the ExperimentHandler attached as well.
+    """
+    def test_QuestHandler(self):
+        nTrials = 10
+        startVal, minVal, maxVal = 50, 0, 100
+        range = maxVal - minVal
+        startValSd = 50
+        grain = 0.01
+        pThreshold = 0.82
+        beta, gamma, delta = 3.5, 0.5, 0.01
+        stopInterval = None
+        method = 'quantile'
+
+        stairs = data.QuestHandler(
+            startVal, startValSd, pThreshold=pThreshold, nTrials=nTrials,
+            stopInterval=stopInterval, method=method, beta=beta,
+            gamma=gamma, delta=delta, grain=grain, range=range,
+            minVal=minVal, maxVal=maxVal
+        )
+
+        responses = makeBasicResponseCycles(
+            cycles=3, nCorrect=2, nIncorrect=2, length=10
+        )
+
+        intensities = [
+            50, 45.139710407074872, 37.291086503930742,
+            58.297413127139947, 80.182967131096547, 75.295251409003527,
+            71.57627192423783, 79.881680484036906, 90.712313302815517,
+            88.265808957695796
+        ]
+
+        mean = 86.0772169427
+        mode = 80.11
+        quantile = 86.3849031085
+
+        self.simulate(stairs, responses, intensities)
+
+        assert np.allclose(stairs.mean(), mean)
+        assert np.allclose(stairs.mode(), mode)
+        assert np.allclose(stairs.quantile(), quantile)
+
+        # Check if the internal grid has the expected dimensions.
+        assert len(stairs._quest.x) == (maxVal - minVal) / grain + 1
+        assert np.allclose(stairs._quest.x[1] - stairs._quest.x[0],
+                           grain)
+        assert stairs._quest.x[0] == -range/2
+        assert stairs._quest.x[-1] == range/2
+
+
+def makeBasicResponseCycles(cycles=10, nCorrect=4, nIncorrect=4,
+                            length=None):
+    """
+    Helper function to create a basic set of responses.
+
+    :Parameters:
+
+    cycles : int, optional
+        The number of response cycles to generate. One cycle consists of a
+        number of correct and incorrect responses.
+        Defaults to 10.
+    nCorrect, nIncorrect : int, optional
+        The number of correct and incorrect responses per cycle.
+        Defaults to 4.
+    length : int or None, optional
+
+    :Returns:
+
+    responses : list
+        A list of simulated responses with length
+        `cycles * (nCorrect + nIncorrect)`.
+
     """
-    responsesToMake=[]
-    for cycle in range(10):
-        for nRight in range(4):
-            responsesToMake.append(1)
-        for nWrong in range(4):
-            responsesToMake.append(0)
-    return responsesToMake
+    responsesCorrectPerCycle = np.ones(nCorrect, dtype=np.int)
+    responsesIncorrectPerCycle = np.zeros(nIncorrect, dtype=np.int)
+
+    responses = np.tile(
+        np.r_[responsesCorrectPerCycle, responsesIncorrectPerCycle],
+        cycles
+    ).tolist()
+
+    if length is not None:
+        return responses[:length]
+    else:
+        return responses