# Factorizing the motion sensitivity function into equivalent input noise and calculation efficiency

Rémy ´ Allard, Angelo ´ Arleo

<figure><img src="image_1.png" /></figure>

## To cite this version:

Rémy ´ Allard, Angelo ´ Arleo. Factorizing the motion sensitivity function into equivalent input noise and calculation efficiency. Journal of Vision, 2017, 17 (1), pp.17. 10.1167/17.1.17. hal-01487795

HAL Id: hal-01487795

https://hal.sorbonne-universite.fr/hal-01487795

Submitted on 13 Mar 2017

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_2.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
# Factorizing the motion sensitivity function into equivalent input noise and calculation efficiency

Rémy Allard

Institut de la Vision, Sorbonne Universités, Paris, France

<figure><img src="image_3.png" /></figure>

<figure><img src="image_4.png" /></figure>

Angelo Arleo

Institut de la Vision, Sorbonne Universités, Paris, France

<figure><img src="image_5.png" /></figure>

<figure><img src="image_6.png" /></figure>

The photopic motion sensitivity function of the energy-based motion system is band-pass peaking around 8 Hz. Using an external noise paradigm to factorize the sensitivity into equivalent input noise and calculation efficiency, the present study investigated if the variation in photopic motion sensitivity as a function of the temporal frequency is due to a variation of equivalent input noise (e.g., early temporal filtering) or calculation efficiency (ability to select and integrate motion). For various temporal frequencies, contrast thresholds for a direction discrimination task were measured in presence and absence of noise. Up to 15 Hz, the sensitivity variation was mainly due to a variation of equivalent input noise and little variation in calculation efficiency was observed. The sensitivity fall-off at very high temporal frequencies (from 15 to 30 Hz) was due to a combination of a drop of calculation efficiency and a rise of equivalent input noise. A control experiment in which an artificial temporal integration was applied to the stimulus showed that an early temporal filter (generally assumed to affect equivalent input noise, not calculation efficiency) could impair both the calculation efficiency and equivalent input noise at very high temporal frequencies. We conclude that at the photopic luminance intensity tested, the variation of motion sensitivity as a function of the temporal frequency was mainly due to early temporal filtering, not to the ability to select and integrate motion. More specifically, we conclude that photopic motion sensitivity at high temporal frequencies is limited by internal noise occurring after the transduction process (i.e., neural noise), not by quantal noise resulting from the probabilistic absorption of photons by the photoreceptors as previously suggested.

## Introduction

The photopic temporal contrast sensitivity function is known to be band-pass peaking around 8 Hz

(Watson, 1986). Using an external noise paradigm to factorize the sensitivity into equivalent input noise and calculation efficiency (Pelli, 1981, 1990; Pelli & Farell, 1999; Figure 1), the present study investigated if the variation in photopic motion sensitivity as a function of the temporal frequency is due to a variation of equivalent input noise or calculation efficiency (or a combination of both).

For the sensitivity to counter-phase flicker (which is equivalent to motion sensitivity, Kelly, 1979; Levinson & Sekuler, 1975), Pelli (1990) found that the equivalent input noise decreased with increasing spatiotemporal frequency at low spatiotemporal frequencies and was constant at high spatiotemporal frequencies (up to 16 Hz and 16 cycles/°). Pelli's interpretation was that sensitivity is limited by neural noise at low spatiotemporal frequencies and by quantal noise (also known as "photon noise") at high spatiotemporal frequencies, which is consistent with the interpretation of van Nes, Koenderink, Nas, and Bouman (1967). Quantal noise results from the probabilistic absorption of photons by the photoreceptors and is therefore spatiotemporally white (i.e., spatially and temporally uncorrelated) up to very high spatial and temporal frequencies, whereas neural noise would decrease with increasing spatiotemporal frequency (Pelli, 1990; Raghavan, 1995). Although Pelli (1990) had to measure contrast thresholds in low and high noise to estimate equivalent input noise, his study focused on equivalent input noise, and he did not report contrast thresholds in low and high noise (or sensitivity and calculation efficiency, respectively). Nonetheless, given that sensitivity falls off at high temporal frequencies, constant equivalent input noise at these frequencies suggests that calculation efficiency falls off at high temporal frequencies. Concluding that the equivalent input noise is constant at high temporal frequencies therefore suggests the fact that the sensitivity falls off at high temporal frequencies is due to a fall-off in calculation efficiency, not a rise in equivalent input noise.

Citation: Allard, R., & Arleo, A. (2017). Factorizing the motion sensitivity function into equivalent input noise and calculation efficiency. *Journal of Vision*, *17*(1):17, 1–12, doi:10.1167/17.1.17.

<figure><img src="image_7.png" /></figure>
<figure><img src="image_8.png" /><figcaption>Figure 1. Linear Amplifier Model (Pelli, 1981, 1990). The impact of the internal noise is quantified as the level of external noise having the same impact as to the internal noise (i.e., equivalent input noise). The efficiency of the noisy contrast-invariant calculation is derived from the smallest signal-to-noise ratio required to perform the task in high noise. Adapted from Pelli (1990).</figcaption></figure>

However, the widely accepted view is that the sensitivity fall-off at high temporal frequencies is due to early temporal filtering (e.g., Kelly, 1961, 1969; Roufs, 1972; Watson, 1986; Watson & Ahumada, Jr., 1985), which should affect equivalent input noise, not calculation efficiency. Early temporal filtering would reduce the effective contrast of high temporal frequencies (Watson, 1986) and a contrast reduction gain affecting the signal and noise by the same proportion would have no effect on the signal-to-noise ratio and therefore should have no impact on contrast threshold in high noise (i.e., calculation efficiency, Pelli & Farell, 1999). Indeed, contrast threshold in high noise is known to be proportional to the noise contrast (slope of 1 in log-log coordinates, Pelli, 1981), so substantially reducing contrast at high temporal frequencies should have a direct impact on contrast threshold in low noise (i.e., sensitivity), but not in high noise (i.e., calculation efficiency), that is, it would affect equivalent input noise. As a result, Pelli's (1990) finding that equivalent input noise is constant at high temporal frequencies seems incompatible with the widely accepted view that the sensitivity fall-off at high temporal frequencies is due to early temporal filtering, which should affect equivalent input noise, not calculation efficiency. A limitation of Pelli's study (1990) was that equivalent input noise was measured at only a few temporal frequencies (0, 4, and 16 Hz), and he did not report sensitivity and calculation efficiency. To test if equivalent input noise is constant at high temporal frequencies, the present study factorized sensitivity into equivalent input noise and calculation efficiency at many temporal frequencies: 0.9375, 1.875, 3.75, 7.5, 15, and 30 Hz.

## Experiment 1: Factorizing motion sensitivity

Adding noise to substantially impair contrast threshold (by a factor of at least 2, Pelli & Farell, 1999)

neutralizes the impact of some factors affecting sensitivity such as additive internal noise (negligible in high noise) and contrast gains (affecting both the signal and noise by the same proportion). The external noise paradigm (Pelli, 1981; Pelli & Farell, 1999) is therefore often used to decompose the sensitivity into factors affecting contrast threshold only in low noise (grouped as equivalent input noise) and factors affecting contrast threshold in low and high noise (grouped as calculation efficiency). Such a model implicitly assumes that the low-noise calculation efficiency (i.e., when internal noise dominates) is the same as the measured high-noise calculation efficiency (when external noise dominates), which are therefore both referred to as “calculation efficiency.” The equivalent input noise can be calculated given the sensitivity and the (low-noise) calculation efficiency.

To measure the equivalent input noise and calculation efficiency, the first experiment therefore measured contrast thresholds (c) for a direction discrimination task in absence of noise and in high noise for various temporal frequencies. Given that the energy threshold (E, proportional to the squared contrast threshold) is known to be linearly related to the external noise energy (N) (Pelli, 1981; Pelli & Farell, 1999), these measurements were used to derive the equivalent input noise (Nᵢ) and calculation efficiency (J). In the current study, the signal was a drifting sine wave grating with randomized initial phase so the linear relationship between the energy threshold (E) and external noise energy (N) can be defined as (Bennett, Sekuler, & Ozin, 1999):

$$E(N) = \left[ \frac{(d' + \sqrt{0.5})^2}{J} \right] (N + N_i). \quad (1)$$

Given the measured energy thresholds in absence of external noise, E(0), and in high noise, E(N_HN), this equation can be used to calculate, for each temporal frequency, the equivalent input noise:

$$N_i = \frac{N_{HN}}{\frac{E(N_{HN})}{E(0)} - 1} \quad (2)$$

and the calculation efficiency:

$$J = \frac{N_{HN}(d' + \sqrt{0.5})^2}{E(N_{HN}) - E(0)}. \quad (3)$$

## Method

### Observers

Three naïve observers and one of the authors participated to the study. They had normal or corrected-to-normal vision.
## Apparatus

The stimuli were presented on a 22.5-inch LCD monitor designed for psychophysics (VIEWPixx) with a refresh rate of 120 Hz and a mean luminance of 50 cd/m². At the viewing distance of 2 m, the spatial resolution of the display was 128 pixels/° of visual angle. The monitor was the only light source in the room. The output intensity of each color gun was linearized psychophysically using a minimum motion technique (Cavanagh, MacLeod, & Anstis, 1987) consisting in gamma correcting the display to null the perceived motion of luminance—and contrast-modulated, counter-phase flickering gratings in quadrature phase (90° offset in both space and time). The observer's vision was blurred to minimize the visibility of the contrast-modulated grating and thereby facilitate the calibration. Display nonlinearity would introduce a flickering luminance grating in phase or in opposite phase with the contrast-modulated grating that would interact with the flickering luminance-modulated grating resulting in a motion percept. The Noisy-bit method (Allard & Faubert, 2008) implemented independently to each color gun made the 8-bit display perceptually equivalent to an analog display having a continuous luminance resolution.

## Stimuli and procedure

The task consisted in discrimination the drifting direction (left or right) of a 0.5 cycles/° vertically oriented grating by pressing one of two keys. The signal was presented for 250 ms plus on and off half-cosine ramps of 125 ms each for a total duration of 500 ms. The spatial window of the signal had a diameter of 4° of visual angle plus a half-cosine of 1°. The initial phase of the signal was randomized on each trial.

For each temporal frequency (0.9375, 1.875, 3.75, 7.5, 15, and 30 Hz), contrast threshold was measured with and without high external noise. The noise used was binarized-filtered noise (spatially low-pass filtered with cutoff at 4 cycles/° and then binarized, Jules Étienne, Arleo, & Allard, 2016) and was refreshed at 120 Hz. This noise had a flat energy spectrum up to 60 Hz, which is greater than the critical frequency fusion so it was considered temporally white, and up to 4 cycles/°, which was three octaves above the signal frequency. The noise contrast was fixed to 50%, resulting in an energy ($N_{HN}$) of 30 µ°² s (up to 4 cycles/° and 60 Hz). To avoid triggering a processing strategy shift (Allard & Cavanagh, 2011; Allard & Faubert, 2014a), the noise was continuously displayed (presented during and between trials) and covered the entire screen (i.e., spatiotemporally extended).

Contrast thresholds were measured using a 3 down, 1 up staircase procedure (Levitt, 1971) with step size of 0.1 log and were interrupted after 12 inversions. Such a

<figure><img src="image_9.png" /><figcaption>Figure 2. Sensitivity in absence of noise (closed symbols) and in high noise (open symbols) as a function of temporal frequency. Each data point is represented as the inverse of the energy threshold (left axis) and contrast threshold (right axis). Different symbols represent different observers (see legend in Figure 3). Lines represent the average across observers.</figcaption></figure>

staircase converged to a criterion level of 79% correct response corresponding to a d' of 1.16. For each staircase, the threshold was estimated as the geometric mean of the last eight inversions. Each of the 12 conditions (six temporal frequencies × two noises) was performed three times in a pseudorandom order, and the threshold for each condition was estimated as the geometric mean of the three threshold estimates (i.e., three staircases).

## Results and discussion

As expected, the motion sensitivity function in absence of noise was band-pass and peaked at 7.5 Hz (Figure 2). In high noise, the motion sensitivity function was low-pass. Based on the threshold measurements in absence of noise and in high noise, calculation efficiency and equivalent input noise were derived according to Equations 2 and 3 and plotted in Figure 3. Calculation efficiency, which is essentially proportional to sensitivity in high noise (Pelli & Farell, 1999), slightly decreased with temporal frequency up to 15 Hz (slope of -0.31 in log-log coordinates) and abruptly fell off between 15 and 30 Hz (slope of -2.6, Figure 3, top graph). The equivalent input noise varied more with temporal frequency (Figure 3, bottom graph) as it gradually decreased up to 7.5 Hz (slope of -1.0) and abruptly rose at higher temporal frequencies (slope of 2.1).

These results show that the sensitivity increase up to 7.5 Hz was due to a decrease of equivalent input noise,
<figure><img src="image_10.png" /><figcaption>Figure 3. Calculation efficiency and equivalent input noise as a function of the temporal frequency derived from the data presented in Figure 2, using Equations 2 and 3. Different symbols represent different observers. Solid line represents the average across observers.</figcaption></figure>

not an increase in calculation efficiency. This is consistent with Pelli's (1990) results in which equivalent input noise decreased with increasing low temporal frequency. The sensitivity fall-off from 7.5 to 15 Hz (a factor of 4.1 in energy units) was mainly due to an increase in equivalent input noise, whereas the fall-off from 15 to 30 Hz (a factor of 33) appears to be due to a combination of increase in equivalent input noise (a factor of 5.6) and decrease in calculation efficiency (a factor of 6.0), but see next experiment below.

## Experiment 2: The impact of early temporal integration on calculation efficiency

As described in the introduction above, early temporal filtering is expected to affect equivalent input noise, not calculation efficiency. This suggests that the lower calculation efficiency at 30 Hz in Experiment 1

would not be due to an early temporal filtering substantially decreasing the contrast of high temporal frequencies, but would rather be due to a reduced ability to integrate or select the relevant information to discriminate the drifting direction (e.g., the fastest processing channel tuned to a frequency lower than 30 Hz so it would be less sensitive to 30 Hz in high noise). However, contrast thresholds in high noise are not necessarily independent of early temporal filtering. An early temporal filter could affect more the signal at a very high temporal frequency than the noise integrated within a channel tuned to the signal temporal frequency, thereby impairing contrast threshold in high noise. Consider Figure 4, for instance. Motion detectors could be equally efficient at processing low and high temporal frequencies (e.g., same tuning function relative to the relevant signal frequency, fourth row) and nevertheless result in different contrast thresholds in high noise due to an early temporal low-pass filter. For a signal at very low temporal frequencies (left column), an early low-pass filter would have no impact on both the signal and the noise being integrated by the relevant motion detector. For a signal at very high temporal frequencies (right column), an early low-pass filter could drastically reduce the effective contrast of the signal and reduce less the noise being integrated by the processing channel due to lower frequencies being less affected by the early filter. As a result, even though an early low-pass filter would not reduce the signal-to-noise ratio at the signal frequency (signal and noise at the signal frequency affected by the same proportion), it could reduce more the signal than the total amount of noise being integrated by the processing channel, which would thereby reduce the effective signal-to-noise ratio within the processing channel and impair contrast threshold in high noise (i.e., calculation efficiency). This example shows that contrast threshold in high noise may not be independent of early temporal filtering; abrupt variations of contrast gains as a function of the temporal frequencies could affect contrast threshold in high noise by having different impact on the signal and noise being integrated by the processing channel. It is therefore possible that the drop in calculation efficiency at 30 Hz could be due to early temporal filtering. The objective of the second experiment was to empirically test if early temporal filtering, which considerably reduces the contrast of high temporal frequencies, may also affect calculation efficiency (i.e., contrast threshold in high noise) at high temporal frequencies.

A temporal filter (G) can be factorized into two filters (Figure 5): the contrast gain at the signal frequency ($G_s$), and the shape of the filter, which is the temporal filter normalized relative to the contrast gain at the signal frequency ($G/G_s$). Even though these two filters could be caused by the same physiological process (e.g., early temporal integration), this theoret-
<figure><img src="image_11.png" /><figcaption>Figure 4. Graphs illustrating that an early low-pass filter (second row) could affect differently the signal-to-noise ratio within processing channels tuned to different signal frequencies. The stimulus (first row) represents the energy spectrum of white noise (blue) and a signal (green) at a low (left), middle (center) and high (right) temporal frequencies. The effective stimulus (third row) corresponds to the energy spectrum of the signal and noise after passing through an early low-pass filter (second row). The bottom row represents the energy being integrated by motion detectors equivalently tuned to the respective signal frequency (fourth row). S and N represent the impact of the low-pass filter on the signal and noise energy being integrated by the motion detector (i.e., energy integrated by the motion detector with the early low-pass filter relative to the energy integrated if there were no early low-pass filter). A gain of 1 represents a low-pass filter having no impact on the energy being integrated by the motion detector (e.g., low temporal frequency, left column). The low-pass filter can affect differently the signal and noise energy integrated by the motion detector ($S \neq N$), thereby decreasing (high temporal frequency, right column) or increasing (middle temporal frequency, middle column) the signal-to-noise ratio ($S/N$).</figcaption></figure>

ical factorization is useful because these two filters have different effects on contrast threshold depending on the origin of the main noise source limiting performance. The contrast gain at the signal frequency ($G_s$) equally affects the signal and noise passing through the filter. As a result, it has no effect on contrast threshold when the main noise source occurs *before* the filtering operation (i.e., external noise or early noise, Figure 6), but it affects contrast threshold by a proportion of $G_s$ when the main noise source occurs *after* the filtering operation (i.e., late noise, Figure 6) because it affects the signal and not the main noise source.

The normalized shape of the filter (i.e., $G/G_s$), on the other hand, has no effect on the signal frequency (gain

of 1 by definition), but modulates the noise passing through the filter. Thus, contrary to the gain at the signal frequency, the shape of the filter has no impact when the main noise source occurs *after* the filtering (i.e., late noise, Figure 6) as it affects neither the signal nor the main noise source, but may affect contrast thresholds when the main noise source occurs *before* the filtering operation (i.e., external noise or early noise, Figure 6) as it does not affect the signal frequency but modulates the total amount of noise being integrated by the processing channel. Nonetheless, the effect of the shape of the filter is likely to be small under most conditions because contrast threshold is mainly affected by noise nearby the signal frequency.
<figure><img src="image_12.png" /><figcaption>Figure 5. Given a signal temporal frequency (s), a temporal filter (G, left graph) can be represented as a combination of two filters: a contrast gain at the signal frequency (G<sub>s</sub>, center graph) and the normalized shape of the filter (G/G<sub>s</sub>, right graph).</figcaption></figure>

Thus, if the shape of the filter varies little within the channel processing the signal frequency (gain near 1), it would have negligible impact. Furthermore, for modest variations around the signal frequency, the shape of the filter would likely have opposite effects on the noise at nearby lower and higher frequencies resulting in a weak net effect. For these reasons, the shape of the filter probably has negligible impact under most conditions as it is generally assumed. Nevertheless, for conditions under which the shape of the filter varies abruptly within the frequency range of the processing channel, it may considerably affect performance when the main noise source occurs before the filtering operation as in high noise.

To test if an early temporal filter can affect contrast threshold in high noise (i.e., calculation efficiency), the second experiment artificially applied temporal integration of various durations to the stimulus. If the calculation efficiency is independent of early temporal filtering, as generally assumed, then artificially applying a temporal integration (i.e., a temporal filter) should have no impact on the calculation efficiency (i.e., contrast threshold in high noise). On the other hand, if artificially applying a temporal integration filter to the stimulus affects calculation efficiency, then this would suggest that early temporal filtering by the visual system potentially also affects the calculation efficiency.

<figure><img src="image_13.png" /><figcaption>Figure 6. Elaborated Linear Amplifier Model (from Figure 1) in which an early temporal filter is now included. Since the internal noise may occur before or after this filtering process, two internal noise sources are represented.</figcaption></figure>

## Method

Three of the four observers who participated in the first experiment participated to the second experiment (participant CJE was not available).

The task and paradigm were the same as in the first experiment with the exception that the noise was always present and an artificial temporal integration was induced by convolving the stimulus with a normalized cosine function of different durations (period of 50, 100, 200, and 400 ms, Figure 7, top graphs) gradually reducing contrast of high temporal frequencies (Figure 7, bottom graphs). For comparative reasons, contrast thresholds in high noise were also measured in absence of an artificial filter as in the first experiment. Note that contrast threshold could not be measured for temporal frequencies at which the temporal filter almost completely removed the signal (i.e., contrast gains close to 0 for temporal frequencies of >15, >7.5, and >3.75 for the 100, 200, and 400 ms integration time, respectively).

## Results and discussion

Figure 8 shows the calculation efficiency (derived using Equation 3 based on contrast thresholds in high noise of the current experiment and contrast thresholds in absence of noise from Experiment 1, which had a negligible impact) as a function of the temporal frequency for different artificially induced temporal integration durations. Different calculation efficiencies were observed for different integration times, which show that an early temporal integration can affect calculation efficiency. This effect is more obvious in Figure 9, which plots the impact of an artificial temporal integration filter (i.e., calculation efficiency relative to the no filter condition) as a function of the temporal frequency. Under many conditions, the
<figure><img src="image_14.png" /><figcaption>Figure 7. Cosine temporal integration filters with periods of 50, 100, 200, and 400 ms (top row, left to right), which were convolved with the stimulus. Bottom row represents the impact of such filters on different temporal frequencies given by the Fourier transform of the filter. Note that filters were normalized to avoid affecting low temporal frequencies (contrast gain = 1).</figcaption></figure>

artificial temporal integration noticeably affected per-
formance, positively or negatively (i.e., efficiency gain
>1 or <1, respectively). These results show that,
contrary to what is generally implicitly assumed, early
temporal integration may affect calculation efficiency
(i.e., contrast threshold in high noise).

The four conditions under which the artificial
integration impaired calculation efficiency (gain < 1)
were at the highest testable frequencies for each
integration time (i.e., 50, 100, 200, and 400 ms at 30, 15,
7.5, and 3.75 Hz, respectively, Figure 9). In these
specific conditions, the contrast gains at the respective
signal frequency were very low (see gains at corre-
sponding frequency and filter in Figure 7, bottom
graphs) as in the high frequency condition in Figure 4,
right column. Even though higher frequencies were

nearly completely removed due to the rapid contrast
gain fall-off (gain close to 0), nearby lower frequencies
were reduced much less than the signal (e.g., noise
about four times greater at an octave below the signal
frequency). As a result, integrating noise nearby the
signal frequency could reduce the integrated noise less
than the signal, thereby reducing the signal-to-noise
ratio and impairing calculation efficiency (e.g., high
frequency condition in Figure 4, right column).
Consequently, the lower calculation efficiencies ob-
served at the highest temporal frequencies due to an
early temporal integration can be explained by a
substantial contrast reduction gain at the signal
frequency and a lower contrast reduction gain at
nearby lower frequencies, thereby reducing the effective
signal-to-noise ratio.

<figure><img src="image_15.png" /><figcaption>Figure 8. Calculation efficiency derived from Equation 3, left axis, thresholds in absence of noise, E(0), were taken from Experiment 1, as a function of temporal frequency for different temporal filter widths (different curves) and for the three observers (different graphs). Different highly saturated colors represent different integration times. Contrast thresholds in high noise are also represented using de-saturated symbols and the right axis, but because they are nearly equivalent to calculation efficiency (Pelli & Farell, 1999), de-saturated symbols are almost always hidden behind the saturated symbols representing calculation efficiency.</figcaption></figure>
<figure><img src="image_16.png" /><figcaption>Figure 9. Calculation efficiency gain (left axis) due to the application of an artificial integration filter, that is, calculation efficiency relative to the no filter condition (black curve in Figure 8). A gain greater than 1 represents a greater efficiency (lower contrast thresholds) in the presence of the artificial filter, and a gain lower than 1 represents a greater efficiency without an artificial filter. Data are averaged across observers, and error bars represents standard errors of the mean. As in Figure 8, symbols in de-saturated colors represent contrast threshold gains (right axis), but are almost never visible.</figcaption></figure>

Interestingly, an artificial filter reducing the contrast at high temporal frequencies could *improve* thresholds in noise under some conditions. The more substantial facilitation effects (a factor up to 3.0 in energy units and 1.7 in contrast units) were observed at 3.75 Hz for a temporal integration of 100 and 200 ms and at 7.5 Hz for a temporal integration of 100 ms. These facilitation effects can be explained by the fact that contrast gain fall-off was accelerating around the signal frequency (see gains at corresponding frequency and filter in Figure 7, bottom graphs), so the shape of the filter reduced the noise contrast at nearby high frequencies more than it increased the noise contrast at nearby lower frequencies. Consequently, the higher calculation efficiencies at some temporal frequencies can be explained by a higher contrast reduction of high temporal frequencies compared to the contrast reduction at the signal frequency, thereby increasing the effective signal-to-noise ratio (e.g., middle frequency condition in Figure 4, middle column). In other words, if noise at high frequencies impairs performance, then a low-pass filter reducing the contrast at these frequencies could improve performance.

General discussion

The target of the present study was to investigate which underlying internal factors are responsible for the variation of photopic motion sensitivity as a function of the temporal frequency by using a noise

paradigm factorizing sensitivity into equivalent input noise and calculation efficiency. The first experiment found that the variation in sensitivity was mainly due to a variation of equivalent input noise up to 15 Hz and was affected by both equivalent input noise and calculation efficiency at very high frequencies (>15 Hz). The second experiment showed that early temporal filtering could explain the drop in calculation efficiency at very high temporal frequencies.

The results of the first experiment are incompatible with the hypothesis that quantal noise was the main limiting internal noise source at high temporal frequencies. Quantal noise is caused by the probabilistic absorption of photons by the photoreceptors resulting in a noise that is temporally white up to temporal frequencies many orders of magnitude higher than temporal acuity (Pelli, 1990; Raghavan, 1995). Given that any temporal filtering occurring *after* the main internal noise source would have the same impact in low and high noise (i.e., early and external noise in Figure 6, respectively), it would not affect equivalent input noise. Since there is no temporal filtering occurring *before* quantal noise (optical factors such as the modulation transfer function of the eye, according to Campbell & Gubisch, 1966, affect spatial, but not temporal, information), equivalent input noise would be temporally white over the range of frequency limited by quantal noise. Furthermore, quantal noise sets a lower limit to the equivalent input noise independently of the processing occurring after the quantal noise (Pelli, 1990). Thus, if the main internal noise source were quantal noise at some temporal frequencies, the equivalent input noise at these frequencies would be constant and would correspond to the smallest level of equivalent input noise measured across all temporal frequencies. The smallest equivalent input noise was measured at 7.5 Hz and gradually increased towards further frequencies (V-shape in Figure 2, lower graph), so the main internal noise source cannot be quantal noise (at least at all other frequencies). On the other hand, the rise in equivalent input noise at high temporal frequencies is compatible with the widely accepted view that the sensitivity fall-off at high temporal frequencies is mainly due to early temporal filtering. As stated above, to affect equivalent input noise, temporal filtering must occur *before* the main internal noise source. We therefore conclude that the main noise source limiting motion sensitivity at high temporal frequencies was noise occurring after some early temporal filtering stage.

Given that the main internal noise source occurs *after* some temporal filtering (i.e., late noise in Figure 6), then the shape of such early temporal filter ($G/G_s$, Figure 5) could affect contrast threshold in high noise (i.e., high-noise calculation efficiency), but not in low
noise (i.e., sensitivity; for explanation, see introduc-
tion of Experiment 2). Consequently, the underlying
assumption of the Linear Amplifier Model (Figure 1)
that the low-noise calculation efficiency is the same as
the high-noise calculation efficiency may have been
violated. Indeed, if a factor affects contrast thresholds
only in high noise (e.g., shape of filter preceding main
internal noise source), then the contrast threshold in
high noise (i.e., high-noise calculation efficiency)
cannot be used to estimate the low-noise calculation
efficiency and thereby the equivalent input noise.
Given that there was some temporal filtering occurring
before the main internal noise source (Experiment 1)
and that the shape of the early temporal filter can
substantially impair contrast threshold only in high
noise at very high temporal frequencies (Experiment
2), then the drop in high-noise calculation efficiency at
30 Hz does not necessarily reflect a drop in low-noise
calculation efficiency.

Nonetheless, the fact that the low-noise calculation
efficiency could have been underestimated does not
imply that it was. Consider the hypothesis that the
low-noise calculation efficiency was the same as the
measured high-noise calculation efficiency, so that the
drop in the high-noise calculation efficiency at 30 Hz
was not due to the shape of the early temporal filter
(assumed to have a negligible impact), but to a late
factor equally affecting low- and high-noise calcula-
tion efficiency (e.g., fastest motion processing channel
tuned to less than 30 Hz). To explain only the rise of
equivalent input noise from 15 to 30 Hz (a factor of
5.6, Figure 2) by an early temporal filter preceding the
main internal noise source, we calculated that a cosine
filter would need a period of 42 ms. Since an artificial
temporal integration of 50 ms was found to have a
considerable impact on high-noise calculation effi-
ciency (drop by a factor of 14.6), it is unlikely that a
slightly shorter integration time (42 ms) had no effect.
These results therefore suggest that the shape of the
early temporal filter, which would not affect contrast
threshold in low noise, impaired threshold in high
noise so that the low-noise calculation efficiency was
greater than the high-noise calculation efficiency. In
other words, the result of the second experiment
suggests that the shape of filter occurring before the
main internal noise source caused the low-noise
calculation efficiency to be underestimated to some
extent.

Now consider the opposite hypothesis: Early
temporal filtering, not low-noise calculation efficiency,
was responsible for the entire fall-off in sensitivity
from 15 to 30 Hz. We calculated that a cosine filter
would need a period of 54 ms to explain the entire
sensitivity fall-off (a factor of 33 in energy units,
Figure 2). Given that an artificial filter of 50 ms was
found to bias the high-noise calculation efficiency at

30 Hz by a factor of 14.6 in energy units (or 3.8 in
contrast units, Experiment 2), a similar integration
time (54 ms) can therefore explain the high-noise
calculation efficiency fall-off by a factor of 6.0 in
energy units (or 2.4 in contrast units). Of course, the
impact of the artificial 50-ms cosine filter and the early
temporal integration on high-noise calculation effi-
ciency may differ depending on the actual shape of the
early filter (not being a cosine) or nonlinear summa-
tion effects (applying two similar filters would not
necessarily result in summing their impact on contrast
threshold). Nevertheless, Experiment 2 suggests that
early temporal filtering could have considerably
impaired high-noise calculation efficiency at 30 Hz.
Given that this temporal filtering occurred before the
main internal noise source (as concluded above), it
affected high-noise calculation efficiency, but not low-
noise calculation efficiency. In other words, an early
temporal filter could explain both the entire sensitivity
fall-off from 15 to 30 Hz and the high-noise
calculation efficiency fall-off. The results therefore
suggest indirectly that the low-noise calculation
efficiency remained good up to the highest temporal
frequency tested (30 Hz). At the very least, the drop in
high-noise calculation efficiency observed at 30 Hz
cannot be taken as evidence of a drop in low-noise
calculation efficiency. Further experiments are re-
quired to directly evaluate the low-noise calculation
efficiency at very high temporal frequencies.

Note that, if low-noise calculation efficiency was
indeed underestimated at 30 Hz, then the equivalent
input noise was also underestimated by the same
proportion because sensitivity is composed of two
factors: equivalent input noise and low-noise calcula-
tion efficiency. Indeed, the drop in sensitivity needs to
be explained by a combination of a drop in low-noise
calculation efficiency and a rise in equivalent input
noise. If the drop in low-noise calculation efficiency was
overestimated (i.e., low-noise calculation efficiency
underestimated at 30 Hz), then the rise in equivalent
input noise was equally underestimated (i.e., equivalent
input noise underestimated at 30 Hz).

The fact that the high-noise calculation efficiency
may not be equal to the low-noise calculation
efficiency highlights a limit of the external noise
paradigm as developed by Pelli (Pelli, 1981; Pelli &
Farell, 1999). The rationale of this paradigm is that
the factors affecting sensitivity (or contrast threshold
in low noise) either affect the impact of internal noise
(or equivalent input noise) or the ability to select and
integrate the information relevant to the task (i.e.,
low-noise calculation efficiency). The equivalent input
noise is therefore estimated by assuming that the low-
noise calculation efficiency is the same as the measured
high-noise calculation efficiency. This paradigm
therefore aims to estimate the impact of factors
affecting contrast thresholds only in low noise (grouped as equivalent input noise) and factors affecting contrast threshold equally in low and high noise (grouped as calculation efficiency). However, the current study shows that some factors (e.g., shape of an early filter as in Figure 5, right graph) modulating the impact of external noise without having any impact on signal and the main internal noise occurring after the temporal filter could affect contrast threshold only in high noise. This violates the assumption that the low-noise calculation efficiency is the same as the high-noise calculation efficiency. The implication of the violation of this assumption is the same as when different processing strategies operate in low and high noise (Allard & Cavanagh, 2011, 2012; Allard & Faubert, 2013, 2014a, 2014b; Allard, Renaud, Molinatti, & Faubert, 2013): It implies that the low-noise calculation efficiency cannot be assumed to be the same as the high-noise calculation efficiency. The difference in the current study is that the violation of this assumption is not due to different processes operating in low and high noise, but to a factor present in both low and high noise (namely, shape of early filter) affecting threshold only in high noise.

The shape of early filter is analogous to the perceptual template in the Perceptual Template Model (Lu & Dosher, 2008), which is a task-specific filter that modulates the impact of the external noise by operating before the main internal noise source. Such a high-level template operating before the main internal noise also has an impact only in high noise (i.e., when there is external noise to filter), but the distinction is that the current study shows that the filtering could occur very early in the visual system (e.g., it could be at the photoreceptor level since an artificial filter applied to the stimulus was sufficient to affect high-noise calculation efficiency) without being task-specific.

Nevertheless, the fact that the shape of early filtering occurring before the main internal noise source can affect performance only in high noise does not always compromise the application of the Linear Amplifier Model. This model remains useful to measure equivalent input noise when there is no filtering occurring before the main internal noise source (e.g., quantal noise). In this case, the shape of the filter would equally affect threshold in low and high noise, so low- and high-noise calculation efficiency would remain the same and it would not bias the equivalent input noise. When the main internal noise source occurs after some filtering, however, the shape of the filter would affect threshold only in high noise. Nonetheless, as mentioned in the introduction of Experiment 2, under most conditions this factor would likely have a negligible impact. The shape of the filter would likely noticeably affect high-noise calcu-

lation efficiency only when it varies substantially around the signal frequency, such as at very high temporal frequencies. In the current study, the rise in sensitivity at low temporal frequencies was much smaller (slope of 0.68 log-log in energy units) than the sensitivity fall-off at high temporal frequencies (slope of -5.1). Given that the shape of the early filter could have biased the estimate of the low-noise calculation efficiency by a factor up to 6.0 (presuming that motion processing is not more efficient at 30 than 15 Hz), the impact of the early filter should be much less at lower frequencies. Furthermore, the fact that an artificial 50-ms cosine filter had a stronger impact on high-noise calculation efficiency at 30 Hz (a factor of 14.6) than the drop in the high-noise calculation efficiency (a factor of 6.0) suggests that the impact of the early temporal filter on the high-noise calculation efficiency at lower temporal frequencies would be lower than the impact caused by the artificial filter (<1.85 in energy units or <1.36 in contrast units, Figure 9). Consequently, it is unlikely that the shape of the early filter substantially affected the high-noise calculation efficiency at lower temporal frequencies.

Interestingly, the sensitivity fall-off at high temporal frequencies due to early temporal integration (i.e., filtering) is analogous to the sensitivity fall-off at high spatial frequencies due to the optical modulation transfer function of the eye (Campbell & Gubisch, 1966). Although these spatial and temporal blurs are analogous, they notably differ in their origin: Optical blur occurs *before* the transduction process, whereas the temporal integration necessarily occurs *after* the transduction process. This subtle difference has important implications for the limiting noise source at high frequencies because it modulates differently the impact of quantal noise. The temporal blur substantially reduces the impact of quantal noise at high temporal frequencies, whereas the optical blur does not. A consequence of this difference is that high spatial frequencies are more likely limited by quantal noise (Pelli, 1990; Raghavan, 1995), whereas high temporal frequencies are more likely limited by neural noise.

In sum, the present study suggests that the variation in motion sensitivity as a function of the temporal frequency is mainly due to early temporal filtering, not to the ability to select and integrate motion. More specifically, we conclude that motion sensitivity at high temporal frequencies is limited by neural noise, not by quantal noise resulting from the probabilistic absorption of photons by the photoreceptors.

*Keywords: temporal sensitivity function, equivalent input noise, calculation efficiency, temporal filtering, external noise paradigm*
## Acknowledgments

We thank Daphné Silvestre for helpful comments. This research was supported by ANR – Essilor SilverSight Chair ANR-14-CHIN-0001.

Commercial relationships: none.

Corresponding author: Remy Allard.

Email: remy.allard@inserm.fr.

Address: Institut de la Vision, Sorbonne Universités, Paris, France.

## References

Allard, R., & Cavanagh, P. (2011). Crowding in a detection task: External noise triggers change in processing strategy. *Vision Research*, *51*(4), 408–416. Retrieved from http://ovidsp.ovid.com/ovidweb.cgi?T=JS&CSC=Y&NEWS=N&PAGE=fulltext&D=medl&AN=21185855

Allard, R., & Cavanagh, P. (2012). Different processing strategies underlie voluntary averaging in low and high noise. *Journal of Vision*, *12*(11):6, 1–12, doi:10.1167/12.11.6. [PubMed] [Article]

Allard, R., & Faubert, J. (2008). The noisy-bit method for digital displays: Converting a 256 luminance resolution into a continuous resolution. *Behavior Research Methods*, *40*(3), 735–743, doi:10.3758/BRM.40.3.735.

Allard, R., & Faubert, J. (2013). Zero-dimensional noise is not suitable for characterizing processing properties of detection mechanisms. *Journal of Vision*, *13*(10):25, 1–3, doi:10.1167/13.10.25. [PubMed] [Article]

Allard, R., & Faubert, J. (2014a). Motion processing: The most sensitive detectors differ in temporally localized and extended noise. *Frontiers in Psychology*, *5*, 426, doi:10.3389/fpsyg.2014.00426.

Allard, R., & Faubert, J. (2014b). To characterize contrast detection, noise should be extended, not localized. *Frontiers in Psychology*, *5*, 749, doi:10.3389/fpsyg.2014.00749.

Allard, R., Renaud, J., Molinatti, S., & Faubert, J. (2013). Contrast sensitivity, healthy aging and noise. *Vision Research*, *92*(0), 47–52, doi.org/10.1016/j.visres.2013.09.004.

Bennett, P. J., Sekuler, A. B., & Ozin, L. (1999). Effects of aging on calculation efficiency and equivalent noise. *Journal of the Optical Society of America, A, Optics, Image Science, & Vision*, *16*(3), 654–668.

Campbell, F., & Gubisch, R. (1966). Optical quality of

the human eye. *The Journal of Physiology*, 186(3), 558–578, doi:10.1113/jphysiol.1966.sp008056.

Cavanagh, P., MacLeod, D. I. A., & Anstis, S. M. (1987). Equiluminance: Spatial and temporal factors and the contribution of blue-sensitive cones. *Journal of the Optical Society of America A*, *4*(8), 1428–1438, doi:10.1364/JOSAA.4.001428.

Jules Étienne, C., Arleo, A., & Allard, R. (2016). Maximizing noise energy for noise-masking studies. *Behavior Research Methods*, 1–13, doi:10.3758/s13428-016-0786-1.

Kelly, D. H. (1961). Visual responses to time-dependent stimuli. II. Single-channel model of the photopic visual system. *Journal of the Optical Society of America*, *51*(7), 747–754, doi:10.1364/JOSA.51.000747.

Kelly, D. H. (1969). Diffusion model of linear flicker responses to ast. *Journal of the Optical Society of America*, *59*(12), 1665–1670, doi:10.1364/JOSA.59.001665.

Kelly, D. H. (1979). Motion and vision. II. Stabilized spatio-temporal threshold surface. *Journal of the Optical Society of America*, *69*(10), 1340–1349, doi:10.1364/JOSA.69.001340.

Levinson, E., & Sekuler, R. (1975). The independence of channels in human vision selective for direction of movement. *The Journal of Physiology*, *250*(2), 347–366, doi:10.1113/jphysiol.1975.sp011058.

Levitt, H. (1971). Transformed up-down methods in psychoacoustics. *Journal of the Acoustical Society of America*, *49*(Suppl. 2), 467–477.

Lu, Z.-L., & Dosher, B. A. (2008). Characterizing observers using external noise and observer models: Assessing internal representations with external noise. *Psychological Review*, *115*(1), 44–82. [Article].

Pelli, D. G. (1981). *The effects of visual noise*. Cambridge, UK. Cambridge University Press.

Pelli, D. G. (1990). The quantum efficiency of vision. In C. Blakemore (Ed.), *Visual coding and efficiency* (pp. 3–24). Cambridge, UK: Cambridge University Press.

Pelli, D. G., & Farell, B. (1999). Why use noise? *Journal of the Optical Society of America, A, Optics, Image Science, & Vision*, 16(3), 647–653.

Raghavan, M. (1995). *Sources of visual noise*. Syracuse, NY: Syracuse University Press.

Roufs, J. A. J. (1972). Dynamic properties of vision-II. Theoretical relationships between flicker and flash thresholds. *Vision Research*, *12*(2), 279–292, doi:10.1016/0042-6989(72)90118-6.

van Nes, F. L., Koenderink, J. J., Nas, H., & Bouman,
M. A. (1967). Spatiotemporal modulation transfer in the human eye. *Journal of the Optical Society of America, 57*(9), 1082, doi:10.1364/JOSA.57. 001082.

Watson, A. B. (1986). Temporal sensitivity. In K. Boff, L. Kaufman, & J. Thomas (Eds.), *Handbook of perception and human performance* (pp. 6:1–6:43).

New York: Wiley. Retrieved from http://ntrs.nasa.gov/search.jsp?R=19870046231

Watson, A. B., & Ahumada, Jr., A. J. (1985). Model of human visual-motion sensing. *Journal of the Optical Society of America, A*, *2*(2), 322–341. Retrieved from http://www.ncbi.nlm.nih.gov/pubmed/3973764