Running head: FORM PRIMING PROJECT

1

A behavioral database for masked form priming
James S. Adelman1 , Rebecca L. Johnson2 , Samantha F. McCormick3 ,Meredith McKague4 , Sachiko Kinoshita5 , Jeffrey S. Bowers6 , Jason R. Perry7 ,Stephen J. Lupker7 , Kenneth I. Forster8 , Michael J. Cortese9 , Michele Scaltritti10 ,Andrew J. Aschenbrenner10 , Jennifer H. Coane11 , Laurence White12 , Melvin J. Yap13 ,Chris Davis14 , Jeesun Kim14 , & Colin J. Davis3,6
1 University of Warwick; 2 Skidmore College; 3 Royal Holloway, University of London;
4 University of Melbourne; 5 Macquarie University; 6 University of Bristol;
7 University of Western Ontario; 8 University of Arizona;
9 University of Nebraska, Omaha; 10 Washington University, St. Louis;
11 Colby College; 12 Plymouth University; 13 National University of Singapore;
14 Marcs Institute, University of Western Sydney

Author Note
Corresponding Author: James S. Adelman, Department of Psychology,
University of Warwick, COVENTRY, CV4 7AL, UK. Telephone: +44 (0) 24 7615 0233;
Electronic mail: J.S.Adelman@warwick.ac.uk
James S. Adelman and Colin J. Davis are the organizers (senior authors) of thisproject. Other authors are in order of contribution. We thank Alexa Banculli, Peter
Bowles, Josef Broder, Helen Brown, Leo Chong, Alison Crumpton, Laura Cunniffe,
Louise Goddard, Laura Grima, Lydia Gunning, Anna Hall, Hyun Kyoung Jung,
Hannah Jenkins, Julie Lee, Bree Loethen, Mayumi Kohiyama, Stuart Miller, Luke
Mills, Nicole Newson, Ann-Marie Raphail, Emma Roscow, Jocelyn Schock, Annabel
Snell, Allison Teevan, Wan Zhen Chua, Louise Warner, Melissa Yeo and Jimmie
Zhang for their assistance in data collection, and Dave Balota for helpful comments.
Data collection at Royal Holloway, University of London was supported by a grantfrom the Experimental Psychology Society.

FORM PRIMING PROJECT

2
Abstract

Reading involves a process of matching an orthographic input with stored
representations in lexical memory. The masked priming paradigm has become astandard tool for investigating this process. Use of existing results from thisparadigm can be limited by the precision of the data and the need forcross-experiment comparisons that lack normal experimental controls. Here, we
present a single, large, high-precision, multi-condition experiment to address these
problems. Over 1000 participants from 14 sites responded to 840 trials involving 28
different types of orthographically related primes (e.g., castfe-CASTLE) in a lexical
1.4.1 decision task, as well as completing measures of spelling and vocabulary. The data
were indeed highly sensitive to differences between conditions: After correction for
multiple comparisons, prime type condition differences of 2.90 ms and above reachedsignificance at the 5% level. This paper presents the method of data collection andpreliminary findings from these data, which included replications of the most widelyagreed-upon differences between prime types, further evidence for systematic
individual differences in susceptibility to priming, and new evidence regarding
lexical properties associated with a target word’s susceptibility to priming. These
analyses will form a basis for the use of these data in quantitative model fitting andevaluation, and future exploration of these data that will inform and motivate newexperiments.

FORM PRIMING PROJECT

3Introduction

The everyday activity of reading involves correctly selecting from one’s
vocabulary the viewed word from among a variety of candidate words with some or
many of the same features. How the relevant candidates are evaluated in terms ofmatches and mismatches in identity and ordering of letters is a major current concern
in visual word recognition research. This concern is reflected in explicit
computational models (e.g., Adelman, 2011; Davis, 2010; Norris & Kinoshita, 2012),
as well as a wealth of experimental research, much of it using the masked form
priming paradigm developed by Forster and Davis (1984,; Forster, Davis, Schoknecht,& Carter, 1987). In these experiments (e.g., Davis & Bowers, 2006; Davis & Lupker,
2006; Grainger, Granier, Farioli, Van Assche, & van Heuven, 2006; Perea & Lupker,
2003), the presentation of a target stimulus for a lexical decision is preceded by a brief
1.4.2 presentation of a potentially related (nonword) prime stimulus. From the extent that
responses to a word target are faster following a related nonword prime thanfollowing an unrelated prime, researchers make inferences regarding similarity
between the processing evoked by the related prime and the processing evoked by averidical presentation of the target.
Many of these experiments have provided indications of important qualitative
differences among different types of primes allowing researchers to distinguishamong classes of models. However, as models have become more sophisticated inlight of these data, it has become problematic to evaluate them by a short list ofqualitative criteria alone. More than one model may produce the correct qualitative
pattern, and modelers have begun to use more quantitative criteria, such as thecorrelation between the observed priming and priming predicted by a given model(e.g., Adelman, 2011; Davis, 2010). However, calculating such correlations on thebasis of the combination of priming effects from several different experiments has
key drawbacks.
First, combining data across experiments lacks the kinds of experimental
controls we would normally expect in our studies. We would ordinarily aim to ensure

FORM PRIMING PROJECT

4

that our estimates of priming of different kinds come from the same population ofparticipants, and ideally from a within-subjects experiment. Moreover, we would
ordinarily use the same targets for each type of priming of interest, in order to avoid
contamination by moderating properties of the target words. Furthermore, we would
not use different equipment, font sizes etc. in investigating different types of priming.
Second, the size of many of the priming effects has not been estimated withgreat precision when this was not needed for the comparison of interest in theoriginal study. Typically, a priming effect (or difference between two priming effects)
does not reach significance if it is not at least 10 ms (even ignoring the issue ofmultiple comparisons), implying a 95% confidence interval on the size of the effectwith a range of 20 ms or so. Given the range of priming effects obtained in lexicaldecision with nonword primes and word targets — the usual paradigm of interest —is only around 50 ms, this level of precision can often be insufficient for the purposes
0.4

of assessing quantitative model predictions. Contemporary models such as Letters in

1.4.3 Time and Retinotopic Space (LTRS: Adelman, 2011), the Spatial Coding Model (SCM:
Davis, 2010) and the Bayesian Reader (Norris & Kinoshita, 2012) readily makepredictions of differences between conditions of less than 10 milliseconds, and this isof particular interest, for instance, in the use of derangements (permutations ofstimulus order which leave no letter in its original position: e.g., Lupker & Davis,
2009; Guerrera & Forster, 2008) where non-significant results of this magnitude havebeen observed. Moreover, these models can make quite similar predictions to one
another; in Adelman’s comparison of the LTRS and SCM models, the averageabsolute discrepancy in predicted priming between the two models was 8 ms.
Third, the estimates of the size of the priming effects are biased upwards by theprocesses involved in selecting experiments for publication. An experiment that has
no significant effects is highly unlikely to be published, so an experiment in which
the noise in the data from the control condition happens to make it unusually slow ismore likely to be published than one where it is unusually fast, which is likely toreach the file drawer. Thus, published priming effects will be on average larger than

FORM PRIMING PROJECT

5

the true effects.
Fourth, relatively few of the effects shown in these studies have been subject to2.6.1 published replication, which is a concern with a false positive rate of 1 in 20,
regardless of other concerns (see, for instance, the special issue edited by Pashler &Wagenmakers, 2012).
Some elements of the mega-study approach (e.g., Balota et al., 2007; for areview, see Balota, Yap, Hutchison, & Cortese, 2012) can appropriately deal withthese concerns. First, as a single large study with within-subjects manipulations, amega-study creates no systematic relationship between lexical predictor variables
and properties of the participants. Second, with more participants and stimuli, a
2.6.2 mega-study offers greater precision of estimates of any effects (provided the increasein quantity of data is sufficient to countermand other sources of variability that might
be introduced, such as site differences). Third, with no particular comparison
reaching significance being needed for the study to be of interest, mega-studies do
not get stuck in the file drawer. Fourth, mega-studies offer (conceptual) replications
that might not otherwise occur for many effects. However, due to the nature ofpriming manipulations — the presence of two stimuli on a trial, the need for abaseline condition, and the fact that most pairs of words are unrelated or weakly
related — a priming mega-study requires a more controlled approach than otherparadigms where an exhaustive selection (of a subset of items with some property)
would be possible and a random selection from among these items would be useful;that is, the various related and unrelated conditions in a priming study must be
selected a priori (for this approach to semantic/associative priming, see Hutchison,
Balota, Cortese, & Watson, 2007; Hutchison et al., in press).

The present study
With these points in mind, the present study was designed to produce a large
masked priming data set that could serve as the basis for a wide range of analyses
that would be useful in assessing models. To do so, researchers from 14 different

FORM PRIMING PROJECT

6

universities collected data using 28 prime types with 420 word targets (and 420
nonword foils). This approach differs from most mega-studies in two major respects:
The initial focus of the study is the different conditions designed into the experiment,
rather than lexical properties of words sampled across their natural distribution, andthe comparisons of these conditions are controlled by counterbalancing rather thanby covarying out potential confounders. In these respects, this study is more like a
version of an ordinary experiment that is enlarged in terms of words and participants
than like other mega-studies.
Prime conditions. Our choices of prime conditions reflect several theoretical
2.4

and empirical motivations. Our goal was to produce a general database of different
types of potentially theoretically relevant primes, rather than to produce yet anotherexperiment that purported to decide between two contemporary theories. We
adopted a large range of conditions that models should account for, because (i) doing
so will extend the utility of the data set to future, not just contemporary, models of
orthographic processing in word recognition; (ii) on the sheer balance of
probabilities, some of the past experimental comparisons almost certainly produced awrong result (of Type I or Type II type); and (iii) having a large number of conditions
is the most constraining approach when models depend on numerical parameters in
a complex way. Conditions where all contemporary models agree on the direction of
the qualitative effects are still important to include in a data set of this nature,
because they avoid the possiblity that modelers will be able to invoke variations inthese numerical parameters across different experiments to accommodate patterns
that would otherwise be incompatible in their models (see Adelman & Brown, 2008a,
for further discussion). There are several sets of conditions for which models like
LTRS can predict several different orderings, depending on the parameters
controlling different processing speeds. However, not all of the orderings that LTRS
can predict will come about if these speed paramters have to take values that workon the uncontroversial effects. Moreover, as yet unexplored interactions between
non-controversial effects and lexical properties of the targets of priming might beFORM PRIMING PROJECT

7

informative, and opening up the possibility of such exploratory analysis is one of the
major motivations for the collection of mega-study data.
One important issue was to select prime conditions having variants thatdiffered in position — initial, medial or final — in order to address claims regarding
the relative importance of exterior and interior letters arising primarily from otherparadigms (e.g., Humphreys, Evett, & Quinlan, 1990).
We also considered it important to include primes created by various amounts
of insertion (e.g., pragkise-PRAISE) and deletion (e.g, prse-PRAISE; also known assuperset and subset primes, respectively, e.g., Grainger et al., 2006; Van Assche &Grainger, 2006). These conditions provide evidence regarding the relative importanceof different positions, the flexibility of positional representation, and the balance of
positive and negative evidence in lexical matching.
We further considered it important to include transposition (e.g.,priase-PRAISE) and substitution (e.g., prnvce-PRAISE) primes involving various
positions. The evidence that transpositions produce more priming thancorresponding substitutions (e.g., Perea & Lupker, 2003) suggests that letter identityand position are encoded separately (i.e., slot-based coding of letter position, e.g.,
McClelland & Rumelhart, 1981, is inconsistent with the data), and is a key motivation
for the recent development of new models of letter identification and lexicalmatching. We also included neighbor-once-removed primes (e.g., prihse-PRAISE)
that combine transposition with substitution of a transposed letter. Evidence that thiscondition produces less priming than the substitution alone (Davis & Bowers, 2006) isinconsistent with coding schemes that base matching on open bigrams (i.e.,
representations of letter pair orders where letter pairs need not be adjacent) alone(e.g., Grainger & van Heuven, 2003).
We also considered more extreme transposition primes (involving manychanges in letter order, e.g., rpiaes-PRAISE), because the absence of significant
priming in such conditions (e.g., Guerrera & Forster, 2008; Lupker & Davis, 2009) has
been taken as informative regarding forms of inhibition that might operate inFORM PRIMING PROJECT

8

addition to the facilitation from the overlap between prime and target.
Our two unrelated conditions — a wordlike pseudoword condition and a more
arbitrary letter string condition — represent two strategies for selecting baselines
from which to calculate priming effects. In contrast to these lower limits, the identity
condition represents a presumed upper limit on the amount of priming within theparadigm.
Individual differences. With the large array of words in this study, there isconsiderable variability in their lexical properties, and these properties could operate
as moderators of the priming effects. Those moderators would be open to analysis byregression of this study’s data. Such use is a further motivation for this kind of study.
The details of such moderating effects can act as additional constraints on models of
the relevant processes. Furthermore, we took the opportunity to collect brief
measures of individual differences in spelling and vocabulary — which aremoderators of priming effects (Andrews & Hersch, 2010; Andrews & Lo, 2012) — toextend the possibilities for additional uses of the data set.
MethodParticipantsParticipants were recruited at 14 different university sites for course credit ormonetary compensation in multiples of 28, according to the counterbalancing scheme
described below. Problems with timing responses with the equipment at the
University of Nebraska, Omaha led to those participants being excluded from further
analysis. Participants whose accuracy at primed lexical decision was below 75% were
2.6.3 replaced1 . Table 1 summarizes the number of participants, including the number
excluded for errors or as excess to counterbalancing, and the form of compensation ateach site. As our goal was to produce as precise an estimate of each priming effect as
possible, each site was asked to provide as many participants as possible within a
1 For this purpose, failure to respond before a 2000 ms timeout described in the Procedure (0.47% of

all trials) was counted as an error. Data are included in the downloadable database for all participants
who were excluded or replaced for the analyses we present here.

FORM PRIMING PROJECT

9

fixed time window.

Design
Lexical decision times were measured in response to words primed with 28
types of primes, detailed in Table 2. Prime type was varied within-subjects, withparticipants within each site spread evenly over 28 counterbalancing lists. All lists
contained all targets, and targets were paired with a different prime type in each list,
with the constraint that each list contained an equal number (15) of each prime type.

Apparatus and Software
1.2

Visual presentation apparatus varied with site as detailed in Table 1, but all

2.1

sites used a 60Hz refresh rate setting. Moreover, an estimate of the typical viewing
distance (as no chin rest or other control on head position was used) was used tomodify the scripts for the DMDX stimulus presentation software (Forster & Forster,2003) to adjust the size of the stimuli so that the width of each character in a target
was approximately 1 degree of visual angle.
Some sites used button boxes for responses to two-alternative choice tasks; the2.2remainder used keyboard presses (left and right shift-keys). Although keyboards are
not particularly precise devices for measuring individual response times, Ulrich and
Giray (1989) showed that such problems can only have their influence by increasing
variance, reducing power, which can be counteracted by increasing sample size, andDamian (2010) showed that the additional variance introduced by keyboards is
negligible relative to human variability for numbers of trials orders of magnitude
fewer than in the present mega-study. All sites used numerical keyboard presses as
responses for the vocabulary task with four alternatives.

0.2

The monitor and button box or keyboard used are listed in Table 1.

0.3

These were the only differences in the manner the experiment was presented at

1.2
2.2

the various sites.

FORM PRIMING PROJECT

10

Stimuli
Four-hundred twenty word stimuli, all six letters long, were selected to betargets in the primed lexical decision task, and for each of these, 27 nonwords werechosen to be primes, one for each of the non-identity prime types detailed in Table 2(where lexicality was determined by reference to CELEX: Baayen, Piepenbrock, &Gulikers, 1995). All word targets had frequency above zero in CELEX, HAL (Burgess,1998) and SUBTLEX (Brysbaert & New, 2009), and had lexical decision accuracy of atleast 80% in the English Lexicon Project (Balota et al., 2007). No target was listed in
CELEX as having a derived or inflected morphemic structure. 120 targets had at least
1.3

one higher frequency orthographic neighbor. Other details of frequency,

2.6.4 orthographic and phonological characteristics of the word targets are given in Table 3.
No target contained the same letter twice, nor did any prime other than therepeated-insertion (123DD456) prime. Some prime type conditions could be createdin more than one way (e.g., as we used six-letter targets, a single medial substitution
could occur in four positions) as indicated in the table; across targets, these
subconditions were used equally often. Inserted or substituted letters were chosen at
random without replacement from those not in the target.
Two of the prime types were designed to be orthographically unrelatedbaselines. The arbitrary baseline was composed of six letters not in the target
pseudorandomly chosen without replacement with the constraint that the prime be anonword. The pseudoword primes were created to be wordlike (and likely
pronounceable) in the same way as the nonword foils, which we will now describe.
Four-hundred twenty nonword foils were constructed using an algorithm thatpseudorandomly replaced two letters of a real word in such a way that the resulting
string contained no repeated letters and each of its trigram frequencies exceeded aminimum value of one per million (based on the CELEX database). The real word
inputs to this algorithm were the 420 word targets. In this way it was ensured that the
nonword foils were well-formed English stimuli with orthographic structures thatclosely matched those of the word targets. The selection was constrained such that

FORM PRIMING PROJECT

11

none of the primes for the nonwords (constructed analogously to the primes for thewords) was a word; this constraint — and the prohibition on repeated letters —1.3

necessitated a custom program. Orthographic characteristics of the nonword foils

2.6.4 are given in Table 3.
To assess spelling ability, a set of items based on inconsistent and unusual
sound-spelling correspondences was used, consisting of 42 words (e.g.,
ELEMENTARY ) and 40 nonwords that were words modified to contain typical spelling

errors (e.g., REFRENCES , BENAFIT, TOUNGE), based on a list from Burt and Tate (2002).
The Shipley (1940) vocabulary test has 40 target words of increasing difficulty(from TALK to PRISTINE), each associated with one correct synonym and three foils.

Procedure
Participants first completed the primed lexical decision task. On each trial, a 300
ms initial presentation of a central fixation cross (+) was followed by a 200 ms blankdisplay, after which a hash (##########) mask was presented for 500 ms. Then theprime was displayed in lower case at five-eighths size for 50 ms, before the target
appeared in upper case until either the participants responded with a left or right
response, for nonword or word, respectively, or 2000 ms had elapsed. If an incorrect
response was given, or 2000 ms elapsed without a response, corrective feedback wasgiven. Instructions preceding this task described the sequence of events omitting
mention of the prime, and indicated the timed nature of the task whilst indicatingthat accuracy should not be unduly sacrificed.
2.6.5

The procedure was similar for the spelling items that followed in a new block— that is, the task was lexical decision — but without the prime and 2000 ms cutoff.
Participants were given new instructions, that these items were chosen to be difficult
to spell, and accuracy was emphasized.
Finally, a computerized version of the vocabulary portion of the Shipley (1940)
test was administered, displaying the target above four numbered potential
synonyms in turn for each of the 40 items, for a numerical key response.

FORM PRIMING PROJECT

12
Results

0.1

All the trial-by-trial data, including those for participants excluded fromanalysis here, are available for download fromhttps://files.warwick.ac.uk/jadelman2/browse#FPP . The data are
available as text files, or an Excel spreadsheet including details of the excluded trials
and the calculation of the condition means.
Overall priming results

1.1

Trials with associated response times of more than 1500 or less than 150 ms

2.6.3 (0.77% and 0.06% of correct trials, respectively, as illustrated in Figure 1) were
2.6.6 discarded — comparably with prior studies — for the purposes of the following
presentation of the data (again, “excluded” trials are included in the full databaseavailable to the database user). Table 4 presents the mean correct response times towords for each of the 28 prime types, and the resultant priming estimates against
each unrelated baseline.
Although the size of the data set (and design matrix for the analysis) is
prohibitive for ANOVA or other linear model analysis using most modern
computational software2 , we were able to use direct computation of the sums of
2.6.7 squares (literally adding up the squares of residuals for particular models, withmodel fitting based on marginal means, which is not how modern software typicallycomputes the values to appear in an ANOVA table) in combination with Clark’s(1973) pseudo-F calculations to produce a reasonable estimate of themean-squared-error (Clark’s Equation 14) in the estimation of the comparisons
between conditions, taking into account random effects associated with both subjects
and items. Applying Tukey’s HSD procedure to adjust for multiple comparisons withthis estimate of the mean-squared error implies that a difference between conditionsof 2.90 ms is significant at the 5% level, 3.24 ms at the 1% level, 3.37 ms at the 0.5%
2 The forms of analysis that could not be performed due to the computational memory requirements

are those whose computation includes the calculation of the pseudoinverse of the design matrix. In
modern software, such as SPSS, SAS, and R, this is routinely used as part of the fitting of linear models,
including mixed effects models.

FORM PRIMING PROJECT

13

level, and 3.66 ms at the 0.1% level.

Comparisons of priming by site
Figure 2 illustrates the quality of agreement between the priming estimates foreach testing site. Even the sites with fewer participants correlate very highly with theaverage of all sites, though (as one would expect) not so well with each other.
However, this fact does not rule out differences in the magnitude of priming (e.g., asite with half the priming effect would show a perfect correlation), which aresuggested by the varying slopes of the regression lines3 . Nevertheless, the agreement
between sites is notably good considering the variation in equipment used at
0.2

different sites. We also specifically compared the RTs from each priming conditioncollected with LCD monitors against those collected with CRT monitors. Although
RTs from the LCD sites were 41.47 ms slower than those from the CRT sites, pairedt(27) = 68.96, the correlations of RTs from each priming condition across the twomonitor types was r (28) = .965 and the correlation of priming effects relative to thearbitrary baseline was r (27) = .960.

Spelling, vocabulary, and site differences
The mean spelling score was 77.7% (SD = 8.8%). The mean vocabulary score
was 74.3% (SD = 9.7%). Spelling and vocabulary were well correlated across2.6.9 participants, r (924) = .422, p < .000000000000001.
The mean for each site in spelling, vocabulary, baseline response time and forthree major priming effects is given in Table 5. Substantial variability is shown in theestimates of priming at this level, despite the good correlations among sites.
Correlations of the spelling and vocabulary variables with each of the condition
mean response times and the priming effects are presented in Table 6, as arecorrelations of the sum and difference of the spelling and vocabulary scores and their
3 The Deming regression in those lines corrects for attenuation or regression dilution due to noise in

the x-observations insofar as its ratio with that in the y-observations can be predicted from sample size.

FORM PRIMING PROJECT

14

z-scores4 with response times and priming effects. There is a clear (expected) patternsuch that responses are faster for those with better spelling and vocabulary (e.g., Yap,
Balota, Sibley, & Ratcliff, 2012). Further, those with better spelling and vocabulary
showed less priming.
Turning to the difference between spelling and vocabulary, spelling had astronger relationship with overall response times. As a consequence, the difference
scores (SpellMinusVocab) also correlated negatively with response times. On
priming, however, if anything, the effect was in the reverse direction. In the final
column of Table 6, there are 20 out of 27 (sign-test: p = .019) positive correlations
between ZSpellMinusZVocab and priming; these results indicate greater priming for
those participants whose spelling was relatively better than their vocabulary.
However, this pattern was weak and none of the correlations was significant in its
own right.

Item-level analysis of targets
1.0
We examined whether targets with particular lexical properties wereparticularly susceptible or immune to priming by correlating lexical properties of theword targets with the priming as measured by substracting the mean of all the
related conditions from the mean of the two control conditions. In addition to log.
word frequency taken from CELEX (Baayen et al., 1995), the following lexical
properties were taken from the English Lexicon Project (Balota et al., 2007): log. HAL
frequency, log. SUBTLEX frequency, log. SUBTLEX contextual diversity, mean bigram
frequency, number of homophones, number of syllables, number of phonemes,
orthographic neighborhood size, phonological neighborhood size, phonographic
neighborhood size, and the Levenschtein based neighborhood variables proposed by
Yarkoni, Balota, and Yap (2008), as detailed in Table 7. The Levenschtein variables arecalculated as the average orthgraphic or phonological Levenschtein distance of the 20
4 With two positively correlated variables, the standard (equal-variance) principal components anal-

ysis gives the sum and difference of the z-scores, divided byare equivalent.

√

2, so for the purposes of correlation theseFORM PRIMING PROJECT

15

nearest words of a given target word; these are known as OLD20 (orthographic
Levenschtein distance 20) and PLD20 (phonological Levenschtein distance 20). The
zero-order correlations in that table show that PLD20 had the numerically strongest
relationship with priming. All variables except homophony had at least a marginal
relationship between all these variables and priming. The correlations among thevarious predictors of priming indicate the influences of these variables on priming
might not be unique.
We addressed the uniqueness of the effects of each of these variables in sixteen
multiple regression analyses that combined each of the four frequency (or contextual
diversity) counts with each of four strategies for including neighborhood variables:
first, including them all; second, including all three (N) measures based on one-letterand one-phoneme different neighbors; third, including only the orthographic andphonological neighborhood size, excluding the phonographic neighborhood variable;
and fourth, using only the Levenshtein-based OLD20 and PLD20 measures. These
analyses are summarized in Table 8.
In analyses where PLD20 was included, it was a significant predictor ofpriming, and was the only neighborhood variable that predicted priming (at the 5%
level). When PLD20 was excluded, orthographic N did usually predict priming. The
only other variables to reach significance were those based on the subtitle-based
corpus. Log. subtitle frequency predicted priming in all analyses in which itappeared, whereas log. subtitle contextual diversity was only significant when
PLD20 was absent from the regression.

Discussion
We have presented the first large single-experiment database of masked form
priming data in order to address four main limitations of using ad hoc databases
composed of several experiments. First, a lack of control exists when experiments are
combined. Although here we used multiple sites with differing equipment, suchdifferences were spread uniformly across conditions, rather than confounded with

FORM PRIMING PROJECT

16

condition. Second, there has been a lack of precision (and power) in priming
estimates from prior studies. In the present study, differences between conditions ofaround 3 ms could be considered significant, even with correction for (378) multiple
comparisons. Examination of subsets of the data showed the preceding concerns to
be valid: Although the data set as a whole had good reliability, and sites correlated
well with one another, there were still substantial differences in estimates ofindividual priming effects across different sites, especially for those with fewer
participants. Third, publication bias might overestimate priming effects, and fourth,past studies had not been subject to replication. We now compare our results withthose of some previous studies.

Exterior vs. interior letters
Examination of conditions in which the difference between prime and target
was the substitution of a single letter provided surprising evidence for the
2.6.10 importance of central letters. Medial substitutions (SN-M: desihn-DESIGN)
produced less priming than initial (SN-I: pesign-DESIGN) and final (SN-F:
desigj-DESIGN) substitutions, with the two ends not differing from one another.
This is consistent with the finding of Perea and Lupker (2003) that final, but not
medial, double substitutions produce priming with five-letter words. However, thisresult contrasts with the finding of Schoonbaert and Grainger (2004) in French that
double substitutions only produced priming when the substitution involved the last
two letters of seven-letter words (and no double substitution produced priming forfive-letter words). It also contrasts with a variety of evidence from letter
identification tasks showing the importance of exterior letters (in the form of higher
2.6.10 accuracy of report Estes, Allemeyer, & Reder, 1976). The condition that preserved
only exterior letters, disrupting the identity of all four interior letters (EL:dzbtkn-DESIGN) produced no priming. Transpositions produced a different
2.6.10 pattern, with initial adjacent transpositions (TL12: edsign-DESIGN) producing less2.6.10 priming than final adjacent transpositions (TL56: desing-DESIGN); medial adjacentFORM PRIMING PROJECT

17

2.6.10 transpositions (TL-M: desgin-DESIGN) differed from neither. Against unrelated
controls, Perea and Lupker also found that final and medial transpositions did not
differ. Schoonbaert and Grainger had in contrast found more priming for medial
transpositions of five-letter words than exterior transpositions. However,
Schoonbaert and Grainger found no such difference with seven-letter words.
Insertions and deletions produced a pattern such that disruptions of the final
2.6.10 character (DL-1F and IL-1F: desig-DESIGN and designl-DESIGN) produced
more priming than medial (DL-1M and IL-1M: dsign-DESIGN anddesrign-DESIGN) and initial (IL-1I: mdesign-DESIGN) disruptions. Given that
insertions and deletions affect both letter identity and position, it would be consistentto suggest that the pattern for these alterations is the combination of the patterns forthe letter identity (substitution) and letter position (transposition) cases.

Insertions, deletions, and prime length

Putting aside the issue of position, even stimuli involving several deletions
2.6.10 (SUB and DL-2M: des-DESIGN and degn-DESIGN) and insertions (IL-2M andIL-2MR: desaxign-DESIGN and deshhign-DESIGN) provided moderate priming
relative to the unrelated baselines, and deletions (DL-1M: dsgin-DESIGN) were less2.6.10 disruptive than substitutions (SN-M: desihn-DESIGN). This is consistent with thepatterns observed by Van Assche and Grainger (2006) and Norris, Kinoshita, and vanCasteren (2010) respectively, despite their different prime and target lengths.
Nevertheless, it is unclear to what extent the length of the unrelated baseline
contributes to these results, as the unrelated baseline (six letters) was not the same asthat of the deletion and insertion primes. Thus it will prove difficult tounambiguously disentangle the roles of stimulus length and insertion and deletion
with the present data alone. Further constraint on theoretical accounts of for theseeffects would come from studies systematically manipulating prime and target
length.

FORM PRIMING PROJECT

18

Transposition vs. replacement
2.6.10Transposition primes produced more priming than equivalent substitution
primes, consistent with earlier findings (TL-M vs. DSN-M: deisgn-DESIGN vs.
dewvgn-DESIGN: e.g., Perea & Lupker, 2003; Schoonbaert & Grainger, 2004). Indeed,a transposition involving a substituted letter (N1R: dslign-DESIGN) did not
produce significantly less priming than the substitution alone (SN-M:
desihn-DESIGN), contrary to an earlier report with shorter (five-letter) stimuli(Davis & Bowers, 2006).
Extreme transpositions

2.6.10

Several of the extreme transpositions (T-All, RH, IH, TH: edisng-DESIGN,
sedngi-DESIGN, idgens-DESIGN, igndes-DESIGN) produced small but
significant priming effects. Previous studies had reported null effects from thesetypes of primes (e.g., Guerrera & Forster, 2008; Lupker & Davis, 2009). The data areconsistent with two possible (and not necessarily competing) explanations of theinconsistency. First, given the size of the effect, the previous null effects could arise
from a lack of power. Second, the difference between studies could be due to the

2.6.11 greater length of the stimuli in the earlier experiments, which means the total
amount of change from target to prime was greater in the prior studies (e.g., T-All
was 4, transpositions in the prior studies, but 3 transpositions in this one).
Unrelated baselines
The present study also used two different forms of unrelated baseline that havebeen employed inconsistently in the literature, one made up of arbitrary unrelated
letters (cbhaux-DESIGN, and the other designed to form a pronounceable
pseudoword (voctal-DESIGN). The latter led to faster responses, and hence lower
priming estimates. Faster responses for pseudoword unrelated primes (and hence
lower estimates) are what would be expected if the prime directly contributes to theword-nonword decision, because the pseudowords are more wordlike, and therefore

FORM PRIMING PROJECT

19

more suggestive of a “word” response. As such, although this result is not necessarily
surprising5 , it does point to yet another concern when comparing or agglomerating
different experiments.

Uses of the database

The kinds of comparisons of conditions discussed above are not the primarybasis for use of this database for establishing empirical patterns — as thesecomparisons are already all listed in Table 4 — and, indeed, selecting subsets of thedata to perform simple comparisons has not been how other mega-studies have beenused (see Balota et al., 2012, for a review). Rather, mega-studies have been used toassess and compare models (e.g., Adelman & Brown, 2008b; Spieler & Balota, 1997),
to consider the role of individual differences (Yap et al., 2012), and to investigate new(continuous) predictors or measures (e.g., Adelman & Brown, 2007; Adelman, Brown,& Quesada, 2006; Yarkoni et al., 2008). We discuss our preliminary findings along
these lines below, and we envisage that many uses of these new data will beanalogous to those with other mega-studies, with the expectation that predictors ofinterest will interact with type of prime.

Individual differences

The overall patterns of faster responses and less priming for those with better(written) language skills are consistent with the prior report of Andrews (2008).
Whilst good spelling was more strongly related to faster responding than good
vocabulary, there was no evidence that good spelling was associated with lessfacilitatory priming over and above the effect of language competence in general, thepattern reported by Andrews and Lo (2012).

FORM PRIMING PROJECT

20

Target differences
1.0
A moderator of whether a target could be primed by an orthographically similar
prime that was identified in one of the earliest studies in the paradigm (Forster et al.,1987) is the neighborhood size of the target. Here, we found that of the neighborhood
variables that predicted priming, the strongest, and the only one accounting for
unique variance, was PLD20. This phonological measure stands in contrast to theorthographic neighborhood variables that are normally of interest in the context of
this paradigm. One possible interpretation is that lexical decisions are made at thephonological level (e.g., Rastle & Brysbaert, 2006), and so even orthographic priming
is sensitive to phonological competition. Alternative interpretations could suggestthat PLD20 is a better indicator of the truly relevant orthographic neighborhood
because that orthographic neighborhood is sensitive to multi-letter graphemes rather
than letters, or that the consonant-vowel pattern is important.
Whether the amount of priming obtained is sensitive to the frequency of thetarget has been a subject of some debate with many experiments (e.g., Forster &Davis, 1984) finding no such effect, but others indeed finding the effect when the
manipulation is sufficiently large, and the word stimuli are all familiar to participants(e.g., Kinoshita, 2006). Here, only with one frequency count — albeit the one that ismost predictive of lexical decision times (Brysbaert & New, 2009) — did we reliabilyfind an effect on priming such that higher frequency words were less susceptible topriming. Even if this means the effect is a real one, it also means it is quite a weakeffect (possibly in part because of restricted range). Given that the effect of frequency
is so weak, and frequency and neighborhood variables are correlated, it would not beso surprising if there were some other yet-to-be-constructed neighborhood variable
that would subsume the effect of both PLD20 and frequency.

5 On the other hand, there are reasons to suppose the wordlikeness of the prime might not contributeto the word-nonword decision directly. First, the pseudoword primes were just as nonword-like as thefoils. Second, the foils were very wordlike, disfavoring a criterion based on wordlikeness rather thanidentification of a single word.

FORM PRIMING PROJECT
Conclusion
The present database is the first of its kind for investigating orthographic(masked form) priming. It should serve as a benchmark data set in a variety ofinvestigations surrounding orthographic processing. An important example is theinvestigation of differences in mean priming for different prime types and theimplications for models, but the uses to which these data could be put include
moderation by individual differences, moderation by properties of items, andsequential effects and variability.

21

FORM PRIMING PROJECT

22
References

Adelman, J. S. (2011). Letters and time in retinotopic space. Psychological Review, 118,
570–582.
Adelman, J. S., & Brown, G. D. A. (2007). Phonographic neighbors, not orthographic
neighbors, determine word naming latencies. Psychonomic Bulletin & Review, 14,455–459.
Adelman, J. S., & Brown, G. D. A. (2008a). Methods of testing and diagnosing
models: Single and dual route cascaded models of word naming. Journal of
Memory and Language, 59, 524–544.
Adelman, J. S., & Brown, G. D. A. (2008b). Modeling lexical decision: The form of
frequency and diversity effects. Psychological Review, 114, 214–227.
Adelman, J. S., Brown, G. D. A., & Quesada, J. F. (2006). Contextual diversity, notword frequency, determines word-naming and lexical decision times.
Psychological Science, 17, 814–823.
Andrews, S. (2008). Lexical expertise and reading skill. Psychology of Learning andMotivation, 49, 249–281.
Andrews, S., & Hersch, J. (2010). Lexical precision in skilled readers: Individual
differences in masked neighbor priming. Journal of Experimental Psychology:
General, 139, 299–318.
Andrews, S., & Lo, S. (2012). Not all skilled readers have cracked the code:
Individual differences in masked form priming. Journal of Experimental
Psychology: Learning, Memory, and Cognition, 38, 152–163.
Baayen, R. H., Piepenbrock, R., & Gulikers, L. (1995). The CELEX Lexical Database(Release 2) [CD-ROM]. Philadelphia: Linguistic Data Consortium, University ofPennsylvania.
Balota, D. A., Yap, M. J., Cortese, M. J., Hutchison, K. I., Kessler, B., Loftis, B., . . .
Treiman, R. (2007). The English Lexicon Project. Behavior Research Methods, 39,445-459.
Balota, D. A., Yap, M. J., Hutchison, K. A., & Cortese, M. J. (2012). Megastudies: WhatFORM PRIMING PROJECT

23

do millions (or so) of trials tell us about lexical processing? In J. S. Adelman(Ed.), Visual word recognition, Vol. 1: Models and methods, orthography andphonology (pp. 90–115). Hove, England: Psychology Press.
Brysbaert, M., & New, B. (2009). Moving beyond Kučera and Francis: A criticalevaluation of current word frequency norms and the introduction of a new andimproved word frequency measure for American English. Behavior Research
Methods, 41, 977–990.
Burgess, C. (1998). From simple associations to the building blocks of language:
Modeling meaning in memory with the HAL model. Behavior Research Methods,
Instruments and Computers, 30, 188–198.
Burt, J. S., & Tate, H. (2002). Does a reading lexicon provide orthographic
representations for spelling? Journal of Memory and Language, 46, 518–543.
Clark, H. H. (1973). The language-as-fixed-effect fallacy: A critique of language
statistics in psychological research. Journal of Verbal Learning and Verbal Behavior,
12, 335–359.
Coltheart, M., Davelaar, E., Jonasson, J. T., & Besner, D. (1977). Access to the internal
lexicon. In S. Dornič (Ed.), Attention and performance VI (pp. 535–555). Hillsdale,
NJ: Erlbaum.
Damian, M. F. (2010). Does variability in human performance outweigh imprecision
in response devices such as computer keyboards? Behavior Research Methods, 42,
205–211.
Davis, C. J. (2010). The spatial coding model of visual word identification.
Psychological Review, 117, 713–758.
Davis, C. J., & Bowers, J. S. (2006). Contrasting five different theories of letter position
coding: Evidence from orthographic similarity effects. Journal of Experimental
Psychology: Human Perception and Performance, 32, 535–557.
Davis, C. J., & Lupker, S. J. (2006). Masked inhibitory priming in English: Evidence
for lexical inhibition. Journal of Experimental Psychology: Human Perception and
Performance, 32, 668–687.

FORM PRIMING PROJECT

24

Estes, W. K., Allemeyer, D. H., & Reder, S. M. (1976). Serial position functions forletter identification at brief and extended exposure durations. Perception &Psychophysics, 19, 1–15.
Forster, K. I., & Davis, C. (1984). Repetition priming and frequency attenuation inlexical access. Journal of Experimental Psychology: Learning, Memory, andCognition, 10, 680–698.
Forster, K. I., Davis, C., Schoknecht, C., & Carter, R. (1987). Masked priming withgraphemically related forms: Repetition or partial activation? Quarterly Journalof Experimental Psychology, 39, 211-251.
Forster, K. I., & Forster, J. C. (2003). DMDX: A windows display program withmillisecond accuracy. Behavior Research Methods, Instruments and Computers, 35,
116–124.
Grainger, J., Granier, J. P., Farioli, F., Van Assche, E., & van Heuven, W. J. B. (2006).
Letter position information and printed word perception: The relative-position
priming constraint. Journal of Experimental Psychology: Human Perception and
Performance, 32, 865–884.
Grainger, J., & van Heuven, W. (2003). Modeling letter position coding in printed
word perception. In P. Bonin (Ed.), The mental lexicon (pp. 1–24). New York:
Nova Science.
Guerrera, C., & Forster, K. (2008). Masked form priming with extreme transposition.
Language and Cognitive Processes, 23, 117-–142.
Humphreys, G. W., Evett, L. J., & Quinlan, P. T. (1990). Orthographic processing invisual word identification. Cognitive Psychology, 22, 517–560.
Hutchison, K. A., Balota, D. A., Cortese, M. J., & Watson, J. M. (2007). Predicting
semantic priming at the item level. Quarterly Journal of Experimental Psychology,
61, 1036–1066.
Hutchison, K. A., Balota, D. A., Neely, J. H., Cortese, M. J., Cohen-Shikora, E. R., Tse,C.-S., . . . Buchanan, E. (in press). The semantic priming project. Behavior
Research Methods. doi: 10.3758/s13428-012-0304-z

FORM PRIMING PROJECT

25

Kinoshita, S. (2006). Additive and interactive effects of word frequency and masked
repetition in the lexical decision task. Psychonomic Bulletin & Review, 13,
668–673.
Lupker, S. J., & Davis, C. J. (2009). Sandwich priming: A method for overcoming thelimitations of masked priming by reducing lexical competitor effects. Journal of
Experimental Psychology: Learning, Memory, and Cognition, 35, 618–639.
McClelland, J. L., & Rumelhart, D. E. (1981). An interactive activation model ofcontext effects in letter perception: Part 1. An account of basic findings.
Psychological Review, 88, 375–407.
Norris, D., & Kinoshita, S. (2012). Reading through a noisy channel: Why there’s
nothing special about the perception of orthography. Psychological Review, 119,517–545.
Norris, D., Kinoshita, S., & van Casteren, M. (2010). A stimulus sampling theory ofletter identity and order. Journal of Memory and Language, 62, 254–271.
Pashler, H., & Wagenmakers, E.-J. (2012). Editors’ introduction to the special section
on replicability in psychological science: A crisis of confidence? Perspectives on
Psychological Science, 7, 528–530.
Peereman, R., & Content, A. (1997). Orthographic and phonological neighborhoods
in naming: Not all neighbors are equally influential in orthographic space.
Journal of Memory and Language, 37, 382–410.
Perea, M., & Lupker, S. J. (2003). Transposed-letter confusability effects in masked
form priming. In S. Kinoshita & S. J. Lupker (Eds.), Masked priming: The state of
the art (pp. 97–120). Hove, UK: Psychology Press.
Rastle, K., & Brysbaert, M. (2006). Masked phonological priming effects in English:
Are they real? Do they matter? Cognitive Psychology, 53, 97–145.
Schoonbaert, S., & Grainger, J. (2004). Letter position coding in printed word
perception: Effects of repeated and transposed letters. Language and Cognitive
Processes, 19, 333–367.
Shipley, W. C. (1940). A self-administering scale for measuring intellectual

FORM PRIMING PROJECT

26

impairment and deterioration. The Journal of Psychology, 9, 371–377. doi:
10.1080/00223980.1940.9917704
Spieler, D. H., & Balota, D. A. (1997). Bringing computational models of word
naming down to the item level. Psychological Science, 8, 411–416.
Ulrich, R., & Giray, M. (1989). Time resolution of clocks: Effects on reaction timemeasurement—good news for bad clocks. British Journal of Mathematical andStatistical Psychology, 42, 1–12.
Van Assche, E., & Grainger, J. (2006). A study of relative-position priming withsuperset primes. Journal of Experimental Psychology: Learning, Memory, andCognition, 32, 399–415.
Yap, M. J., Balota, D. A., Sibley, D. E., & Ratcliff, R. (2012). Individual differences invisual word recognition: Insights from the English Lexicon Project. Journal of
Experimental Psychology: Human Perception and Performance, 38, 53–79.
Yarkoni, T., Balota, D., & Yap, M. (2008). Moving beyond Coltheart’s N: A new
measure of orthographic similarity. Psychonomic Bulletin & Review, 15, 971–979.

Excluded Due to Error or Equipment

5
0
2
1
3
4
0
1
3
3
29
0
0
0

Subjects Tested / 28

RHUL 217 7.75
Skidmore 197 7.04
Warwick 119 4.25
Melbourne 66 2.36
Macquarie 65 2.32
UWO 60 2.14
Bristol
59 2.11
WUSTL 57 2.04
Arizona 31 1.11
MARCS 31 1.11
Nebraska 29 1.04
Colby 28 1.00
Plymouth 28 1.00
Singapore 28 1.00

Subjects Used

Excess to Counterbalancing

Complete Sets of 28 Used
7
7
4
2
2
2
2
2
1
1
0
1
1
1

Compensation
Cash or CC
Cash or CCCCCCCCCash or CC
CC
Cash or CC
CC
Cash or CC
CC
Cash or CCCC
CashMonitorSamsung 793DF CRT
Dell 1907FPVt LCD
Sony CPD-G200 CRT
Sony E230 CRT
NEC 4FG Multisync CRT
LG Flatron W2242TQ LCDAOC F1770 CRT
iMac MA590LL integral LCD
NEC 14” Multisync C500 CRT
Mitsubishi DV997FDB CRT
Dell 1908FPt LCD
Dell P2210t LCD
Hanns-G AG172D LCD
Viewsonic E72f+ CRT

Custom DMDX PIO-12 BB
Dell L100 KB
NMB RT2158TWUK KB
Custom DMDX PIO-12 BB
Custom DMDX PIO-12 BB
HP KB38211 KB
Viglen KU-0325 KB
Apple KY6310LVLVZSA KB
Custom DMDX PIO-12 BB
Custom DMDX PIO-12 BB
Dell U 4739 KB
Dell SK-8115 KB
Viglen KB-0325 KB
Dell SK-8110 KB

RT Device

Subjects Tested

Note: MARCS = Marcs Institute, University of Western Sydney; RHUL = Royal Holloway, University of London; UWO = University of Western Ontario; WUSTL = Washington University, St. Louis. CC = Course Credit.
KB = Keyboard. BB = Button Box.

16 196
1 196
5 112
9 56
6 56
0 56
3 56
0 56
0 28
0 28
0
0
0 28
0 28
0 28

FORM PRIMING PROJECT
27

Table 1
Number of participants from each contributing site.

FORM PRIMING PROJECT

28

Table 2
Prime types forming conditions of the experiment. Where multiple codes are indicated, equal
numbers of targets participate in each of these sub-conditions. Where d or D is indicated arandom letter not present in the target is used; where d is indicated more than once, the same
letter is not reused, where D is indicated more than once, the same letter is reused.
Prime type

Code relative to 123456

Abbreviation

e.g.: DESIGN

Identity
Initial transposition
Medial transposition
Final transposition
2-apart transposition
3-apart transposition
Medial deletion
Final deletion
Central double-deletion
All-transposed
Transposed-halves
Half
Reversed-halves
Interleaved-halves
Reversed-except-initial
Initial substitution
Medial substitution
Final substitution
Neighbor-once-removed
Central double-substitution
Central insertion
Central double-insertion
As above, repeated letter
Central quadruple-substitution
PrefixSuffixUnrelated pseudoword
Unrelated arbitrary

123456213456132456/124356/123546
123465143256/125436
15342613456/12456/12356/12346
12345
1256
214365456123
123/456321654415263165432d234561d3456/12d456/123d56/1234d6
12345d12d356/13d456/124d56/123d46
12dd56123d456
123dd456
123DD456
1dddd6d123456
123456ddddddddddddd

ID
TL12
TL-M
TL56
NATL-24/35
NATL25
DL-1M
DL-1F
DL-2M
T-All
TH
SUB3
RH
IH
RF
SN-I
SN-M
SN-F
N1R
DSN-M
IL-1M
IL-2M
IL-2MR
EL
IL-1I
IL-1F
ALD-PW
ALD-ARB

designedsigndesgindesingdegisndgsien
dsign
desig
degn
edisngigndesdessedngi
idgensdngisepesigndesihndesigjdsligndewvgndesrigndesaxign
deshhign
dzbtknmdesigndesignl
voctalcbhauxFORM PRIMING PROJECT

29

Table 3
Properties of lexical decision targets, according to Elexicon web interface.
Orthography
Words
Orthographic N
Bigram frequency
OLD20

Nonword foils

Min.

Max.

Mean

Min.

Max.

Mean

0
442.8
1.45

8
4187.6
3.00

1.29
1798.11
2.12

0
390.4
1.60

6
3937.42.95

0.61
1732.14
2.29

Min.

Max.

Mean

Mean (log.)

62
0.12
0.07

283,001
501.33
85.35

Min.

Max.

Mean

0
0
1.00
3
1

35
6
4.00
7
3

2.96
0.71
2.01
5.09
1.82

Frequency (words)
HAL frequency
SUBTLEX freq.
SUBTLEX CD

13,258 8.08
18.73 2.33
5.39 2.16

Phonology (words)
Phonological N
Phonographic N
PLD20# of phonemes# of syllables

FORM PRIMING PROJECT

30

Table 4
Mean correct response times (RT) for each prime type for word targets.
Code
ID
DL-1F
IL-1F
TL56
TL-M
DL-1M
SN-F
SN-I
TL12
IL-1M
IL-1I
SUB3
IL-2MR
DL-2M
SN-M
N1R
NATL-24/35
IL-2M
T-All
DSN-M
RH
NATL25IH
TH
ALD-PWRFEL
ALD-ARB

12345612345
123456d
123465132456/124356/123546
13456/12456/12356/12346
12345d
d23456213456123d456
d123456
123/456
123DD456
1256
1d3456/12d456/123d56/1234d6
12d356/13d456/124d56/123d46
143256/125436
123dd456
21436512dd56321654153426415263456123
dddddd165432
1dddd6
ddddddRT

Priming-ARB

Priming-PW

634.48642.93643.51644.70645.74647.60647.71648.00648.13648.16650.49651.34651.68652.25654.48655.40656.97657.74660.39662.23663.73667.25668.26668.36672.37674.30674.82677.17

42.69
34.23
33.66
32.46
31.42
29.56
29.45
29.16
29.03
29.00
26.67
25.83
25.48
24.91
22.68
21.77
20.20
19.42
16.77
14.94
13.44
9.91
8.90
8.80
4.80
2.86
2.34
0.00

37.89
29.44
28.86
27.67
26.62
24.77
24.66
24.37
24.23
24.21
21.88
21.03
20.69
20.12
17.88
16.97
15.40
14.63
11.98
10.14
8.64
5.11
4.11
4.01
0.00
-1.94
-2.46
-4.80

FORM PRIMING PROJECT

31

Site (# of sets)

Spelling

Vocabulary

Baseline ARB

Baseline PW

ID Priming(-ARB)

1LD Priming(-ARB)

TL Priming(-ARB)

ID Priming(-PW)

1LD Priming(-PW)

TL Priming(-PW)

Table 5
Mean accuracy on spelling and vocabulary trials (%) for each contributing site, with baseline
RT (ms) from unrelated prime trials and priming effects (ms) for identity, medial
one-letter-different and medial transposed-letter primes.

Arizona (1 set)
Plymouth (1 set)
UWO (2 sets)
Macquarie (2 sets)
Bristol (2 sets)
Warwick (4 sets)
Skidmore (7 sets)
RHUL (7 sets)
Colby (1 set)
Melbourne (2 sets)
MARCS (1 set)
WUSTL (2 sets)
Singapore (1 set)

71.17
74.48
75.70
75.94
77.07
77.47
77.56
77.96
79.14
79.36
79.75
81.34
82.10

64.55
72.86
69.73
74.42
73.53
71.36
76.11
73.71
80.18
73.88
77.95
81.56
76.07

739.44711.30682.20625.00683.13683.36710.69624.24732.19708.64649.53664.54705.33719.20695.47666.06614.44688.54682.20707.94623.86724.39694.74642.04664.41
693.17

58.34
64.65
38.01
40.25
32.35
34.08
39.32
45.43
27.94
59.07
43.77
35.59
66.80

42.41
45.90
34.03
24.51
19.96
17.79
25.19
30.48
26.42
47.12
27.86
29.45
37.50

55.82
32.45
36.51
34.31
15.67
19.62
31.73
32.58
17.57
48.57
36.17
23.48
48.98

38.10
48.82
21.88
29.69
37.76
32.92
36.57
45.04
20.14
45.17
36.27
35.46
54.64

22.18
30.07
17.89
13.95
25.37
16.63
22.44
30.09
18.62
33.22
20.37
29.33
25.35

35.58
16.62
20.37
23.75
21.07
18.47
28.97
32.20
9.77
34.67
28.67
23.35
36.83

Average (33 sets)

77.70

74.30

677.17672.37

42.69

22.68

31.42

37.89

17.88

26.62

Highest − lowest

10.93

17.01

115.20109.05

38.86

29.33

40.15

34.50

19.27

27.06

FORM PRIMING PROJECT

32

Table 6
Correlations between individual difference variables — spelling, vocabulary, their sum andtheir difference — and condition response times and priming effects (against the arbitrary
unrelated prime baseline) at the subject level.

Prime Type

SpellPlusVocab

SpellMinusVocab

ZSpellPlusZVocab

ZSpellMinusZVocab

Spelling

Vocabulary

SpellPlusVocab

SpellMinusVocab

ZSpellPlusZVocab

ZSpellMinusZVocab

Priming (ms)

Vocabulary

RT (ms)Spelling

Correlation with:

ALD-ARB
ALD-PW
DL-1F
DL-1M
DL-2M
DSN-M
EL
ID
IH
IL-1F
IL-1I
IL-1M
IL-2M
IL-2MR
N1R
NATL-24/35
NATL25
RF
RH
SN-F
SN-I
SN-M
SUB3
T-All
TH
TL12
TL-M
TL56

-.231
-.222
-.235
-.204
-.217
-.221
-.209
-.209
-.205
-.212
-.223
-.235
-.197
-.227
-.240
-.203
-.194
-.216
-.214
-.207
-.223
-.226
-.205
-.167
-.201
-.222
-.199
-.207

-.156
-.135
-.135
-.134
-.131
-.126
-.137
-.145
-.139
-.123
-.112
-.125
-.109
-.128
-.144
-.128
-.099
-.132
-.112
-.119
-.146
-.160
-.127
-.111
-.140
-.137
-.146
-.122

-.228
-.210
-.217
-.198
-.204
-.203
-.203
-.208
-.202
-.196
-.195
-.211
-.179
-.207
-.225
-.194
-.171
-.204
-.191
-.190
-.217
-.227
-.195
-.163
-.200
-.211
-.203
-.192

-.053
-.066
-.077
-.051
-.065
-.073
-.052
-.045
-.047
-.069
-.089
-.087
-.069
-.077
-.072
-.055
-.076
-.063
-.081
-.068
-.056
-.046
-.059
-.040
-.042
-.063
-.035
-.065

-.230
-.212
-.219
-.200
-.207
-.206
-.205
-.210
-.204
-.199
-.198
-.214
-.181
-.210
-.228
-.196
-.174
-.207
-.194
-.193
-.219
-.229
-.197
-.165
-.202
-.213
-.205
-.195

-.070
-.081
-.093
-.065
-.080
-.088
-.067
-.060
-.062
-.083
-.103
-.102
-.082
-.092
-.089
-.069
-.089
-.078
-.095
-.082
-.072
-.062
-.073
-.052
-.057
-.078
-.049
-.079

-.027
.007
-.048
-.030
-.024
-.048
-.038
-.062
-.039
-.021
-.005
-.063
-.022
.007
-.062
-.074
-.046
-.038
-.032
-.021
-.012
-.056
-.119
-.056
-.014
-.055
-.039

-.042
-.034
-.039
-.045
-.053
-.040
-.020
-.041
-.060
-.077
-.059
-.084
-.056
-.024
-.056
-.103
-.053
-.080
-.058
-.022
.003
-.057
-.084
-.032
-.030
-.018
-.057

-.041
-.018
-.051
-.045
-.046
-.052
-.034
-.060
-.059
-.060
-.040
-.088
-.047
-.011
-.070
-.106
-.059
-.071
-.054
-.025
-.005
-.067
-.119
-.051
-.027
-.042
-.058

.016
.039
-.005
.018
.030
-.004
-.015
-.015
.024
.057
.053
.026
.034
.030
-.000
.035
.010
.043
.028
.003
-.013
.006
-.025
-.019
.016
-.031
.020

-.041
-.017
-.051
-.044
-.045
-.052
-.035
-.061
-.058
-.058
-.038
-.087
-.046
-.010
-.070
-.105
-.059
-.070
-.053
-.025
-.005
-.067
-.120
-.052
-.026
-.043
-.057

.013
.038
-.008
.015
.027
-.008
-.017
-.019
.020
.053
.050
.020
.031
.029
-.005
.027
.006
.038
.024
.001
-.014
.002
-.033
-.023
.014
-.034
.016

Note — Critical value for correlation at 5% is ±.065; 1% ±.085; 0.5% ±.092; 0.1% ±.108.

FORM PRIMING PROJECT

33

Table 7
Correlation matrix of lexical target properties.

Priming
CELEX
HAL
SUB-WF
SUB-CD
Orth N
Phon N
PhGr NOLD20
PLD20
BG Freq
Len Phon
Len Syll
Homophones

Priming
CELEX
HAL
SUB-WF
SUB-CD
Orth N
Phon N
PhGr NOLD20
PLD20
BG Freq
Len Phon
Len Syll
Homophones

Priming

CELEX

HAL

SUB-WF

SUB-CD

Orth N

Phon N

1.000
-.109
-.091
-.155
-.137
-.151
-.163
-.106
.215
.242
-.054
.126
.135
-.074

-.109
1.000
.772
.780
.800
.048
.108
.018
-.168
-.177
.104
-.147
-.083
.042

-.091
.772
1.000
.780
.778
-.019
.073
-.041
-.074
-.096
.117
-.062
.001
.011

-.155
.780
.780
1.000
.985
.062
.166
.019
-.188
-.217
.123
-.220
-.081
.028

-.137
.800
.778
.985
1.000
.067
.149
.022
-.196
-.215
.132
-.212
-.090
.013

-.151
.048
-.019
.062
.067
1.000
.362
.826
-.637
-.385
.329
-.158
-.126
.054

-.163
.108
.073
.166
.149
.362
1.000
.364
-.430
-.672
.019
-.611
-.415
.433

PhGr N

OLD20

PLD20

BG Freq

Len Phon

Len Syll

Homophones

-.106
.018
-.041
.019
.022
.826
.364
1.000
-.537
-.349
.201
-.139
-.169
.000

.215
-.168
-.074
-.188
-.196
-.637
-.430
-.537
1.000
.611
-.407
.380
.401
-.133

.242
-.177
-.096
-.217
-.215
-.385
-.672
-.349
.611
1.000
-.193
.665
.537
-.255

-.054
.104
.117
.123
.132
.329
.019
.201
-.407
-.193
1.000
.053
-.012
-.093

.126
-.147
-.062
-.220
-.212
-.158
-.611
-.139
.380
.665
.053
1.000
.506
-.244

.135
-.083
.001
-.081
-.090
-.126
-.415
-.169
.401
.537
-.012
.506
1.000
-.183

-.074
.042
.011
.028
.013
.054
.433
.000
-.133
-.255
-.093
-.244
-.183
1.000

Note — Critical value for correlation at 10% is ±.080; 5% ±.096; 1% ±.125; 0.5% ±.137;
0.1% ±.160.
CELEX = log. CELEX frequency (Baayen et al., 1995); HAL = log. HAL frequency(Burgess, 1998); SUB-WF = log. subtitle (Brysbaert & New, 2009) frequency, SUB-CD =log. subtitle contextual diversity (Adelman et al., 2006); Orth N = orthographic neighborhood size (Coltheart, Davelaar, Jonasson, & Besner, 1977); Phon N = phonologicalneighborhood size; PhGr N = phonographic neighborhood size (Adelman & Brown,
2007; Peereman & Content, 1997); BG Freq = mean bigram frequency; OLD20, PLD20= orthographic/phonological Levenshtein distance, average of smallest 20 (Yarkoni etal., 2008); Len Phon = length in phonemes; Len Syll = length in syllables; Homophones= total number of entries with same pronunciation.

FORM PRIMING PROJECT

34

Table 8Regression coefficients in models predicting priming (average all related prime types minus
average all control prime types) by target lexical properties.
Neigh. measures:

All

Orth N, Phon N, PhGr N

Freq. Measure:

CELEX

HAL

SUB-WF

SUB-CD

CELEX

HALIntercept
CELEX
HAL
SUB-WF
SUB-CD
Len Phon
Len Syll
BG Freq
Homophones
Orth N
Phon N
PhGr N
OLD20
PLD20
R-squared

−5·84
−1·11

−0·65

4·98

2·55

20·22†−1·41†

26·63∗−1·14

SUB-WF

Neigh. measures:

−3·14
1·19
0·00
−1·02
−1·61
−0·01
1·80
6·78
11·82∗
8·40%

Freq. Measure:

CELEX

HAL

SUB-WF

Intercept
CELEX
HAL
SUB-WF
SUB-CD
Len Phon
Len Syll
BG Freq
Homophones
Orth N
Phon N
PhGr N
OLD20
PLD20
R-squared

20·45†−1·43†

27·00∗

31·53∗

0·42
4·33
0·00
−1·07
−2·80∗
−0·46
1·76

5·23%

SUB-CD

6·10%

5·66%

29·66∗CELEX

HAL

SUB-WF

SUB-CD

−8·13
−1·10

−2·84

2·75

0·34

−1·12
−5·10 ∗ ∗5·04%

−4·66∗−0·34 −0·19
4·41
4·25
0·00
0·00
−1·36 −1·37−2·65† −2·64†−0·44 −0·47
1·62
1·65

OLD20, PLD20−1·36†

5·04%

29·22∗−5·01 ∗ ∗−3·94†
−3·01
0·18
1·01
4·08
0·00
0·00
−1·00
−0·88
−1·59
−2·70†
−0·03
−0·50
1·84
1·78
6·93
11·89∗
8·00%
5·25%

Orth N, Phon N

0·43
0·67
3·77
4·03
0·00
0·00
−1·54 −1·73−1·66† −1·77∗−0·43 −0·38

31·12∗−1·33†−4·40∗−2·71 −2·57
0·82
0·98
0·00
0·00
−0·58 −0·73−1·61 −1·66−0·06 −0·01
1·95
1·93
7·20
7·48
11·94∗ 12·01∗
7·64% 7·72%

SUB-CD

−0·13
4·14
0·00
−1·97
−1·71∗
−0·37

5·93%

−4·41∗−4·76∗
0·03
3·97
0·00
−1·99
−1·68∗
−0·40

5·49%

−2·53
0·27
0·00
−1·21

−2·46
0·40
0·00
−1·22

8·44†
8·89†
12·06 ∗ ∗ 12·01 ∗ ∗
7·34%
7·41%

−3·07
0·63
0·00
−1·45

−3·96†
−2·90
0·46
0·00
−1·52

8·28†
8·31†
11·83 ∗ ∗ 11·96 ∗ ∗
8·12% 7·72%

Note — *** = p < .001; ** = p < .01; * = p < .05; † = p < .1. Abbreviations as in Table 7.

FORM PRIMING PROJECT

35

0.00250.0020Density0.00150.00100.00050.0000

0

500

1000

1500

Correct Response Time

Figure 1. Histogram of all correct response times within the two-second time-out.

2000

FORM PRIMING PROJECT

720 760
760
740
720

Arizona(1 set)

720
700
680

680 720720 760

620 660

640 680

680 720680 720600 640680 720700 740660 700

680 720660 700

660 700.47.55.68 .63 .72 .63 .70 .70 .71 .73 .59 .59 .73

Bristol(2 sets).56.72 .76 .81 .64 .86 .82 .80.58.85 .77 .86

Colby(1 set).67.81 .70 .71 .64.58.69 .58 .72

760
740
720

660
640
620

Macquarie(2 sets)

680
660
640

Condition Mean Response Times (ms)

36

.54

.64

.81 .80 .78 .83 .82 .88 .67 .81 .77 .89
MARCS(1 set)

720
700
680

.77 .68 .80 .79 .80 .59 .78 .75 .84
Melbourne(2 sets)

720
700
680

.80 .94 .87 .91 .76 .86 .84 .95
Plymouth(1 set)

640
620
600

.84 .81 .83 .64 .83 .80 .87
RHUL(7 sets)

720
700
680

.86 .94 .74 .92 .81 .97
Singapore(1 set)

740
720
700

.89 .72 .88 .80 .92
Skidmore(7 sets)

700
680
660

.72 .91 .84 .97
UWO(2 sets)

720
700
680

.70 .67 .77
Warwick(4 sets)

700
680
660

.81 .94
WUSTL(2 sets)

700
680
660

.88
Average(33 sets)

660 700

620 660

680 720560 600

580 620640 680

640 680

560 600

620 660

640 680

620 660

620 660

600 640

Condition Mean Response Times (ms)Figure 2. Comparison of priming effects at the different sites using condition mean
correct RTs at each site. Above/right of diagonal: Correlation coefficient between
condition means at each pair of sites and (weighted) average of sites. Below/left ofdiagonal: scatterplot of these condition means with Deming regression; thistechnique differs from ordinary linear regression in that it allows for noise in thex-observations of known size relative to the y-observations (rather than assuming
there is no noise in the x-observations) — here this ratio was set to that implied if
differences were only caused by sample size.

620 660

700
680
660

660
640
620

720
700
680

600
580
560

620
600
580

680
660
640

680
660
640

600
580
560

660
640
620

680
660
640

660
640
620

660
640
620

640
620
600

660
640
620