Discussion of Plans for Designing the Recall Period for the 

Consumer Expenditure Interview Survey 

David Cantor 

Westat and Joint Program for Survey Methodology 

Discussion prepared for the 2010 Consumer Expenditure Surveys Methods Workshop 
1 

The Issue paper on the recall period describes the background and ongoing research that 

is being conducted as part of the redesign of the Consumer Expenditure Interview Survey (CEQ). 

The paper discusses redesign plans as they relate to the recall period and relevant design 

parameters. The discussion below is organized into two sections. In the first section two issues 

are discussed that are particularly important to consider when designing an effective recall 

strategy. The second section addresses selected questions posed in the Issue paper. 

Important CEQ Design Features Related to the Recall Period 

The recall period is related to a number of other aspects of the survey interview, 

including the burden placed on the respondent, the mode of data collection, the use of proxy 

reporting and the questionnaire. The plans for the redesign of the CE are covering virtually all of 

these topics in one form or another (BLS, 2010). Two that are particularly important to consider 

for deciding on the best recall strategy is the burden placed on the respondent and the 

performance of interviewers when implementing the survey. 

Burden and the Recall Task 

The burden placed on respondents is a critical driver in the response process associated 

with retrospective recall. The recall and dating of events is one of the most difficult tasks that 

are asked of the survey respondent. If respondents are not motivated to perform these tasks, data 

quality will suffer. A corollary to this is that recall can be significantly improved by increasing 

the engagement and effort of the respondent. The event history calendar (EHC) is successful 

partly because it actively engages and encourages the respondent to systematically think about 

events over the reference period (Belli and Callegaro, 2008: 36-37). Similarly, a common 

methodology of cognitive interviewing is to conduct detailed debriefings of respondents to assess 

the quality of information reported during the main part of the interview. The premise is that if 

given the time and proper assistance, most individuals can significantly improve on their 

memory search and retrieval. 
2 

It is clear the redesign team recognizes the significant burden placed by the CEQ. The 

emphasis on splitting up the current interview and addressing proxy reporting are clear examples 

of this. On the current CEQ, the respondent is asked to report on a wide range of expenses, 

including those for other household members. Given the difficulty in completing the recall task, 

it is hard to imagine respondents have much time to devote to a thorough search of memory for 

all items. The survey documentation states the average interview length is 1 hour. When I took 

the interview as part of preparation for this workshop, the interview lasted 2.5 hours and we did 

not finish. My interviewer was not a well-practiced field staff and I was also asked a module 

that is only asked on the first visit. Nonetheless, my sense is that 1 hour is a significant 

underestimate of the actual burden when all of the questions are read as required and for anyone 

who takes the response task seriously. 

The interview should be limited to less than an hour. This parameter could change if 

there are new methods in place that significantly ease the response task. For example, if there 

were a way to significantly increase the use of records on the survey, one could increase the 

overall length of the survey. In terms of cognitive burden, respondents will begin to fade after 

30 to 45 minutes when repeatedly asked to recall the detailed information that is required by the 

CEQ. This is not based on any scientific study, so the parameter of 1 hour may not be wellgrounded. It is based on conducting numerous cognitive interviews and focus groups. We have 

learned through this experience that to go beyond 1 hour in a cognitive interview is usually not 

worth the effort. Respondents become fatigued and are just not able to maintain the cognitive 

effort to produce quality data. Note that this experience is based on paid volunteer respondents 

who have made a special trip to a central location just to participate. If these respondents 

become fatigued at 45 minutes to 1 hour, a less motivated person would not last as long. 

Monitoring Interviewers 

The interviewer plays a central role when collecting auto-biographical data. Just as these 

questions require significant effort on the part of the respondent, they also require the interviewer 

to work. The minimum requirement is that they read all of the questions as intended. For 

example, the current CEQ interview relies on a large number of examples, or memory cues, 
3 

when asking questions about expenditures. The idea is that cues will help respondents search 

their memory in ways they may not have otherwise done. When asked a global question about a 

class of expenses, there may be a tendency for respondents to conduct cursory memory searches. 

This is sometimes called failures of “metamemory”, where respondents prematurely abort 

searches because of a false global feeling that they have nothing to report (Biderman, et al., 

1985). Methods to promote recall, such as the use of extensive cueing are ways to combat this 

tendency. 

Assisting respondents in an exhaustive memory search conflicts with the interviewer’s 

other main task of getting a high cooperation rate. Respondents may not be receptive to sitting 

through the entire interview, with all protocols being read. I suspect this is partly how the 

current interview is only taking 1 hour to complete. Interviewers may be taking shortcuts 

through the instrument to get a completed interview. 

There is evidence that increased monitoring of activities can be a deterrent to 

interviewers from taking shortcuts. Experimental and quasi-experimental research on the 

National Crime Victimization Survey (NCVS) found that centralized telephone interviewing 

produced significantly higher rates of victimization than de-centralized field interviews in two 

separate field experiments (Hubble and Wilder, 1988; Biderman, et al., 1985: 111-155). The 

assumption is that because the centralized telephone interviewers are being monitored, 

interviewers are more likely to read through the screener which elicits reports of victimization. 

An important component of the CEQ redesign, at least as it relates to promoting recall, is 

to institute methods to train, monitor and provide feedback to interviewers on an ongoing basis. 

Dijkstra et. al., (2008) discuss the importance of these aspects of the design when administering 

the EHC. The monitoring can be done through use of paradata from the computerized interview 

(e.g., section timings) and use of Computer Audio Recorded Interviews (CARI). The Census 

Bureau is now instituting CARI within their household surveys. Interviews can be recorded, 

either all or on a sample basis. Supervisors would review the interviews to check interviewer 

practices and provide feedback on performance. 
4 

Selected Questions Posed by the Issue Paper 

The remainder of this discussion will address selected questions posed by the Issue paper. 

Q1. What are the key aspects of recall period that CEQ should address during the redesign 

process? 

The Issue paper focuses on the length of the recall period as a key issue to be assessed. 

Research on the CEQ (Neter and Waksberg, 1964; Silberstein and Jacobs, 1989; Sudman and 

Ferber, 1971), as well as on other topics (medical events – Cannell, 1965; hunting and fishing - 

Chu, et al., 1992; and victimization – Bushery, 1981) all come to the same conclusion. Longer 

recall periods lead to more measurement error. For the CEQ, there are two important 

characteristics related to error. One is whether the expense is reported at all. The second is the 

detail associated with the event. The two critical details for the CEQ are the date of the purchase 

and the amount of money spent. 

Prior to and/or in conjunction with experimenting with different recall periods, the CEQ 

research agenda should concentrate on the optimal methods to promote recall. Research has 

shown that significant gains in validity can be realized through the use of methods that assist in 

the recall of events (Belli and Callegaro, 2008; Biderman et al., 1985). It would be useful to 

experiment with several different approaches to increase recall. As noted above, the current 

CEQ methodology uses a list of global questions covering each of the major expenditure areas. 

Within each area, the interviewer reads a set of specific cues to assist the respondent to define 

and remember relevant expenditures. The respondent is provided a show card that lists the 

information that is being read by the interviewer (Exhibit 1). 

Elaboration and/or modification of this methodology might look at several areas. One 

should be the types of cues provided to the respondent. Memories can be triggered by not only 

specific references to items, but also activities associated with purchases (e.g., shopping; 

vacations; commuting to work). Consideration might be given to changing, modifying or 

supplementing the cues that are currently being used. This might also imply a different 
5 

organization of the interview – for example by shopping trip or even retail outlet. A second area 

would be the use of a calendar. A calendar might be a particularly useful for improving the 

dating of events. It might be as simple as having the respondent provide meaningful dates for the 

reference period and training the interviewer to probe using the calendar when going through the 

list of cues. More elaborate protocols could also be applied (e.g., Belli, et al., 2008). 

A third area would be formalizing the role of the interviewer. A protocol should be 

developed for how the interviewer should move through the interview, with specific emphasis 

on promoting recall of expenditures. This would include directions on the proper pace and how 

to administer effective probes. It might also include methods to assist the respondent when 

trying to estimate or reconstruct expenses (e.g., recall of episodic memories vs. recurring events). 

A fourth area of research is the use of visual aids. The material shown to respondents of the 

CEQ should be optimized for memory search. For example, research using eye-tracking and 

response latencies (Redline, et al., 2009) has found grouping responses within major 

subheadings, such as the CEQ Information Card, leads to some confusion. Formatting the 

visuals aids like this should be tested and coordinated with what the interviewer is saying. One 

might also consider other types of displays, such as pictures of items, which might be easier to 

coordinate with interviewer questions. 

Q2. How should CEQ approach balancing low item-incidence levels and sample size 

requirements with a shorter recall period? 

If the CE is to move to shorter reference periods, and maintain the same level of 

precision, the sample size will have to be increased. It is possible that there will be some gains 

in precision per sampled unit with shorter reference periods, especially for smaller expenses. 

This gain would be achieved by an increase in the total expenditures reported per month. 

However, it is unlikely that this gain will make up for the reduction in the number of interviews 

that can be completed if no increase in the budget is realized. 

It is possible to address this problem by trying to incorporate more self-administration 

into the interview process. For example, if it were possible to convert the interviewer-
6 

administered instrument to an automated self-administered instrument, there could be substantial 

cost savings related to data collection. Going forward with this type of instrument would require 

significant investment in development and testing. The National Institutes of Health has been 

developing such an instrument for conducting a 24 hour dietary recall (Subar, et al., 2010). For 

purposes of the CEQ this development may not be within the Gemini Project Timeframe. 

Nonetheless, this should be an opportunity to begin this development process by focusing on 

effective memory aids (e.g., cues; frames of reference; visual displays). 

Assuming the interviewer is still heavily involved in the CEQ, there are designs that 

could be used to mix reference periods both within and across surveys (see discussion below). 

However, at a general level, the decision on the length of the reference periods is a function of 

two general issues. One relates to the overall project goals. As discussed below, evaluation of 

the reference periods should be completed using some measure of total survey error (e.g., mean 

square error). An important issue to be considered is whether priority should be given to 

minimizing error for estimates of level or estimates of change. The main evaluation measure 

seems to be the estimate of level. This is what is needed for the CPI weights and a primary 

measure of quality seems to be the ratio between the CE and the National Accounts (BLS, 2010). 

However, measuring change also seems to be an important criteria. When developing evaluation 

strategies to choose the reference period, it is important to design studies that could measure both 

types of errors. The length of the recall period may affect estimates of change and level in 

different ways. Some argue, for example, that if there is constant bias over time, estimates of 

change will relatively unbiased. Kobilarcik et al (1983) provides an example of this logic when 

evaluating the reference period of the National Crime Survey (NCS). A 3 month reference 

period had a lower MSE for estimates of level, but there was a strong suspicion that this was not 

the case for estimates of change. Since the highest priority for the NCS were estimates of 

change, a 6 month period was used. 

A second important issue will be deciding on the expenditure categories that have highest 

priority. Prior work shows very large differences in effects of the length of the recall period by 

the amount of the expenditure (Neter and Waksberg, 1964; Sudman and Ferber, 1971). Smaller 

purchases were much more subject to recall error than larger purchases. If this pattern holds for 
7 

the redesigned CEQ, decisions will need to consider priorities across the different expenditure 

categories. 

There are designs that can be used to mix different reference periods. One way is to 

change the content of the interview, either within the CEQ or between the CEQ and the CE Diary 

Survey. For example, there might be two different CEQ interviews. One would be conducted 

more frequently (e.g., monthly) and collect expenses that occur frequently. The other would be 

conducted less frequently, perhaps every six months, and would be used for low incidence items. 

One could imagine a similar split between the CEQ interview and the CE Diary Survey. This 

would essentially shift many of the more frequent purchases to the Diary Survey and keeping the 

less frequent purchases to the interview. The sample of the Diary Survey would be increased to 

account for this shift. The Interview Survey would be done fewer times (e.g., twice) and have a 

longer reference period. 

It is also possible to mix reference periods within the same interview. There are a 

number of variations on this methodology. Two examples are: 

Method A - Conduct the interview asking for all infrequent purchases for an extended 

period of time (e.g., 1 year, 6 months, last month). Frequent purchases would be asked 

for the last month. 

Method B. Conduct 3 monthly interviews for the same housing unit followed by a fourth 

interview 12 months later.. All four interviews would ask about frequent purchases over 

a one month period. The first three interviews would ask about infrequent purchases over 

a one month period. The fourth interview would use longer recall periods for the 

infrequent purchases (e.g., 3 months, 6 months, 12 months). 

There are, of course, a number of variations across these two models (eg. Different number of 

interviews; different recall periods). The critical difference in the two methods above is the 

extent the recall period is bounded by a prior interview. Method A does not have a bounded 

interview, while method B maintains a bounded interview for most interviews. We expect there 

to be significant telescoping effects for unbounded interviews across all types of expenses. Of 

these different models, therefore, Method B is preferable because it has the tightest control over 

telescoping. 
8 

A more unconventional option would be to vary the reference period within the same 

panel and use the information to statistically adjust the data from the longer reference period. 

For example, one could conduct consecutive monthly interviews and then conduct longer followups (e.g., 3 months or 6 months). One could then adjust the data from the longer reference 

periods to resemble the shorter reference periods (e.g., Kobilarcik, et al., 1983). The advantage 

of this is that it may not require as many additional interviews as the direct collection methods 

proposed above. The disadvantage is that it isn’t clear a robust adjustment model could be 

developed. There are a number of factors that would have to be taken into account, including 

any variations across important population groups and expense categories. Additional survey 

factors would also have to be considered, such as telescoping patterns and panel conditioning. 

Finally, it would be important to account for the sampling variability of the adjustment factors. 

Prior to planning an adjustment program, calculations would be needed to assess how precise the 

adjustments would have to be in order to significantly increase precision of the estimates. 

Q3. Are there any other tradeoffs that CEQ should balance when dealing with the topic of 

recall period? 

Q4. What are best-practices for determining ideal the recall period for the various 

expenditure categories within the CEQ questionnaire? 

Q6. What should the next steps be to explore and research this issue for a possible change in 

CE methods? 

The standard methodology for optimizing the length of the reference period has been to 

conduct field experiments. These experiments typically compare the performance of two or 

more recall periods that vary in length. For example, the current design of the CEQ is based on a 

study that examined the reporting of jobs and expenses for alterations and repairs (Neter and 

Waksberg, 1964). This study varied both the length of the reference period, as well as the 

bounding status of the interview. The Issue paper mentions that a field experiment is currently 

ongoing that is examining different recall periods. 
9 

Ideally, the R&D plan should sequence the testing so that final design decisions are based 

on a field experiment testing the recall period using an enhanced interviewing protocol. As 

noted above, the effect of the length of the recall period will partly be a function of the methods 

used to conduct the interview. For this reason, it would be important to conduct a final 

experiment on the length of the recall period once the protocol has been fully developed. The 

initial set of research activities should focus on methods to enhance recall, regardless of the 

length of the reference period. 

The research should make every effort to base the evaluation on a measure of total survey 

error. One such measure, for example, might be the mean square error (MSE). By definition, 

any measure of total survey error requires the collection of a gold standard to estimate the bias 

term. The gold standard that is used could vary by the scope and magnitude of the research. For 

small scale research (e.g., cognitive interviews; small field tests), one might use debriefing 

interviews which probe extensively on the time period. These debriefings could use methods 

similar to those used for an event history calendar. These smaller scale studies might also ask 

respondents to retrieve records as a way to validate reports that were given during the interview. 

Respondents would be administered the interview using recall without records and the debriefing 

would then use extensive debriefing, as well as review of selected records (e.g., credit card bills; 

computerized bank records) to measure the amount of error in the reports. 

Validating data for larger scale research, such as a study that compares reference periods 

of different lengths, is more difficult. One possibility would be to sample purchases from 

different businesses and conduct the CE with those individuals who made the purchase. This 

“reverse record check” design is not ideal because it only offers a limited view of the universe of 

purchases any individual might make (Marquis, 1984). It may also be difficult to locate 

individuals based on a record of purchases --- it would require getting cooperation from a retailer 

willing to recruit customers for a study. Investigation into different vendors who provide 

consumer information may also be another source of validation data. While these lists may be 

imperfect, randomization distributes the error evenly across treatments and could be used in the 

comparison if the error is not substantial. 
10 

A second possibility would be to ask respondents to keep a diary of their purchases over 

all or some portion of the target recall period. Once turning in the diary, respondents would be 

interviewed using the CE protocols. The information from the diary would be used as the gold 

standard, based on the assumption that the information was recorded close to the time 

expenditures were made. Ideally, the diary would be delivered and picked up in short enough 

intervals to insure that data quality is maintained. For example, one could follow the CE Diary 

Survey procedures and drop off and pick up diaries once a week. The advantage of this method 

is that it would cover a wide range of expenses including those that are of most concern for the 

CE (ie, frequent purchases). A more labor intensive method would be to conduct frequent 

interviews with respondents, using very short reference periods (Rips, et al., 2003; Millen, et al., 

2005). The disadvantage of these methods is that they are also subject to error, such as nonresponse and telescoping. It may also be the case that completing the interviews would lead to a 

greater likelihood the purchases will be remembered in a subsequent interview. Nonetheless, 

this type of design has been found to be effective in investigating issues related to the quality of 

retrospective recall. 

If it is not possible to collect a gold standard based on externally collected data, then one 

could make the assumption that higher expenditures are generally indicative of better reporting. 

This is based on the idea that for a bounded reference period the major concern are errors of 

omission, rather than over-reporting due to external telescoping. A similar set of assumptions 

were used during the redesign of the NCS described above (Bushery, 1981; Kobilarcik, et al., 

1983).

Is the “more is better” assumption justified for expenditures? The global comparison 

between the CE expenditures and the National Accounts seems to support this. Neter and 

Waksberg (1964) found that higher levels of expenditures were reported for shorter recall 

periods for the smaller expenses. For example, for their lowest category of expenses, the 1 

month recall period produced approximately 40% higher expenditures than a three month recall 

period. However, this was not the case for the larger expenditures, which did not show a 

statistically significant difference between the 1 and 3 month reference periods. Theoretically 

this makes sense because larger expenditures should be less subject to recall error. However, it 
11 

isn’t clear how this assumption would apply to expenses that are not recalled from episodic 

memory, such as regular expenditures like bill payments. 

One also has to be cautious about the “more is better” criterion when using enhanced 

interviewing and recall methods. These methods place greater demands on the respondent to 

actively search their memories. This might encourage external telescoping or at least bring in 

events that may not be eligible. For example, Kreuter et al., (2008) tested interleafed vs group 

filters. The former asks a single filter, which if answered positively leads to additional questions. 

The latter asks all filter questions at once. As expected, the group filter led to more reports of 

target characteristics. However, the group version did not prove to have a higher rate of 

validation when checked against external data sources. One possible explanation is that the 

group version encouraged respondents to report ineligible events which led to more external 

telescoping. 

7. How would you design a survey, or set of surveys, to collect detailed monthly data on a 

comprehensive range of expenditures? The proposed design should address recall period 

as well as other relevant survey design issues, while meeting the requirements specified in 

the CPI Requirements Document. 

With the existing information and cost constraints, it is hard to improve upon the current 

design of having two separate surveys. The logic used when it was first developed of using the 

interview to collect infrequent, larger purchases and the Diary Survey for the more frequent, 

routine purchases still rings true. To reduce burden on the Interview Survey, the design should 

seriously consider putting additional items on the Diary Survey. If necessary, the sample size for 

the Diary Survey would be increased to maintain required precision for the less frequent 

expenses. The Diary Survey should incorporate reporting by other members of the household, 

including children. It would be preferable to do this by having individual diaries for each person 

in the household, including children 12+. The Diary Survey would have several different modes 

of response, including the use of the internet. 

The Interview Survey would collect expenses for the less frequent purchases and institute 

a protocol that promotes recall. Interviewers would encourage respondents to use whatever 
12 

electronic or paper records they have. The interview could be done while examining these 

records on the computer or by reviewing bills. If it is possible, multiple respondents would be 

encouraged to respond, perhaps to the point of returning to the household if there is another 

person with significant expenses to report. 

After the first interview, respondents would be given the option of completing the survey 

on the web, in a self-administered format. As with the in-person survey, the survey would 

include protocols to promote recall. For example, there have been attempts to administer a selfadministered EHC for sensitive behavior (Martyn, 2008). A more sophisticated example of this 

type of application is an automated self-administered 24 hour dietary recall (e.g., Thompson, et 

al, 2010; Subar, et al., 2010). This is a computerized interview available on the internet that is 

designed to conduct large-scale nutritional studies. Development of a similar type of instrument 

to collect data on expenditures could be developed for either an application for the diary and/or 

for an interview using a longer reference period. In the context of the Interview Survey, those 

that do not want to complete on the web or do not have access, an in-person interview would be 

completed. Similarly, non-responders would be followed up in-person. 

Interviewers would be monitored using paradata. There would be follow-up when issues 

come up that violate the interviewing standards. For respondents, there would be selective 

follow-up of self-administered responses that indicate cursory or lack of effort in the response 

(e.g., missing data; low levels of expenditures; short timings). 
13 

References 

Belli, R.F. and M. Callegaro (2008) “The Emergence of Calendar Interviewing. A Theoretical 

and Empirical Rational.” pp. 31 – 52, in Belli, R.F., Stafford, F. and D. Alwin (Ed) 

Calendar and Time Diary Methods in Life Course Research. Sage: Los Angeles 

Belli, R.F., Stafford, F. and D. Alwin (2008) Calendar and Time Diary Methods in Life Course 

Research. Sage: Los Angeles 

Biderman, A. D., D. Cantor, J. P. Lynch, and E. Martin (1986). Final Report of Research and 

Development for the Redesign of the National Crime Survey. Prepared for the Bureau of 

Justice Statistics. Washington, DC: Bureau of Social Science Research, Inc. 

BLS (2010) Gemini Project Vision Statement. Paper provided for the 2010 CE Design 

Workshop. 

Bushery, J. (1981) “Recall Bias for Different Reference Periods in the National Crime Survey.” 

1981 Proceedings of the Section on Survey Research Methods, American Statistical 

Association. 

Cannell, C. (1965). Reporting of Hospitalization in the Health Interview Study. Vital and Health 

Statistics, National Center for Health Statistics, 2(6), U.S. Public Health Service. 

Casey, W. (2010) CPI Requirements of CE. Paper prepared for the 2011 Design CE Workshop. 

Cantor, D., and Lynch, J. (2005). Exploring the effects of changes in design on the analytic uses of the 

NCVS data. Journal of Quantitative Criminology 21: 293-320. 

Chu, A., Eisenhower, D., Hay, M., Morganstein, D., Neter, J. and J. Waksberg (1992) 

“Measuring the Recall Error in Self-Reported Fishing and Hunting Activities.” Journal 

of Official Statistics 8: 19-39. 

Dijkstra, W., Smit, J.H. and Y.P. Ongena (2008) “An Evaluation Study of the Event History 

Calendar.” pp. 257 - 275, in Belli, R.F., Stafford, F. and D. Alwin (Ed) Calendar and Time 

Diary Methods in Life Course Research. Sage: Los Angeles 

Hubble, D., and Wilder, B. E. (1988). Preliminary results from the National Crime Survey CATI 

Experiment. Proceedings of the American Statistical Association: Survey Methods Section, 

New Orleans, LA. 

Kreuter, F., McCullouch, S., Presser, S. and R. Tourangeau (2008) “The Effects of Asking Filter 

Questions in Interleafed versus Grouped Format.” Paper presented at the Annual Meeting 

of the American Association for Public Opinion Research. 
14 

Kobilarcik, E.L., Alexander, C.A., Singh, R.A. and G.M. Shapiro (1983) “Alternative reference 

periods for the National Crime Victimization Survey.” 1983 Proceedings of the Section on 

Survey Research Methods, American Statistical Association. 

Marquis, K. (1984) “Record Checks for Sample Surveys” pp. 130-147 in Jabine, T.B., Straf, 

M.L., Tanur, J.M. and R. Tourangeau (eds) Cognitive Aspects of Survey Methodology: 

Building a Bridge Between Disciplines. Report of the Advanced Research Seminar on 

Cognitive Aspects of Survey Methodology. National Academy Press: Washington D.C. 

Martyn, K. (2008). “Adolescent Health Research and Clinical Assessment Using SelfAdministered Event History Calendars.” Pp. 69 - 86, in Belli, R.F., Stafford, F. and D. 

Alwin (Ed) Calendar and Time Diary Methods in Life Course Research. Sage: Los 

Angeles

Millen, A.E., Midthune, D., Thompson, F., Kipnis, V., and A. F. Subar (2005) The National 

Cancer Institute Diet History Questionnaire: Validation of Pyramid Food Servings, 

American Journal of Epidemiology 163(3): 279 – 288. 

Neter, J. and J. Waksberg (1964) “A Study of Response Errors in Exepnditures Data from 

Household Interviews. Journal of the American Statistical Association, 59: 18-55. 

Redline, Cleo, Tourangeau, Roger, Couper, Mick, Conrad, Fred, and Ye, Cong. 2009. “The 

Effects of Grouping Response Options in Factual Questions with Many Options.” Annual 

Conference. Federal Committee on Statistical Methodology. 

http://www.fcsm.gov/events/papers2009.html accessed November 1, 2010. 

Rips, L., Conrad, F. and S. Fricker (2003) “Straightening out the Seam Effect in Panel Surveys.” 

Public Opinion Quarterly 67: 522 – 554. 

Silberstein, A.R. and C. Jacobs (1989) “Symptoms of repeated interview effects in the Consumer 

Expenditure Interview Survey.” In Kasprzyk, D., Duncan, G., Kalton, G. and M.P. Singh 

(eds) Panel Surveys. Pp. 289-303. 

Subar, A.F., Crafts, J., Zimmerman, T.P., Wilson, M. Mittl, B., Islam, N.G., McNutt, S., 

Potischman, N., Buday, R., Hull, S.G., Baranowski, T., Guenther, P.M., Willis, G., Tapia, 

R., and F.E. Thompson (2010). “Assessment of the accuracy of portion size reports using 

computer-based food photographs aids in the development of an automated selfadministered 24-hour recall”, Journal of the American Dietetic Association 110:55–64 

Thompson, F.E., Subar, A.F., Loria, C.M., Reedy, J.L. and T. Baranowski (2010) “Need for 

Technological Innovation in Dietary Assessment.” Journal of the American Dietetic 

Association, 110: 48-51. 
15 

Exhibit 1. Example of Cues from CEQ Information Booklet