CONSUMER FINANCIAL PROTECTION BUREAU 

INFORMATION COLLECTION REQUEST – SUPPORTING STATEMENT B 

COLLECTIONS OF INFORMATION EMPLOYING STATISTICAL METHODS 

 

 

TELEPHONE SURVEY EXPLORING CONSUMER AWARENESS OF AND 

PERCEPTIONS REGARDING DISPUTE RESOLUTION PROVISIONS IN 

CREDIT CARD AGREEMENTS 

 

(OMB CONTROL NUMBER: 3170-XXXX) 

1. Respondent Universe and Selection Methods

A telephone survey (including both landline and cellphones) will be conducted to 

explore consumer awareness of and perceptions regarding dispute resolution 

provisions in consumer credit card agreements. The respondent universe for this 

survey will consist of the approximately 160 million Americans who own credit 

cards. The survey will initially reach out to 7,000 potential respondents, in an effort 

to ultimately collect information from approximately 1,000 credit card holders 

(assuming a response rate of approximately 15%). Interviews will be conducted in 

English or Spanish, according to the respondent’s preferred language.1

The proposed telephone script is attached as Appendix A, revised based on user 

testing of the original and comments received as part of the 60-day Federal Register 

Notice process; two consultation rounds with interested parties representing both 

financial service organizations and consumer groups; focus group testing with two 

sets of consumer panels; and extensive user testing. The justification of each 

question is attached as Appendix B. 

The CFPB expects that the survey sample will be selected through random-digit 

dialing of residential and cellular telephones to obtain data from a target sample 

size of approximately 1,000 credit card holders across the country. Unlike an 

Internet-based or mailed survey, a telephone call requires real-time interaction, 

which minimizes the possibility that respondents will supplement their knowledge 

 1 In 2010, the U.S. Census released a report on language use in the United States in 2007. Language 

Use in the United States: 2007. United States Census Bureau. Released April 2010. The study found 

that 80% of the population 5 years and older spoke only English at home. Regarding the remaining 

20%, the population that spoke a language other than English at home, three-fifths spoke Spanish or 

Spanish Creole (defined as including Spanish, Spanish Creole, and Ladino). The remaining 

population was in a group of diffuse categories: “Other Indo-European languages,” “Asian and Pacific 

Island languages,” or “Other languages.”

Spanish language interview templates will be produced following OMB approval of this proposal and 

will be subsequently provided to OMB.
before responding to survey questions by searching the Internet or reviewing their 

credit card agreements. 

2. Information Collection Procedures

The sample design for conducting the approximately 1,000 telephone interviews will 

be based on a list-assisted random digit dialing telephone sample. In such case, 

analyzing the vast majority of the survey questions would involve calculating ratios 

based on data from the survey, for example how many interviewees (out of 1,000), 

picked a particular option. In general, confidence intervals are not known without 

assuming a particular functional form for the standard error of the population. In 

the case of ratios, and after making appropriate assumptions, one can use a 

Bernoulli distribution. The maximum variance, when both variables are equally 

likely, is 0.25.2 Based on that, a 95% confidence interval can be computed, with a 

confidence interval of at most +/- 3.1%.3 The confidence interval is actually 

narrower, given that answers are not equally likely. The exact bounds of the 95% 

confidence interval, however, will not be known until after the data are collected. 

The confidence interval described above is applicable for testing a single hypothesis. 

The CFPB will utilize the Bonferroni correction, or other widely acceptable and 

similar statistical methods, to arrive at the correct confidence intervals while 

testing many hypotheses simultaneously.4

Regarding landline telephone numbers, telephone numbers are ten digits long 

(AAA-EEE-XXXX), where the first three numbers are the area code, the second 

three are the exchange, and the last four digits are the number within the 

exchange. The area code, three-digit exchange numbers, and the first two digits of 

the four-digit suffix specify a “100-bank” containing 100 telephone numbers. ICF 

International will sample telephone numbers (using randomly-generated two-digit 

strings to complete the telephone number) from 100-banks that have been identified 

as containing at least three listed residential numbers, to ensure a relatively high 

hit-rate.

 2 This calculation is based on a binomial distribution where both outcomes are equally likely. 

3 See Peter Stopher, Collecting, Managing, and Assessing Data Using Sample Surveys, Cambridge 

University Press, Cambridge UK (2012). There are 1,000 respondents, the 95% level of confidence is 

equivalent to +/- 1.96 standard errors based on the normal distribution of the ratio statistic, and the 

maximum variance of the population is 0.25. This results in a confidence interval of the estimated 

statistic +/- x, where x = 1.96

�1000/0.25 ≈ .031.

4 If researchers are testing a sufficient number of null hypotheses, then at least one of them is likely 

to be rejected based on pure luck of the draw. Bonferroni correction, or a similar method, is utilized 

to ensure that a statistically significant result is not simply due to researchers testing so many 

hypotheses that some just happened to be true. 
The cellphone sample of telephone numbers will be drawn separately from 

telephone exchanges dedicated to cellphones.5

ICF International will normalize the collected data by geographic region, where 

feasible. 

The initial sample sizes would be large enough to yield the required number (1,000) 

of interviews nationwide.

Sampling would be conducted independently within each stratum. 

ICF International will seek a 30% to 70% sample split between cellphone-only 

households and households with a landline (the latter group including both 

landline-only households and ‘dual-users’ – those with both a cell and landline 

phone).6 This ratio reflects a balance between great representativeness of the 

sample population and cost, as cell phone interviews are significantly more 

expensive than traditional landline interviewing, due to legislative restrictions that 

require dialing of cell phone numbers and a higher rate of refusals, vis-à-vis 

landlines. 

ICF International will vary call attempts across the days of the week and times of 

day including weekdays, weeknights, and weekends. Groups of calls will be made 

at three separate times over a number of weeks, which should improve the 

likelihood that those who are travelling during some portion of the collection 

timeframe are available to participate in the survey and mitigate the risk that an 

external event alters consumers’ perceptions on a large scale. ICF International will 

also set their telephone dialing system to display a toll-free number that consumers 

can call back, in the event that consumers “screen” their calls. 

 5 Upon initial contact and introduction with cellphone respondents, the interviewer will immediately 

ask questions to determine that the person is not in a situation that could pose a safety risk to that 

individual (e.g., driving). If the contacted individual is found to be in a situation that could pose a 

risk, the interviewer will terminate the call and call back at another time.

6 The most recent data published from the National Health Interview Survey in December 2012 

shows that 35.8% of American homes relied solely on cellphones. Further, 15.9% of American homes 

received “all or almost all” calls on cellphones despite having a landline telephone. 

Demographic differences may alter these numbers, though. For example, 60.1% of adults aged 25-29 

lived in households with only cellphone telephones. 58.2% of adults renting their home relied solely 

on cellphones – twice the rate of adults owning their homes, 23.2%. 75.9% of adults living only with 

unrelated adult roommates were in cellphone-only households. Blumberg, Stephen J. and Julian V. 

Luke. Wireless Substitution: Early Release of Estimates From the National Health Interview 

Survey, January-June 2012. U.S. Department of Health and Human Services, Centers for Disease 

Control and Prevention, National Center for Health Statistics. Released 12/19/12 available at 

http://www.cdc.gov/nchs/data/nhis/earlyrelease/wireless201212.PDF. 
ICF International will attempt to contact sample members seven times (e.g., across 

multiple call groups), to ensure that the survey does not capture just those easier to 

contact and most likely to be home on the first attempt. 

When contact is made with an answering machine or voicemail, ICF International 

will leave at least two messages during the field period. The message would explain 

that the household has been selected to participate in a national study by the CFPB 

and ask that the contractor’s toll-free number be called to schedule an interview. 

The message will explain that participation is voluntary. The message will also 

reference a notice on the CFPB website, so prospective respondents can verify the 

legitimacy of the survey. The website would not collect any data from consumers or 

offer any tools for scheduling beyond displaying the contractor’s toll-free telephone 

number.

Once contact has been made via landline telephone numbers, one adult in the 

selected household will be chosen at random for participation in the survey. ICF 

International will make the selection based on the household’s “most recent 

birthday.” The “most recent birthday” method asks for the credit-card holding adult 

within the sampled household who, at the time of respondent selection, has the 

most recent birthday (the callers will not request the date of the actual birthday). 

Only that randomly-selected person will be eligible to participate in the survey. 

Randomization processes will not be used with cellphone respondents, where there 

is presumably a greater correlation between telephone numbers and specific 

individuals.

The CFPB will obtain approximately 1,000 replies to the survey. Given the unique 

nature of the survey, as described above, it is difficult to predict what the 

respondent reply rate will be with a reasonable degree of accuracy. The CFPB 

estimates that the response rate will be 15% or less, given that dispute resolution 

provisions are unlikely to be salient topics for potential respondents.7 The effect of 

 7 Certain commentators raised concerns about the original projections regarding response rates 

reflected in the CFPB’s prior submission to OMB. See ̧e.g., (AFSA Aug. 6, 2013 pp. 5-6; Shestowsky 

and Welsh, Aug. 8, 2013).

The CFPB’s projections are within the range of other estimates of expected response rates. See, e.g.,

Floyd J. Fowler, Jr., Survey Research Methods p.58 (4th Ed. 2009) (“[E]ven conscientious 

organizations sometimes achieve response rates in the 30% to 50% range – particularly when the 

subject of the survey is not immediately engaging. The difficulty of achieving high response rates is 

one of the intrinsic limits of [random digit dialing] as a sampling approach.”); Robert M. Groves, et. 

al., Survey Methodology pp. 187-88 (2d Ed. 2009) (describing non-response rates for the Survey of 

Consumers, conducted by the University of Michigan Survey Research Center, as growing “from 

percentages in the 30s in the 1980s to percentages near 60 more recently.” “Again, the lesson is clear: 

response rates have declined over time.”). Cf. Panel on a Research Agenda for the Future of Social 

Science Data Collection; Committee, et al., Nonresponse in Social Science Surveys: A Research 

Agenda p. 1-6 (2013) advanced copy available at http://www.nap.edu/catalog.php?record_id=18293 

(“Based on the above evidence, we are able to conclude that non-response rates continue on a long-
the reply rate could be minimized, however, by careful study of the demographics of 

participants to assure representative weighting across education, race/ethnicity, 

age, income, and gender. 

Multiple telephone calls to potential survey respondents, references to the CFPB 

website, and the prenotification letter to landline users, should help maximize the 

survey reply rate. 

The CFPB anticipates that questions about consumer income may result in a 

significantly higher non-response rate than any other survey questions. Question 

23, which seeks income information, is the second-to-last question, followed only by 

a query about the consumer’s gender. Accordingly, data from any “drop outs” may 

still be used. Consumers will also be instructed that they may refuse to answer or 

otherwise skip any questions that they do not to answer.

Each interviewer will be monitored for quality assurance throughout the course of 

the project:

• Audio monitoring will be used to maintain the quality of interviewer 

performance relating to matters such as:

o Initial contact and recruitment;

o Reading the questions as written;

o Reading (or not reading, where appropriate) response categories 

according to study specifications;

o Whether open-ended questions are properly probed;

o Whether ambiguous or unclear answers are properly clarified;

o Avoiding bias, either by comment, rate of speech, or by tone of voice; 

and

o General professional conduct. 

• The interviewers’ data entry will also be monitored to ensure that the 

interviewer enters the correct code, number, or verbatim response. 

As referenced above, the survey will collect data at three separate points in time 

over a series of weeks. In addition to assisting with gathering a wide demographic 

sample, this three-part collection also allows for contingency planning, in the event 

that design parameters fail to meet expectations in initial data collections. In that 

regard, if quality assurance monitoring or initial test results yield extremely 

problematic results (such as dramatically inadequate participation), the contractor 

 

term upward path, but we are concerned that solid evidence about the reasons for the decline is still 

elusive”) (emphasis in original).

As described above, the proposed survey has been simplified and shortened extensively, which 

should increase response rates. 
may suggest changes to the survey’s questions or format. In such limited 

circumstances, additional rounds of data collection may be added to ensure that 

consistent standards are used across the entire collection.

The CFPB anticipates that this will otherwise be a one-time survey. 

3. Methods to Maximize Response Rates and Address Issues of Non-Response

As referenced in greater detail above, at least seven attempts to contact potential 

respondents will be made before the calls are abandoned. Calls will be made at 

different times of day and different days of the week. 

A prenotification letter will be sent to those landline sample members for whom an 

address can be identified using outside data sources. The letter will emphasize the 

voluntary nature of the inquiry and encourage participation by stressing the value 

of the survey and how the results will be used. The letter will not, however, 

reference credit card agreements or dispute resolution provisions, so that consumers 

do not conduct research before the interview. A copy of the proposed advance letter 

is attached hereto as Appendix C.

The questionnaire was extensive tested to identify problems that could deter 

respondents from completing the survey:

• It was initially tested internally by the CFPB with a small group of CFPB 

employees to identify any problems that could deter respondents from 

completing the survey. 

• The Initial Questionnaire was made available with the CFPB’s original 

Federal Register Notice and request for comment describing the proposal on 

June 7, 2013. (Vol. 78, No. 110, Page 34352.) The CFPB received 

approximately 22 comments in response.

• The CFPB met with two sets of external stakeholders on July 8, 2013 to 

discuss their general feedback to the proposal and Initial Questionnaire. 

• In January 2014, the CFPB sought (and subsequently received) OMB 

approval to perform consumer focus groups and usability testing relating to a 

revised version of the proposed survey.

• On February 18, 2014, ICF International performed two ninety-minute focus 

groups with credit card holders in the Bethesda, Maryland area. The focus 

groups explored consumers’ general familiarity with issues raised by the 

proposed survey and the vocabulary used by consumers when they discuss 
them. The focus group panels were diverse with respect to age, ethnicity, and 

education.

• On March 11 and 12, 2014, the CFPB met with three sets of external 

stakeholders to discuss their general feedback on the Initial Questionnaire, 

after having revised the document to incorporate information learned from 

the focus group testing. 

• From March 24 to March 28, 2014, ICF International user-tested revised 

versions of the proposed survey in individual calls with a diverse set of 

consumers.

ICF International will emphasize to its interviewers that their manner of approach 

at the time of first contact is often the most important factor in securing the 

interview. Interviewers will rely on a carefully-designed introduction emphasizing 

trust and the importance of the survey, shown in Appendix A. Similarly, 

interviewers will be trained on how to answer the most likely questions for the 

survey. A hard copy of draft Frequently Asked Questions will be provided to each 

interviewer at their station. Interviewers will be trained to answer all questions in 

an open, positive, and confident manner. If a caller is reluctant, the interviewer 

could provide a reference to the CFPB website, so prospective respondents can 

verify the legitimacy of the survey.

When respondents refuse to participate, the interviewer will seek information 

regarding the refusal to participate. The interviewer will enter this answer and any 

other details relating to the refusal into the computer-assisted telephone 

interviewing software. 

4. Testing of Procedures or Methods

As discussed above, ICF International used focus group testing, followed by user 

testing to gather information and tailor survey questions. 

Pre-test interviews will also be conducted with nine or fewer respondents prior to 

formal administration of the survey, in order to test all of the survey systems, the 

survey instrument, and the computer-assisted telephone interviewing software.8

The pre-test will be conducted under conditions identical to the planned conditions 

for the administration of the survey. The issues to be addressed in the pre-test will 

include: question sequencing and skip patterns; completion time; the randomization 

of certain question elements; checking that invalid responses are not entered; and 

the full range of procedures. 

 8 Such testing will include providing all possible responsible categories for each question to identify 

embedded errors, as well as skipping issues. 
Following the pre-test interviews, the interviewers will report problems or 

observations they have with the project team. That feedback, combined with other 

data gathered from the pre-test, will be used to revise procedures, the computerassisted telephone interviewing software, and the survey questions, as appropriate. 

5. Contact Information for Statistical Aspects of the Design

The following individuals have reviewed technical and statistical aspects of 

procedures that will be used to conduct the proposed survey.

Will Wade-Gery 

Division of Research, Markets and Regulations

Consumer Financial Protection Bureau

(202) 435-7700

william.wade-gery@cfpb.gov

Alexei Alexandrov

Division of Research, Markets and Regulations

Consumer Financial Protection Bureau

(202) 435-7700

alexei.alexandrov@cfpb.gov

Kelvin D. Chen

Division of Research, Markets and Regulations

Consumer Financial Protection Bureau

(202) 435-7700

kelvin.chen@cfpb.gov